Why AI confidently lies — and how to spot it in time

An AI system can write a convincing answer and still get the name of a law, a medication dosage, or the date of an event wrong. The problem is not that the model has “decided to deceive” the user. It has no intentions in the human sense at all. Its task is to continue the text in the most likely way, drawing on patterns in its data and on how the request is phrased.

This creates an uncomfortable paradox: the better a model is at writing, the easier it is to mistake an invention for a fact. Polished style, a confident tone, and a long explanation are not evidence. For users in Kazakhstan, this is especially apparent when asking about local taxes, registration rules, prices, public services, and medicines: the model may have little up-to-date data and sometimes substitutes similar rules from another country for Kazakhstan’s.

What actually happens inside

A language model is trained on a huge amount of text. When generating an answer, it receives context and estimates which piece of text is most likely to come next. It then selects that piece, adds it to the answer, and repeats the process. That is how a paragraph, email, or set of instructions gradually takes shape.

The model does not open an encyclopedia in its head with a card labeled “truth.” It stores complex statistical relationships between words, concepts, and styles. As a result, it may correctly continue a sentence about a familiar fact but get a rare detail wrong. For a model, the line between “I know” and “this sounds true” is not as clear as it is for someone checking a document.

There are technical reasons too. Training data contains errors, contradictions, outdated pages, and texts with no sources. A request may be ambiguous. And if a user asks for five studies, the model may treat the number five as an instruction to provide exactly five titles—even when some of those studies do not exist.

Generation settings also affect the likelihood of invention. With more randomness, answers become more varied, but the risk of inaccuracies rises. With less randomness, text is usually more predictable, but that does not turn it into a verified reference source. A model can repeat the same mistake with complete confidence.

Which inventions can be dangerous

The most harmless version is a wrong film release date or a mix-up involving an official’s position. Such a mistake is irritating but rarely changes anyone’s life. The situation is different when someone makes a medical, legal, or financial decision based on the answer.

Consider this request: “I have a temperature of 39°C. Can I take two different medicines at the same time?” A model might name an incompatible combination, give the wrong dosage, or overlook a symptom requiring urgent medical attention. Even a good answer here remains general information, not a doctor’s consultation.

A legal hallucination sounds more authoritative. A user asks: “Which article of the RK Code allows me to return this product?” The model may cite a real code but invent the article number or transfer a rule from Russian legislation. Worse, it may fabricate a court decision and provide a link to a page that does not exist.

False conditions are dangerous in financial matters. For example, an AI may confidently say that a particular transfer to Kazakhstan is subject to a specific tax or promise a guaranteed return on a bond. Rates, limits, and requirements change; an answer without a date and a link to an official source should not be treated as grounds for action.

There are everyday examples too. AI may invent that an airline allows a certain item in carry-on luggage, name a nonexistent Almaty–Bishkek bus route, or provide an old schedule for a Public Service Centre. These mistakes cost less, but they can still ruin a trip and waste time.

Why the answer sounds so confident

Confidence in the writing is a presentation feature, not a measure of accuracy. The model was trained on explanations, reference works, news, and dialogues, so it is good at reproducing the shape of an expert answer: an introduction, arguments, a conclusion, and a list of sources. If there is no reliable fact available at the right point, it may fill the gap with a plausible-sounding construction.

Precise details without confirmation are especially suspicious: regulation numbers, book pages, DOI numbers for academic papers, company names, and direct quotations. A model can assemble a plausible combination from familiar elements. The result may be a real author, an article in a real journal—and a nonexistent publication within it.

Context can also push the model toward an error. If you ask, “Why does Law X violate consumer rights?”, the question already assumes that Law X exists and violates those rights. The model may fail to check the premise and immediately start proving it. The more insistently a user asks it to “stop hedging and give a specific answer,” the stronger this effect becomes.

Signs that AI is probably making things up

The first signal is excessive specificity where the model provides no source. The text gives a date, clause, and statistic but no official page—or includes a link that leads only to a website’s homepage. Open it. If the document is not there, that is not a minor issue.

The second signal is internal inconsistency. At the beginning of the answer, an event is dated 2022; later, it is said to have happened in 2023. One paragraph gives one amount, while the table gives another. You can ask the model a short follow-up question: “Check whether there is a contradiction between points 2 and 4.” But repeating the request does not replace external verification.

The third is evasiveness around a simple fact. Instead of saying “I don’t know,” the model uses phrases such as “researchers believe” or “according to some reports,” with no study title or author behind them. Be cautious when a complex subject is explained too smoothly and without caveats: real regulatory or scientific information often includes conditions, exceptions, and an effective date.

It helps to ask: “Which parts of your answer can’t you confirm?”, “As of what date is this information current?”, and “Give links only to primary sources.” The model may still make mistakes afterward, but it becomes harder for it to hide gaps behind general language.

How to verify answers in practice

Break the task into two steps. First, ask AI to explain the topic or prepare a draft. Then separately verify the facts that affect the decision. For Kazakhstan’s laws, look for the text on an official government resource; for a schedule, check the carrier’s website; for medicine, consult the instructions and a doctor. The publication date and the last-updated date matter.

Do not copy your own assumption into a prompt as if it were an established fact. Instead, write: “Check whether this rule exists and provide an official source.” Ask the model to separate confirmed information from assumptions. For calculations, provide the initial data and verify the result with a regular calculator or spreadsheet.

Comparing several models can help catch some errors. If different systems independently name the same rule, date, or formula, confidence increases, but the statement does not become true: the models may have been trained on the same erroneous text. The next step is therefore to compare the answer with the primary source.

In QueryWise, you can ask one question to several models and see the differences in one place. The current top four in the ranking are Claude Fable 5, GPT-5.6 Sol, Kimi K3, Grok 4.5; Claude Fable 5 has a score of 9.8. We added this comparison on launch day: answers often matched on simple questions, while queries involving local rules produced discrepancies immediately. It is a useful way to spot a questionable fact before it finds its way into an email or work document.

QueryWise has no subscription: a question costs 10–75 ₸ on average, new users get three free questions, and the interface works in Russian and Kazakh. The service saves time during an initial comparison, but it does not replace verification by a doctor, lawyer, accountant, or official government body.

A simple rule for important decisions

The higher the cost of a mistake, the less acceptable it is for an answer to be your only source. A post idea, translation, or draft email may be fine after a quick editorial review. A medical symptom, legal article, tax calculation, or contract term requires a primary document and a specialist.

AI is useful as a conversation partner, research assistant, and editor. But replace the question “Does this sound convincing?” with “What is this based on?” If you cannot find a source, the claim remains a hypothesis—even when the text is written without a single stumble.