What these systems cannot do
Fluent text is easy to mistake for a mind. The limits below are about the systems, not about your prompts being "wrong."
They do not check the world unless you add a check
A language model predicts text. It does not, by itself, open a laboratory, read a meter, or confirm that a cited paper exists. If you have not added a tool or a person to do that checking, a confident answer can still be false. This is not a rare glitch at the edge of an otherwise truthful mind. It is a consequence of how the models work.
The practical habit is simple. For any claim that has to be right (a legal date, a medical fact, a citation, a number you will act on) treat the model as a drafter, not as a source. Read the actual source. If no source is offered, that is information too.
They do not have a stable self, and they do not care
A model can write in the first person. That is a pattern in the text, not evidence of a self that persists between sessions and holds aims of its own. It does not want to help you, and it does not want to harm you. It produces the continuation that its weights and its input make likely. If a product around the model stores a memory or applies a policy, that is the product, not a soul in the weights.
This matters because people can be moved by a sentence that sounds loyal, scared, or loving. The honest stance is to be polite if you wish, and to remember that you are talking to a function. Anthropomorphism is a human habit. It is a poor guide to what the system will do next.
They do not reliably know when they do not know
People often hope that a model will say "I don't know" when the data is thin. Sometimes it does, especially if you ask it to. Often it will produce a fluent guess instead, because fluent guesses were a successful way to continue text during training. Uncertainty is not always marked in the output, even when it should be.
You can ask for sources, for a lower-confidence phrasing, or for a list of what would change the answer. Those prompts help. They do not install a guaranteed honesty module. An agent can look things up, and can still misread what it finds.
They do not replace the energy, the machines, or the judgment
A model can propose a plan for a robot, a power system, or a piece of research. Someone still has to provide the watts, the hardware, and the decision to act. Type 2 Civilization and Kardashev II are about the energy. Robot Motion is about the machines. This site is about the software that sits on top, and about not asking that software to be the whole stack.
Judgment stays with the people who use these tools. That is not a slogan about human uniqueness. It is a statement about responsibility. If a model drafts a wrong plan and you run it, the consequences are yours to handle. The useful systems will be the ones that make good work easier to start, and that leave the checks in place. They will not be the ones that ask you to stop looking.
Common questions
Will the next generation of models remove these limits?
Some limits can be reduced with better training, better tools, and better checks. The core point on this page still holds unless the mechanism changes: a system built to predict text is not, by itself, a reliable window on the world.
If a model sounds certain, should I trust it more?
No. Certainty in the wording is not the same as a check against a source. Treat confident style as a writing choice, not as evidence.
Are these systems useless, then?
No. They are useful for drafting, summarizing, exploring code, and starting a plan, especially when you can verify the result. The limits tell you where that usefulness stops, not that it was never there.