Coverage, Not Complexity

There is an observation that sooner or later everyone makes who works with AI more than on weekends. In an hour, a model takes apart a problem you would have spent a month on. And right after that it can't put together an extract from your internal document, something yesterday's intern would handle.

What makes AI so unstable?

Out of habit we sort tasks by complexity: this one is simple, that one is hard. And we expect the tool to handle the simple ones with confidence and stumble on the hard one, the way a person would.

But with AI the logic is different: what matters is how much similar material people have written and made available in texts, above all in the training data. I call this scale coverage.

An olympiad problem is hard, but it has been written up thousands of times: it has a genre, canonical solution techniques and a criterion of a correct answer known in advance. Writing your internal policy is a simple task, but it is written up nowhere: the document is tailored to your business, and only three people will use it internally.

Three layers of novelty

The argument about whether AI can handle the new is pointless until three different things, all of which get called novelty, are pulled apart.

The first layer is a new instance of a known genre. The rules are set, the answer format is set, the criterion of correctness is set. Here the model works with confidence.

The second layer is a new combination of what is already described: carrying an idea from one industry into another. The model has every industry in its knowledge base at once, and it will find an analogy between fields foreign to each other faster than any specialist.

The third layer is new knowledge: a question nobody asked before you, where it is unknown in advance what a correct answer should look like. The model simply won't ask that question by itself. It will cope under a person's step-by-step guidance and carry out the steps. But in that case the question and the criterion of correctness are held by the person, not by the model. This is a property of the design, not a flaw of the current version.

The most honest example of the third layer

A model closed an open mathematical problem from Erdős's list, one that had stood unsolved for decades. A strong result.

Just look at who did what. Erdős asked the question eighty years ago. The problem came from a collection of Erdős's open problems curated by mathematicians, already precisely stated. Other mathematicians checked the answer, and then improved on it. The model went from a ready question to a checkable answer, and on that stretch it beat people.

Nobody asked it to pose the question or to decide whether the question mattered. That is the boundary between the second layer and the third: solving a problem that has been posed and posing a problem that didn't exist are different kinds of work.

"But novelty tests are already being passed"

A fair objection: "There are tests built from problems the model has never seen, and this year it solves them with scores above the human level."

The objection falls away once the layers are separated. Such a test is made of new instances in a known genre: the transformation rules are set, the answer format is set, the criterion of correctness is fixed in advance. That is the first layer, and closing it doesn't contradict the thesis. About the third layer no test can say anything: one cannot be built, because it is unknown in advance what counts as a correct answer.

A practical criterion

In practice many people run into a simple observation. When you have a clear picture and understanding of what you want, and have more or less built the whole thing in your head, AI is great to work with and can even look like an alternative to a person. But when you don't know what you want yourself, or you are tired and not ready to think the task through, the result will be mediocre.

And here a simple rule can be stated.

Before handing a task over, look at how much similar material has been written and is openly available. A lot: hand it over, and the result will be strong regardless of complexity and of how precisely you thought it through. Nothing: you'll get plausible emptiness, and the simplicity of the task won't save you.

For tasks that involve real novelty the criterion is different: is there a way, known in advance, to check whether the answer is correct, and are there similar lines of reasoning for solving the problem? If there are, go ahead, the tool will cope with the hard stuff too. If not, this is your territory. There the machine is the packager, not the author. And the key criterion here is how clearly and precisely you yourself picture the end result and can put into words what you want.

A rare process the model does solve

Take a rare internal process that appears in no source but reduces to a known class of problems: an assignment problem, a matching problem, a standard contract. It looks like "my unique case, the model is powerless here". And the model copes, and copes well.

Because the model hasn't read about your process. It has read about the class of problems your process reduces to, and it found the connections and the similarity. The axis runs between what fits into what has been described and what doesn't, and not at all between "mine" and "someone else's". So the first thing worth doing with a task is to try to reduce it to a class. If the reduction works, the quality of the answer goes up many times over, and you haven't changed a single line in the model.

Who closes the gap

People have a gap between the described and the undescribed too: in an unfamiliar field we are just as helpless. The difference is who closes it.

A person closes it themselves: asks themselves the question, reads, tries, makes mistakes, asks others, and reaches an acceptable level in a field they knew nothing about. And from then on they keep using that skill.

A model has two paths, and both lie outside it. The first is further training: it is done by other people's hands, once every few months, centrally, and not on your data. The second path is you: you formulate what you want, provide the data, build the working infrastructure, check the result. It works here and now, and all the practical value of AI today rests on it.

The second path has a price that press releases don't mention: the setup has to be repeated every time. The AI model itself remembers nothing; all of that is crutches made of external files. A scenario you have debugged doesn't move to the neighbouring task by itself.

What this has to do with general intelligence

Let's put it together. The strength of the tool is proportional to how much people wrote about the task before you. The tool cannot move this axis from the inside: the weights don't change after training. The gap gets closed either by the factory once every few months, or by a person, by hand, every time.

General intelligence, by any reasonable definition, is the ability to take on what you haven't seen and weren't designed for, to carry a task through to a result under changing conditions, and also to form a lasting skill on your own and use it in the future. Today's tool can't do the last part: a person closes that gap for it.

Of course, nobody has proof that general intelligence is impossible with current technology. I am describing how the tool behaves today, and why talk of general intelligence arriving soon fits poorly with that behaviour.

And the condition under which I will admit I was wrong. A system poses a question on its own, decides on its own that the question matters, and specialists in the field accept its result as new knowledge rather than as a well-solved assigned task. When that appears, the thesis will have to be rewritten, and I will rewrite it.