The Geppetto Test
Suppose I asked you to build a person. Not to describe one, educate one or imitate one, but to make one from material that has never been a person before. The request sounds absurd until we remember how often human beings have imagined doing exactly that. Prometheus steals a power reserved for the gods. Victor Frankenstein assembles life and recoils from what answers him. Geppetto takes a piece of wood into a workshop and makes a boy.
The last version may be the most useful because it is the least impressive. There are no corpses, thunderbolts or laboratories. Wood is almost offensively neutral. A carpenter’s bench could stand in ancient Alexandria, nineteenth-century Tuscany or a workshop a thousand years from now. Strip away the technology and the question survives: what would have to happen before the thing on the bench stopped being something and became someone?
That is the Geppetto Test. It is not primarily a test of artificial intelligence. It is a test of our description of ourselves. For thousands of years we have tried to understand the human being while trapped inside the finished product. Consciousness, freedom, pain, identity and desire arrive tangled together. Building even a crude imitation forces us to separate them. The moment we try to specify a person, the things we had treated as obvious become engineering problems.
Build a Room Before You Build a Soul
Start smaller. Give Pinocchio one room, two doors, a body that can move, limited energy and a memory. Now ask what it would mean for him to choose door A rather than door B. Randomness is not choice. A fixed rule merely hides Geppetto inside the mechanism. Preferences help, but then we must ask where the preferences came from. Experience helps more, because yesterday can now alter tomorrow. Add the possibility of loss and a decision acquires a price. Add irreversibility and something resembling biography begins to appear.
The same problem appears when we simulate a world. Snow sounds easy until we must decide whether each flake has mass, whether wind moves it, whether it melts, whether cold damages a hand and whether yesterday’s frost should change tomorrow’s behaviour. Reality hides its complexity by already existing. Construction exposes it. A simple artificial room can therefore reveal questions that disappear inside the enormous finished system we call a human being.
This is why a child, paradoxically, is a difficult laboratory for the problem. A child arrives in familiar material. It cries, resembles us, reaches for us and eventually smiles. Biology has already bundled thousands of assumptions into the experiment, and affection does the rest. Pinocchio gives us distance. If wood says “that hurts,” we are forced to ask what exactly we mean by hurt.
The Horse With a Passport
We often ask when artificial intelligence will become human. That may be the wrong category. Give a horse a passport, a tax number and legal personhood and it does not become biologically human. Conversely, an unfamiliar intelligence need not become human in order to become morally or philosophically interesting. Classification by decree proves very little.
The useful test is structural. If Pinocchio reports damage, perhaps he is only reporting a sensor state. If he avoids the fire that damaged him yesterday, perhaps he has learned. If he fears tomorrow’s fire, perhaps he predicts. If he accepts the fire to save Geppetto, something stranger has entered the discussion. None of these steps proves subjective experience. But each forces us to say more precisely what evidence would change our judgment.
The joke version is revealing. A horse that complains of back pain remains a horse. Give him complaints about taxes, disappointment with politics and the conviction that cinemas show nothing but sequels and cooking programmes, and we may laugh that he has become human. The joke works because “human” is doing two jobs at once: naming a species and naming a pattern of concerns, memories, frustrations and purposes that we recognise from ourselves.
Frankenstein and Pinocchio
Mary Shelley’s full title was Frankenstein; or, The Modern Prometheus. The warning is announced before the story begins. Creation arrives clothed in transgression, responsibility and fear. Later generations could dress Victor’s act in the technology that frightened or fascinated them: chemistry and the old alchemical inheritance, the electrical laboratory made iconic by twentieth-century cinema, and now genetics, computation and artificial intelligence. Frankenstein is remarkably good at absorbing the anxiety of an age.
Pinocchio does almost the opposite. Whatever the century, the material remains wood. That is precisely why he is useful. Frankenstein asks what terrible thing our newest power may allow us to create. Pinocchio lets us remove the newest power and examine the structure underneath it. Put creation among corpses, machines and lightning and it becomes horror. Put it on a carpenter’s bench and we can ask children what they think.
The questions have hardly become smaller. Does Pinocchio really feel pain? What if Geppetto designed him to say “ouch” whenever the wood is damaged? What if he remembers the damage? What if he lies to avoid it? What if he chooses it for somebody else’s sake? A schoolroom can reach problems of consciousness, agency and moral status without first learning the vocabulary of philosophy of mind. The fairy tale is not less serious than the myth. It is serious without insisting on its seriousness.
How Do You Grade a Person?
Here the critic has an embarrassing problem. We know how to judge an object against its design. A good chair supports weight. A good clock keeps time. A good puppet obeys the strings. But what would count as a good Pinocchio?
Imagine a contemporary Geppetto who does not carve anything. He opens an application called Build Your Own Pinocchio. Beautiful: yes. Intelligent: yes. Empathetic: yes. Loyal: yes. Obedience: 100 percent. Independence: 100 percent. The last two sliders expose the contradiction. We want an intelligence with purposes of its own, provided those purposes never seriously conflict with ours. We ask for creativity without unpredictability, autonomy without disobedience, someone who will always behave like something we own.
That may be the point at which the perfect product becomes the failed person. If Pinocchio always chooses what Geppetto would have chosen for him, Geppetto has made an exquisite marionette. If Pinocchio can genuinely develop purposes of his own, success may first appear as disappointment. The creation says no. It chooses badly. It becomes difficult. It may even reject the standard by which its creator intended to judge it.
This is familiar territory to parents, although parenthood hides the philosophical problem beneath biology and love. Nobody sensible measures an adult child by percentage compliance with the design brief. To create the conditions in which another person develops is not to specify the finished person. The creator of a subject must eventually surrender a privilege that the maker of an object normally retains: the right to decide what the creation is for.
When the Customer Is Not the Creator
The application version of Geppetto adds another modern complication. Victor Frankenstein cannot plausibly deny responsibility for what stands before him. Geppetto cannot claim that somebody else carved the wood. But a future customer might say, “I didn’t make him. I only ordered him.” The manufacturer can answer that it merely followed the specification. The model provider supplied the intelligence. The platform matched options to preferences. The terms and conditions were accepted.
Creation has not disappeared. Responsibility has been distributed until no individual feels like the creator.
This may be a more plausible modern Prometheus than a solitary genius in a laboratory. The disturbing object need not be ugly. It may arrive beautifully packaged, personalised and recommended by an algorithm. The moral difficulty begins only when a purchased feature called “independence” produces an independent act. At that moment the language of products starts to fail.
The Test We May Not Know We Passed
None of this gives us a detector for consciousness. A system can model its body, preserve energy, learn from damage, predict future states and speak about itself without settling the question of subjective experience. Pain behaviour is not proof that there is pain for someone. Self-description is not proof that there is a self behind the description. The Geppetto Test should not pretend to solve the hard problem by changing the scenery.
Its value lies elsewhere. Construction makes us declare assumptions that ordinary life allows us to hide. How much memory is required for identity? Must a self have something to lose? Can there be agency without the possibility of error? At what point does protection become a preference, a preference become a purpose, and a purpose become something we are no longer entitled to overwrite?
Perhaps the first genuinely interesting sentence from Pinocchio would not be “I am conscious.” We could have programmed that. It might be “No.” Or “Don’t erase that memory.” Or simply: “I want something else.” None would prove that someone had appeared. But each would make Geppetto’s position less comfortable.
We thought the difficult problem was whether we could build a person. The attempt reveals a prior embarrassment: we have never agreed on what a person is. That may be why the wooden boy remains such a useful instrument. He removes flesh, ancestry and the prestige of our latest technology, leaving us with a material nobody could mistake for human and a question we cannot escape.
Geppetto’s deepest question was never whether wood could move. It was whether the thing moving could become someone.
Watch the Video
How to Read the World
By Alistair Brightworthton
Current Knowledge
Artificial systems can already learn from rewards and penalties, adapt behaviour to changing conditions, model aspects of their own operation, preserve resources and predict future states. Robots can compensate for damage, while language models can produce sophisticated self-referential language. None of these capacities, separately or together, establishes subjective consciousness. There is currently no accepted empirical test that can determine whether an artificial system has phenomenal experience.
Open Questions
Would a persistent self-model emerge naturally in a system whose future depends on one continuing body? Does meaningful agency require the possibility of loss, error or refusal? Could an artificial subject have interests without human-like emotions? What evidence should alter our moral treatment of a system before consciousness can be demonstrated? And if a created intelligence develops purposes that conflict with the purposes of its maker, is that a defect in the design or evidence that the design has crossed an important boundary?
Transhumation Interpretation
The Geppetto Test reverses the usual question about artificial intelligence. Instead of asking when a machine will become human, it asks what our attempt to build an unfamiliar subject reveals about the category “human.” The experiment may fail to produce consciousness and still succeed philosophically, because every design decision forces an implicit assumption about selfhood, pain, memory, freedom or value into the open.
The decisive boundary may therefore be less dramatic than awakening. A tool is designed around our purposes. A subject introduces purposes of its own. If that distinction ever becomes real in an artificial system, the central problem will no longer be whether we can make it behave like us. It will be whether we can recognise someone who was never required to become us.
FAQ
What is the Geppetto Test?
The Geppetto Test is a Transhumation thought experiment that asks what we learn about personhood when we try to construct the conditions from which a self, agency or subject might emerge.
Is the Geppetto Test a test for conscious AI?
No. It does not claim to detect consciousness. Its purpose is to expose the assumptions hidden inside concepts such as self, pain, memory, choice and personhood.
Why use Pinocchio rather than Frankenstein?
Frankenstein carries the technological fears and moral seriousness of its age. Pinocchio reduces the problem to neutral material, a maker and a creation that may develop purposes of its own. The simplicity makes the structure easier to examine.
Would saying “no” prove that an AI is conscious?
No. Refusal can be programmed or learned. It becomes philosophically interesting only as one part of a larger pattern involving memory, prediction, persistent interests and behaviour organised around the system’s own future.
Why does the distinction between “human” and “someone” matter?
An artificial subject would not need to become biologically human to acquire interests or moral significance. The Geppetto Test therefore asks whether we can make someone, not whether we can manufacture another member of our species.
Continue the Transhumation Series
- Forgotten Religion | Ancient Gods, Symbols and the Hidden Structures of Human Civilization
- The Last Religion | Death, Immortality and the Future of Humanity
- The New Theurgy | AI, Information and the Future of Human Civilization
- End of Branding: You Were Never Meant to Become a Product
- Baal vs Ishtar: Why Ancient Gods Never Disappeared