The Last Millimeter: When the Screen Becomes the Problem

The Last Millimeter

We spent decades improving the world behind the screen. What happens when the only thing left to improve is the fact that we are standing in front of it?

There is a particular cruelty to the window of a good pastry shop.

Everything has been arranged to defeat distance. The éclair is close enough to inspect. You can see the glaze catching the light, the small imperfection where the chocolate has settled, perhaps even the condensation beginning to form around something colder behind it. Nothing important has been hidden from you. Light crosses the boundary freely. Information crosses it. Desire certainly crosses it.

Only you cannot.

Put your finger against the glass and something peculiar happens. Your fingertip can occupy almost exactly the same place, from your point of view, as the object you want. A millimeter or two separates success from failure. The distance is so small that it feels almost insulting.

That may turn out to be one of the strangest problems technology has ever created for itself.

The Window We Learned to Forget

The earliest computer interfaces did not pretend very hard.

There was a machine and there was you. You sat in front of it and issued instructions. Keys were pressed, commands entered, pointers moved. The separation was obvious enough that nobody needed to discuss it. The computer was over there. You were over here.

Then we began removing the evidence.

The keyboard gave some of its authority to the mouse. The mouse surrendered part of it to touch. Touch began giving way to gesture, voice, gaze and movement. A child using a tablet does not necessarily experience the glass as a command surface in the way an early computer user experienced a keyboard. The child touches the thing itself. Or, more precisely, behaves as though this were true.

That distinction matters.

Good interfaces have a curious ambition: they seek their own disappearance. The door handle succeeds when we stop thinking about the handle and simply enter the room. A steering wheel succeeds when the driver thinks about the bend rather than the rotation of a mechanical control. A touchscreen succeeds when we forget that our finger has touched nothing except glass.

Technology improves by making mediation feel less like mediation.

For most of computing history, that has seemed like an uncomplicated victory.

It may not remain one

When Looking Becomes Being There

Virtual reality makes the old problem unusually visible because it does not merely place an image before the eye. It attempts to coordinate perception and action closely enough that the brain begins treating a constructed environment as somewhere rather than merely something.

Turn your head and the world turns correctly around you. Reach and a virtual hand follows. Move closer to an object and its apparent size changes as expected. Sound acquires direction. Increasingly sophisticated haptics can add another layer of correspondence.

None of this requires the virtual table to become a physical table.

That is the important part.

Human perception is not a customs officer demanding proof of material authenticity before allowing an experience across the border. Give the perceptual system sufficiently coherent relationships between vision, movement, sound and action, and something remarkable can happen: an artificial environment begins to acquire the psychological character of a place.

But then reach for the table.

Your hand passes through it.

For one second the machinery returns.

The illusion has not necessarily failed because the graphics were poor. It has failed because one part of your body asked a question that the system could not yet answer.

And so, naturally, we improve the answer.

The Better the Machine, the Stranger the Mistake

More resolution. Less latency. Better tracking. Spatial audio. Haptic gloves. More accurate reconstruction of hands, faces and rooms.

There is a temptation to imagine that the culmination of these technologies will be spectacular: dragons indistinguishable from animals, impossible cities, planets constructed while we watch.

Perhaps.

But there is another, quieter measure of technological mastery.

A table.

A completely ordinary table that looks right when you approach it, sounds right when something is placed upon it and feels right when your fingers touch its edge. A window through which afternoon light enters at precisely the angle it should. A human hand whose movement contains none of the tiny errors that remind you that it is being rendered.

The better such systems become, the less impressive they may need to look.

And that is when the remaining errors become strange.

Imagine a wall-sized display of a room continuing seamlessly from the room in which you are standing. The perspective is perfect. The light matches. People on the other side respond naturally. Sound behaves correctly. Nothing announces itself as artificial.

You approach.

Closer.

Closer still.

Until your nose almost touches the surface.

After perhaps a century of computation, billions of transistors, global networks, machine learning, motion tracking and extraordinary feats of engineering, the most obvious problem remaining might be one of the oldest objects humans know.

A sheet of glass.

You are still on this side.

Remove the Glass

Fine.

Remove it.

This is where the thought experiment becomes more interesting, because our instinct is to imagine that removing the final pane completes the journey. The barrier has gone. The frame is empty. The hand passes through.

But the hand remains.

So does the eye.

So does the skin that distinguishes contact from its absence, the proprioceptive system that tells you where your limbs are, the vestibular system that helps establish orientation, and the nervous system through which the event becomes available to you at all.

We removed one interface and exposed another.

This does not prove that the human body is merely a screen, terminal or replaceable biological peripheral. That conclusion would run far ahead of what we know. We do know, however, that our experience of embodiment is more negotiable than ordinary intuition suggests. Experimental illusions involving virtual limbs and bodies demonstrate that the sense of what belongs to us can be influenced by carefully coordinated sensory information.

That should make us cautious in both directions.

It would be premature to declare the body an interchangeable carrier.

It may be equally premature to declare it the final border.

Crossing one pane does not abolish boundaries.

It changes what the next boundary is made of.

The Screen Was Never Just the Screen

Perhaps this is why the history of interfaces looks different when viewed backwards.

The keyboard was not simply replaced because keyboards were bad. The mouse was not a historical mistake. Touchscreens will not become absurd merely because something eventually replaces them. Each was an extraordinarily successful answer to a particular distance between intention and consequence.

Then that distance became noticeable.

So we shortened it.

Again.

And again.

The pattern suggests an uncomfortable possibility. We may not actually be travelling toward a world without interfaces. We may be travelling through a succession of interfaces that become transparent enough to reveal the one underneath.

Keyboard.

Mouse.

Controller.

Glass.

Eye.

Skin.

Body.

And here we should stop.

Not because we have reached the answer, but because this is exactly where knowledge ends and interpretation begins.

Want to Know More?

How to Read the World

By Alistair Brightworthton

Current Knowledge

Immersive technologies can produce powerful experiences of presence, and coordinated visual, motor and tactile information can influence how people experience virtual environments and even aspects of bodily ownership. Interfaces have also moved repeatedly toward more immediate forms of interaction, reducing the perceptual distance between intention and action.

None of this demonstrates that consciousness can exist independently of the biological systems currently associated with it.

Open Questions

How far can technologically mediated presence go before the distinction between operating a representation and inhabiting an environment becomes psychologically unimportant?

How flexible is the human sense of embodiment?

If increasingly sophisticated interfaces can make external machinery disappear from experience, is there a principled point at which that process must stop?

And most importantly: is the body the observer standing before the final screen — or is it another screen whose transparency we have never had reason to notice?

We do not yet know.

Transhumation Interpretation

Transhumation does not require the claim that human beings will escape matter.

The more interesting pattern may be almost the opposite.

Again and again, we preserve a function while changing the thing that carries it. Information leaves one medium and appears in another. Controls disappear while control remains. Interfaces are abandoned while the relationship they enabled survives in a new form.

We do not escape carriers.

We learn to change them.

Perhaps the body is different. Perhaps somewhere beneath skin, perception and nervous system there really is a boundary that cannot be crossed without destroying the very thing attempting to cross it.

That possibility should remain open.

But so should the other one.

One day we may remove the keyboard, the controller, the headset and finally the glass, only to discover that the object we confidently called ourselves was standing in the same historical position as every interface before it.

Perfectly natural.

Completely indispensable.

Until something made the next boundary visible.

You have reached the edge of the world.

The edge was you.

For now.

Continue the Transhumation Series

• Forgotten Religion | Ancient Gods, Symbols and the Hidden Structures of Human Civilization
• The Last Religion | Death, Immortality and the Future of Humanity
• The New Theurgy | AI, Information and the Future of Human Civilization
• End of Branding: You Were Never Meant to Become a Product
• Baal vs Ishtar: Why Ancient Gods Never Disappeared