If I give you a glass of water, what do you notice?

You might think, “It’s just a glass of water — what should I notice?”

There are many things to notice. You can see the amount of water in the glass, but you can also feel it in the way the water sloshes about when you pick up the glass. You can feel the weight of the glass. It has substance.

Now think about your hand. How did you grab the glass? This might depend on the size and the shape of the glass. Does it have a handle? A stem? There are many ways to hold any kind of glass.

Our hands have an incredibly rich and expressive repertoire, and we improvise constantly without the slightest thought. Just think about how you hold your toothbrush, eat food, or write with a pen.

Now take out your phone. What do you notice? What do you feel?

The screen is made out of glass, just like the glass of water. But does it give you the same feedback as other objects in the real world? How do you hold it, grab it, or make use of it?

When using mobile devices, we are actually just moving objects below a screen made of glass. This is what Bret Victor calls “Pictures Under Glass”. What other object in the world behaves like this?

We are always peeking through a rectangular screen, a black mirror, into the digital world. We’re not engaging in a full sensory experience. We have reduced ourselves to peering eyes and swiftly moving fingers with a singular focus on the digital world.

We are walking around with supercomputers in our pocket — which itself is pretty amazing. But it’s also pretty disappointing that we reduce our interaction with computers to a pair of eyes and a single finger, when we have an entire body at our command.

Is this the way we have to use computers? Will our future be a world of screens, our minds bound to them, and unable to see the world around us?

This isn’t the future I want, and it’s not just me: many others have thought along the same lines.

The Marble Answering Machine, created by Durrell Bishop in 1992, features small marbles that act as a container for your information that are usable by all kinds of different devices — move the marble from your answering machine to your phone to return the call, place the marble next to an audio speaker to hear a song or voice notes, or next to your display to see relevant information in a message, such as maps or invitations. I really like how this changes the nature of a digital system into something more direct and immediate.

Mark Weiser’s approach on ubiquitous computing uses a shift to many digital devices and systems, which are simultaneously and seamlessly integrated into everyday activities without the user necessarily being aware of it — in contrast to today’s trend of having a single device that you carry wherever you go. Weiser’s main argument is that computers should be ubiquitously integrated into the environment, so that we focus on solving our problems and not on the computer.

Ivan Poupyrev is working on transforming everyday objects such as clothes and furniture into interactive surfaces, like a jacket you can use to answer a phone call or a plant you can play like a musical instrument. Any object can be turned into a gesture interface — without modifying the actual object. I love the vision of deeply connecting the digital and physical worlds without having to rely on our phones.

Dynamicland — a research group in Oakland, California — are turning the computer world upside down, by not focusing on a single computing device for everyone, but by building a communal computer that can be used by people working together. They are inventing an entirely new medium, where people work together with real objects in the real world, brought to life by technology built into the room. It’s a space where everyone gets their hands on everything, where you play, trade ideas, and work together with all the skills you possess.

I’m not saying we should throw out our computers, phones, and tablets. They’re good. They’ve allowed many people to use computers to their advantage. But if this is all we have for the next few decades, it would be a failure of our collective imagination.

What we need is a dynamic medium that we can see, feel, and manipulate in a direct way. I’ve mentioned just a few interesting approaches, but many more exist. The good ideas are already out there — it’s just a matter of getting society to adopt them.