form 282

Google Soli

Interview: Jessica Krejci

Simulation is applied throughout the entire design process, not just to detect possible errors at an early stage, but also to open up new areas of application for (innovative) technologies. One example of this is Google’s Soli project: a specially manufactured interaction sensor uses (high-frequency) radar to track hand movements to control products.

The gestures imitate movements from our known physical world to make it easier for users to communicate, learn, and remember the procedures. We spoke to Leonardo Giusti, head of the design team at Google ATAP (Advanced Technology and Projects) who is also responsible for the Soli project. With a thesis on human-computer interaction at the Università degli Studi di Firenze, research work at MIT and practical work experience at Sano Intelligence and Samsung, Giusti demonstrates interesting approaches to the use of simulation in the design process.


With Soli it seams like simulation replaces reality. Can simulation be a vision for the future?

I think with Soli we have an interesting take on simulation. In Soli we have used radar technology to track the human hand and recognise gestures. In order to make it easy for people to learn and remember the gestures, we have designed them as a simulation of physical tools. Typically, we try to anchor them as something people are already familiar with like pressing a button. But if you look at the interface from a historical point of view – not just Soli, but also other interfaces – this is actually a very typical approach. Think about touch user interfaces for example, they started as skeuomorphic simulation for the physical reality; in the 1990s a button looked like a button, right? However over time, as people got more and more familiar with these kind of touch interactions, this initial convention could be broken. So nowadays, a button doesn’t have to look like a button anymore. This gave the designer freedom to come up with new interaction patterns that can be designed to elaborate the unique quality of the underlying technology.

I believe that simulations are design elements in the context of an interactive system and are basically a temporary phase. Usually, that is what happens in the beginning of the process to get people familiar with a new technology. But as the technology evolves and as people get more familiar, the simulation begins to evolve to form a new reality itself with its own rules and conventions. This is great as it provides the designer with the opportunity to experiment and explore new possibilities that exist only on that specific medium. That’s why I think you achieve better and more interesting design solutions.


Is it possible to translate this back into the physical world?

I think it might be possible to translate this specific interaction that you developed in a different medium back to the physical world. If you think about Soli, we can use inspiration gained from the design of touch interfaces to come up with new chances, for example, to translate simulation from one medium to another taking different shapes as they evolve.


Coming from a material reality it might seem that our world is becoming a virtual reality. Do you think this is why simulation takes a more important role today?

This is a very interesting question and I have been thinking about this. I can only give you my personal answer to this: in short, I would say yes, but I don’t think this is a kind of sure simulation or sort of a collective hallucination that we all live in. If you think about what is happening in the past years, they viewed technology as innovation that made it possible to embed computational capabilities into everyday objects and I think this is going to change the way we think and make sense of our material reality.

For example, we are used to think about our material reality as fixed, you buy a car and its functionalities don’t change over time. With software it is completely different: when you buy new software you get updates, it can be changed and it is really malleable. When hardware and software are coming together in everday products, I believe that this difference between the material and the digital world is in fact collapsing and becomes less and less relevant. A good example is Tesla: they change the driving features of the car through a software update. So the physical experience of driving a car can be actually changed over night.

You could say that our material reality is somehow getting closer to the virtual reality. Meaning that our reality inevitably has possibilities that are not yet expressed. The material reality or physical artefacts can actually change over time and are not fixed anymore. They don’t have to be the same as when they came out of the factory, because even after being shipped the material reality or the material experience of these products can evolve. So in this sense, we can actually talk about a virtualisation of the material reality.


This also creates new opportunities for the product life cycle.

If you really want to push this into the future, you can think about this new development in programmable matters and the fact that you can also control the atomic structure of a material through software. So when you buy a product, you don’t just have one product, but something that can become many different things, things that are not yet expressed.


How would you define simulation?

Generally speaking, I think we can define simulation as a kind of imitation of the behaviour of a system in an environment that you can control, where you can change some variables and see what happens. Usually, you have a model that you want to study and then you put this model into action and observe how it behaves over time. And it doesn’t matter if this happens in a software environment or in the real world, as long as you have a model that represents certain aspects of a system and this model can evolve and play out.


So the goal of simulation is to test your ideas?

Yes, it is basically testing the idea and trying to understand what are potential problems and opportunities before you actually build the final system.


What is the difference between a prototype and a simulation in this context?

A prototype is a model of an artefact or a system that you want to study. A simulation is when you take this model and you put it into action. So in the design of interactive systems – which is what I do – you usually put this model into action with users in the loop. So the way we use this type of simulation is that we use it to discover problems and opportunities in relation to people.


Do you always start with a specific question?

That is one thing you can do when you have a prototype. But I think we can also use simulation in different ways in the design process and in a more creative way. For example, in Soli we actually used simulation even before we had the system working. As we were actually trying to understand what gestures made sense, we were asking people to confirm these gestures via a device. To see what they will do in that specific situation, we had a person hidden behind the scene to simulate the gesture with the device – this is called Wizard of Oz and is a typical methodology in interaction design. This led us to actually support this different interaction even before the engineering finalised a working system, which is extremely helpful because it can help us designers orient and also influence the way technology is taking.


How are prototypes and simulation different compared to an experiment?

If you think about traditional scientific methods and the way experiments are done in natural science or psychology, I would say that they may or may not involve a simulation process. Sometimes you do experiments where you have two conditions and you try to understand if they support your hypothesis or not. But in social sciences you can use simulation and ask people to pretend and act in a certain way and this type of approach is what we mainly use in the design interaction system. We ask people to pretend something. So, I would say that simulation is more a technique and a process and an experiment is more the approach and the methodology you use to derive your results.


How real does a simulation have to be in order to be accepted by users as a potential reality? Or vice versa.

I believe that as long as you set the proper context, it doesn’t really matter how close to reality your prototype or simulation is. In the design of interfaces, designers sometimes just use paper prototypes and ask people to simulate a certain task. As humans we are really good at pretending. From a design point of view it is important to understand that different types of prototypes will lead to different types of results. Of course, as you move closer to the shipping deadline, you want the fidelity of your simulation to go up and get as close as possible to the final product.


Would you say that design and engineering define simulation differently?

Simulation always depends on the domain you are in, but in my experience for interaction designers the main difference is that I usually use simulation with humans in the loop. For me it is important to have the final user as part of the simulation. Engineers use simulation to define and set the properties of a system that may not include a human being. At Soli, for example, we used simulation a lot to test how people interact with the radar; but our radar engineers used simulation to find out how the radar behaves in different enclosures.


Thank you so much for the interview.


form 287
Women and Design

form Design Magazine

Grid one form287 cover 700px cover Order now


Verlag form GmbH & Co. KG
Sophienstraße 26
60487 Frankfurt am Main

T +49 69 153 269 430
F +49 69 153 269 431

Subscription Service

Questions to your subscription invoice, to our subscription program or a change of address?

T +49 69 153 269 438
F +49 69 153 269 439


Sign up now and get access to exclusive ticket draws, monthly news about the magazine as well as information on exciting design events and publications.

Sign up

Design Magazine
Established 1957