Interview with Patrick Tresset: Simulating Humanness
Humans are said to be creatures of habit. This saying refers to routine behaviours that are hard to abandon, having become established over longer periods of time. As well as influencing our everyday actions, such habits may also exist in the world view shaped by our personal and cultural contexts. Calling this into question is not easy, but also in design, reversal and subversion may be effective means of drawing attention to social wrongs, highlighting alternative views or developing practical improvements.
What are you actually investigating when building robots that are able to draw portraits of human beings?
I have been working on the same idea for the past 16 years, so there have been different phases.
Originally, it was purely artistic research. In 1999, when I was still a painter, I started thinking about how to make machines produce drawings. I had lost my passion for making art, leading me almost naturally to explore the possibilities of machines doing what I no longer could. What first motivated my research entails that to this day there is still an underlying biographical element to my work and the robots. The robots can also be seen as being some form of self-portraits. I realised rapidly that to get interesting or truthful results, processes have to be simulated. Rather than trying to get a precise or premeditated result it would be better to simulate processes such as perception, motor control, feedback etc. and see what happens. At that time it was an intuition. What I learned during this first phase was that it would be possible to get interesting results and that it could be my artistic practice. I also understood that I needed to acquire better research skills, and knowledge of computer sciences, and other fields such as perception, cognition and so on.
Then, I joined the masters programme at the Goldsmiths College Computing Department as a mature student in 2004. During the two years of my studies I worked on the same idea of producing drawings with computation. I acquired computational and research skills and became acquainted with technologies such as computer vision and researched into psychology and perception. My final project and thesis was developing a software, which produced simulated line drawings of faces. I published a couple of research papers and some drawings in the press. This attracted some attention to my work. However, although the drawings were produced by the software, I did not feel that they were interesting drawings, just perhaps good illustrations. It was not clear to me back then why I did not see them as artistic work. This project provided a proof of concept for the next stage.
Following this, I was encouraged to embark on a doctoral research. We got a three-year research grant from the Leverhulme Trust in 2009. This grant was to investigate the sketching process through computational modelling. In the application for the grant, we did not include robotic components. However, after the first year of research, when I investigated how people draw and how people perceive drawings, it became obvious that the model had to be embodied in a robot. One of the reasons for this is that a drawing is the result of an ensemble of gestures. It is the memory of the artist’s actions and this affects how the drawing is perceived by the observer. In a certain way, the complexity of robotics simplified the problem of developing a computational model. The questions shifted to developing a system capable of producing drawings that have the same effect on the viewer as those produced by a human, without being pastiches.
One of the theories about the appreciation of drawings is that the observer perceives the intentions of the artist – not only the conscious intentions but also the covert “intentions” that govern perception and motor control. The drawing is perceived in such a manner due to the characteristic of each trace which contains in its visual appearance the history of the movement that created it. This implies that to be appreciated as interesting, a drawing should be the result of processes that could be perceived as intentional or deliberate and performed in the physical world. For the marks to have recoverable memory, they have to contain in themselves traces of the velocity, pressure, and direction of the drawer’s hand.
I first began to experiment with robotics in 2009, then I exhibited a robot for the first time in 2010. Although the drawings produced by the robots were not corresponding with my ambitions, the work attracted a lot of international attention. I got the first satisfying results in 2011, just before having a solo show at Tenderpixel Gallery in London. Since then, the robotic installations have been exhibited widely including major museums such as the Centre Pompidou, Tate Modern, Israel Museum, Victoria and Albert Museum, Science Museum, and other events such as Ars Electronica, Update_5, London Art Fair, Kinetica and Istanbul Biennial. Recent exhibitions include the MMCA, Bozar, Laznia Center and La Maison d’Ailleurs. Forthcoming exhibitions include a solo show in London.
What resources flew into the building of the robots?
The components I use are relatively low-cost made by a Korean company called Robotis. I purchase the parts and then assemble the arms. This affordability enables me to develop installations with multiple robots. Also during the testing phase the risk of damaging the robot is not a problem, so it does not require time-consuming care which slows down the development when working with more expensive robots. I am now beginning to get some parts manufactured specifically to build my robots.
I write the software, or more precisely, the system that controls the robots’ behaviours. This system relies on existing open source libraries and frameworks for machine learning, computer vision, and robotics, and the main code is programmed in a programming language named Python. I use the same system for all my work; drawings, paintings, videos and for controlling the robots’ behaviours. This computational system has been growing and evolving since 2008.
What were the main challenges while developing the robots?
It is very different to develop a software (with no body) or a system that controls a robot’s behaviour. A robot exists and acts in the real, physical world – and reality is very complex and unpredictable, almost defective. As a result, even when all bugs have been removed, there is still plenty of scope for things to go wrong. Making a robot do something is interesting in its own right. You realise how highly sophisticated and adaptable animals are, including humans. While trying to manipulate a robot, you understand more and more what it means to be human.
For example, after building my first arm and getting it to move, the next step was to get it to draw a line. Although this seems like a very simple action, it is not easy to program without some experience in robotics. Even though we do not realise, but a lot of factors influence the mark-making process such as the friction of the tip of the pen on the paper, the roughness of the paper, or the weight of the pen. All these elements influence the dynamic of the arm and the forces that need to be applied to trace a line. As the robots draw from observation, it adds another layer of unpredictability due to changes of light and other factors.
As the robots I develop are exhibited in public and sometimes for a lengthy exhibition, they have to be very reliable and produce relatively interesting drawings most of the time.
Do the Paul robots differ from each other?
The Pauls have numbers – Paul II to Paul X so far. And then, some have letters, for example Paul III.a, b, c, d, e, f, etc. The different numbers are dependent on the style of the school desk standing for the robot’s body. Most of the Pauls have the same morphology: a body constituted by a vintage school desk, bolted on it a left arm on which is attached a black BIC pen. Also attached to the table there is a motorised eye. So far, the computational system controlling the robots has been constantly updated. Therefore, it is the same for each Paul.
How was the overall reaction of the exhibition visitors to the installations?
When I first showed a robot in public, I was surprised by the people’s fascination and how they connected with it; especially a robot that drew from observation. People are not scared by robots. When in their presence, people perceive robots as being somewhat alive, as having agency. As soon as a robot is in public, it becomes an actor; an actor that can simulate living beings with stylised behaviours that evoke humanness. This is what I play with when creating the installations I exhibit in museums and galleries. I am still surprised how easily we allow ourselves to be fooled by a bit of machinery just because it looks at us and can perform an activity we can identify with.
Drawing is definitely one of these activities that we can intimately relate to. It is often seen as an indicator of intelligence that we can trace from prehistoric times. When there is a drawing on a rock or a cave wall, we immediately instinctively know that intelligent humans created it. When we discovered the drawings in the Lascaux cave, what we initially considered as primitive beings became our close ancestors. It is perhaps the same mechanism that puzzles, amazes, and fascinates people when they see robots drawing, especially drawing humans.
Are you developing the Paul robots further?
Yes, robots are constantly evolving. However, I am planning to redesign the system’s architecture to be able to easily integrate technologies such as reinforcement learning, the aim being for the robots to be able to develop their own style. I am just finishing an artwork in collaboration with the Polish artist Goshka Macuga titled Before the Beginning and After the End for her solo show at the Prada Foundation in Milan where I will be exhibiting some new drawing robots which are ambidextrous and mobile and also presenting large scale scrolls covered with drawings. The show is titled To The Son Of Man Who Ate The Scroll and it runs until 19 June 2016. There are other upcoming exhibitions in Canada, France, and Belgium. And I am involved in a new collaboration with Oliver Deussen’s team from the University of Konstanz, where I will be doing a residency in the next few months.