Lilla LoCurto and Bill Outcault have been invited as artists-in-residence at the University of North Carolina, Colleges of Computing and Informatics and Art and Architecture. While there they’ve been collaborating with faculty and students throughout the campus in diverse areas such as software, architecture, engineering, machine vision, psychology and dance on a project titled the willful marionette, for which the artists also received a Guggenheim Fellowship in 2014. The artists’ goal was to build, from the scanned image of a human figure, a stringed marionette that will respond in real time to spontaneous and interactive human movement, engaging with a viewer by reading their movements and expressions. The marionette was completed in May, 2014.
LoCurto & Outcault’s work has focused on the tenous nature of existence through the digital remapping and reconfiguration of the human body. Their practice has employed, among other technologies, computer graphics, three-dimensional scanning and motion-capture to explore issues related to this. In prior works, the artists animated the three-dimensional scans of dancers by attaching motion-captured choreography to them, creating virtual marionettes in video animations. The marionette’s strings in the willful marionette are manipulated by motors and software. There are two depth sensors that read and analyze the behaviors and gestures of participants that, in turn, cause the marionette to react. The puppet’s subsequent actions are designed to elicit further responses, creating an exchange focusing on the frailty and insecurities of the human participant and raising issues of contemporary relevance. LoCurto & Outcault’s interests in this project are more phychological than mechanical and the marionette has simple movements that show it to be aware of a person’s actions and to engage with them subtly. The artists’ intention was not to create so much a perfectly functioning robot but rather to imbue an obviously mechanically actuated marionette with the ability to solicit a physical and emotional dialogue with a viewer.
A marionette’s ability to abstract human characteristics and their disconnected relationship to us lends them qualities that are unnerving and fascinating at the same time. The marionette has always been a reflection of our own actions and it has played an important role throughout human history. It has always though, been a distortion of humankind. Their heads are bigger, fingers and noses longer and their movements exaggerated, making them “less human.” What would be considered inappropriate behaviors by human actors are acceptable, even expected, in marionettes. This and their performative aspect, in its scale and its historical implications, have been extremely intriguing to the artists. Contemporizing the marionette and its theater using robotics and computer vision to make it “more real” creates a changed dialogue with the viewer and allows the interaction between them to become psychologically more intimate and potentially more vulnerable. Additionally, by utilizing several advanced technologies in the service of a traditional art form, this project bridges a gap in the contemporary art experience between the static viewing of the past and the increasingly interactive and dynamic experience to be found in work that allows the art and the viewer to interact in unpredictable ways.
A meaningful exploration of emerging technology
Computer vision has become commonplace and we are increasingly surrounded by machines that can not only see and sense, but soon, reason. the willful marionette is a link in this technological evolution. The marionette is an analogue device and will still be operated by strings, however its manipulator will be a machine. Through external sensors and its mechanical operator, the marionette will engage with a viewer in unpredictable and provocative ways. This will add to a discourse on the development of a mechanical consciousness and, as human control over this consciousness decreases or is abdicated, our relationship to it.
The relationship between technology and culture
the willful marionette employs a historically traditional and accepted medium, the marionette theater, to approach the dialogues it wishes to inspire. Through the marionette’s confrontation and physical exchange with the viewer, the project broaches issues of mechanical surveillance, our increasing communication with computer devices and the possibilities of artificial intelligence. Through the responses of the marionette, the participants become aware of being “watched” as they communicate with the puppet. As these responses become quite varied, participants are also required to confront the reality of the marionette’s intellect in its ability to react to them. The accessible and ineractive nature of the willful marionette affords numerous opportunities for public engagement. Being based around a marionette gives the project an immediate psychological familiarity that allows it to be shared in venues outside the main institution. Its technological nature and the process of developing the interactive apparatus could provide a number of iterations or prototypes and create opportunities to actively solicit from the public feedback and input as to possible actions and character attributes of the marionette.
Produced data for other artists, technologists, or arts organizations
In addition to the three-dimensional scan data, the willful marionette produces data in the mechanical manipulation of the marionette, the machine vision technology in reading the participants, the algorithmic in the software to actualize the project and the psychological in determining the marionette’s actions and the participants’ related behaviors.
The artists thank the following people and departments at UNCC for their invaluable contributions to the project: (College of Computing and Informatics): Mary Lou Maher, Chair, Software and Information Systems; William Ribarsky, Chair, Computer Science; Kazjon Grace, post-doctoral researcher; Alexander Adams; Mohammad Javad Mahzoon; Trevor Hess; Yueqi Hu; Lena Lee; (College of Arts+Architecture): Ken Lambla, Dean.
This text is based on Lilla LoCurto & Bill Outcault’s artist statement and a series of exclusive materials provided by the artists in private © Lilla LoCurto & Bill Outcault; adaptation by Sabin Bors / November 8, 2014.