ARTWORK

Tinker is a theatrical performance by real-time, autonomous artificial characters set on the facade of the Stone Opera House for the LUMA Binghamton 2017 projection arts festival. Designed as a theatrical stage for dancing characters, the historical Stone Opera House is the perfect canvas for a narrative play by hand painted, self-animated characters. Tinker was developed to explore the idea of autonomous dance, where the characters decide and choreograph their own movements in real-time. Set to a progression of musical works from historic to modern these digital characters choose their motions within each music genre, resulting in novel performances at each showing.

The Stone Opera House is a historic building located in Binghamton, NY. The building was designed in 1892 by Charles M. Stone who felt that central New York was in need of a decent theater for entertainment. At its high point from 1900-1920, the building hosted famous performers including Sarah Bernhardt, Ethel and John Barrymore. At one point President Roosevelt spoke here himself. Despite its fame the building is currently in need of major repairs. The LUMA Projection Arts Festival revitalizes the Binghamton area by bringing these treasured spaces back to life. TINKER in particular was designed as a narrative, musical play to reflect and honor the theater on which it is performed.

CHARACTER ANIMATION

The now forgotten plays that once took place in the Stone Opera House provided the inspiration for a virtual dance performance. For Tinker we developed live, artificially intelligent dancers to perform a new play. Unlike other projection mapping projects which are animated and pre-rendered, the goal of Tinker was to allow the characters themselves to decide and choreograph their own actions. Each performance is unique. As the performance is a 10 minute play for a live audience, there is an expectation of a narrative arc. Narrative cohesion was provided with music that was used to influence, but not strictly determine, the dance motions chosen by avatars. Each character imagines and transitions from its current motion to the next dance move by choosing from a variety of motions based on the current scene.

Character animation is based on the free Carnegie Mellon motion capture library which contains pre-recorded moves for several dance styles. Custom software was developed to enable character animation with real-time performance. This technique disrupts the common method of facade projection mapping by studios which develop pre-rendered animations. One difficulty with live avatars is that character behavior can be unpredictable, for example when characters decide to dance off-stage. To counter this the notion of ‘center stage’ was created to direct motions back on stage. In addition, to improve group choreography, a lead dancer is occasionally selected to guide the others in a dance motion that is performed synchronously.

MODELING & LIGHTING

The set and stage were developed from photographs of the real building, which were rectified (made flat), and then used to develop a 3D virtual of the Stone Opera House. By building the 3D model in parts, the building could be deconstructed and moved in sync with the music toward the end of the piece. Since the entire piece is a real-time, digital performance, stage lighting was added with the use of custom C++/OpenGL shaders designed to mimic stage spot lights. Animated characters were then integrated into the virtual model to perform in front of the virtual, stage lit set.

MUSIC AND NARRATIVE

The musical performance and narrative of Tinker is a sound collage on the theme of transition from the classical to modern world. Musical selections determine the narrative, and include Rachmaninoff, Dvorak, Tango, Polka, and Waltz which give way to Dub step, DJ and experimental tracks. The word Tinker is a casual reference to bricolage, the French term for improvisation developed by Levi-Strauss, and also a reference to the magic and mischief of the sprite Tinker Bell from Barrie’s Peter Pan. Following the theme of theatrical performance the first half of Tinker has ballet performers with theatrical lighting set to classical music. Toward the middle and end, the music becomes digital and experimental and the performers switch to karate while the architectural facade breaks away to expose structure and form. This transition expresses the 21st century shift from the industrial to the post-modern world.

FINAL PERFORMANCE

The final performance was presented at LUMA Binghamton 2017 projection arts festival to an audience of several thousand viewers. Thanks to the LUMA organizers and an amazing ground crew, Tinker was setup in two days with a high lumen projector and sound equipment across the street from the Stone Opera House. The live performance was then synchronized with the building and presented multiple times that evening for festival attendees.