Project Description
The central theme of my art is to create what I call a “zero-interface” experience that is a meditation on bridging the natural human gesture with the virtual world created by logic. My thesis contains three parts–hand drawing, body composing and Kinect IO. Hand drawing captures the user’s motion and creates a swarm drawing correspondingly in real time. Body composing allows the user to generate sound in real time using his or her shadow to precisely trigger music notes under the edge of a motion path. Kinect IO creates an entertaining and immersive environment that lets people control a set of digital objects with simulated physical properties by using their body to interact within a virtual environment. Above all, my project explores the “zero-interface” which builds the sensation of a strong connection between code and visuals, virtual and reality, computer and human.
Body-Composing and Hand-drawing are both screenbased installation and browser-based interaction work.
Kinect IO is a screen-based installation written in C++.
  • Body Gesture--- Kinect IO
  • Body Gesture --- Body Composing
  • Body Gesture--- Hand Drawing
Content Detail:

Body Gesture --- Body Composing
Body Composing is an installation that utilizes hand and body movement to compose music in real-time. Viewers control the grid of mapped Midi-notes on the screen.The participant sees 2 layers. The first is semi-transparent note-mapping grid on top of the screen. The second is the filtered user image.The movement of the body is represented as a bright outline which triggers which the note to play. By waving their hands, the viewer creates music in the air. Natural gestures create a dialogue with the computer and produce expressive audio-visual artwork. Body composing is written in Actionscript 3.0.
Body Gesture--- Hand drawing
Hand-drawing is an screen-based installation that uses a camera to capture a user's hand motions and and transform them into an aesthetic painting. The viewer creates a drawing pattern with natural waving gestures. Several stokes, each with with their own behavior, follow the position of the moving objects in front of camera. The experience changes as the painting develops. At first, the viewer will experience the joy of using natural movement to control the drawing and then as the number of strokes increases, the underlying drawing begins to reveal. The viewer then experiences the unique, non-repeatable process of a particular artwork. The work is built upon Hype framework developed by Branden Hall.
Body Gesture ---Kinect IO
Kinect IO, an interactive installation, presents an entertaining and immersive environment. Viewers control a set of digital objects with simulated physical properties by using their body to interact within a virtual environment. Using the Microsoft Kinect and custom software written in OpenFrameworks, information about the viewer’s position is used to build virtual skeletons. When participants enter the active standing area, they need to face to the Kinect and do the calibration pose as instructed for 5 Seconds to get the pose detected. After the user’s body is learned by computer, the virtual skeleton will appear on the screen. The user can then interact with virtual objects that rain down on them, or make a dance with their skeletons. Multiple participants are supported at the same time.