Falling (2013) is a real-time interactive system, implemented through Max/MSP in conjunction with the PS2 Gametrak controller ("tethers"), that lets the user express music and graphics through motion and intuition.

Please check out this VIDEO DEMO.
(The music track I used was Rachmaninoff's Vocalise)

Falling explores the "normal" and the "abnormal." It takes place in a pair of parallel "worlds": an aural world of music, and a virtual 3-D world filled with little balls. The "normal" is represented by the default world where the music plays at regular speed and the balls are safely bound to the ground by gravity. The only upward force is powered by the music; changes in the fundamental frequencies trigger random balls to shoot upward, but only to be pulled back down by gravity eventually.

However, pulling of the tether strings in different directions may cause various "anomalies" to occur: in the audio domain, these include pitch/tempo shifts, reshuffled audio, and filter effects; in the visual domain, these correspond to shifting gravity, rotating planes, and flashing colors, respectively.

Below is a quick summary of these "anomaly" correspondences between the audio, the visuals, and the motions:
Left String
1) x-axis movement (pulling string right/left)
<=> filter cutoff frequency for audio (right = lower, left = higher)
<=> color diversity of the balls (right = more uniform, left = more diverse)
2) y-axis movement (pulling string forward/backward)
<=> sample playback rate for audio (forward = higher, backward = lower)
<=> gravity in the virtual world (forward = larger, backward = smaller)
3) z-axis movement (pulling string up/down)
<=> audio volume (up = louder, down = softer)
Right String
4) x-axis: same as #1) above
5) y-axis movement
<=> row number in a 2D wave-table for the audio (forward = row 2, center = row 1, backward = row 3)
<=> virtual world dimensions (forward = smaller x-dimension, backward = smaller z-dimension)
6) z-axis: same as #3) above

The left and right strings are independently playing the same music track (at slightly different speeds), so the user can experiment with alternating between the two and creating various combinations of effects.

The Patch
Below is a screenshot of the Max patch. In order to use the system, one would need to plug in the Gametrak controller into the computer and make sure that the patch can receive its inputs.


You can DOWNLOAD the zip file containing the patch, the music track, and a README document that will walk you through the process.