Outline: Visual Imagination and Interaction in Music

Introduction

Thesis Statement and Hypothesis

Background

Perception

Sound, Visual, and Perception

Music, Visual, and Perception

Music Visualization in History 

Sheet Music

Color Organ

Futurism and Cinema

Technology

Music, Film and Music Video

Music Digitalization

Interaction

Multimedia platform

Current Situation

Problems

Lack of Imagination

Interaction between Audience and Music
Digital Music Explosion/Identity of A New Artist

Case Study

Animated Music

Disney: Fantasia

Matteo Negrin: Music Painting

Forss: City Ports

Interactive Music

Labuat: Painting A Song

Ellie Goulding: Lights

Placebo: The Never-Ending Why

The Johnny Cash Project

Projection Project

Cirque du Soleil: Zarkana

Methodology

From Music Structure to Visual Forms

Lyrics and Typography

Motion

Interface and Interaction

Solution

Project Brief

The project

Review (conclusion)

Technology

Music, Film and Music Video

Music and Film has influenced each other since the music was used in movies. Music has impacts on visual perception and vice versa. 

• In 1920s, music became an important part in films.  It guides viewers into narratives. It goes along with storytelling, mood, and clarifies the plot. 
• The influence of visual information on the perception and interpretation of music performances can indeed change the way viewers experience music.  
• In 1980s, the born of music video and music television (MTV) has a profound impact on music industry. Viewers are receivers; they receive the obvious value, and the messages on visual, and couldn’t get the metaphors in music.

Music Digitalization

In nowadays, information travels fast in digital world, and so is music. Music can be composed, updated and spread online, and can be heard a million times over night. 

In other words, it is easy to get famous, but it is also easy to fall out.

Interaction

The interaction between audience and performers has changed through technology. Audience participation is not a new concept. In 1880s, music was performed in music halls where sing alongs and audience participation took place. 

• 1880s: Music halls
• 1920s: Mass singing performances survived on sports events and clubs
• 1960s: The Beatles was born and its yellow submarine revives the music hall ideal of audience participation. 
• 1980s to present: TV, gigs, sports events, clubs, or even online.

John Cage: Creating music was a process that initiated by the composer or performer, but completed by the audience.

Example: Bobby McFerrin

As technology advanced, performances with spectacular lighting, stereo, and visual effects are common nowadays. But the more I experience it, the more I feel like we are only receivers to these technological platforms. 

Example: Superbowl 2012 and 1993

Multimedia Platform

Internet has enormously changed the way music is distributed. 

Music has to be assigned for a new value. "The product is no longer the music in and off itself. The product is the musician’s story and the experience of being part of it,”-Scooter Braun, Justin Bieber’s manager.

The role of graphic designers in collaborating with musicians can be as important as musicians itself. I believe in the co-creating environment, graphic designers and musicians can growing together, feeding and informing the other.

  

Note: Before the music dies (Documentary) 

History in Music Visualization

Sheet Music

Sheet Music is written representation of music. It is a visual language readable by musicians and performers. Before the fifteenth century, it is preserved on manuscripts. After printing became available, the effect of printed music was similar to the effect of the printed word, in that information spread faster, more efficiently and to more people that it could through manuscripts.

Color Organ

In 1895, Alexander Wallace Rimington built the clavier à lumières, or keyboards with lights. His device formed the basis of the moving lights. In 1960s and 1970s, the term color organ became popularly associated with electronic devices that responded to their music with light shows.

Example: Debussy Clair de lune

Futurism and Cinema

“The dream of creating a visual music comparable to auditory music found its fulfillment in animated abstract films by artists such as Oskar Fischinger, Len Lye and Norman McLaren”-William Moritz

Futurists have another visual approach to express the content of sound with typography. 

In Filippo Marinetti's Zang Tumb Tumb and  Les mots en liberté, he visualized the sound by type treatment. The book of Zang Tumb Tumb celebrated the Battle of Tripoli with expressive typographic methods that combined sound poetry and visual prosody. 

In Les mots en liberté, tremendous visual effects can be seen through the composition of typography. Visual imagination was required to conceive of such a work and to integrate its visual-verbal-tactile effects so thoroughly.    

1921 Hans Richter "Rhythmus 21"

Documentary: Before the Music Dies

Before the Music Dies (2006)
Director: Andrew Shapter
Writers: Andrew Shapter, Joel Rasmussen

Although this documentary is not directly related to my thesis, it's a great glimpse of how major music industry has changed.

Focused on the mainstream music industry, the industry has dramatically changed. It is from analog to digital, from radio to MTV, and to Internet, and from global record companies to independent companies.

Music is the emotional identity for human beings. In the old days, when radio is the main platform to distribute music, musicians moved us with their throats, their voice, and instruments. It is their experience that we have connection with that moves us. 



In the 1980s, we reach the age of videos when MTV appears. Visuals dominate music. The music industry began to look for young, good looking faces rather than real music talents. As we reached the 1990s, new technology and Internet creates a whole new platform for musicians. Everyone can produce music at home. It’s easier than the old days that new musicians can promote themselves online and gain exposure. They can do everything on their own. But it is also harder to find the target audience, and vise versa because there are so many musicians out there.

From the current situation we have, I came to conclude three problems we have right now:

• Audience lost their imagination in music due to the appearance of MTV and music videos. 
• The connection and interaction between audience and musicians has been changed as new technology approaches. 
• It’s harder but also easier for a new musician to build his musical identity in the age of Internet.    

Interface and The Role of A Graphic Designer

Another article I read about is "The Rise of the Creative Thing."
http://brandsplusmusic.blogspot.com/2010/05/rise-of-creative-thing.html
http://brandsplusmusic.blogspot.com/2010/06/collaborating-on-creative-things.html

Internet has enormously changed the way music is distributed. In the old days, there was a record holding the music. But the rise of the Internet makes music being copied easily, and costs nothing, or nearly nothing. The old value of producing and composing music has disappeared. Music has to be assigned for a new value. "The product is no longer the music in and off itself. The product is the musician’s story and the experience of being part of it,” said Scooter Braun, who is Justin Bieber’s manager.

In the article, the author suggested that musicians should have at least one of these three approaches:

Personally be able to create more than just music.

Hire people or work with a team who can supplement what they don't/can't do.

Collaborate with artists in other media so that together they create multi-dimensional packages/experiences.

The package of music should be considered as an identity of the music and musician. The package not only refers to the physical one but also the use of multimedia platforms such as music videos, website, or even games.  Neither musicians should do it all to promote themselves or collaborate with developers, or designers to make it happen.

There is Will.I.AM talking about the death of music.

Audience Participation: Super Bowl Halftime Show

Music innovator John Cage once said that music could no longer be seen as something separate and detached from its listeners and from its context. Rather, creating music was a process that initiated by the composer or performer, but completed by the audience.

As technology advanced, performances with spectacular lighting, stereo, and visual effects are common nowadays. But the more I experience it, the more I feel like we are only receivers to these technological platforms. For example, in 2012 Super bowl halftime performance, the special effects of projecting and lights, and performers are spectacular. But there is a one-way direction to the audience. Another Super Bowl halftime performance in 1993 shows different interaction between audience and performer Michael Jackson. In “Heal The World,” audience in the stadium is part of the show. They are given certain color papers so they can lift them up to become the portraits of children. This corresponded to what John Cage’s idea of the performance should be completed by the audience.

Example of Audience Participation

Recently I am researching on the interaction between audience and performers, and I found this wonderful piece. This is so simple, and no matter what kind of music you love, you still can get the idea.

I began to wonder if the visual can do the same thing to trigger audience to be part of the performance. It would be not only audience react to the music and performers, but also create their visual reaction to the whole performance.

Interactive Visual Music Examples

Labuat: Painting A Song    

In2009, Herraiz Soto & Co., an independent creative agency based in Barcelona, created an interactive website “Labuat: painting a song" allowing viewers to draw their own version of the song. 

Ellie Goulding: Lights

It is designed by HelloEnjoy, an interactive studio in Brighton, UK.  At HelloEnjoy, designers believe in creating interactive 3D experiences that are both intuitive and visually engaging.Their goal is to demand very little from the user to reach broader audiences, and at the same time reward them with high-end graphics and aesthetics.

In Lights, the viewer can control the light spot flying faster or changing directions by clicking the mouse. The creative approach was based around the concept of flying over an infinite landscape which reacts to the sound and evolves along with the music. This allows user interaction but also delivers a great experience if the user chooses not to interact.

Placebo: The Never-Ending Why

The Johnny Cash Project


The Johnny Cash Project is a collaborative online project, which allow fans to draw and complete the video. According to the statistics, over 250,000 people from over 170 countries contributed to the project. Although it is a crowdsourcing project, the video is still filled with the same quality. Frames go by very quickly and the color tone maintains black, gray, and white throughout the whole video.

3D Mapping

Disney's Fantasia (1940)

Fantasia is a good example of combining abstract and concrete forms to music. There are eight segments that took from various masterpieces of classical music. The mysterious and playful characters interpret classical music with vivid imagination and movement. It is a 1940 American animated film produced by Walt Disney and released by Walt Disney Productions.

• The Nutcracker Suite
• The Rise of Spring
• The Toccata and Fugue