The Wonderful World of Sound

Let's talk about the role of audio in video art. There are completely silent video works and there are fully soundtracked, audio-reactive video works, and there are works somewhere in between. The entire spectrum is valid.

Context is King

Personally, I tend to think of audio for video works as extremely contextual. For example, if there's a piece that is exhibited in a gallery space with other works in close proximity, I'd prefer if it were silent. Sometimes galleries will have headphones connected to the piece for viewers to use, but that's a less-than-ideal experience on the part of both the viewer and the artist. Viewers may need to wait and take turns to fully experience the work. If the artistic intent is to fully engage the viewer, that may be hard in a crowded gallery context.

But if we're talking about a screening in a theater space with other works as part of an experimental film program, ideally there would be some kind of soundtrack. If you do not provide a soundtrack, the soundtrack is the audience shifting in their seats and clearing their throats. If that's what you want, John Cage, go for it.

Keep in mind that soundtrack can be interpreted very broadly here. Stan Brakhage's Mothlight is technically silent, but the intention was to hear the projector whip the film around, creating a fluttering sound to accompany the images of insect parts. As the work has shifted to other media from its film origins, it has lost this quality.

Museum spaces will sometimes have an entire room dedicated to screening experimental film and video pieces, often running on a loop. Having a soundtrack here is fine, but given the exhibition space, something more like an installation feels more contextually appropriate. Since you need to be in the physical space itself to experience the work, why not have a score in quadraphonic sound? Or do something interesting with the speaker placement? Or have dueling soundtracks to the same work that recontextualize the visual based on where you are standing?

And it's hard to know how, when, or under what context your work will be viewed. You may submit the same work for a film festival as you would for a gallery show. You may have some wiggle room to ask for certain concessions from the exhibition space. Your piece may just show up on whatever monitor they have free. It may be viewed online. It may be viewed with dollar-store earbuds on a decade-old smartphone or a PC hooked up to a Dolby 7.1 surround sound system.

Progressive Priorities

Given these bouts of chance, I like to approach sound in my work as a layer of progressive enhancement. Progressive enhancement is a concept that comes from the web development world. As a way to address all of the different capabilities of different computers, display resolutions, and web browsers, PE dictates that there are priorities for content delivery. For web pages, this is generally the text content of the page, followed by imagery, style, and interaction.

For my work, the visual is the priority. I hope that in most cases, the visual alone can stand on its own as an impressive piece. A soundtrack can often enhance things that I've worked on, but it shouldn't be necessary.

Given that my work is primarily viewed online, I certainly have no control over whether the viewer has sound capabilities on their device. Take out the technical compontent: I certainly have no control over whether the viewer is deaf or hard of hearing. The piece should still work.

With an eye towards progressive enhancement, you can make a video piece work for a gallery wall, phone on a subway, film festival screening, or a museum installation. A 7.1 sound mix will fall back to 5.1 and further to two-channel stereo, depending on exhibition capabilities.

This is how I view sound, anyway. A lot of video artists have a different priority order. These artists may primarily work in live visuals for concerts or theater. In that context, the sound is definitely the priority and the visual is used to enhance the sound. In that case, beat synchronization may be the top thing you aim for, followed by video content and other effects.

My Approach

Because my priority is visual first, very rarely do I create with a specific score in mind. Nine times out of ten when I'm working, I am either listening to nothing or some conversational podcast. Only when I'm happy with the visual do I move on to the next stage: the edit.

In the edit, when I'm trimming things and making minor adjustments I will add a soundtrack. In some ways, this is a more traditional narrative film approach to motion and sound: capture the performance and then enhance it with a score.

For my daily videos, this part actually comes as part of the upload. My videos start life on TikTok before heading to other platforms. And they start on TikTok because TikTok has an extensive library of sounds that I can use. If a visual reads as a certain vibe, I can find something there to match that mood; enhance it.

While I am also a musician, very rarely do I score my own video pieces. It's not what I default to, though, because music creation and video creation use two very different parts of my brain. It feels like that, anyway. They're two different workflows at the very least.

The new year is coming up. Maybe I should make it a resolution to score more of my own work.