A multi-disciplinary designer’s journey in field recording, sound design, and music.

Musical Sound Design for Installations

Posted: April 12th, 2018 | Author: | Filed under: interactive audio, sound design, theory

Want a challenge? Try to play back interface sounds on the show floor at CES. [Intel Booth, CES 2012.]

Want a challenge? Try to play back interface sounds on the show floor at CES. [Intel Booth, CES 2012.]

For those that might not know,  for the last decade I earned my living designing digital installations: Multi-touch interactive walls, interactive projection mapping, gestural interfaces for museum exhibits, that sort of thing. Sometimes these things have sound, other times they don’t. When these digital experiences are sonified, regardless of whether they are imparting information in a corporate lobby or being entertaining inside of a museum, clients always want something musical over something abstract, something tonal over something mechanical or atonal.

In my experience, there are several reasons for this. [All photos in this post are projects that I creative-directed and created sound for while I was the Design Director of Stimulant.]

Expectations and Existing Devices

It’s what people expect out of computing devices. The computing devices that surround me almost all use musical tones for feedback or information, from the Roomba to the XBox to Windows to my microwave. It could be synthesized waveforms or audio-file playback, depending on the device, but the “language” of computing interfaces in the real world have been primarily musical, or at least tonal/chromatic. This winds up being a client expectation, even though the things I design tend not to look like any computer one uses at home or work.

Roomba

Yes, I strapped wireless lavs to my Roomba. The things I do for science.

Devices all around us also use musical tropes for positive and negative message conveyance. From Roombas to Samsung dishwashers, tones rising in pitch within a major key or resolving to a 3rd, 5th, or full octave are used to convey positive status or a message of success. Falling tones within a minor key or resolving to odd intervals are used to convey negative status or a message of failure. These cues, of course, are entirely culture-specific, but they’re used with great frequency.

The only times I’ve ever heard non-musical, actually annoying sound is very much on purpose and always to indicate extremely dire situations. The home fire alarm is maybe the ultimate example, as are klaxons at military or utility installations. Trying to save lives is when you need people’s attention above all else. However, even excessive use of such techniques can lead to change blindness, which is deep topic for another day. Do you really want a nuclear engineer to turn a warning sound off because it triggers too often?

The Problem with Science Fiction

Science fiction interface sounds often don’t translate well into real world usage.

This prototype "factory of the future" had to have its sound design elevated over the sounds of compressors and feeders to ensure zero defects. [GlaxoSmithKline, London, England]

This prototype “factory of the future” had to have its sound design elevated over the sounds of compressors and feeders to ensure zero defects…and had to not annoy machine operators, day in and day out. [GlaxoSmithKline, London, England]

My day job has been inexorably linked to science fiction: The visions of computing devices and interfaces that are shown in films like Blade Runner, The Matrix, Minority Report, Oblivion,  Iron Man, and even televisions shows like CSI set the stage for what our culture (i.e., my client) sees as future-thinking interface design. (There’s even a book about this topic.) People think transparent screens look cool, when in reality they’re a cinematic conceit so that we can see more of the actors, their emotions, and their movement. These are not real devices – they, and the sounds they make, are props to support a story.

Audio for these cinematic interfaces – what Mark Coleran termed FUI, or Fantasy User Interfaces – may be atonal or abstract so that it doesn’t fight with the musical soundtrack of the film. If such designs are musical, they’re more about timbres than pitch, more Autechre than Arvo Part. This just isn’t a consideration in most real-world scenarios.

Listener Fatigue

Digital installations are not always destinations unto themselves. They are often located in places of transition, like lobbies or hallways.

I’ve designed several digital experiences for lobbies, and there’s always one group of stakeholders that I need to be aware of, but my own clients don’t bring to the table: The front desk and/or security staff. They’re the only people who need to live with this thing all day, every day, unlike visitors or other employees who’ll be with a lobby touchwall for only a few moments during the day. Make these lobby workers annoyed and you’ll be guaranteed that all sound will be turned off. They’ll unplug the audio interface from the PC powering the installation, or turn the PC volume to zero.

This lobby installation started with abstract chirps, bloops, and blurps, but became quite musical after the client felt the sci-fi sounds were far too alienating. [Quintiles corporate lobby, Raleigh NC]

This lobby installation started with abstract chirps, bloops, and blurps, but became quite musical after the client felt the sci-fi sounds were far too alienating. Many randomized variations of sounds were created to lessen listener fatigue. There was also one sound channel per screen, across five screens. [Quintiles corporate lobby, Raleigh NC]

Music tends to be less fatiguing than atonal sound effects, in my experience, and triggers parts of the brain that evoke emotions rather than instinctual reactions (in ways that neuroscience is still struggling to understand). But more specifically, sounds with without harsh transients and with relatively slow attacks are more calming.

Randomized and parameterized/procedural sounds really help with listener fatigue as well. If you’re in game audio, the tools used in first- and third-person games to vary footsteps and gunshots are incredibly important to creating everyday sounds that don’t get stale and annoying.

The Environment

Another reality is that our digital experiences are often installed in acoustically bright spaces, and technical sounding effects with sharp transients can really bounce around untreated spaces…especially since many corporate lobbies are multi-story interior atriums! A grab bag of ideas have evolved from years of designing sounds for such environments.

This installation had no sound at all, despite our best attempts and deepest desires. The environment was too tall, too acoustically bright, and too loud. Sometimes it just doesn't work. [Genentech, South San Francisco, CA]

This installation had no sound at all, despite our best attempts and deepest desires. The environment was too tall, too acoustically bright, and too loud. Sometimes it just doesn’t work. [Genentech, South San Francisco, CA]

Many clients ask for directional speakers, which comes with three big caveats. First, they are never as directional as the specification indicate. A few work well, but many don’t, so caveat emptor (they also come with mounting challenges). Second, their frequency response graphs look like broken combs, partially a factor of how they work, and so you can’t expect smooth reproduction of all sound. Finally, most are tuned to the human voice, so of course musical sound reproduction is not only compromised sonically, but anything lower than 1 kHz starts to bleed out of the specified sound cone. That’s physics, anyway – not much will stop low-frequency sound waves except large air gaps with insulation on both sides.

The only consistently effective trick I’ve found for creating sounds that punch through significant background noise is rising or falling pitch, which lends itself nicely to musical tones that ascend or descend. Most background noise tends to be pretty steady-state, so this can help a sound punch through the environmental “mix.”

One cool trick is to sample the room tone and make the sounds in the same key as the ambient fundamental – it might not be a formal scale, but the intervals will literally be in harmony with one another.

Broadband background noise can often mask other sounds, making them harder to hear. In fact, having the audio masked by background noise if you’re not right in front of the installation itself might be a really good idea. I did a corporate lobby project where there was an always-running water feature right behind the installation we created; since it was basically a white noise generator, it completely masked the interface’s audio for passersby, keeping the security desk staff much happier and not being intrusive into the sonic landscape for the casual visitor or the everyday employee.

Music, Music Everywhere

Of course, sometimes an installation is meant to actually create music! This was the first interactive multi-user instrument for Microsoft Surface, a grid sequencer that let up to four people play music.

Of course, sometimes an installation is meant to actually create music! This was the first interactive multi-user instrument for Microsoft Surface, a grid sequencer that let up to four people play music.

These considerations require equal parts composition and sound design, and a pinch of human-centered design and empathy. It’s a fun challenge, different than sound design for traditional linear media, which usually focuses on being strictly representative or on re-contextualized sounds recorded from the real world. Listen to devices around you in real life and see if you notice the frequency (pun intended) with which musical interface sounds are commonplace. If you have experiences and lessons from doing this type of work yourself, please share in the comments below.

Tags: , , , | No Comments »

Lighthouse Winds

Posted: April 22nd, 2015 | Author: | Filed under: field recording, gear, nature recording, sound design

lighthouseBaja

My past winter holiday involved a sea kayak crossing to Las Islas de Los Todos Santos, a pair of islands four nautical miles offshore of Ensenada, México. We were greeted – and partied with – a nearly toothless lighthouse keeper, and slept in an old lighthouse built in the 1930′s.

We had two days of 15-25 knot winds, and as you might imagine, a lighthouse is a roughshod place. The winds were howling through the old windows and making amazing sounds.

Only one problem: I had a small sea kayak with no room to even pack a handheld field recorder. As I’ve said many times before, the best field recorder is the one you have with you, and this case, my only option was my iPhone. In glorious, shimmering mono.

Today’s sound are of these howling winds, recorded with the Voice Memos app on iOS. I’m not about to make a habit of using my iPhone as a field recorder, even with aftermarket microphones, but hopefully this goes to show that sometimes you do the best with what you have. Especially if the sounds and location are literally once-in-a-lifetime events.

Tags: , , , , , | No Comments »

Desert Train

Posted: April 21st, 2015 | Author: | Filed under: field recording

I’m in the Mojave National Preserve. Massively underrated location, more Joshua trees per acre than Joshua Tree National Park. Gorgeous. Quiet.

My girlfriend is photographing wildflowers in a shallow roadside arroyo. The road follows a set of train tracks; there are small bridges over each arroyo, wash, and ditch. I’m a little bored.

I hear a distant train.

“Where the hell is my field recorder?!?!”

I rummage through the back seat of our car, packed with disorganized camping gear. I violently toss out three huge bags to get at the small Pelican case that holds my Sony PCM-D50. The train gets closer.

I switch on up the D50: No power. “F#&%!!!” I dump the dead batteries into the desert sand, slam fresh batteries in. I toss the Pelican case in the sand and sprint to the small concrete bridge over the arroyo. I slate the take as I run. The train is now visible and almost at the bridge, arriving from my right. I’m rolling. I’m ready…or so I think, having never recorded a train close up before.

The train has two locomotives at the front: They absolutely overload the mics and kick in the D50′s horrendously useless limiters. “S#!%!!!”

But then the cars start rolling by, at least 30dB less loud than the engines. I’m taken aback by the loudness difference and the relative quiet of the cars’ wheels. I’m only 18″ away from the rails; the center of the wheels are at my eye level, elevated above the wash I’m standing in. The old freight cars make a solid chack-chack-chack rhythm, sometimes a galloping sound like a 12-legged horse. The modern liquid container cars produce a smooth, buttery whoosh as they pass. The final engine passes by, screaming like a spacecraft in a sci-fi movie.

I think, for a moment, that I will have no photo to accompany this sound on my blog. Then I do my absolutely ugliest, uncoordinated happy dance, seen only by the ravens and the bees.

Tags: , , , , , | No Comments »

Pedalboard Madness

Posted: May 6th, 2014 | Author: | Filed under: gear, music, sound design, synthesis
Sometimes sound design requires thinking inside of multiple boxes.

Sometimes sound design requires thinking inside of multiple boxes.

I’ve developed a small collection of handmade and boutique electronic effects and instruments over the years, like the Grendel Drone CommanderLite2 Sound PX, and many more (perhaps the subject of another post). Longtime readers may recall that I just love supporting independent makers and small cottage industries: That’s where all the weird, truly innovative stuff happens, and I (like many of you, dear readers) am more interested in cool sound design possibilities than straight-up distorted guitarrrrrrrr sounds.

Beyond this, I’ve also been expanding my collection of effects pedals. My latest three are definitely the weirdest: The Great Destroyer and Robot Devil from Dwarfcraft Devices, and the Wow & Flutter from Snazzy FX.

 

Read the rest of this entry »

Tags: , , , | No Comments »

Lite2Sound

Posted: April 2nd, 2014 | Author: | Filed under: field recording, gear, sound design
Rare Waves' Lite2Sound PX, by Eric Archer: A photonic microphone!

Rare Waves’ Lite2Sound PX, by Eric Archer: A photonic microphone!

I’ve previously written about the heavily-built, wickedly cool Grendel Drone Commander synth from Eric Archer. I check his site, Rare Waves, from time to time for new handmade electronic toys, and I was really intrigued by his newer Lite2Sound PX unit. This small device, in Eric’s words, “extracts audio from ambient light.” It’s a photodiode amplifier. Or a photosensitive microphone. Point it at light, it makes sound. It runs off a 9-volt battery, has a volume control, and a headphone jack. Simple, exciting, and a whole new world of sonic insanity. You can buy them as kits or, as I did, fully assembled.

Sounds pretty straightforward. If you just point it at bright, broad light sources, it’s kind of disappointing. It’s when you start listening to artificial lights in otherwise dim environments that some serious magic starts to happen. My experiments were conducted in and around high tech computer equipment, running an 1/8″ mini jack from the headphone output into my Sony PCM -D50 recorder.

Lights inside of PCs, modulated by fans…and further modulated by speaker grills as I passed the Lite2Sound from side to side. Ethernet network activity lights. Server disk access indicator lights. A close up of the power button of an XBox 360 while booting up. Pulsing lights of devices in standby mode. Halogen lamps behind spinning desk fans.

Lightly armored for future fieldwork!

Lightly armored for future fieldwork!

The resulting sounds were astounding in their range: Static, glitches, distorted synth pads, pure sinewave tones, sawtooth-like tones, and much more. You can’t control it, really. It’s a tool of discovery, and its very nature encourages constant experimentation. It was so small and so perfectly complemented a handheld field recorder, I just wanted to take it everywhere and point it at everything! It imparted the same joy as when you start recording with contact microphones, or hydrophones: A new way to listen to the world around you. The more I used the Lite2Sound, I put it in a small plastic container (hacked with an XActo knife for access to controls and the headphone jack) in order to keep the components better protected.

Lite2Sound is a pretty narrowly-focused device and how useful it is to you depends on your taste for the unpredictable. Me, I adore this thing. Hell, I bought two (for future stereo photo-phonic insanity). It encourages constant experimentation, weighs nothing, and I can see using its output in both sound design and musical contexts. Eric Archer nails it again with an odd concept and a rock-solid, focused execution that results in a toy that just begs to be played with.

Tags: , , , , | 3 Comments »

Milk Frother

Posted: May 16th, 2013 | Author: | Filed under: found sound objects, sound design

milkFrother

This thing was given to me as a Christmas gift. I immediately wanted to not froth milk with it, but to record it. With a hydrophone.

It was initially disappointing…until I put it into a metal pan and realized that its interaction with the pan, not the water, was far more interesting. The hydrophone was still in the water, but the frother was used in the water, inside the pan, and outside the pan as well, at varying speeds.


[Aquarian Audio H2a-XLR hydrophone into Sound Devices 702 recorder]

Tags: , , , | No Comments »

Tunnel Rain

Posted: December 2nd, 2012 | Author: | Filed under: field recording, nature recording, sound design

“Construction 129,” a never-completed gun emplacement overlooking the entrance of the Golden Gate, Fort Cronkhite, Marin Headlands, California.

When a winter storm was forecast for the San Francisco Bay Area, I decided to head to the Marin Headlands and try my luck recording bad weather in the abandoned gun emplacements and military fortifications of Fort Cronkhite. I had previously found some great metal sounds at Fort Baker, also in the Headlands, and I was also inspired by Tim Prebble’s high-wind-sub-bass-generator post.

Unfortunately for me, the storm track was slower than predicted, and I wasn’t able to record in peak wind, although some moderate rain started to fall. Such things happen, even with the best of intentions. So, what do you do when your primary plan doesn’t pan out? You reset your expectations and record what you can. I mean, c’mon, I’m in these amazingly evocative ruins! Surely there’s something that’s audibly interesting!

So, I decided to record rain in the tunnels. A recent re-viewing of Ridley Scott’s Prometheus reminded me that tunnels and caverns in films often feature water drops that are drenched in heavy reverb. I pressed the “record” button in several settings and closed my eyes: Was I in the bowels of an ancient spacecraft? The Mines of Moria? The hull of a leaky, empty freighter?

The gun emplacement at Construction 129. No weapon was ever installed.

Occasional wind blasts forced me to enable a high pass filter at 120Hz to try to remove some of the ambient rumble that didn’t help the recording. In addition, there are no fewer than three foghorns in the Marin Headlands, so I had to make an edit every 50 seconds to eliminate the lonely bleat of the Point Bonita lighthouse horn.

There is always something worth recording. You just need to let your disappointment go, reset your expectations with some beginner’s mind, and just start rolling.


[Sennheiser MKH 30/50 mid-side stereo pair into Sound Devices 702 recorder]

Tags: , , , , , , , | 2 Comments »

Bar Walla

Posted: November 12th, 2012 | Author: | Filed under: field recording, sound design

The Uptown, Mission District, San Francisco. Photo source unknown.

The Uptown is the closest bar to my office, and is a classic Mission District hipster dive bar. One hot, Indian Summer day in the fall, it was filled with patrons, windows flung open…but the jukebox was off. And everyone was concentrated by the bar and front door, leaving the back area empty.

Anyone who tries to record diffuse crowd sounds, or “walla,” knows that this is a golden moment. Human voices, but little intelligible conversation, no background music, not too far away from the noise source. I ordered a beer, sat as far away from everyone as I could, and started rolling on my handheld recorder.

I did a little trickery by taking a segment of the recording, swapping the left and right channels, and layering it with another segment, to effectively double the number of people in the room. Luckily there wasn’t too much background noise to also get multiplied. Perhaps not the most interesting of moments on its own, but the little details of the cash register ring, squeaky door hinges, and the general density of the human sounds represents (to me, anyway) a surprisingly hard-to-capture scene without the intrusion of music.


[Sony PCM-D50 recorder, capsules at 120°]

Tags: , , , , | No Comments »

Swords to Soundshares II

Posted: October 31st, 2012 | Author: | Filed under: field recording, sound design

Battery Yates, Fort Baker, Sausalito, California.

Over a year ago, I posted some scraping metal sounds from the steel doors of Fort Baker’s Battery Yates, right at the Golden Gate of the San Francisco Bay. I recently unearthed some metal hits from that same session: They’re heavy, resonant, and the concrete rooms behind them certainly lent the sound some nice air in the low end. (Read the previous installment if you want to learn more about this location and how it was recorded.)

And, since today is Halloween, the original sounds are presented in both their original form and pitched down by an octave for extra heaviness and spookcreeptacularness.

Enjoy the clanky, boomy fun…and happy Halloween!

[MKH 50 microphone into Sound Devices 702]

Tags: , , , , , , | No Comments »

Recording Single Waves

Posted: February 20th, 2012 | Author: | Filed under: field recording, nature recording, sound design

Look carefully: Deep impact zone, short period, wind parallel to the shoreline. Not a good day for recording single waves. So...what is?

It’s easy to get recordings of roaring surf – which can often just sound like constant white noise – but it can be tough to get relatively clean recordings of individual waves. I’ve found that picking a good day and place to record single wave hits are a combination of marine forecasts and location, and that the size of the waves really doesn’t matter.

The only online article I’ve seen that mentions both field recording and marine forecasting has been published by Tim Prebble. In that excellent article, he describes finding gentle-surf days to record his amazing Seal Vocals SFX library. Sometimes finding those small- to moderate-swell days are sometimes the best days to record single wave crashes…but that’s not the only factor.

Single-wave recording conditions are largely defined by a mix of swell, wind, and location.

Swell is due to the large movements of water over great difference; swell is not the same as waves, which are usually wind-driven. Swell is usually forecast based on buoys offshore, and is forecast days in advance. However, I find that 2-day swell forecasts are the most accurate. Swell is defined by wavelength and amplitude, just like audio: Wavelength is is measured in seconds (usually called period), and amplitude (usually called height) is based in feet or meters. Ideally, you want swell with long periods, because that means there’s a greater volume of water moving, but tall swell is also your friend, because that also influences breaking-wave heights at the shore. (Tip: Waves come in “sets” of big and small, between 3 and 8 waves each, depending on if there are multiple wavetrains that are in or out of sync. That means record for far longer than you’d otherwise think in order to get a good range of big vs. small waves over time.)

Getting back to the topic of wind, not all wind is bad (and we all know what wind does to field recordings: huge amounts of badness.) Offshore winds (those that come off of the shore – yes, that is totally confusing!) will might wave faces up longer than they’d be up otherwise, causing a moment of quiet before the wave breaks (or “closes out,” in surf parlance) and a potentially taller wave face. Wind coming onshore (onto the shore from the water) often smears the wave tops and can cause premature toppling…not to mention the fact that the wind will be blowing directly at your microphone capsules. (Tip: You can’t be at the shore without shorebirds, often noisy ones. Bird feeding activity, and therefore noise, may peak before an oncoming storm. Watch the forecast and choose clear weeks with low-pressure days, when it requires more energy to fly, when possible to reduce avian noise. Gulls are complete bastards in this regard!)

Finally, this brings us to location. Some of this is taste: Do you want to record breaking waves against sand, or against rock? Very different sounds will come out of each; bigger crashes will happen on rocks as opposed to sand. But beyond that, know a bit about your location. What direction is the swell and wind coming from, relative to where you’ll be facing? How does the land curve around, creating refraction waves? What is the slant of the land below the shoreline? This last one is especially critical, as the gentler the slope, the bigger the wave. The “impact zone” – where the waves actually break – can be deep or narrow, and a narrower impact zone will generate fewer audible breaking waves at once. Knowing if a beach has a “dumping break” is useful: It makes it terrible to launch from or surf near, but that also means the waves break right at the shoreline, which is far better for audio recording. Just watch out for your gear. (Not a tip, but trivia: Waves always break when the water depth is 1.3x to 1.5x the wave height, as the horizontal energy of the swell gets forced upwards to create breaking-wave heights near shore.)

Never turn your back to the sea. The sea is huge, powerful, indifferent, and above all, unpredictable. Rogue and “sneaker” waves are not legends or myths: They occur regularly, anywhere in the world. Don’t let a recording session end in tragedy.

A final note on technique: If you’re on a beach, angle the mic a bit more downwards than you’d think would be wise, and you’ll get nice sounds of the sand being pulled back as the breaking waves recede back into the ocean. Not always good to have in every recording, but this adds a lot more character. On a beach with fist-sized rocks, this can be especially cool!

I had a recent session at a sandy beach in the Point Reyes National Seashore that exemplified some of the better (but not quite perfect) conditions that I describe above. First, the wind was blowing at 20 knots…but from the opposite direction of the beach I was on, which was backed by hills. That meant delayed wave breaks and being protected from the wind. There was a southerly swell that came right into the dumping-break beach (I can tell the period is averaging 14 seconds by looking at the waveforms of the audio!), so it was all breaking swell and no wind-driven waves. It was a low-pressure day with a long-term clear forecast, and it wasn’t winter, when the masses of migratory birds come through Northern California. Only one swallow’s call made it into the recording.

The recordings aren’t all that dramatic when you consider the breaking waves were only 3-4′ high. But listening to this loud on headphones, eyes closed, I find that the sense of scale is quite exaggerated without an accompanying visual reference…a little Waves LoAir or layering some low-frequency booms can definitely can tip it into cinematically-big breaker territory if desired. However, even in a 90-second clip, there are only three really distinct wave “claps” as the bigger waves close out at the beach.

[Sennheiser MKH 30/50 mid-side stereo pair into Sound Devices 702 recorder]

Tags: , , , , , , , | 3 Comments »