FROM TWO DIMENSIONS TO THREE:
Creating 21st century audio at Sfjazz

June 1, 2023 | by Richard Scheinin

Joe Lovano performing at SFJAZZ (photo by Scott Chernis)

You know the feeling.

It’s that moment in a jazz club when the music clicks into high gear — unexpectedly, like a change of weather. Suddenly, it’s almost tangible. It’s intimate, charged, vibrating inside you — an amazing feeling. Well, that’s what it felt like last November when saxophonist Joe Lovano performed with his Classic Quartet at the SFJAZZ Center — and, truly, you didn’t even have to be there. If you were at home, donning a decent set of headphones or hooked up to your TV sound system while streaming the live broadcast on the Fridays Live concert series, you would have felt the music. There was the impact of Lovano’s enormous, fleecy tone… the electric snap and chatter of the drums… the unerring pulse of the bass… the piano’s encompassing bell-like chords. All of that would have gathered you up into the current of the music, as if you were in the room, the sound waves arriving inside your head, connecting you to the performers, and taking you on a trip.

For several years now, SFJAZZ’s technical staff has been refining the way it delivers sound to the thousands of listeners who tune into its weekly concert broadcasts. Masanori Yura, the organization’s GRAMMY-winning Senior Audio Engineer, is conversant with all the evolving technologies for real-time audio broadcasts. “But for me, it’s the feeling — the vibe — that is more crucial than the sound quality itself,” he says. Yura has spent 25-plus years collaborating with jazz musicians like Charles Lloyd, Terence Blanchard, Kenny Garrett, Dee Dee Bridgewater and Herbie Hancock, and he understands that musical inspiration is hard to pin down, a little mysterious. Likewise, he feels, the broadcasts shouldn’t only be about technical perfection. Inevitably, intuition enters the equation, a bit of mystery. He is shooting for “the sound that makes you dance. The sound should deliver the same energy to the listener outside the hall as it does for the audience in the room.”

Masanori Yura at SFJAZZ

Randall Kline, co-producer of the series and Executive Artistic Director of SFJAZZ, wants the broadcasts to deliver a “natural sound. We’re trying to make that happen with artificial means — all that technology,” he concedes. But the goal is “to create an experience that replicates your being in the hall, where you shut your eyes and hear all the nuances, because there are no distractions, visually speaking. It’s pure sound.” He pauses, then adds, “And maybe we can even give you an enhanced experience — something that’s like being in the hall, but even a little more.”

When the pandemic began in March 2020, SFJAZZ was forced to cancel hundreds of concerts. What to do? It pivoted toward digital programming, launching the streaming series then known as Fridays at Five. Drawing on its extensive video archive — hundreds of shows — it streamed SFJAZZ performances by the likes of Herbie Hancock, Wayne Shorter, Rhiannon Giddens, Julian Lage, Cecile McLorin Salvant, Kamasi Washington, and dozens of others. The videography was immersive. The camera would move ever faster closer to the stage until it felt as if you were in the front row, staring straight up at the musicians — or, seconds later, as if you were crouching between them onstage, looking directly into their faces or maybe even reading the time on their wristwatches. The distance between audience and artist had seemingly vanished.

The sound quality was excellent, as well — but lately, it’s gotten even better. Audio-wise, as Kline says, the series is on the cusp of offering something more.

A lot of meticulous work — much thought and intention — goes into the audio production. As the concert begins, Yura or another member of the audio team that includes Martin Carmona and Alex Espolet, sits at a console in the venue’s Digital Lab, overseeing what amounts to a multi-track recording session that unfolds over the next hour or two. Tracking inputs from all the instruments on stage, the engineer mixes them in real time using ProTools, and this multi-track mix is instantly transmitted by the software as a two-track broadcast mix. There’s a spatial dimension to it, mirroring the physical set-up on stage: If you close your eyes, as Kline likes to do, you can hear the piano panned to the left-hand side of the stage, the drums on the right, the singer at dead-center. “You have all this detail, and it already sounds pretty great,” says Ross Eustis, co-producer of the series and the Associate Digital Director at SFJAZZ. Also a professional trumpet player, he brings a musician’s ear to the project: “It’s like you’re in the 90th percentile for sound quality. Most people would listen to that and say, `I like it. That’s what it should sound like.’”

But for Lovano’s show, Yura and his team further enriched the audio by adding an unusually sensitive microphone to the process — an ambisonic microphone, hanging from the catwalk 35 feet above the stage. This state-of-the-art device has four capsules — it’s basically four microphones in one — and registers 360 degrees of sound. Audio engineers like to say it captures the “air” in the concert hall. Listening at home, when this “air” is added to the underlying sound mix, you can “hear the room” — feel the energy of the hall, its reverberance, as sound waves scatter throughout the venue.

This month, the ambisonic microphone will become a regular feature of SFJAZZ broadcasts. The effect is “like being in a movie theater,” explains Yura, clearly excited by the development. “You feel all those moving sounds, moving from left to right, front to back. It’s kind of like being in the audience — like you’re in the hall. When there’s applause, you feel which side is clapping louder, if it’s coming from your left or from your right. You hear the movement.” It’s something like hearing in 3-D, he says: “There’s more depth of sound.” 

This experiment is the latest element of a process that dates to September 2021, when live shows resumed at the SFJAZZ Center and weekly live broadcasts became the norm. Master percussionist Zakir Hussain kicked off the rejuvenated streaming series, re-dubbed Fridays Live, and an impressive array of artists has since been featured: Hiromi, Ravi Coltrane, Dee Dee Bridgewater, Terence Blanchard, on and on. In the coming weeks, SFJAZZ members can tune in to watch the likes of Samara Joy, the young GRAMMY-winning singer (June 16); the Sun Ra Arkestra (July 21), the Preservation Hall Jazz Band (Aug. 18), and vocalist José James, performing the music of Erykah Badu (Aug. 28). Don’t forget your headphones or home sound system; all these shows will feature the latest ambisonic refinements to the audio mix. (The Friday broadcasts, which repeat Saturday mornings, are available free to SFJAZZ members. You can also sign up for $5 per month.)

The designers of the SFJAZZ Center foresaw these digital developments.

A decade ago, when the complex opened, this writer spoke to Sam Berkow, the New York-based acoustician who created the sound system and acoustical design for the venue: “There’s a modern need for a new type of performance space,” he explained. “Traditionally, they’ve been built to satisfy two groups: the audience and the performers. Now the halls are being built with three groups in mind: the audience, the performers, and the production team, who represent the audience outside the venue. Every performance space now has become a sound stage.”

Sam Berkow

He recalled how Kline had assigned him “the impossible task” of designing a hall where the sound would feel unprocessed and natural, regardless of who was performing — whether it be a solo pianist, an acoustic jazz group, or an amplified jazz fusion band. Audience expectations had already gone “through the roof” for quality audio, he said. It was happening everywhere: in automobiles, movie theaters, sports arenas. “So the challenge Randall presented us with was ‘hit a home run. Don’t just get up there and get on base.’ We wanted to design a space where people would go, `Wow, that place feels great. I’m glad I saw that show there.’”

Back on the phone a decade later, Berkow — who also designed the acoustics for Jazz at Lincoln Center in New York and has a long history of collaborations with members of the Grateful Dead — reviews the unique acoustical fingerprint of the SFJAZZ Center. Its natural sound has become the backbone of the enhanced audio heard on streaming broadcasts.

Performance spaces should be designed to optimize the musical styles they present. As a longtime student of jazz, Berkow knows that great jazz musicians have highly personal sounds: the tone of Ben Webster’s tenor saxophone is very different from that of John Coltrane’s — or Lovano’s. In designing the acoustics for the SFJAZZ Center, Berkow strove to ensure that the room would allow for such differences to be distinguished. The room couldn’t sound tubby or boomy, like so many rock venues. And as a starting point, it had to be quiet, free of extraneous noise from heating systems or the street: “You create an environment where people can focus and hear the nuanced tone within the overall sound.”

There were other essentials: The sound should be “tonally balanced for the audience, no matter where they’re sitting” in the 700-seat hall. “And we had to make the stage work for the musicians,” whose instruments are amplified. That meant limiting the reflections of sound inside the hall — controlling the sound waves, to keep them from bouncing off the walls and bleeding back through the microphones on stage. That would impair the musicians’ ability to clearly hear one another.

Berkow describes the sound of a jazz group — or other acoustic ensembles, like a string quartet or even a chamber orchestra — as “compact.” It benefits from diffusion — the scattering of the energy throughout the hall, spreading it out over a wide angle, and thereby limiting the reflections. “Think of it as lots of little arrows going out in all directions,” he says.

To bring these acoustical requirements in line with the overall design of the venue, Berkow consulted with architect Mark Cavagnero, theater designer Len Auerbach, and technicians at the renowned Meyer Sound lab in Berkeley.

The decision was made to line the performance space with “a slew of diffusive materials” to scatter and absorb the sound — on the wall behind the stage, in the acoustical canopy above the stage, and along the walls that line the entire venue. The end result isn’t some kind of magic: “There’s still a ton of digital processing of the sound; it’s just done with the goal of not hearing it.” Berkow is well-known for designing a software program called SMAART that “optimizes the interaction between the sound system and the sound of the room itself. But it’s meant to be invisible. If people hear something that sounds processed, we’ve failed. At SFJAZZ, it’s about as natural as it can get.” He describes the hall as “acoustically intimate. You should feel like you’re hearing the sound from the stage, and not just from the sound system. That’s the challenge. When people shut their eyes and point to where the music is coming from, nine times out of ten they point to the stage, and that’s quite a compliment to the room.”

Before the concert begins, three audio engineers are on hand. One controls the sound onstage — what the musicians hear. Another adjusts the mix for the audience in the hall. The third, in charge of the broadcast, sits in the Digital Lab — alone at the console with 24 faders that control up to 64 inputs. (A fourth member of the team, in charge of videography, sits in front of yet another console in yet another room, preparing to sync the broadcast audio feed with the streaming video.)

Video engineers Robert Bambey and Albert Wong in the SFJAZZ Video Suite

A small forest of microphones — as many as 40 for a big band — will capture the instruments on stage, and there are another half dozen microphones positioned around the hall, including above the stage, where the ambisonic mic hangs from the catwalk.

At soundcheck, the engineers make their ballpark settings. And inevitably, when the show begins, the broadcast engineer starts making adjustments. If the drummer is playing louder than expected, the engineer brings the drums down in the mix. “You just jump in,” Yura says. “We’re constantly adjusting, cutting down certain frequencies. It’s a challenge sometimes,” he admits. Like most musicians, or artists, generally, Yura is a chronic self-critic. Asked if he’s satisfied with the broadcasts, he says, “It depends on the show.” Still, he and his colleagues enjoy the flexibility and control that comes with mixing in real time: “If the bass amp is blasting onstage, you’ll hear a lot of bass in the house mix — not enough clarity. But for the broadcast mix, you can go right in and clean that up.”

The half-dozen off-stage microphones — and the ambisonic mic, in particular — are what capture the ineffable aspects of the performance. This is where the mystery comes into the process — the energy, the “air” in the room. In a recording studio, the engineer positions a mic directly on the piano; the sound that you hear comes straight from the instrument. But when you watch a show at the SFJAZZ Center, there’s “all that air between the piano and you. You hear across the distance,” Yura says. “The ambisonic mic lets you hear the air, the distance, so it’s more like you’re sitting in the hall.”

Sometimes the room microphones pick up sounds “that are not part of the music, like the sound of a cup falling down or footsteps, a woman’s high heels,” adds Eustis. “Things happen. There is a sound that comes from people being inside a space together, and there are little accidents that are part of the human experience of the concert.” He mentions old live albums by Bill Evans and Ahmad Jamal: “The whole time there’s the sound of glasses clinking; you don’t really even think about it. But it’s there and it gives it atmosphere. So what is the SFJAZZ atmosphere that’s going into the broadcast? That’s what we’re figuring out.”

The experiments continue. Yura, Carmona and Espolet have been working with “dummy head” microphones and other binaural audio technologies that closely mimic — or even exaggerate — human hearing. As you turn your head, you may hear the rustling of fabric as the person seated next to you shifts in her seat — or maybe a whistle from an audience member behind you, egging on the musicians onstage. These things happen, and, applied judiciously, they could end up in future audio mixes.

Right now, though, Yura is fascinated by the ambisonic mic and the possibilities it affords. It fills in the fronts and backs of the audio, bringing a richness to the mix. He uses words like “warmth” and “depth” and “aliveness” to describe the sound he’s looking for. “Slowly but surely, we’re getting closer to what we’re trying to achieve,” he says. “It’s like going from two dimensions to three.”

You could sense that third dimension when trumpeter and composer Terence Blanchard performed at the SFJAZZ Center last August. He put the hall through a road test, you might say, with a new kind of hybrid ensemble: His own electric quintet (the E-Collective), plus a string quartet (the Bay Area-based Turtle Island Quartet), and a pair of opera singers who have collaborated with him on operatic productions in St. Louis and New York (soprano Karen Slack and baritone Will Liverman). Blanchard, who came of age as a member of Art Blakey’s Jazz Messengers in the early ‘80s, is continually pushing his horizons — the perfect guy to challenge Yura, the audio team, and the acoustical capabilities of this performance space.

Terence Blanchard performing during SFJAZZ Gala 2022 (photo by Scott Chernis)

It was electro-acoustic opera: Whoever heard of that? But if you tuned in last summer, you know how good it sounded. The entire ensemble performed selections from Blanchard’s opera Fire Shut Up In My Bones, which premiered at the Metropolitan Opera in the fall of 2021. The music was regal, moving like a procession in an ancient court, building to a roiling intensity. When the singers left the stage, Blanchard’s electric quintet — accompanied by the string quartet — concluded with a piece titled “Kaos,” which Blanchard composed in response to the 2014 police killing of Eric Garner in New York. The music was like an electronic bath: soaring amplified trumpet, percolating bass, thrashing drums, ice-chilled harmonies from keyboards and guitar. Wearing your headphones, you could feel the sounds hurtling toward you in waves, across the distance — through the air of the hall. The sound was warm, clear, alive. After the audience finished applauding, the broadcast continued for another minute or two. You heard the far-off murmurings of the crowd, still excited, as everyone moved toward the exits. You heard the sounds of their footsteps, the clacking of high heels. Atmosphere.

Mission accomplished.