Building re.flow Introduction: Why use Dolby Digital Plus for web-based projects?

Building re.flow Introduction: Why use Dolby Digital Plus for web-based projects?

150 150 Jason Marsh

Dolby Digital Plus gives you spatial audio for media projects beyond just the classic video use case. This is the first in a series of posts describing the creation of “re.flow” by Jason Marsh at the request of Dolby Laboratories to show off the possibilities of spatial audio with nothing other than a modern browser. As of this writing, Dolby Digital Plus is available in Microsoft Edge on Windows 10, and hopefully more browsers soon.

DolbyAudio_Vert_white_on_blackDolby Digital Plus in the browser is significant not only because it affords playback of multi-channel content when connected to a home theater or headphones on systems that provide surround virtualization, but because it provides a premium consistent audio experience to the browser by incorporating a rich set of metadata about the audio that is part of the Dolby Digital Plus codec. This way, Dolby encoded content sounds great regardless of the system it’s being played back on.

The guidelines for the project were quite simple: create a WebGL project that features Dolby Digital Plus 5.1 surround audio. WebGL is used to create highly dynamic, smoothly animating graphics like you would expect from gaming consoles.

It took me under five minutes to realize that this was the perfect project for me: I have a Masters in Music composition, I’ve been programming forever, years ago I produced binaural projects for Bose, and I’ve been doing WebGL programming full-time for the past year working on Virtual Reality (WebVR). A project made in heaven, right?

The original pitch I made was a silly sketch of planes floating in space, each synchronized to an audio track, and each with a custom visualizer for that track. Luckily, to their credit and not mine, for some reason the Dolby folks trusted me.

re.flow for Dolby Laboratories, 3D object view
re.flow for Dolby Laboratories, 3D object view

Before reading further, you have to try the end result. The actual experience is hard to describe, and since it is a web project, why bother? Use the Microsoft Edge browser on Windows 10, if you have it, or the other modern browsers including Chrome if you don’t. The audio won’t be as good without the Dolby processing in Microsoft Edge, but it will all work.

Go here now if you haven’t already done so: http://dolby.flow.gl

Did you like it? I can hope so!

This post is the first of four posts describing various aspects of the project.

 

The main aspects of this project, and tools used were:

Audio Production:

  • Producing the music, 16 clips in all, in Ableton Live and hiring the vocalists.
  • Obtaining 5.1 surround studio hardware.
  • Taking stereo mixes exported from Ableton Live into Adobe Audition CC 2015 to create the 5.1 panned ec3 files.
  • Wrapping the ec3 files into mp4 files that the browser will be able to use.

Audio Programming

  • Building the music sequencer using the Web Audio Context (all in JavaScript) to simultaneous play 8 tracks, synchronized to the millisecond
  • Building the sequencer user interface
  • Saving the sequencer to a database in the cloud

WebGL design and programming

  • Creation of the 3D scene using ThreeJS and GLAM
  • Implementation of animated textures in GLSL shaders
  • Animation engine to sync up the spatial audio mixes to the graphics
  • Pickers and interface elements within the 3D scene
re.flow for Dolby Laboratories, sequencer view

re.flow for Dolby Laboratories, sequencer view

Finally, though not within the scope of the original project, I went ahead and created a WebVR version so that I could experience it all in Virtual Reality. While this is not ready for prime time and only works on pre-beta browsers, it is a great experience and ultimately a powerful application of all these technologies.

So into the details we go:

Building re.flow Part 2: Music and Surround Sound Production

Building re.flow Part 3: Web AudioContext and Sequencer

Building re.flow Part 4: WebGL Visuals