Grand Theft Audio

TUNING IN

As the name implies, much of Grand Theft Auto’s gameplay is spent hopping from one hot car to the next. But cruising downtown in a boosted four-door coupe isn’t quite the same without some East Coast rap on the stereo. One of GTA’s most popular audio features is the virtual in-car entertainment. There are 17 radio stations players can flick between, so they can bob along with their homies to N.W.A’s Appetite for Destruction, or wind the windows down and have a gang sing-along to Queen’s Radio Ga Ga. There’s even talkback radio, spoof DJs, advertisements, jingles and phone-in guests. Some of the stations are curated by artists, and Flying Lotus is so into it, he’s updating his FlyLo FM station with more exclusive game-only tracks for the reissue.

It’s a rich musical environment in itself, but there’s also an adaptive non-linear, interactive score — equivalent to about 12 albums’ worth of music — created by German electronic music group, Tangerine Dream, that underpins the entire game.

Conner: “We’ve always been fans of the band and having licensed a few of their album tracks on previous games, we knew their sound would complement our vision for GTA V. When we approached them to work with us on an interactive score, we gave Edgar Froese a demo of the game and he loved it. Both he and wife and artist, Bianca Acquaye, visited us at the Rockstar North studio a few times to discuss the scoring process — it was so important they had a true experience of the game, and got a real feel for it, before starting work. Edgar heard a prototype interactive score track we had working on a few GTA missions and knew exactly what to do from there. The man is a legend and bringing his sound into this new interactive technology was very exciting. It helped that Edgar has already scored countless films, so he knew exactly how to convey the emotion for each mission.”

With a mind-boggling amount of both licensed and original music in play, mastering the music was always set to be a significant challenge, as Conner explains: “Levelling off so many different styles of music with hundreds of tracks that can literally flick from one to another anytime the player decides, is no easy feat. I’ve been mastering music for over 20 years so I’ve learned a trick or two, but GTA V did pose a challenge for a number of reasons.

“Every licensed track pre-1999 was easy; everything after 2008 was a volume-war. I had to de-master tracks as they were already crushed to the limit. I deliberately processed the radio content a lot more than on previous games in the series so players could hear a clear difference between the score and the radio.

“As far as consistency on the radio goes, this time round the DJ dialogue turned out to be the biggest job. We recorded them in so many different locations; we even recorded one actor in their own home. To complicate things even more, a number of different mics were used, which meant each recording sounded different.

“I really wanted the DJs to sound mega-bassy, like real radio, but I had to pull back on that in the mix because it was taking over. I couldn’t allow the DJs to have more punch than an explosion, so it was all about giving the radio and score its own place in the mix. As GTA V has multiple sounds in a huge, living world, I had to pull out some frequencies in the music across the board, in order to give the player a good overall sound and eliminate ear fatigue.”

COUNTING VOICES

Talking about talking, recording GTA V’s dialogue is a mammoth production in itself. With not only 150,000 lines of script to record, but 700 actors voicing them, it’s probably the longest line of credits you’ll ever see.

Morton: “The process starts very early in development and continues right to the end. Really, you could write a book about it, but in summary, the game city demographics are looked at and the ‘virtual’ population decided on — which types of people are needed, the areas they live in, and so on. Models are made for the ‘peds’ (pedestrians) and then we set about planning voices. Dan Houser, co-founder of Rockstar will come up with a mini-character brief and backstory for each one, which then goes to the ped writing team who create a bespoke script for each one — about 200 lines of dialogue on average. The actors are cast, and recording time booked. Dealing with about 700 actors means recording peds has to begin months before the game is complete. And there are often revisions as game features continually evolve during development.

“We ran several large recording sessions throughout production, often recording 100 peds in a week (most only take an hour). We had three studios working simultaneously; each one with a director from Rockstar present plus an audio person like me to ensure what’s recorded is exactly what’s needed. As well as getting actors in and out on time, the producers make sure everything runs smoothly. After recording comes ‘take’ selection, and then the lines are edited, each voice mastered and the lines added to the game. It’s kinda manic sometimes, but Rockstar has been building huge scale games for years and the New York production team is expert at this stuff.” 

Central to the believability of GTA’s dialogue is its street cred. Not many script writers can grasp the slang of one gang, let alone an entire population as diverse as Los Santos, San Andreas — so Rockstar just hired the real deal.

Morton: “Dan Houser, Michael Unsworth and Rupert Humphries wrote GTA V. There’s always a huge amount of detail put into the characters from the main players through to the pedestrians that walk the street. Dan has always gone to great lengths to make sure that populations are as varied as they are in real life, and that even the smallest character amounts to more than just a ‘random guy on the street’. When we recorded the gang peds, some of the actors would actually be gang members in real life and would let us know when lines in their script were things they wouldn’t say. We always ended up with fantastic results when guys like that would take the script and make it their own.
“We’ve found, in general, whether it’s for a game or for a movie, you’ll get far better results when you treat your actors like humans — they are assets rather than tools. Actors can often bring things to roles you haven’t anticipated, and while as director you have the final say, you’ll often get nuggets of ‘performance gold’ by working with actors rather than just commanding them. After all, they want to do their job well.”

Conner: “For the main player characters in GTA V, we used a great deal of motion capture — the actors would physically play out the scene together and their movements, facial expressions, and dialogue would all be recorded together. When you can capture an ensemble performance with the right actors you get incredible results — far more detailed than recording everything separately. It’s obviously a lot more resource-heavy than the old way where we didn’t capture that data, but definitely worth it. Ned Luke, Steven Ogg and Shawn Fonteno’s performances really brought their characters to life.”

ALL THE RAGE

In a game like Grand Theft Auto, most everything you see makes a sound — people on the street, birds in the sky, cars in the distance… the coffee cup hitting the floor as a man flees a gunfight. The mix and 3D positions of these sounds relative to the player’s orientation must be continually calculated frame by frame to determine a correct and immersive aural experience for the player. Add that to the dialogue and music handling and you have quite a technical headache. Playback priorities must be carefully managed to create a viable run-time mix balance — you can’t replay everything, nor should you.

Enter RAGE, the audio engine governing the lot: it provides replay logic controls, culling of sounds according to distance and priority weighting, and providing the team a virtual digital surround mixing system.

Conner: “RAGE Audio was designed and built by our in-house team, meaning it’s perfectly suited to these sorts of games. It calculates and tracks about a thousand virtual sounds or voices at any one time, 96 of which will be rendered in the mix. When sounds are added, the volume of each file is calculated. The engine then uses that information in conjunction with what else is happening in-game to work out which sounds/voices should be heard. For instance, a loud sound happening miles away may not be heard over a quieter sound happening close up. It’s a great system for prioritising sounds automatically and working out what is, and isn’t important to the player.

“Many sounds are created in real-time from tiny components rather than using a single pre-built sound effect. The benefits are zero memory-wastage, and it also gives the sound designers flexibility because the sound variation is created as it’s triggered.

“We included some real-time synthesis in GTA V by using AMP — a modular ‘connect-the-cables’-type interface, fully integrated with RAGE Audio. The benefit of synthesis over samples is creating infinitely variable sounds. Sounds that would normally be quite static, if represented by recorded effects, suddenly have life breathed into them, and their behaviour appears more realistic. It’s also very memory efficient. You can do a lot with noise and filters, for instance, creating the sound of air conditioning units. “AMP is also used for DSP. In GTA V you can switch between three characters during missions. They’re often talking to each other via headsets so the voices heard through an earpiece need to be changed on-the-fly as the player switches roles. We designed a headset effect chain to process the voices at run-time as required.”

Morton: “Vehicle sounds are obviously a big deal — from the huge array of cars and trucks through to aeroplanes. The sounds of the cars are again made of tiny components recorded with a multiple-microphone setup. The vehicles sound different close-up than at distance — as they would in real life, and there are other details such as the sound of an engine cooling down after you’ve been thrashing your vehicle and so on. It also meant we could use the components of one vehicle as part of other vehicles. The engine recordings are processed with a granular playback engine so that as the car speeds up or slows down, the sound is perfectly in sync with the vehicle and follows the behaviour of the player. It was a massive step up in quality and realism from the old-school game method of pitching loops or just using static recordings.”

LEAVING THE ROCKSTAR LIFE

As they enthuse about Grand Theft Auto, it’s abundantly clear that working on such a notable world-class game series has been an amazing adventure for the duo. But they’ve since left Rockstar, and now they’re bringing all that expertise to bear in a new venture, Solid Audioworks — the production company they recently co-founded to provide a full audio service; including dialogue, sound effects and score, as well as movie post-production. But surely leaving Rockstar-dom can’t have been an easy decision.

Conner: “For me, after almost 20 years working on the GTA series, seeing the process travel from generation to generation, post-‘V’ seemed like the perfect moment to depart, go out on a high, re-charge the creative batteries and try something completely new. There are so many other games/films I want to work on and tick off the bucket list.

Morton: “Leaving certainly wasn’t a decision made lightly. For me, leaving the Rockstar family was about being able to work from my own studio, closer to my own family, and expand my horizons project-wise. There are lots of interesting games being developed — both indie and AAA — I really wanted to try something different. First of all, I fell into doing film work, which led to bumping into Craig at a pre-shoot meeting. We started discussing working more together, and that subsequently led to us setting up Solid Audioworks. As to what’s next, we will be busy on some film post production until the end of the year, after that we’re very much looking forward to getting back into what we love — games and game audio. Watch this space.”