ABC Goes To Rio
Joshua Craig was ABC Radio’s sole tech stationed in Rio for the Olympics. Here’s his diary of how the ABC pulled off a broadcast amongst bomb threats, angry Frenchmen, and a temporary Olympic village.
4 November 2016
Story: Joshua Craig
The ABC is not your average broadcaster; being resourceful is a useful skill — scratch that, it’s a required one. You have to build the systems the corporation wants with what it has available. If you’re lucky, you might be able to wangle a few extra bits of kit out of the coffers. At the Rio Olympics, the ABC wanted to program both Local and Digital radio, with different content every so often, and occasionally from the same studio. They wanted to have ‘off tube’ (see box item) calling capability at as many sites as possible, with most of it happening in Australia to cut down on travel and accommodation costs in Rio. But importantly, the broadcast had to sound as if the ABC had a full presence in Rio. To add to the challenge, I was the lone wholly technical person getting on a plane to Rio.
This was my first project of this scale for the ABC. I came onto the project in January 2016 after past staff had already begun planning. It would have been beneficial to have been in on the discussions earlier, but you play the hand you’re dealt. My role was to design a system that runs primarily on IP with alternate path IP and ISDN redundancy across seven locations on two continents. The only thing that would give away a person’s location was the small delay when having a conversation.
International radio FX run under everything so it sounded as if the commentator was at the stadium. I’ve got to say, the commentators are very talented to be able to bring such excitement to the broadcast even when they aren’t physically in the stadium. We help where we can by pumping the FX in their ears. Reporters in the field have the option of mobile reporting on several systems including Tieline Report-It, mobile VOIP, or from any of the ABC hardline locations spread across Rio, if mobile becomes an issue. Redundancy is the name of the game, so we also set up Skype as a backup for two-way communications if other systems failed. We didn’t need to use Skype in anger during the Olympic Broadcast — the primary contribution systems held up.
Ultimately, we succeeded (with a lot of help from some amazing people within the ABC organisation). We broadcasted a total of about 200 hours across the 16-day period. All three major broadcast locations, including the IBC (International Broadcast Centre) studio in Rio and Australian-based ‘Riofern’ and Ultimo studio 217/218 fulfilled the brief of simultaneously programming local and digital radio with different content. We had a grand total of three off air events of less than a minute, and each was caused by issues out of our control — a power outage, and two open internet changes which reset our VPN tunnels. I have learnt a lot from the experience and seeing how the other broadcasters operate.
DAY -10 TUESDAY 26 JULY
BRINGING RIO TO OZ
This is Riofern, where some of our Sydney crew called home for a couple of weeks. The mobile broadcast truck was technically located at Seven/NEP in Eveleigh, right next door to Redfern, but Rioleigh just didn’t quite have the same ring to it.
This was the first setup day, soon technicians Aaron Hull and Ryan Unwin would install the setup prepared at the lab in Ultimo. The truck would eventually take host broadcaster feeds — supplied by the Olympic Broadcasting Services (OBS) in Rio and transmitted back to Australia by Seven for its TV coverage — of various venues so our commentators could call them. This approach to calling international games ‘off tube’ is becoming more common and even the monsters of broadcast like NBC use this technique. It obviously saves money, keeps the broadcasters more alert and comfortable, and allows for instant ‘travel’ between venues.
FAVELA HIGHWAY
After close to 30 hours of travelling I was keen to get to the Media Village. It was my first time in Brazil, and the number of shanty towns/favelas we passed was an eye opener. Despite the country’s best efforts to hide them, all you could see out the window of the bus were mud, brick and wood houses built on any surface available.
On the way to the Media Village we passed the IBC and the Barra de Tijuca Olympic Park. Originally another favela, the residents were kicked out to build the Olympic park, though some fought back and retained their land holdings on the site. The city begrudgingly built them some new little houses to live in on the edge of the park.
After about a 90-minute trip through Rio, Barra and most of Recreio we finally arrived at the Media Village. It was (and still was at the end of the regular games) a work in progress — the rooms were unfinished and initially we didn’t have hot water — unwelcome news when you’ve just spent 30 hours in transit.
GET SET!
After a couple of days, Aaron and Ryan had the Riofern truck set up. Field testing started towards the end of the day once I had my rig set up in Rio — I was 13 hours behind in Rio, so they easily beat me to it.
At my end, gear setup was far from the hardest part; most of the services the ABC had ordered from the Olympics providers were not working or had limited functionality. I spent most of the day trying to figure out how to log faults with OBS. It really hit me that the language barrier was going to make things a lot harder to get done. This day could have been reduced to one phrase, ‘não está funcionando’.
DAY -9 WEDNESDAY 27 JULY
GET SET!
After a couple of days, Aaron and Ryan had the Riofern truck set up. Field testing started towards the end of the day once I had my rig set up in Rio — I was 13 hours behind in Rio, so they easily beat me to it.
At my end, gear setup was far from the hardest part; most of the services the ABC had ordered from the Olympics providers were not working or had limited functionality. I spent most of the day trying to figure out how to log faults with OBS. It really hit me that the language barrier was going to make things a lot harder to get done. This day could have been reduced to one phrase, ‘não está funcionando’.
DAY -8 THURSDAY 28 JULY
TEMPORARY DASH
The off tube positions in the NEP truck were nearing completion and testing had begun to ensure everything was working at that site. My focus was finding my IBC ‘land legs’. It was a very unusual place. The building was a temporary structure; its only use was to host broadcasters during the games, after which, it would be demolished. Many parts of the construction were reusable, including cabling (data, fibre, electrical), air con units, and some of the modular walls. The building consisted of two double-height levels divided into six sections each. The combined Seven/ABC section only took up about 1/3 of one of the sections on Level 2. NBC took up two entire sections and the BBC occupied just shy of one section. OBS also commandeered a couple of sections. Being the host broadcaster, it was responsible for providing all of the AV elements to all other broadcasters.
DAY -7 FRIDAY 29 JULY
INSIDE THE BBC
I was still soldiering through IP issues. We were using several IP connections in the IBC, one of which was not working. This particular service was meant to receive incoming audio from our local outside broadcasts — ie. the audio from the venues in Rio. Without this working, I couldn’t get audio contribution into the IBC to forward to Sydney. Game over! It was time to go see the telcos and see if anyone could fix it.
I got to go downstairs and meet my BBC colleagues at both Radio 5 Live and the World Service. We were popping a Tieline Merlin into their system so we could share broadcast assets such as staff, venues and other tidbits. These lines also functioned as further redundancy in case all of the ABC redundancies failed; we could put the BBC coverage to air in Australia if we had serious issues.
The BBC section was amazing. The crew brought full radio and TV capabilities with them including news, research, audio booths, master controls, AV routing setups and much more. An expensive endeavour, I’m sure. It was one of the biggest broadcasters at the games, with approximately 500 people on the ground.
DAY -6 SATURDAY 30 JULY
COPACABANA BOYS
After more setup at the IBC, in the evening I was invited to ride along with the Seven crew to shoot some cutaway footage down at Copacabana. One of my Tieline rigs was going to be set up at the hotel opposite the beach volleyball arena, so it was a good chance to scope out the location.
Dan Sweetapple from ABC TV was the tech looking after the site. We’d worked together quite a bit in the lead up to the games to ensure ABC staff had technical assets, where possible, in each of the precincts. The rig down there was mostly used for radio crosses for NewsRadio and Current Affairs (AM, The World Today, PM). It was also a hardline redundancy in case our mobile broadcasting methods didn’t work correctly or failed.
After getting back to the hotel, I was due to meet the sports broadcasters Quentin Hull and Alister Nicholson arriving the next morning… at 2a.m.
DAY -5 SUNDAY 31 JULY
RIO TRANSIT
I couldn’t believe someone thought it would be a good idea to spread out the venues that much! Travel times were massive. It took about one hour to get from Copacabana to the IBC in Barra, about the same to the Olympic stadium, 30 minutes from the IBC to the media village — it killed my setup times! I knew it was going to be an issue before we went, but I didn’t think it would be that bad.
DAY -3 TUESDAY 2 AUGUST
THE VILLAGE
This room was used to interview various members of Team Australia. Although we were promised a nice quiet, soft room over at the Athlete’s Village, that had fallen through the cracks too. Hypercardioid headsets would have to do, which didn’t sound too bad. It all didn’t matter at that point anyway as there was no IP connection. At least the Team Australia IT people were super helpful and spoke English. The trip wasn’t all a loss; McDonald’s had a popup restaurant at the Village, the best meal I had inside ‘The Bubble’ (the secure zone surrounding the Olympic precinct for accredited people).
DAY -2 WEDNESDAY 3 AUGUST
FIRST LIVE BROADCAST
I went back to the Village to chase the previous day’s problems. There was no sign of a solution yet — the IP address I had set on the unit was correct — and I didn’t have time to return to it. Thankfully I was able to access it remotely once we were underway the next day.
That night we did our first broadcast — olympic football commentated by Ned Hall back in Riofern. It went well, with no major issues other than the ones I knew about; a good test run pre-games.
DAY 0 FRIDAY 5 AUGUST
THE OPENING CEREMONY
First thing in the morning I went down to the Aquatics Centre to make sure the problems down there were fixed — both ISDN and IP hadn’t been working. Back onto the support people. IP was working! Yes! However, ISDN was still out… No! I had a closer look and discovered the Cat 5 cable hadn’t been terminated properly. Nipping that off and re-terminating it did the trick and ISDN was finally working!
The opening ceremony went without a hitch. It was long, and the parade of nations was particularly tedious. We had a couple of reporters on the ground filing pieces wirelessly from in and around the stadium. Surprisingly, the 3G/4G data network was pretty good in Rio, meaning VOIP calls via Facebook, Whatsapp or Apple worked much better than standard calls.
That night, as one of the radio reporters was heading back to their vehicle, they stumbled upon a young boy who had recently been shot dead. It was only one kilometre down the road from the stadium. Maracana — where the stadium is — is in the North Zone, and apparently shootings are a regular occurrence.
DAY 1 SATURDAY 6 AUGUST
FIRST SESSION: IP IN THE DRINK
Day 1 had finally arrived. It was almost a relief after the setup period. We still had issues with IP, but I reconfigured the codecs to handle the unreliable network — more redundancy, less bandwidth. I was originally running 24-bit/48k at 128kbps with an additional redundant packet stream (twice the data). To help reduce network loss I increased the compression to 96kbps (reducing the bandwidth) and lifted the redundant packet streams to two (three times the data). It was technically 13% more total bandwidth taking into account the redundant streams, but that additional redundancy gave the system far greater stability. It still sounded good, and considering most of our listeners are on AM radio, it was fine.
Good thing I was close by the first session at the swimming, because about an hour out from going to air the IP dropped off — gone, with no ability to connect even though I could still see the address. After about 10 minutes of putting in my details with the support staff they finally took it seriously and said they were sending someone down to the venue who spoke reasonable English.
By the time I walked into the venue, sports broadcaster Gerard Whateley still hadn’t seen anyone. On the phone the support staff assured me someone was there, but when I got to our broadcast position there was no sign of anyone with only 25 minutes until we went to air. As I was calling them again, someone from the Brazilian telco — Embratel — turned up, but he didn’t speak any English. Using Google Translate, we sorted out a new IP address, made the hookup and got to air.
DAY 2 SUNDAY 7 AUGUST
HOLY MERDE!: FRENCH DIPLOMACY
Gerard called me around noon telling me to get down to the Aquatic Centre because they were having a problem with the French broadcasters. He said that one of them was looking to start a fight!
At the time I was monitoring a minor IP issue and needed it to solve itself before I left. It finished in a timely fashion and I headed out. When I got there, two of the French broadcasters were hysterical. They were claiming we had stolen their ISDN line, which we hadn’t. Time pressure is normal in this game, and we were again only about 30 minutes out from broadcast. I defused the situation by proving to them we hadn’t stolen their ISDN line. They were right to think someone had, as part of the test showed our ABC Ultimo Master Control Room was able to dial their ISDN number — it connected, just not to any of our gear. I stuck around for a bit (Quentin was monitoring everything back at the IBC) to help communicate with the support staff who wouldn’t run another line until after the current session. Luckily, the French were able to get a 4G connection up, which sorted them for that session.
DAY 7 SATURDAY 12 AUGUST
NBC — THE BIG TIME
While I was helping a colleague at NBC fix a small problem with his Tieline device I got to see their facilities in the IBC. NBC is the ‘royal’ broadcaster for the games. It programs six or seven US networks during the Games period and has a lot of pull all over the world. NBC runs a broadcast system considered second to none. In fact, the IBC is built around NBC’s gear, and its systems are built to fit into containers. It is an amazing place: several full TV broadcast studios, sound studios, 4K rooms, massive Master Control Room, catering, news offices, it’s all there. I’ve never seen anything like it.
Al Craig (a Sydney resident and AT contributor) runs NBC’s commentary system, with Australia’s very own Tieline forming its backbone. I was running four iMix consoles in Rio for the ABC, Al was running over 20 at the IBC, the Rio venues and at some studios in the US. While it seems remote off tube commentary is the future of broadcast, NBC has already been doing it for 10 years. With Al’s system, a broadcaster can have three locations simultaneously contributing to the same commentary as if they were in the same room/venue, even across continents. I could talk about this for hours.
DAY 8 SATURDAY 13 AUGUST
THE ‘BOMB’ IN THE IBC
Towards the end of the night, the IBC was evacuated because of a device found in the men’s bathroom just around the corner from our broadcast position. It was found by a member of the Channel 7 crew. Judging by the photo, it looked pretty suspicious. However, the security forces performed a controlled detonation on it (the third such activity performed during the games) and later released a statement stating that ‘no explosives were found’.
DAY 10 MONDAY 15 AUGUST
POWER STRUGGLES
Finally, after six days of trying various fixes, we bypassed the extreme noise issue we’d been having at Riofern with the Studer console. The night before we had isolated it to the internal analogue input card, but it had been hard to track down as the symptom was happening intermittently. I sourced an Audient eight-channel preamp to bypass all our on air inputs and routed them into the AES. Still, it shouldn’t have happened.
The day started well but just after our first cross, Riofern lost all power, knocking both Seven and our site off the air. It was a major power outage that took out both main backups at the Seven/NEP building, the UPS and generator were offline. We had our own UPS backup which we could have used to go to air via ISDN (as network services also went down with the power), but instead chose to use staff at other sites in the next hour of updates.
Not long after power was restored at the site, sports broadcaster, Jim Maxwell had a medical emergency and was rushed to hospital suffering from a suspected stroke. All the team at Riofern handled the situation very well, and we chose to move broadcasting away from that site again while the team composed themselves.
DAY 15 SATURDAY 20 AUGUST
CALLING IT IN
Alister called the women’s triathlon action from the IBC — just like the marathon, walks, and cycling road race. Often the courses for those races cover very large areas and it’s logistically easier to call the action from a single location on the finish line or at the IBC using host broadcaster feeds.
DAY 16 SUNDAY 21 AUGUST
CLOSING DOWN
The final event was the closing ceremony, which was being commentated on from Australia. The only element required from me was FX, so I could basically dismantle the majority of my system in the IBC. One of my favourite parts of the games occurred in the closing ceremony when the Prime Minister of Japan, Shinzo Abe, popped out as Mario during the handover to Tokyo. Here’s hoping Japan’s infrastructure will be a little more reliable.
RESPONSES