Pyeongchang winter olympics: using interactive ar technology to enliven yle’s olympic studio
The success of the women’s ice hockey team, Rukajärvi’s skilful snowboarding, Pärmäkoski’s pile of medals and Niskanen’s gold – the Pyeongchang Winter Olympics offered several great moments for sports fans. In addition to the actual sports the viewers got to enjoy Yle’s expert studio analyses, which Keho Interactive spiced up with versatile virtual graphics created with AR technology. The viewers were delighted with, for instance, virtual snowfall, three-dimensional ski track maps and attention-grabbing live interviews where the athletes in South Korea were teleported to the studio as holograms.
Petra Manner and Yle Sports expert Kalle Lassila discussing the upcoming analysis before the broadcast.
The preparations for the project with Yle Sports began roughly about six months before the Olympic Games. The intention was to bring new kinds of useful virtual elements to the broadcasts without drowning them in special effects. Yle has used AR technology before – for instance in the presidential election broadcast earlier this year – but on a small scale. However, this time the crew set out to do something different by improving virtual graphics and thus making them more interactive instead of them simply being static props. Keho Interactive moved its virtual studio to Pasila for the duration of the Olympic Games and Yle’s own team was responsible for the work in South Korea – everything was in place, let the show begin! The beginning of the Olympic broadcasts was an exciting time since the project included several new elements for the team to master.
Keho Interactive in the house!
The zero hour draws near: pictures of building the studio before the Olympic Games.
By using AR (augmented reality) technology it’s possible to display virtual content on top of a video feed in real time. In this process most of the effort goes into modelling the lens and tracking its movements, yet it’s also the most rewarding part of the work. The difference between a virtual studio and a traditional green screen comes from multidimensionality: in a virtual studio one can move the camera and change filming angles just like when filming in a normal studio. While tracking the camera’s movements the optics need to be carefully modelled into the virtual lens – if the optics of the virtual camera don’t match the optics of the normal camera, moving, zooming or focusing the image will break it. The general-purpose lens modelling method developed by Keho Interactive provides a basis for a diverse range of virtual systems, and can therefore be used in various projects in the future.
The virtual camera moves in a game engine, which is used to create virtual 3D objects visible to the naked eye, such as the ski track maps that demonstrate variations in altitude. The media server plays the virtual videos and images through the game engine, and it’s also used to erase video background as well as to enhance colours. The mixer places the virtual elements on top of the video feed, and as a result we have, for instance, the hologram of Sami Jauhojärvi from South Korea discussing with Petra Manner in the studio. The viewers were able to see Jauhojärvi’s hologram but Manner could only hear his voice. However, there was a monitor outside the frame where Manner could see her interviewee. This made it possible for the interviewer to look straight at the interviewee thus creating a seemingly natural interview situation.
Practice run: Krista Pärmäkoski as a hologram in Yle’s Olympic studio before the broadcast.
Creating these live hologram interviews – something that had never been done before in Yle’s broadcasts – turned out to be one of the most amazing moments of the entire project. The idea of doing them arose on the evening preceding the morning broadcast when the interview was due to air, which meant there was only approximately forty-five minutes to test the idea in practice. By utilising the existing technology and their own software, together with Yle’s team in South Korea Keho Interactive was able to teleport Jauhojärvi’s interactive hologram to the studio in real time while Jauhojärvi himself was standing in front of a green screen more than seven thousand kilometres away. The next live interview aired already on the following day when Kalle Palander’s hologram visited Antti-Jussi Sipilä in Yle’s studio.
Several hologram appearances were carried out during the Olympic Games and most of them were seen in Yle’s “Korea in an Hour” night show. Due to the time difference, doing live interviews wasn’t always possible, which is why recorded material from earlier in the day was used in the broadcasts. As time went by and the team gained more and more experience, the colours of the holograms were enhanced and the slight delay disappeared almost completely. Yle’s experts in Pasila played an important part in the project by handling several crucial things such as sound technology and video mixing.
Several hologram interviews appeared especially in the “Korea in an Hour” show. Here is the host of the show, Kristiina Kekäläinen.
Achieving interactivity in the broadcasts was above all a team effort. Keho Interactive created the required virtual graphics and was ready to display them when needed, the interviews were conducted in South Korea, Yle’s television crew oversaw the broadcasts, and the director kept things in hand by making sure that everything was in the right place at the right time.
Coordinating the ski track maps provides a prime example of seamless cooperation: the presentations were planned beforehand so every turn of the map and every highlight on the ski track appeared in sync with the performance of the studio hosts. For example, this way a certain premeditated question presented during the broadcast could act as a que to Keho Interactive’s expert within hearing distance, who was then able to highlight the correct part of the ski track at the right moment by using a tablet. It is possible to develop a user-friendly version of the interface being used to do this, so in the future studio hosts might be able to coordinate the virtual effects by using their own tablets.
A three-dimensional ski track map that displays the routes and variations in altitude.
The Winter Olympics project executed with Yle provided yet another step forward in utilising AR technology in television and was also a rewarding and educational experience for Keho Interactive. The next aim is to concentrate especially in improving three-dimensionality. AR technology is here to stay and Keho Interactive will be there to create new kinds of solutions to meet the demand. In the future we might see virtual graphics in television broadcast which are even more multifaceted and interactive than before – possibly even ones in which viewers could participate from home. Time will tell.
Written by Johanna Honkonen