With augmented reality (AR) technology, instead of simply reading about a natural national monument in danger of losing its protection, you can walk around its red rock formations, see cliffs with ancient etchings appear in front of you and look at hundreds of years of cultural indigenous history at risk of being lost to mineral extraction. Instead of viewing two-dimensional images of spacesuits since the Apollo 11 mission that first landed people on the moon, you can see these high-tech outfits standing on your living room floor and examine them through your phone screen from every angle. With AR, storytellers can create content that literally places readers into a scene. Journalists have been using this relatively new technology for roughly five years, but AT&T (No. 1 on 2019 Top 50 Companies for Diversity) is now applying 5G to improve this technology. AT&T is teaming up with the Washington Post to help expand reporters’ use of AR for journalism.
Immersive journalism is not new for the Washington Post, which published its first AR story in 2015 and now delivers AR content using its smartphone app, but the partnership with AT&T — and the improvements 5G technology will bring to it — are. Jeremy Gilbert, who works in the newsroom with a focus on how to implement new technologies, spoke to Fair360, formerly DiversityInc about the potential this new partnership has.
“The relationship with AT&T will hopefully allow us to do something we haven’t even been able to do yet,” Gilbert said. “We haven’t been able to do it at the scale that we want; the speed that we want. So, what we’re really hoping is that the AT&T relationship and potentially use of 5G will allow us to do more and better AR.”
Immersive technologies: a crash course
In order to understand how 5G can improve immersive journalism, it is important to grasp how these technologies work. Immersive technologies, including augmented reality (AR), virtual reality (VR) and mixed reality (MR), all work by superimposing virtual information and objects onto the user’s reality. Virtual objects can be created in a variety of ways including photogrammetry — a process of combing many two dimensional images to create a three dimensional object — and volumetric capture — which fuses multiple videos to create three-dimensional moving object.
Coding allows these images to be visible through a smart phone’s camera,
respond to the movement of the phone and even interact with the real-life scenery around it.
In AR, the virtual object or information appears layered on top of the user’s surroundings but doesn’t interact with the real objects (think Pokémon Go!). In MR the user’s physical environment is mapped and analyzed and the virtual object or information is anchored in the real world, meaning it can interact with its real-life surroundings (think the hologram of actor Jon Hamm at 2017 Sundance Film Festival). VR is a completely immersive experience, where the virtual experience completely replaces a user’s physical environment — like a 3D, 360-degree video viewed on a mobile device or headset (like the Oculus Rift, a popular VR headset device).
The effects of AR, VR and MR lend themselves well to storytelling.
The benefits of 5G come in when it comes to speed. This fifth generation of wireless technology expands the bandwidth of these networks, which is necessary in a world where it is increasingly common for each person to have multiple devices connecting to a wireless network. As companies like AT&T experiment with the potential of 5G, one intriguing possibility is the use of temporary, high speed mesh networks. The ability to have high speed capabilities outside of normal coverage areas would be especially useful when journalists are in the field capturing images for immersive storytelling. Instead of going to a remote location, gathering images and then waiting to get back to the studio to see if the captures were sufficient, photographers and videographers can transmit and process in near-to-real time allowing them to view a rough draft of the project even as they gather more information.
“Most of the objects we might want to capture are not necessarily easy to transport into your studio,” Gilbert said.
Edge computing is the process of moving computation and data storage units closer to where they are needed. While cloud data centers are distributed throughout the country, edge computing involves individual mobile devices that can be much closer to where individual users want to process data. Mo Katibeh, AT&T’s chief marketing officer told Fair360, formerly DiversityInc this capability also helps ensure a trip to capture images of a national monument in Utah won’t be in vain when reporters return to the studio in Washington, D.C. — even if there are mistakes.
“5G will allow them to capture that information and in real time, using another technology called edge computes — which is bringing the cloud closer to you — … that will happen in more real-time and allow the editors who are monitoring this to say, ‘Oh, I need you to redo something,’” Katibeh said.
Katibeh also said 5G improves the viewer experience by lowering the latency time, or the time between the action that sparks a response and the response. In short, the lower the latency, the faster the response and more realistic the AR, VR or MR footage.
“If, as a human, you’re using virtual reality or augmented reality and the latency is taking more than about 10 milliseconds, then you start feeling nauseous because the image is jittery, versus the way that your brain is processing the video that you’re seeing,” Katibeh said.
A virtual walk in others’ shoes
While 5G is helping to expand the performance of AR, AR itself is helping to expand what journalism can do and what effects it can have. Katibeh said AT&T sought a partnership with the Post because the paper has established itself as an outlet that seeks to apply cutting-edge technologies to its reporting. But the Post has not been the only outlet experimenting with this technology, and its impact can be more than just “cool” and entertaining.
John Pavlik is a researcher, author and Rutgers University journalism professor who focuses on how technology impacts journalism. This past fall, he published “Journalism in the Age of Virtual Reality: How Experiential Media Are Transforming News,” in which he discusses how immersive technologies impact the effects of journalistic storytelling. Pavlik said he does not use the term “paradigm shift” lightly, but believes AR and VR have the power to completely change journalism, just as photography did when it first began appearing in newspapers.
“I think we could see a paradigm shift in journalism toward one in which journalism starts to become more participatory so that the public becomes part of the journalism in a more active way,” Pavlik said.
The New York Times is another notable outlet that has experimented with harnessing the storytelling power of immersive technology. The first VR documentary the Times published was “The Displaced” in 2015, a story featuring children who had been displaced by war and violence throughout the world. The documentary was striking because it allowed viewers to not just view these heart wrenching stories but be part of them. Good journalists write details or capture photos or videos that help place the reader or viewer into the story, and VR and AR take that journalistic tenet a step further.
“One thing that we have seen with immersive storytelling formats … is that they create empathy for subjects in a way that is often deeper than words or images alone can do,” Gilbert said. “That you really feel the experience in a different way that you would if you were just reading or just watching it.”
A 2017 study by S. Shyam Sundar and Jin Kang at Pennsylvania State University and Danielle Oprean at the University of Missouri found immersive journalism increases viewers’ empathy for the story’s subjects. It also sparks action. Jeremy Bailenson, a cognitive psychologist and director of Stanford University’s Virtual Human Interaction Lab and other researchers found in an experiment that those who watched his VR film “Becoming Homeless” were more likely to sign a petition in support of affordable housing.
“Sometimes it is just frankly easier to show than tell,” Gilbert said.
Showing stories through immersive technologies can allow them to take on more nuance and context, Pavlik said.
“I think we’re going to see journalism make a tangible leap forward, closer to the truth,” Pavlik said. “That’s the long-term goal of any journalism: to tell the truth more fully. And I think that’s what these tools have the real ultimate capability to do.”
Katibeh said the tech, media and business worlds are merely at the dawn of discovering all immersive technology and 5G can do.
“The future is up to our own imaginations,” he said. “What I can’t wait — about 5G and AR — is to see how businesses, how application developers, how innovators are going to take this technology and dream up entirely new ways of using it.”