PDF Download
PubReader
Export Citation
Email the Author
Share Facebook
Share Twitter
Cited by 0 Article
Metrics (View:507)
Introduction
Augmented reality (AR) refers to an interactive experience that enhances the real world via the use of computer-generated perceptual information encompassing visual, auditory, haptic, and other sensory elements. Though AR is largely synonymous with mixed reality or extended reality (XR), it differs from virtual reality (VR). While the surrounding environment in VR is completely virtual, that in AR is only partially virtual through the addition of virtual layers to the real environment.
The first example of AR in history is perhaps a prototype simulator introduced in 1929 by Edward Link, a pioneer in the development of simulation technologies (Zhao, 2009). Since then, over approximately a century, AR has emerged as a prominent technology, experiencing increased popularity and growth across various fields. The technological developments in AR have played a key role in transforming the nature of communication in many business areas, providing new possibilities and applications that were previously only imagined. Two notable examples in marketing include 1) Wanna and 2) IKEA Place app. Wanna offers a realistic and engaging visualization experience that allows users to virtually try on different retail products through their smartphone. This example shows that AR is a potent tool for e-commerce as it increases consumer involvement and lowers uncertainty associated with online product purchases (Nanda, Xu, & Zhang, 2021; Xing & Chen, 2017). IKEA Place app, on the other hand, enhances furniture and décor experiences by providing a comprehensive visual immersion that incorporates realistic light and shadow rendering. Each product in the IKEA Place app is meticulously proportioned and rendered in three-dimensional format, ensuring cohesive integration within the customer’s environment.
The use of AR technologies has also been prevalent in music industry. The advance of digital technologies (e.g., the internet, smartphones, and music streaming platforms) has offered a playground for AR to come into play in the music video creation, an essential part of music marketing and promotion. One of the earliest examples of AR in music videos is Beastie Boys’ “Intergalactic” released in 1998. The video features a giant robot, created by using AR technology, that interacts with the surroundings and the band members. More recent examples are artists Taylor Swift and Billie Eilish, who have released music videos that allow viewers to interact with the content in new ways. For instance, in Taylor Swift’s “Look What You Made Me Do” music video, viewers can click on different parts of the screen to reveal hidden messages and clues. In Billie Eilish’s music video “Therefore I Am”, viewers can use their smartphone to scan objects in the video to make new three-dimensional elements appear in their camera’s field of view. Furthermore, the adoption of AR has been extended to significantly transform the training and education of future generations in music. The Microsoft Hololens opens a new horizon for piano education by developing a holographic system capable of instructing young trainees in piano playing without losing emotional connections with them.
Another area where the AR technologies have been popularly adopted in the music industry includes live music events such as concerts and festivals. What is increasingly popular is so called XR concerts wherein AR technologies are immersed into the live concert performance through the collaboration between social media platforms and event promoters. A prime example of the XR concert is U2’s eXPERIENCE + iNNONENCE Tour of 2018. During this tour, concertgoers had the opportunity to witness a remarkable spectacle. A colossal iceberg projection appeared on an impressive 80-foot screen. Meanwhile, those who were using their smartphones were treated to a truly immersive experience as Bono, the lead vocalist of the band, floated over their heads.
The idea of using AR technologies in the live music concert, however, comes at a critical juncture for the music industry, where stakeholders such as artists and audiences hold diverse perspectives on it. Some attendees argue that the use of smartphones during live music events detracts from the overall experience and leads to a diminished sense of presence and engagement in the moment. Correspondingly, some artists, like Beyoncé, and Bruno Mars, enforce no-phone policies to preserve the original artistic quality of their performances. Despite the presence of such concerns, the XR concert market has been expanded in recent years and continues to grow remarkably from USD 105.58 billion in 2023 to USD 472.39 billion by 2028 (Mordor Intelligence, 2023). This evidence indicates that many others embrace AR technologies as an avenue for capturing and disseminating memories more effectively in the live performance setting. Given the recent surge in popularity of the AR technologies used for the XR concerts, this paper introduces two such cases, namely, Snapchat and Gorillaz. In the sequel, we detail the AR technologies used in each case and discuss their potential to be an effective communication vehicle in live music events.
Two Cases of Augmented Reality in Live Music Events
Snapchat is a multimedia chatting application launched by Snap Inc. in September 2011. Originally focusing on person-to-person photo sharing, Snapchat has evolved as a multipurpose platform that features users’ chronological stories and shows brands’ short-form contents. Having over 750 million monthly active users, Snapchat boasts a strong community of music enthusiasts. This is evidenced by the fact that 85% of Snapchat users claim to use the app to enhance their live music experiences (Mendelson, 2023).
At the Snap Partner Summit 2022, Snapchat announced its exclusive partnership with Live Nation, designating itself as the sole AR partner for sixteen major events. These include renowned festivals like Lollapalooza, Bonnaroo, the Governors Ball, and the Electric Daisy Carnival (EDC), the last of which is one of the largest electronic dance music festivals in the United States of America. The partnership with Live Nation empowers Snapchat to develop unique AR lenses and effects through its branded AR creative studio, Arcadia, marking a significant stride in the integration of AR into the live event experience (Spangler, 2022). This collaboration further taps into Snapchat’s AR capabilities, transforming the app into a twenty-first-century playbill for festival goers.
Snapchat’s partnership with Live Nation aims to go beyond experimentation, seeking to elevate live music performances to new heights. Another partnership with Disguise, a key player in live event visualization and production acclaimed for its innovative RenderStream technology, enables Snapchat to integrate AR visuals into onstage productions. This innovative approach results in a fusion of the artist’s creative vision and AR, producing an immersive experience that dissolves the boundaries between the virtual and real worlds. The intricately woven AR visuals has been an integral part of live concerts.
In 2023, the EDC, attracting over 100,000 music fans, marked the debut of Snapchat’s four new AR filters, which include:
-
Friend FindAR Lens, which allows attendees to locate their friends within the crowd.
-
AR Compass Lens, which shows the line-up for each stage, including current performers and upcoming acts.
-
Night Owl Lens, which brings EDC’s iconic Owl to life, blending with the open air around festival goers.
-
Daisy Lens, which uses ground segmentation to make digital flowers sprout from the ground.
Presented in Figure 1 are visual images of these four AR filters. According to Snapchat, the primary goal of these four AR lenses is to enhance the overall festival experience for attendees. The aim is to leave a lasting impression and to ensure that the lenses complement rather than detract from the main performances.
Gorillaz is an English virtual rock band formed in 1998, along with Nexus Studios, a leading real-time animation specialist. Gorillaz consists of two human members, a musician Damon Albarn and a comic artist Jamie Hewlett, and four animated members such as 2-D (vocals, keyboard), Murdoc Niccals (bass guitar), Noodle (guitar, keyboards, vocals), and Russel Hobbs (drums). Since its inception, Gorillaz has been productive and successful, releasing a total of eight studio albums ranging from Gorillaz in 2001 to Cracker Island in 2023 and clinching several music awards (e.g., Grammy, MTV Video Music, NME, and MTV Europe Music).
Gorillaz hosted an AR concert in New York City and London in December 2022 to present its single “Skinny Ape.” A multitude of fans gathered to witness the animated rock band live, against the London and New York skylines, from their mobile phones (Oleaga, 2022). After downloading and installing the free application Gorillaz Presents from the App Store or Google Play, attendees were able to view the band’s live performance on their mobile devices while overlaying three-dimensional (3D) images of the band members within the app. Animations and dynamic visual elements responded to the music and movements of the band members within the attendee’s current location, creating a mixed-reality experience for spectators. Moreover, the use of diverse camera angles played a crucial role in deepening audience engagement and creating a heightened sense of immersion. Through the integration of AR technology, Gorillaz adeptly employed multiple strategically positioned cameras surrounding the virtual stage. This enabled the capturing of various perspectives and viewpoints, encompassing the band members and the virtual environment. The incorporation of these camera feeds within the AR concert experience granted viewers the flexibility to seamlessly transition between different camera angles and even see a panoramic 360-degree view of the virtual stage. By intentionally selecting these camera angles, Gorillaz could enhance the concert encounter. Several snapshots portraying the Gorillaz’s AR concert are presented in Figure 2.
The development of the Gorillaz live location-based AR concert is facilitated by the introduction of Google’s ARCore Geospatial API, an AR developer platform that enables users to remote attachment of content to any location captured by Google Street View, and thereby create AR experiences on a global scale. Content creators and artists can turn cities into live canvases by building and publishing transformative and robust location-based, immersive experiences.
Geospatial Creator, powered by ARCore and Google Maps, facilitates the visualization and placement of digital content in real-world locations, resembling the functionality of Google Earth or Google Street View. It incorporates features like Rooftop anchors, which simplify the process of anchoring virtual content with 3D Tiles and reduces development time and effort. These tools enable the creation of world-anchored, cross-platform experiences on supported Android and iOS devices. Immersive experiences developed in Adobe Aero can be easily shared through QR code scanning or links, eliminating the need for full app downloads. Geospatial Creator empowers users to experience their creations in the physical world through real-time localization and augmentation.
When viewed through a device’s camera, digital elements such as 3D models and text labels seamlessly blend with the user’s perception of the physical environment (Oleaga, 2022). Directed by Jamie Hewlett, the co-creator of Gorillaz, together with an Emmy-nominated director Fx Goby, the “Skinny Ape” performances leveraged the AR Core Geospatial API to render cultural experiences in public spaces.
Such a pioneering implementation of AR and localization-based marketing can revolutionize cultural events by delivering customized, real-time promotional experiences. By fusing the corporeal and virtual realms, the Gorillaz show exemplified the potential of these technologies not solely in out-of-home media, but also in immersive and AR advertising, thereby bridging the chasm between the virtual and physical worlds while enhancing interactions between brands and audiences. However, it is crucial to note that the significance of Gorillaz’s AR concert extends beyond technology, as their performances are characterized by a fusion of diverse genres and an uncategorizable style (Oleaga, 2022). Fans who were unable to attend the in-person concerts in New York and London in 2022 were able to enjoy the concerts from the comfort of their homes until June 2023 through the Gorillaz app (Anima, 2023).
Discussion
AR has quickly become one of the most influential concepts in the music industry. In general, AR integrates virtual elements into real-world environments, resulting in enhanced sensory experiences. In the context of live music events, AR can be used in a range of different ways. As evidenced by the previous two cases, AR may encompass interactive visuals projected onto stage set-ups, augmented sculptures, or holograms of band members appearing alongside live performances. Through the utilization of holographic projections, digital enhancements, and/or interactive elements, artists can blur the line between reality and the virtual world as well as expand the boundaries of artistic expression (Madan, 2016). Therefore, AR serves as a potent catalyst that transports viewers to new imaginative territories, dismantles traditional constraints between performers and spectators, and enables artists to enhance performances with smartphones serving as formidable conduits for this (Sakhuja, 2021). All these functionalities of AR indicate that AR can be used as a vehicle to facilitate effective communications between artists and audiences.
Another important appeal of AR technology, as highlighted by Wu and Kim (2022), lies predominantly in its simplicity and accessibility. Unlike VR, which immerses users entirely in a simulated world using specialized equipment like headsets, AR generates virtual experiences simply through a smartphone or tablet apps, making them more user-friendly and widely accessible. This makes AR an ideal choice for festival and concert attendees, who typically carry their phones with them. The simplicity of AR not only enhances the overall event experience but also fosters widespread adoption. AR’s intuitive ability to overlay digital content onto the physical world ensures that even first-time users can readily participate and benefit from its functionalities.
The future of live music events lies in the integration of technology and entertainment, with AR at the forefront. As AR technology advances rapidly and the usage of mobile devices increases, new trends emerge and the drive to find new ways to engage attendees arises. Musicians can utilize AR to enhance and evolve their onstage presence with digital effects and holograms, by directing the audience’s focus to the music being presented. In addition, the easiness of sharing and using AR contents through social media platforms can foster connections among concertgoers and enable to identify individual user preferences that can be used to generate personalized concert recommendations.
The two cases, Snapchat and Gorillaz, described in this study provide valuable insights into the innovative use of AR technologies in live music performances. Notable here, is the transformative power of AR at the XR live concerts. AR can bring a new level of excitement to the concert experience by incorporating virtual elements into the physical stage. The real-time visual effects, animations, and responsive three-dimensional objects that are synchronized with a band’s music altogether foster deep engagement and interactivity among attendees, allowing them to customize their visual journey and actively participate in the concert.
The XR live concerts can exceed physical and geographical limitations, allowing fans from around the world to engage with their favorite artist through AR-enabled live streams and apps. This integration of remote participation expands the accessibility and inclusivity of the concert experience, democratizing access to live performances for a broader audience.
AR technology also offers exciting opportunities for storytelling during live concerts, as artists can merge real-world elements with virtual narratives to create immersive and emotionally resonant storylines that unfold throughout the performance. This combination of storytelling and AR technology adds depth and memorability to the concert experience, further enhancing audience engagement, increasing loyalty, and creating a strong emotional bond between the musician and their audience.
However, it is also important to acknowledge the potential limitations of the XR live events. Technical constraints and equipment prerequisites, like Internet access, can present accessibility hurdles for certain audiences. Correspondingly, substantial production expenses and complex implementation processes such as developing an AR software platform or copyrights, may impede widespread adoption within the music industry.
The possibilities arising from the successful integration of AR technology into live performances go beyond just the XR concerts. This case study has implications for the wider music industry and presents new opportunities for artists. It challenges traditional conceptions of concerts and encourages further investigation into augmented experiences that bridge the physical and virtual domains.
Conclusion
AR has become a revolutionary strength in live music performances, transporting attendees to new venues and environments. It serves as an essential tool for artists and event promoters to engage and connect with their audience in new ways. The integration of AR through festival apps and social media platforms ensures accessibility around the world. It also has the potential to redefine entertainment and unlock new dimensions of music, art, and human connection.
Snapchat’s and Gorillaz’s success with AR concerts/festivals exemplifies the power of collaboration between artists, animators, and technology experts. Their achievements underscore the significance of embracing interdisciplinary collaboration to unlock the full potential of AR technology in concert production.
In conclusion, AR has become a transformative communication approach, propelling music performances into uncharted territories. It empowers musicians and event planners to forge deeper connections with their audiences, blurring the lines between the real and the virtual world.
