How can we showcase the possibilities of Performance in Immersive Technologies?
Performance and technology have always been intertwined, unfortunately in the past decade the budget cuts to art institutions and funding have relegated technology exploration as something that needs to be done by others. There are some clear examples on how technology is helping Performance advance, from 360.º videos to the use of motion capture, to the latest piece by the San Francisco ballet. Unfortunately, these are isolated cases that are outside the possibilities of most Performance companies, who do not have the ability to move in this direction and in many cases do not even know where to start. The technological advancements and budgets have created a significant division on the possibilities and the opportunities to even R&D some of the work.
Partners involved
The Problem
The performing arts sector, rich in tradition and talent, faces modern challenges that hinder its evolution:
- Technological Hesitation: Many organisations within the sector lack the infrastructure or expertise to integrate new technologies, often due to the absence of dedicated R&D departments.
- Creative Limitations: A widespread lack of understanding of the potential of immersive and XR technologies limits their adoption, curtailing innovation in performance creation.
- Economic Barriers: High costs associated with producing traditional and technologically enhanced performances restrict the ability to innovate, impacting long-term sustainability.
The Solution
The use of Mixed Reality (MR) enriches live performances by blending digital elements with real-world interactions, introducing a new era of performing arts:
- Motion and Volumetric Capture: Enabling performers to extend their expressions into the digital realm, where their movements are captured and translated into captivating visual effects that interact with live audiences.
- Interactive Digital Set Designs: Transforming static stages into dynamic environments that react to performer interactions and audience inputs, making each performance unique and engaging.
- Audience Participation Tools: Offering personalised MR devices that allow audiences to influence the narrative flow or explore multiple perspectives of the performance, turning passive viewing into an interactive experience.
Target Audience
This use case serves a variety of professionals in the performing arts sector:
- Theatre Directors and Producers: Harness XR technologies to enhance narrative complexity, creating deeply immersive theatrical experiences.
- Stage Designers and Technical Crew: Implement XR for dynamic, cost-effective set designs that are digitally enhanced and interactive.
- Choreographers and Dancers: Integrate XR to blend live dance with virtual elements, transforming traditional performance into interactive art.
- Musicians and Composers: Utilize XR to produce multisensory concerts, potentially also enabling remote collaborations.
- Actors and Performers: Engage with XR to interact within enhanced digital environments, broadening their performance techniques.
- Drama and Performing Arts Schools: Incorporate XR in educational syllabi to equip students with modern tools that are increasingly relevant in global arts industries.
- Motion Capture
- Volumetric Video
- Performance
- AI
- Theatre
Performing Arts Use Case: Key Milestones
Here’s a look at the Performing Arts Use Case’s journey. This timeline highlights the evolution of the mixed reality (MR) piece of created by SATORE Studio.
Pre-Pilot
1- Definition of the Use-Case Concepts
Concept 0: Mixed Reality Immersive Theatre Experience: Budget constraints in Performing Arts underscore the need for efficient new workflows, which this concept addresses by defining and testing immersive storytelling techniques using technologies like movement, stylization, USD, digital humans, audio, and controls.
Concept 1 Movement: Understanding the capabilities and limitations of different movement capture technologies is crucial for directors, choreographers, and artists, as it fundamentally impacts performance. Baselining these systems for single or multiple performers in various spaces, along with their capture and processing needs (both post and live), is therefore essential.
Concept 2 Stylisation: Utilise multiple styles, in order to understand from the audience the different levels of acceptance and also what are the other elements that are important when trying to create a world that improves the quality of the experience.
Concept 3 Universal Scene Description (USD): Using USD while creating a small experience, while at the same time using another game engine, to compare and understand the limitations of USD.
Concept 4 Digital Humans (DH): Building on the understanding of movement capture, it’s vital to explore the boundaries of stylization to avoid the uncanny valley while maximizing character customization for production needs like hair, makeup, and costume. This also involves investigating the potential of AI to create believable digital actors.
Concept 5 Audio / Music / Sound / Speech: Create an experience that is fully driven by Audio, using music, sound and speech. This will give us insights into how to best use audio within the experience, but also how it should be created and the different solutions that it can attain.
Concept 6 Controls: Create interconnected solutions with other software, in order to use their capabilities, in order to influence how the experience is being driven.
Concept 7 World Building: Use multiple techniques that create variation and exploration on the world creation. Each technique gives a different look and feel, as well as the opportunity to combine them, based on the expertise of the user.
Pilot
1- Coming back to Concept 0
The pilot, which is based on the findings of Concept 0, is a mixed reality (MR) experience that immerses users in the life journey of Christian Aiken Isaac, known as Aiken, as they engage with him across various stages of his life.
