Get an inside look at our mixed reality Crack XR app with Oliver Ellmers
We’ve launched our first mixed reality app, Crack XR.
At Crack Magazine, we’re interested in the latest innovations in technology and how they can be used to elevate music, or amplify the works of groundbreaking artists. Over the past decade, we’ve experimented with apps to animate landmark covers such as our Aphex Twin and Jehnny Beth issues. Now we’ve worked with Innovate UK – a government-supported scheme “driving innovation and development during the Covid-19 pandemic” – to create our latest venture in tech: Crack XR.
The app features immersive performances from Bermondsey-raised rapper Flohio – who raps the lyrics to Sweet Flaws – rising London rapper Brian Nasty with the love song Heart Emoji, and PC Music innovator A.G. Cook; he reimagines The Darkness with guests Hannah Diamond and Sarah Bonito. All three performances are brought to life in extended reality, and can be watched anywhere using your phone. For now, the app is only available on iOS and you’ll need a WiFi connection to download the performances. Check out Crack XR here.
Visual artist Oliver Ellmers helmed the technical direction and development of the app. It’s been over six months in the making, over the course of the pandemic. Below, he gives us an inside look into the creative process.
View this post on Instagram
What was the thinking behind Crack XR?
Due to Covid-19, stages and live performances were inaccessible and/or not happening. We wanted to give people access to live music performances while they were unable to attend events, from the comfort of their own homes. Mobile technologies offer a fantastic delivery method for any sort of digital multimedia as most people have a modern smartphone and a network connection available to them.
We wanted to do something different from simply creating music videos or 2D video streams of artist performances to people’s smartphones or tablets.
Why did you choose augmented reality in particular?
We chose augmented reality technology embedded within our own mobile app as a delivery method for artists’ performances because it enables us to create immersive experiences that people can interact and engage with – from wherever they are on their smartphones. With this brief in mind, we applied for IUK and were granted funding.
Once you were successful, what came next?
We had to figure out how we were going to create the 3D artists! 3D scanning, photogrammetry and volumetric capture became our main areas of research. We went for volumetric capturing, which is a 3D scanning technique that records animated three-dimensional digital assets. Over six months of research and development with various volumetric capture workflows and techniques, we ended up settling on a cutting-edge workflow using modern depth-sensing technology and game engine technology.
How did you go about filming the performances?
We worked with 3D designer George Jasper Stone to create the environments for the artists to perform on and within. We set up a permanent capture rig in Tottenham Hale for creating our captures. This allowed us to create volumetric captures of artist performances and integrate them into various outputs such as augmented reality, virtual reality, interactive multimedia systems, and for use in live television, film and VFX productions.
What’re you taking away from the project?
We have since developed our own production workflows and pipelines to efficiently create these sort of experiences, and formed our new studio Good Measure. Now that the Crack XR app is out in the wild, we have a platform for distributing immersive augmented reality artists’ performances, and a new platform for us to engage with our followers in new and meaningful ways.
Download the Crack XR app via the Apple App Store
ADVERTISEMENTS