Disney Research Invites You to Share the Magic of Augmented Reality with a Seat on the Magic Bench


Share this Article

As an important part of The Walt Disney Company, the Disney Research company’s mission is to increase the value for the company by developing and delivering scientific and technological innovation to create ‘the science behind the magic.’ Far more than singing, dancing Mousketeers, these talented researchers complete basic and application-driven research in many topics, including animation, robotics, computer graphics, human-computer interaction, and more. Disney Research has also utilized 3D printing technology many times, most recently to design bendable machines and bring animated characters to life; the researchers even developed their own 3D copier last summer.

Now Disney Research is using advanced technology to allow groups of people to enjoy augmented reality (AR) together, but unlike the shared reality HoloViveObserver system, you don’t have to hold a device or wear a bulky, head-mounted display. The Magic Bench, a custom software and hardware platform, is actually a combined augmented and mixed reality (MR) experience, and allows users to not only see and hear animated, virtual characters, but feel them as well.

“One of the compelling ideas behind mixed reality is this ability to share the same space as an animated character,” said Moshe ‘Mo’ Mahler, principal digital artist at Disney Research. “And this is where an exciting new form of storytelling begins.”

Want to hang out with your friends and get stuck in the rain with a giraffe, or have an elephant hand you a glowing, gold orb? The Magic Bench makes it possible! It actually instruments the environment in the combined experience, instead of an individual person running the show – this lets multiple people enjoy the ‘walk up and play’ experience together.

“This platform creates a multi-sensory immersive experience in which a group can interact directly with an animated character. Our mantra for this project was: hear a character coming, see them enter the space, and feel them sit next to you,” said Mahler.

Anyone who sits on the Magic Bench will be able to see themselves in a mirrored image on a large display that’s in front of them, which creates a third person point of view. A depth sensor then reconstructs the environment, so instead of superimposing one video feed onto another, the participants can occupy the same 3D space as a computer-generated object or character.

“The bench itself plays a critical role. Not only does it contain haptic actuators, but it constrains several issues for us in an elegant way,” Mahler explained. “We know the location and the number of participants, and can infer their gaze. It creates a stage with a foreground and a background, with the seated participants in the middle ground. It even serves as a controller; the mixed reality experience doesn’t begin until someone sits down and different formations of people seated create different types of experiences.”

While the experience does require the physical bench, a color camera and depth sensor from a Microsoft Kinect were used to create a HD-video-textured, real-time 3D reconstruction of the bench, the participants, and the environment. The scene is then reconstructed using an algorithm, which aligns the depth sensor information with the RGB camera information.

To make the whole thing look as normal as possible, a modified algorithm creates a 2D backdrop, which gets rid of any depth shadows in areas where the depth sensor doesn’t have a corresponding line of sight with the camera. Then, both the 2D and the 3D reconstructions are put into position in the virtual space, and then populated with 3D effects and characters to create a composite capable of interacting with virtual light and shadows. 

A large team of scientists from Disney Research Pittsburgh wrote a paper on their work, titled “Magic Bench – A Multi-User & Multi-Sensory AR/MR Platform.” Co-authors include Mahler, Ali Israr, James Krahe, Shawn Lawson, Jake Marsico, John Mars, Jim McCann, Kyna McIntosh, and Alexander Rivera.

The abstract reads, “Many MR interactions are generated around a first-person Point of View (POV). In these cases, the user directs to the environment, which is digitally displayed either through a head-mounted display or a handheld computing device. One drawback of such conventional AR/MR platforms is that the experience is user-specific. Moreover, these platforms require the user to wear and/or hold an expensive device, which can be cumbersome and alter interaction techniques. We create a solution for multi-user interactions in AR/MR, where a group can share the same augmented environment with any computer generated (CG) asset and interact in a shared story sequence through a third-person POV. Our approach is to instrument the environment leaving the user unburdened of any equipment, creating a seamless walk-up-and-play experience. We demonstrate this technology in a series of vignettes featuring humanoid animals. Participants can not only see and hear these characters, they can also feel them on the bench through haptic feedback. Many of the characters also interact with users directly, either through speech or touch.”

This week, the Disney Research Pittsburgh team demonstrated its Magic Bench at the SIGGRAPH 2017 conference in Los Angeles. Discuss in the Magic Bench forum at 3DPB.com.

[Sources: Disney Research, EurekAlert / Images: Disney Research]


Share this Article

Recent News

EOS & AMCM Join Forces with University of Wolverhampton to Establish UK Centre of Excellence for Additive Manufacturing

3D Printing News Unpeeled: Better Elastomers, Mailbox Keys and Origami Networks


3D Design

3D Printed Art

3D Printed Food

3D Printed Guns

You May Also Like

3D Printing Unpeeled: New Arkema Material for HP, Saddle and Macro MEMS

A new Arkema material for MJF is said to reduce costs per part by up to 25% and have an 85% reusability ratio. HP 3D HR PA 12 S has been...

3D Printing News Briefs, January 20, 2024: FDM, LPBF, Underwater 3D Printer, Racing, & More

We’re starting off with a process certification in today’s 3D Printing News Briefs, and then moving on to research about solute trapping, laser powder bed fusion, and then moving on...

3D Printing Webinar and Event Roundup: December 3, 2023

We’ve got plenty of events and webinars coming up for you this week! Quickparts is having a Manufacturing Roadshow, America Makes is holding a Member Town Hall, Stratafest makes two...

Formnext 2023 Day Three: Slam Dunk

I’m high—high on trade show. I’ve met numerous new faces and reconnected with old friends, creating an absolutely wonderful atmosphere. The excitement is palpable over several emerging developments. The high...