It looks like the depth-sensing time-of-flight camera technology that was developed and popularized for projects like the Xbox Kinect and the ZCam are already showing up in the next generation of smartphones. With Google announcing at CES 2016 that their Project Tango will bring computer vision, depth sensing, and motion tracking technology to Lenovo consumer mobile devices this year, full 3D video is just one of dozens of apps that will soon be available. With depth-sensing technology, smartphones will instantly become hyper-aware of their immediate surroundings and can be used for everything from virtual reality applications to a version of Google maps with walking directions inside of buildings to enabling accurate sizing suggestions when purchasing clothing and shoes online.
Range imaging time-of-flight cameras are highly advanced LiDar systems that replace the standard point-by-point laser beams with a single light pulse to achieve full spatial awareness. The camera can sense the time that it takes light to return from any surrounding objects, combine it with video data and create real time 3D images that can be used to track facial or hand movements, completely map out a room or even remove or overlay 3D objects or backgrounds from an image. The technology can be adapted as a high-speed 3D laser scanner, which can capture more than 150 images per second. Realistically, the technology could turn the everyday smartphone into an incredibly advanced, handheld 3D scanner.
While 3D video and high-speed laser scanners are not new technology, they have never really been adopted by mainstream users due to cost. Typically, full-use 3D cameras and laser scanners can cost $10,000 and increase in price to over $200,000 for higher-end models. But the cost of the technology is rapidly dropping thanks to impending consumer-grade products like those developed by Google’s Project Tango not to mention Intel RealSense and the upcoming Apple PrimeSense depth-sensing cameras. With this new technology hitting smartphones, a world of new, life-changing apps and functionality will be available to virtually everyone.
One of the hurdles that needs to be cleared before the full range of depth-sensing camera features can be adopted by mainstream users is the ability to process and compress the large amounts of data required to generate a fully-realized 3D image. But with advances in smartphone processors and data compression, app developers are already hard at work developing for the new platform. While this may require hardware manufacturers to heavily invest in software development to manage the new apps, many of the industry’s biggest names are already on board.
Depth sensors’ ability to capture large amounts of 3D information will completely change the way that digital photography is done. Users would be able to seamlessly remove objects, isolate single objects, and even remove and replace the entire background of a picture. Navigation software like Google Maps would be more accurate than ever by combining GPS data with the ability to accurately sense the user’s immediate surroundings; they could offer the ability to navigate inside of buildings. Devices could even be developed that allow the visually impaired to navigate the world around them, and the hearing impaired could have sign language translated in real time.
Another major application could completely alter the way that we buy and shop for clothing by offering highly accurate sizing without having to step into a store or physically try anything on. Depth sensors on smartphones would turn a 3D selfie into incredibly accurate body measurements that would work with retailers to allow their clothing or shoes to be virtually tried on to verify fit. There are already apps available using this technology, but with a 3D camera in every customer’s home, it could cause the online clothing market to explode. It would even allow high-end retailers the ability to offer custom tailoring or even the fabrication of customized, bespoke clothing.
Naturally, virtual reality and 3D printing would suddenly be accessible to just about everyone. VR enabled smartphones paired with VR headsets would be able to generate complete 3D environments and allow users to interact with them in real time. Games and apps could also be developed to automatically create levels and interactive environments based on a user’s surroundings, turning a living room into a game level or a virtual workspace. And the ability to capture incredibly detailed 3D images means that almost any real world object can be 3D scanned, converted into a 3D printable file and sent directly to online 3D printing services providers. Not only would that turn rapid prototyping into instant prototyping, but it would allow artists and designers to create something and instantly offer 3D printed copies of their work to the rest of the world.
Of course all of those abilities are probably a while out in terms of product availability, but the smartphone cycle moves very quickly. Because consumers usually replace phone models every eighteen months, hardware manufacturers and software and app developers are always pushing to release new devices with the best new toys. With the first handful of depth-sensor enabled devices due out later this year, it isn’t hard to expect that 3D cameras will be standard on smartphones within the next few years. Are these features you are interested in for your smartphone? Discuss in the Smartphones Go 3D forum over at 3DPB.com.
Here is a video explaining how Google’s Project Tango works and what it can be used for:
You May Also Like
3DPOD Episode 46: Engineering CAD with nTopology CEO Brad Rothenberg
Max and I had a really great time talking to nTopology CEO Brad Rothenberg. Brad started nTopology as a series of tools for creating lattices, but it became much more...
3DPOD Episode 45: Sarah Goehrke, Additive Integrity, and Women in 3D Printing
Today, we’ve got the whirlwind that is Sarah Goehrke on the 3DPOD. Max and I had a great time talking to the former 3DPrint.com Editor-in-Chief about her editorial business, Additive...
3DPOD Episode 44: Jennifer Coyne of Wabtec & John Barnes
Today, we talk once again with John Barnes, but we also have Jennifer Coyne of the Wabtec Corporation joining the program. Wabtec is a huge Pittsburg-based train company that makes...
3DPOD Episode 43: Powder Bed Fusion Innovations with Aerosint’s Edouard Moens de Hase
Edouard Moens de Hase talks to us today about Aerosint. For the last few years, they have been working on improving powder bed fusion. The company has an innovative drum...
View our broad assortment of in house and third party products.