2d3d.ai: 3D Reconstruction from a 2D Image Using a Neural Network

Share this Article

As artificial intelligence begins to take a real hold in the technology department, we are able to loosen the reins a bit as humans and relax while the innovations we have programmed go forth and create, transform, and improve.

Peter Naftaliev, from 2d3d.ai

That’s the recent situation for Peter Naftaliev, an AI consultant who blogs at 2d3d.ai and works at Abelians, a small firm of former Israeli intelligence Corps staffers, as he brings us another way to improve on going from 2D into 3D seamlessly, thanks to futuristic technology, along with building somewhat reluctantly on previous work in ‘3D Scene Reconstruction from a Single Image’ (noted somewhat inferior single object reconstruction, but impressive natural scene image) and a recent Computer Vision and Pattern Recognition paper, “Mesh R-CNN.”

Regarding the most recent project in reconstruction, Naftaliev states:

“This is the highest quality 3D reconstruction from 1 image research I have seen yet. An encoding-decoding type of neural network to encode the 3D structure of a shape from a 2D image and then decode this structure and reconstruct the 3D shape.”

Featuring a transparent background and an input image of 128×128 pixels, the base resolution is 64x64x64 voxels and ‘can produce output in any required resolution (!) without retraining the neural network.’ Naftaliev also points us to the corresponding paper, ‘Learning Implicit Fields for Generative Shape Modeling,’ by Zhiqin Chen and Hao Zhang, as they ‘advocate’ for using generative models and also intend to improve the visual quality of the resulting shapes.

“By replacing conventional decoders by our implicit decoder for representation learning (via IM-AE) and shape generation (via IM-GAN), we demonstrate superior results for tasks such as generative shape modeling, interpolation, and single-view 3D reconstruction, particularly in terms of visual quality. Code and supplementary material are available at this https URL,” state the authors.

“Our implicit decoder does lead to cleaner surface boundaries, allowing both part movement and topology changes during interpolation. However, we do not yet know how to regulate such topological evolutions to ensure a meaningful morph between highly dissimilar shapes, e.g., those from different categories. We reiterate that currently, our network is only trained per shape category; we leave multicategory generalization for future work. At last, while our method is able to generate shapes with greater visual quality than existing alternatives, it does appear to introduce more low-frequency errors (e.g., global thinning/thickening).”

 

  • The first column is input image.
  • The second column is the AI 3D reconstruction.
  • The last column is the original 3D object of the car – ‘ground truth.’

While the neural network here was trained over cars, Chen and Zhang also use other examples like chairs and airplanes. Naftaliev explains that input-output images and voxel resolutions can be switched ‘for any required implementation.’ And if you are wondering how that last car was reconstructed, welcome to the power of AI. After training over a multitude of examples, the software knows how to present the proper image. If you are looking for tools for 3D reconstruction, try exploring classic photogrammetry techniques and two examples: Agisoft and AutoDesk – Recap.

“This type of software can benefit from the current AI research. Reconstruction of simple planes even if they are not completely seen in the image, handling light reflections or aberrations in the image, better proportion estimations and more. All these can be improved using similar neural network solutions,” says Naftaliev in conclusion.

It seems that we could be on the cusp of being able to easily extract STL files from 2D images and drawings. If repeatable and easy then this would let anyone draw an object that could be turned into a 3D printable file. At the same time, many images can be used to reverse engineer, remix and improve objects that then can be 3D printed. One of the main things holding back 3D printing is that few know CAD and this approach could give many more people the ability to create 3D shapes that could be printed. This could have serious and far-reaching positive impacts on 3D printing.

With a focus on learning, educating, and sharing, Naftaliev strives to understand the open-source community, and what makes them thrive. Check out some of their other projects here. If you are interested in artificial intelligence and the impacts it is making within the open-source and 3D printing community, read about other related stories such as living architectures, human-aware platorms, image recognition, and more.

What do you think of this news? Let us know your thoughts! Join the discussion of this and other 3D printing topics at 3DPrintBoard.com.

3D shapes generated by IM-GAN, our implicit field generative adversarial network, which was trained on 643 or 1283 voxelized shapes. The output shapes are sampled at 5123 resolution and rendered after Marching Cubes (from ‘Learning Implicit Fields for Generative Shape Modeling,’)

[Source / Image: 2d3d.ai]

Share this Article


Recent News

3D Printing News Briefs, November 28, 2020: Thinking Huts, nScrypt, Alloyed, ASTM International

RYUJINLAB, INC Launches Low-cost Metal 3D Printing Service for General Public



Categories

3D Design

3D Printed Art

3D Printed Food

3D Printed Guns


You May Also Like

Sponsored

Meltio Engine Jumps over the Limits of Metal 3D Printing by Enabling Hybrid Fabrication

MELTIO has officially presented today the new version of the MELTIO Engine, a fabrication module which enables 3D printing of full density metal parts when integrated with CNC machines, robots,...

3D Printing for Preppers: The Virtual Foundry’s Metal 3D Printing Filament

Foreshadowing the expansion of bound metal printing by several years, Bradley Woods developed the idea of metal 3D printing filaments in 2014 when he obtained his first 3D printer kit....

Sponsored

Additive Manufacturing 2.0: The future of metal manufacturing starts now

It’s increasingly clear: The way we make things is changing. As more companies realize the advantages that come with additive manufacturing – like tooling-free manufacturing, ability to create highly complex...

3D Printing Financials: Revenue Up in First Nine Months of 2020 for SLM Solutions; Q3 Earnings Down

For the third quarter that ended September 30, German metal 3D printer manufacturer SLM Solutions reported revenues decreased by 13% to €14.8 million compared to last year’s €17 million. Along...


Shop

View our broad assortment of in house and third party products.