Enlarge / A poison dart snake frog, rendered as a 3D model by Magic3D.

Nvidia

Friday will be a day for researchers from Nvidia Announces Magic3DAn AI model called. can generate 3D models from text descriptions. You can enter a prompt such: “A blue poison-dart frog sitting on a water lily,” Magic3D It generates a 3D A mesh model complete with colored texture in just 40 minutes. Modifications can make the model even more beautiful. can used in CGI art scenes and video games.

In its academic paperNvidia frames Magic3D As a result of DreamFusion?, text-to-3D In September, Google researchers revealed a new model. Similar to DreamFusion using a text-to-image model to generate An image in 2D that is then converted into volumetric. NeRF Data, (Neural Radiance Field) Magic3D It uses a two-stage procedure that takes a coarse model with low resolution and optimizes the result to higher resolution. The paper’s authors claim that the result is a two-stage process. Magic3D Method can generate 3D DreamFusion can create objects twice as fast as DreamFusion.

Magic3D can Also, you can perform prompt-based editing 3D meshes. Given a low-resolution 3D It is possible to modify the model and base prompt text You can also modify the model. Also, Magic3DThe authors show that they can preserve the same subject through multiple generations (a concept called coherence) while using the style and technique of a 2D image, such as a cubist paintings, to convey their thoughts. 3D model.

Nvidia didn’t release any Magic3D Code along with the academic paper.

The ability to generate 3D from text It feels like a natural development in today’s dissemination modelsThe neural networks can be used to create new content by combining data from multiple sources with intense training. Only 2022 has seen the rise of capable. text-to-image models DALL-E, Stable Diffusion, and the rudimentary text-to-video generators from Google and Meta. Google also launched the aforementioned text-to-3D Two months ago, model DreamFusion was created, and people have since adopted it. adapted similar techniques Open source model that is based on Stable Diffusion.

As for Magic3DResearchers behind it believe it will be possible to create anything anyone wants. 3D models Without the need for Training in special subjects. The technology can be refined and used to speed up the development of video games (and VR) and possibly find applications in special effects. for Film and TV. They write at the end of the paper: “We hope with Magic3D, we can democratize 3D synthesis and open up everyone’s creativity in 3D content creation.”