Meta Shares Study on Meta 3D Gen, a Generative AI System to Create 3D Assets From Text Prompts

Meta has shared a brand new analysis paper a few generative synthetic intelligence (AI) mannequin that may generate 3D belongings. Dubbed Meta 3D Gen, the AI mannequin generates high-quality 3D fashions from textual content prompts in lower than a minute. The corporate highlighted that the AI mannequin may create end-to-end high-resolution textures and materials maps utilizing a mix of texture era and text-to-mesh strategies solely developed in-house. Notably, this method is totally different from those by Tencent and Stability AI, which released image-to-3D fashions.

Sharing its analysis paper, which is presently hosted on Meta’s web site, the corporate claimed that its 3D Gen mannequin can render 3D fashions with excessive immediate constancy and guaranteeing top quality. The mannequin can be mentioned to help bodily primarily based rendering (PBR), which permits the generated fashions to be relighted in several real-world purposes. Notably, the analysis paper isn’t peer verified, so a few of the claims made might be exaggerated. There is no such thing as a approach to discover out except the mannequin is launched to the general public.

The paper additionally explains the approaches used to develop the AI mannequin. Meta has developed two in-house programs — Meta 3D AssetGen and Meta 3D TextureGen. AssetGen is accountable for producing 3D objects from a given textual content immediate. In Stage I, it renders the 3D mannequin utilizing its understanding of the immediate, and in Stage II it additional refines the 3D mannequin to reconcile the view-based textures and completes these components that aren’t seen in any views. TextureGen is accountable for discovering and making use of the fitting textures on the mannequin.

This technique of 3D mannequin era has a number of advantages for the generated 3D belongings. For example, as a result of it makes use of textual content prompts, there isn’t any requirement for reference photographs and customers can generate solely distinctive fashions. The PBR help additionally signifies that these fashions might be taken out of the system and instantly utilized to different environments. The paper claims the era output of those fashions are additionally reasonable.

Meta mentioned that the 3D Gen AI mannequin has a win charge of 68 p.c in single stage, which signifies that 68 instances out of 100, the AI can generate the fitting 3D asset primarily based on the textual content immediate. The corporate additional claims that the mannequin outperforms numerous business baselines when it comes to immediate constancy and visible high quality for complicated prompts.