iTWire - Nvidia announces generative AI models and NIM microservices for OpenUSD language, geometry, physics, and materials (2024)

Through new OpenUSD-based generative AI and Nvidia-accelerated development frameworks built on theNvidia Omniverseplatform, more industries can now develop applications for visualising industrial design and engineering projects, and for simulating environments to build the next wave ofphysical AI and robots.

The new offerings includeNvidia NIMmicroservicesfor AI models that can generate OpenUSD language to answer user queries, generate OpenUSD Python code, apply materials to 3D objects, and understand 3D space and physics to help accelerate digital twin development. In addition, new USD connectors to robotics and industrial simulation data formats and developer tools let users stream massive, fullyNvidia RTXray-traced datasets to Apple Vision Pro.

“The generative AI boom for heavy industries is here,” said Nvidia vice president of Omniverse and simulation technology Rev Lebaredian. "Until recently, digital worlds have been primarily used by creative industries; now, with the enhancements and accessibility Nvidia NIM microservices are bringing to OpenUSD, industries of all kinds can build physically based virtual worlds and digital twins to drive innovation while preparing for the next wave of AI: robotics.”

Generative AI comes to USD with Nvidia NIM
The world’s first generative AI models for OpenUSD development, developed by Nvidia, will be available asNvidia NIM microservices. The models enable developers to incorporate generative AI copilots and agents into USD workflows, broadening the possibilities in 3D worlds and helping speed the adoption of USD across a new range ofindustrial sectors, like manufacturing, automotive and robotics.

The microservices available in preview are:

USD CodeNIM— answers general knowledge OpenUSD questions and automatically generates OpenUSD-Python code based on text prompts that can then be inputted into an OpenUSD viewing app, such as usdview from Pixar, or an Nvidia Omniverse Kit-based application to visualise the corresponding 3D data.

USD Search NIM— enables developers to search through massive libraries of OpenUSD, 3D and image data using natural language or image inputs.

USD Validate NIM— checks the compatibility of uploaded files against OpenUSD release versions and generates a fully RTX-rendered, path-traced image, powered byNvidia Omniverse Cloud APIs, or application programming interfaces.

Newly announced NIM microservices that will be available soon are:

USD Layout NIM— enables users to assemble OpenUSD-based scenes from a series of text prompts based on spatial intelligence.

USD SmartMaterial NIM— predicts and applies a realistic material to a computer-aided design object.

fVDB Mesh Generation NIM— generates an OpenUSD-based mesh, rendered by Omniverse Cloud APIs, from point-cloud data.

fVDB Physics Super-Res NIM— performs AI SuperResolution on a frame or sequence of frames to generate an OpenUSD-based, high-resolution physics simulation.

fVDB NeRF-XL NIM— generates large-scale neural radiance fields in OpenUSD using Omniverse Cloud APIs.

Foxconn, a global manufacturing leader with more than 170 factories worldwide, is already benefiting from Nvidia's computing platform, using NIM microservices and Omniverse to create adigital twinof a factory under development.

“Digital twins will help us accelerate the next wave of industrial manufacturing and autonomous machines,” said Foxconn chief digital officer and smart manufacturing platform head Dr Zhe Shi.“Nvidia Omniverse and the new NIM microservices will democratise the ability to develop digital twins and help our teams build physically based virtual factories faster than ever.”

WPP, world leader in marketing and communications services company, is an early adopter of USD Search and USD Code NIM microservices, implementing them in its generative AI-enabled content creation pipeline, built on Nvidia Omniverse, for customers such as The Coca-Cola Company.

“The beauty of the innovation is how compatible it is with the way we work, and that it leverages open standards — accelerating not only future work, but allowing us to continue to build on and extend the usefulness of all our previous investments in standards like OpenUSD,” said WPP chief technology officer Stephan Pretourius. “Using Nvidia NIM microservices with Nvidia Omniverse has made it possible for us to launch innovative new production tools with companies like The Coca-Cola Company at unprecedented speed.”

USD connectors bring generative AI to more industries
A series of new USD connectors for robotics data formats and streaming to Apple Vision Pro opens the portals of OpenUSD interoperability and advanced authoring to more industries.

Nvidia andSiemens, a global leader in industrial automation and software, are extending theircollaborationto facilitate more industrial workloads using OpenUSD. Siemens will integrate OpenUSD pipelines with its Simcenter portfolio of simulation technologies to support evidence-based decision-making and collaboration among key stakeholders.

This integration enables high-fidelity, real-time, photorealistic visualisation of complex simulation data, providing deeper insights into a product’s performance within its real-world operating environment. The work will build on Siemens’ efforts to incorporate Omniverse into its Teamcenter Product Lifecycle Management portfolio.

Nvidia also released a connector from Unified Robotics Description Format to OpenUSD, letting roboticists seamlessly bring their robot data across applications, including for design, simulation andreinforcement learning.

To further advanceOpenUSD ecosystemexpansion, Nvidia announced the OpenUSD Exchange software development kit, enabling developers to build their own robust OpenUSD data connectors.

New developer tools and APIs to stream large-scale OpenUSD scenes from an application built on the Omniverse platformtoAppleVision Provia theNvidia Graphics Delivery Networkare now available inearly access.

“OpenUSD is revolutionising the way we create and interact with 3D content,” said Pixar chief technology officer and OpenUSD chairman alliance Steve May. “Now, with these new services and APIs for OpenUSD built by Nvidia, we expect to see accelerated growth and adoption of USD, paving the way for new users and industries to more easily engage with our ecosystem.”

OpenUSD Ecosystem Momentum
Last year, Nvidiacofounded the AOUSDalong with Pixar, Adobe, Apple and Autodesk. Through the AOUSD, Nvidia and other collaborators haveannounceda new OpenUSD release, progress on an OpenUSDcore specificationand new members.

Availability
USD Search, USD Code and USD Validate NIM are available in preview on theNvidia API catalog. The OpenUSD to URDF connector is now available withNvidia Isaac Sim.

Developers can get startedintegrating generative AI into OpenUSD workflows with new Omniverse developer toolsand a reference workflow forbuilding a generative AI-enabled synthetic data pipeline with OpenUSD.

Learn more about the latest in generative AI and accelerated computing bytuning in to Nvidia founder and CEO Jensen Huang’s fireside chatsat Siggraph, the premier computer graphics conference, running through 1 August in Denver.

About Nvidia
Nvidia is the world leader in accelerated computing.