NVIDIA Cosmos Expands: 5 Blueprints to Train Robots on Synthetic Data

Robots learn the way people do: by watching things happen. The problem is that the interesting things — a forklift tipping, a spill spreading across a factory floor, a car cutting across three lanes — rarely happen on camera, and staging them is expensive or dangerous. NVIDIA's answer, announced October 21 at GTC Berlin and detailed in an October 22 company blog post, is to generate the training data instead: five new Cosmos Data Factory blueprints that turn ordinary footage and still images into synthetic training sets for robots, self-driving cars, and industrial inspection systems.

The expansion covers the whole pipeline, not just data. Alongside the blueprints, NVIDIA released four new Cosmos 3 Nano simulation checkpoints for testing robot and vehicle policies in repeatable virtual scenarios, extended its Cosmos Framework training tools as agent skills to more open models, and updated its Cosmos NIM microservices, which the company says deliver up to twice the inference speed in validated configurations. The whole stack is open and free to use — the bill arrives as GPU time, not a software license.

What Cosmos is, in one paragraph

Cosmos is NVIDIA's platform for "world models": AI systems that understand and simulate the physical world so robots and autonomous vehicles can predict what happens next instead of just reacting to what a camera sees. The current generation, Cosmos 3, launched May 31, 2026, as an omnimodal foundation model — it takes in text, images, video, audio, and robot-action data, and can generate all of them back. It comes in two sizes: Nano, at 16 billion parameters (an 8-billion-parameter Reasoner paired with an 8-billion-parameter Generator), and Super, at 64 billion parameters. Both are distributed under the OpenMDW 1.1 license, which permits commercial use with attribution — products built on Cosmos must carry a "Built on NVIDIA Cosmos" credit. Unlike many "open weight" releases that quietly restrict commercial deployment, Cosmos is usable in a product from day one, provided you read the attribution and use terms first.

NVIDIA says the Cosmos platform ranks in the top five on six open-model leaderboards. That is the company's own framing, not an independent ranking — it is a claim about standing within a category NVIDIA helped define, so treat it as marketing context rather than a benchmark result.

The five blueprints: synthetic data for the cases cameras never catch

Cosmos Data Factory is the data-preparation half of the platform: tools and agent skills for labeling, curating, augmenting, and evaluating training data at scale. The October expansion adds five cloud-native blueprints, available through GitHub, Microsoft Azure, Nebius, and CoreWeave:

• Autonomous Vehicle Dataset Generation: curates and labels driving data, then adds synthetic examples of hard-to-capture scenarios — the near-misses and edge cases that matter most for safety and appear least in real logs.
• Image Attribute Augmentation: varies how objects look to improve image search and recognition training.
• Event Video Generation: turns still images into synthetic videos of rare or unsafe events — a spill, a fall, a hazard — without staging the real thing.
• Defect Image Generation: generates printed circuit board, metal, and glass defect images for inspection models, where real defective samples are scarce by definition.
• Video Data Augmentation: changes weather and lighting in existing footage while preserving scene structure, so a model trained on sunny-day video still works in rain.

Alongside the blueprints, Cosmos Curator — NVIDIA's GPU-accelerated tool for curating video training datasets — now handles multimodal, multisensory inputs and adds new processing blocks. In practice that means a dataset can mix video, audio, and sensor streams in one curation pass instead of three separate ones.

The customer examples NVIDIA cites are specific enough to check. Porsche is using Cosmos Data Factory through Voxel51 on Microsoft Azure to build a shared autonomous-vehicle data flywheel across Volkswagen Group brands; NVIDIA says the approach is expected to cut repeated physical test drives by up to 90% and shorten development cycles by up to 85% — note "expected," these are projections, not measured results. ANYbotics generated more than 1,500 synthetic video clips with the blueprints, validating outputs through Encord's platform, to train industrial-inspection robots to recognize liquid spills and changes between facility visits. Ultralytics retrained its YOLO model with real images plus targeted synthetic defects from Cosmos Data Factory, and reports improved defect localization and recall — the improvement is Ultralytics' own report, not an independent measurement.

Training and adapting: the framework is now agent skills

With data prepared, developers can train their own architecture or start from a pretrained model. At GTC Berlin, NVIDIA extended Cosmos Framework — its open, modular set of model training and optimization tools — as agent skills to more open models, alongside Cosmos 3 itself. Developers and agents can now adapt models including GR00T, Alpamayo, QWEN, Pi0.5, and other compatible Hugging Face models through fine-tuning and reinforcement learning, then optimize them with distillation and quantization. On the inference side, HD Diffusers, vLLM and vLLM-Omni, and SGLang all work now, and the whole flow can be wired through Cosmos Framework skills instead of done manually.

The worked example NVIDIA gives is a distillation story: robotics research lab MicroAGI post-trained Cosmos 3 on videos of humans manipulating objects, then distilled its 32-billion-parameter reasoner down to an 8-billion-parameter Nano model with, in NVIDIA's account, no loss in accuracy for its object-manipulation task. "No loss" is the company's summary of MicroAGI's result on one task, not a general guarantee — distillation results vary sharply by task, and NVIDIA publishes no broader comparison here. Developers can also label data and post-train Cosmos 3 on Roboflow for their specific environments, then deploy to Roboflow-hosted or on-premises NVIDIA GPUs to bring video reasoning into factories and warehouses.

Simulation checkpoints and faster inference

The third piece of the expansion is testing. NVIDIA released four new Cosmos 3 Nano (16B) example checkpoints for multiview, multisensory data augmentation and simulation, with recipes and skills included, so developers can adapt the models to train and evaluate their own policies for autonomous vehicles and bimanual robots in interactive simulations before real-world testing. The point is repeatability: a policy that must handle a rare scenario can be run against the same synthetic scenario hundreds of times, which real-world testing cannot offer.

For deployment, updated NVIDIA Cosmos 3 NIM microservices deliver up to 2x faster inference with no quality loss in validated configurations, according to NVIDIA, while continuous video analysis enables sub-second alerts. The "validated configurations" qualifier matters: the 2x figure applies to the setups NVIDIA tested, not every deployment. For vision AI developers, the new Metropolis Blueprint for video search and summarization (VSS) 3.3, built on Cosmos NIM microservices, turns one prompt into a video AI application and helps teams monitor 40% more video streams with the same GPU, per NVIDIA. ADNOC is already using Fogsphere's MistIQ agentic platform, built with Cosmos and the Metropolis VSS Blueprint, to detect safety hazards in real time across 65 sites — and for rare events, Fogsphere uses Cosmos Data Factory to generate synthetic training data instead of physically staging dangerous situations.

What it actually costs

Here is the pricing story, and it is the part most coverage skips: there is no price list for Cosmos, because there is nothing to buy. The models are open under OpenMDW 1.1, the framework is open on GitHub, and the blueprints are cloud-native workflows — NVIDIA does not sell Cosmos as a hosted, pay-per-token API, so it publishes no per-token rate. The cost is the compute underneath.

That compute has three main shapes. First, the blueprints run on partner clouds — Microsoft Azure, Nebius, and CoreWeave — and you pay those providers' GPU rates for the time your data jobs consume. Second, self-hosting: Cosmos 3 Nano is sized for a single workstation-class GPU, and NVIDIA's own examples run on an RTX PRO 6000, while the Super model needs Hopper or Blackwell GPU clusters. Third, local development hardware: NVIDIA's DGX Spark, the 64GB desktop AI system that went on sale October 23 for $4,999, is the machine NVIDIA positions for running models like these locally — and up to eight of them can be clustered for 1TB of unified memory, per NVIDIA. None of those three costs appears on any Cosmos page, because they are not Cosmos prices. Budget for the GPUs, not the software.

One more cost to name honestly: synthetic data is not free of validation work. Every generated clip still needs checking — ANYbotics validated its outputs through Encord, and NVIDIA's own examples lean on validation platforms for a reason. A team that generates 10,000 defect images and trains on them unchecked has bought a faster way to learn the wrong thing. The blueprints save the cost of staging rare events; they do not remove the cost of verifying the synthetic data teaches what you think it teaches.

Pros and cons

Pros: genuinely open stack — models, framework, blueprints, and recipes under permissive terms with commercial use allowed; covers the full chain from data generation through training, simulation, and deployment; simulation checkpoints make rare-scenario testing repeatable; runs on partner clouds or your own hardware, with no vendor API lock-in.

Cons: no hosted API means no pay-as-you-go option — you provision GPUs yourself or pay a cloud provider; the headline numbers (90% fewer test drives, 2x faster inference, 40% more streams) are NVIDIA's own projections or validated-configuration figures, not independent measurements; synthetic data still needs validation, which is real work the blueprints do not do for you; the 16B Nano model still needs workstation-class hardware, and the 64B Super needs a cluster.

How the pieces fit together

FAQ

Is NVIDIA Cosmos free? The software is: the Cosmos 3 models, the Cosmos Framework, and the Data Factory blueprints are free to use, with the models under the OpenMDW 1.1 license permitting commercial use with "Built on NVIDIA Cosmos" attribution. What is not free is the compute — cloud GPU time on Azure, Nebius, or CoreWeave, or your own workstation or cluster.

Can I use Cosmos 3 commercially? Yes, under the OpenMDW 1.1 license, which permits commercial and non-commercial use with attribution. The license is not Apache 2.0, so read the attribution and use terms before shipping a product — the difference matters for a startup, not for a hobby project.

What hardware do I need to run Cosmos 3 Nano? The 16B Nano model is sized for workstation-class GPUs — NVIDIA's examples run on a single RTX PRO 6000. The 64B Super model needs Hopper or Blackwell GPU clusters. For local development, NVIDIA positions the $4,999 DGX Spark as the entry point, with clusters of up to eight units for larger workloads.

How is Cosmos different from a video generator like Sora or Veo? Video generators produce footage for people to watch. Cosmos is a world model: it predicts how a physical scene evolves, including robot actions and sensor inputs, so a robot or vehicle can plan and test behavior in simulation before acting in the real world. The generated video is a byproduct of the simulation, not the product.

Do I still need real training data if I use the blueprints? Yes. The blueprints augment and extend real data — Ultralytics retrained YOLO with real images plus synthetic defects, and ANYbotics validated synthetic clips against real-world data. Synthetic data covers the cases cameras rarely catch; it does not replace the ground truth that teaches the model what real looks like.

Key takeaways

• NVIDIA's October 21 Cosmos expansion adds five Data Factory blueprints, four Cosmos 3 Nano simulation checkpoints, agent-skill training tools, and faster NIM inference — one open stack from data to deployment.
• The blueprints generate synthetic training data for the cases real cameras rarely catch: rare driving scenarios, unsafe events, and scarce defects in circuit boards, metal, and glass.
• Everything is free to use under the OpenMDW 1.1 license, which permits commercial use with attribution — the real cost is GPU time on Azure, Nebius, CoreWeave, or your own hardware.
• The headline figures are NVIDIA's own: 90% fewer test drives and 85% shorter development cycles are projections, and 2x faster NIM inference applies to validated configurations, not every setup.
• Synthetic data still needs validation — the blueprints remove the cost of staging rare events, not the work of checking that the generated data teaches the right thing.

Physical AI has spent years bottlenecked on data: the most important training examples are the ones that almost never happen. Cosmos's October expansion does not solve that problem outright, but it makes the workaround — generating the missing data — a standard, open, and free part of the toolchain. As of October 25, 2026, the question is no longer whether synthetic data is available to robotics teams. It is whether they validate it well enough to trust.