LATEST
AI Tools

NVIDIA Nemotron-Labs-Sigma: The Free Model That Writes in Parallel

Updated November 20, 2026
12 min read
Editor's Note: This article was human-reviewed for accuracy
Abstract teal and black waves flowing across a dark background, evoking parallel streams of text

Quick Answer & Key Takeaways

  • • NVIDIA's Nemotron-Labs-Sigma is a free, open-weights language model that writes text in parallel blocks instead of one token at a time.
  • • It ships in 3B and 8B sizes, runs on your own NVIDIA GPU, and the paper behind it claims about 3.5 times the throughput of standard decoding.
  • • There is no hosted API and no independent verification of those numbers yet.

Every language model you have used writes the same way: one token, then the next, then the next. NVIDIA's Nemotron-Labs-Sigma breaks that habit. It writes whole blocks of text at once, cleaning up noise until words emerge, and NVIDIA's paper says the trick delivers about 3.5 times the throughput of standard decoding. The model is free, its weights are open, and it runs on your own NVIDIA graphics card. There is no hosted API, no launch blog post, and no independent test of the speed claims yet. This article covers what Sigma is, what the paper actually measured, what it costs to run, and where it falls short.

What Nemotron-Labs-Sigma actually is

Nemotron-Labs-Sigma is a continuous diffusion language model, released in 3B and 8B sizes. Instead of predicting the next token, it starts from Gaussian noise in a 16-dimensional continuous token-embedding space and denoises it, block by block, until text appears. The denoiser network and the token embeddings are learned jointly, and the model is trained blockwise through likelihood optimization. That is the design described in the paper Large Language Continuous Diffusion Models (arXiv:2610.02665), written by researchers at NVIDIA, Cornell University and Georgia Tech, submitted on October 2, 2026 and revised on November 12, 2026.

The practical difference is parallelism. A normal model commits to one token before it can produce the next. Sigma refines a whole block of tokens together, using DDPM or DDIM samplers with self-conditioning, while tokens it has already committed reuse a standard key-value cache. Fewer denoising steps per block means faster output at some cost to quality, and the smooth continuous representation is what makes few-step distillation possible. The model card puts it plainly: sampling quality holds up well down to few denoising steps per block, and the default block is 64 tokens, configurable down to 32.

Sigma is not a chatbot out of the box. The 8B model on Hugging Face is a base, pretrained model, intended as a starting point for research on parallel decoding, fine-tuning and distillation, not as an instruction-following assistant. For chat and instruction following, NVIDIA points users to the instruct variant. The weights landed on Hugging Face on November 12, 2026, with no accompanying NVIDIA blog post and no listing on the company's build.nvidia.com service, according to an independent write-up that examined the release. The weights were first noticed publicly in the r/LocalLLaMA community on November 16, 2026.

The four variants, and which one to download

The release is a small family, not a single checkpoint. The base model and the instruct model are the two main downloads, and distilled versions trade a little accuracy for much faster generation at a fixed step count. The table below uses each model card's own specifications.

VariantSizeContextBest for
Nemotron-Labs-Sigma-8B (base)8B4,096 tokensResearch, fine-tuning, distillation
Nemotron-Labs-Sigma-8B-instruct8B8,192 tokensChat and instruction following
Nemotron-Labs-Sigma-3B3BSee model cardSmaller GPUs and quick tests
Nemotron-Labs-Sigma-distill-8B-nfe48B4,096 tokensFast generation at a fixed 4 denoising steps

Two details matter before you download anything. First, the distilled variants are trained for one fixed step count, and that count is baked in: the NFE 4 variant runs at 4 denoising steps only, and the bundled inference code does not expose a step override. Second, the base model works best with few-shot or benchmark-style prompting, while the instruct model is the one to use if you want it to answer questions. The model card is explicit that the base model is not an assistant, so anyone who loads it expecting a chat interface will be disappointed.

What the benchmarks say

Every number in this section comes from NVIDIA's own paper and model cards, and none of it has independent verification yet. That is the honest frame: these are the authors' measurements on their own hardware, reported in their own paper. Treat them as a claim about what the architecture can do, not as a settled result.

The headline claim is speed. Measured on a single NVIDIA GB200 using SGLang with one request at a time, the 8B model averaged 3.46 times the throughput of its own autoregressive decoding across GSM8K, HumanEval and MBPP, compared with 3.04 times for NVIDIA's earlier masked-diffusion model, Nemotron-Labs-Diffusion. The paper's abstract adds that one-step speculative decoding on the distilled 8B model yields a further gain of about 3.5 times over the autoregressive counterpart on the same GB200 hardware. The distilled models also claim roughly 8 times acceleration on math tasks and 4 times on coding, producing 8 and 4 tokens per forward pass respectively, with negligible accuracy loss.

On quality, the regularised Sigma-8B averaged 71.16% across six math and coding benchmarks after pretraining, against 70.22% for the matched-size Nemotron-Labs-Diffusion-8B. The distilled 3B model running just 8 denoising steps scored 84.62% on GSM8K, against 85.57% for its full teacher run at 256 steps. The published six-task means for the distilled 8B variant at 2 denoising steps are lower, as expected: GSM8K 80.17%, MATH 55.72%, HumanEval 48.17% and MBPP 37.52%, for a six-task mean of 53.02%. The trade is visible in the numbers. Fewer steps buy speed and cost quality, and the paper's own tables show how steep that slope is.

The model card adds a concrete throughput figure of its own: roughly 120 to 140 tokens per second for 256 generated tokens with the DDPM sampler at 32 denoising steps per 64-token block, on a single NVIDIA Hopper H100 in bfloat16. That is a useful anchor, but note the conditions. Single-request serving on a GB200 or H100 is not the same as a busy production cluster handling many users at once, and the paper's comparisons are against the model's own autoregressive baseline, not against the fastest hosted models.

What it costs

The model itself costs nothing. The weights are free to download under the NVIDIA Nemotron Open Model License, which the model card states permits commercial and non-commercial use, and NVIDIA hosts no service, so there is no per-token bill, no subscription and no API key to buy. That is the whole pricing story for the model. The real cost is the hardware it runs on.

What you pay forUSDGBP (approx.)CAD (approx.)Notes
Nemotron-Labs-Sigma weights (3B or 8B)$0£0CA$0Open weights, NVIDIA Nemotron Open Model License
Hosted API or cloud service$0£0CA$0None exists; NVIDIA hosts no service
Your GPU and electricityVariesVariesVaries8B in bfloat16 needs about 16 GB for the weights alone

The hardware bar is real but reachable. The model needs a CUDA-capable NVIDIA GPU running in bfloat16, with PyTorch 2.4 or newer and Transformers 4.55 or newer, on Linux. Supported architectures are Ampere, Blackwell, Hopper and Lovelace. CPU inference is not supported: the model card states the diffusion sampler is numerically degraded on CPU and produces unusable output. The 8B weights take about 16 GB in bfloat16 by simple arithmetic, 8 billion parameters at 2 bytes each, so a 24 GB consumer card such as an RTX 4090 has room for the weights plus the key-value cache, while the 3B version should fit on mid-range cards. Those VRAM figures are estimates based on the published parameter counts, not NVIDIA's own published requirements, because NVIDIA has published no official hardware requirements beyond the GPU and precision.

For comparison, our Recursive Flint review covers a free 40B coding model built to run on a laptop, and our Nemotron 3.5 Transcribe review covers NVIDIA's earlier free Nemotron release, a transcription model that names every speaker.

How to run it

Running Sigma is a download, not a sign-up. The model loads through the standard Transformers library with trust_remote_code, and no custom kernels, flash-attention or training framework are needed for inference. The model card's own example is short:

from transformers import AutoModel, AutoTokenizer
repo = "nvidia/Nemotron-Labs-Sigma-8B"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(repo, trust_remote_code=True)
model = model.cuda().to(torch.bfloat16).eval()

Generation takes a sampler, a time schedule and a step count. The card recommends the DDIM sampler on the log_snr grid with 32-token blocks, and warns that the DDPM sampler degenerates at low step counts. Below roughly 16 steps the pretrained models lose quality noticeably, so for few-step generation the card points you to the distilled variants, which need neither guidance nor score-temperature tuning.

Denoising stepsClassifier-free guidanceScore temperature
DDIM-3230.3
DDIM-1630.4
DDIM-830.6
DDIM-420.8

These settings apply to both model sizes, base and instruct, according to the model card. The two tuning knobs worth understanding are classifier-free guidance and score temperature, which the paper identifies as essential for high-fidelity reasoning and coding. Lower step counts need stronger score temperature to stay stable, which is why the table raises it as the steps fall.

Sigma vs the alternatives

Sigma enters a crowded class of small, openly licensed models, and its pitch is different from all of them. Most open models compete on benchmark scores per dollar. Sigma competes on how the text is produced. The table compares it with the closest options, using each model's own published specifications.

ModelHow it writesPriceAPI access
Nemotron-Labs-Sigma 8BContinuous diffusion, blocks in parallel$0, open weightsNone; self-host only
Nemotron-Labs-Diffusion 8BMasked (discrete) diffusion$0, open weightsNone; self-host only
Standard open 8B models (for example Qwen3-8B)Autoregressive, one token at a time$0, open weightsVia third-party hosts
Hosted frontier modelsAutoregressivePer-token API ratesYes, managed API

The comparison that matters most is inside NVIDIA's own family. Sigma is the successor idea to Nemotron-Labs-Diffusion: continuous embeddings instead of discrete masked tokens, which the paper says degrades more gracefully at low step counts and allows embedding-space steering to control the quality-diversity trade-off. The paper reports Sigma ahead of the masked model on standard benchmarks after pretraining, with the largest consistent gains on coding, and ahead on throughput as well, 3.46 times against 3.04 times over the same autoregressive baseline. Against ordinary autoregressive open models, the trade is different. Those models have years of tooling, quantised GGUF builds, Ollama and LM Studio support, and large communities. Sigma has none of that yet: no quantised builds are known, and the model card's own runtime notes assume plain PyTorch with trust_remote_code. Our DeepInfer explainer covers the other end of the same problem, making standard models cheaper to serve, while our Inkling 2 Small review covers a much larger open model with a published per-token price.

What it cannot do yet

The limits are as informative as the benchmarks, and most of them are stated in NVIDIA's own documentation. The model is English only, with a 4,096-token context on the base model and 8,192 on the instruct model, which is small next to the 32K or 128K windows that are now routine. Reinforcement-learning post-training, a standard step in making today's leading models useful, is left as future work in the paper. There are no published safety evaluations, and the training data is not distributed: the card describes an internal NVIDIA corpus of about 1.3 trillion tokens, roughly 1 trillion of autoregressive initialisation followed by 0.3 trillion of diffusion training, and notes it is the same corpus used for Nemotron-Labs-Diffusion.

The release itself is unusually quiet. One independent write-up counted five downloads of the distilled 8B variant in 30 days and zero likes as of November 17, 2026, and found no blog post, no build.nvidia.com listing and no launch thread. That is a third-party snapshot, not NVIDIA's own reporting, but it matches the public record: the paper and the model cards are the announcement. There is also no hosted endpoint to try before downloading, so the only way to test Sigma is to install it and run it yourself. Finally, the paper's own caveats apply. The throughput figures were measured under a single-request serving setup, and the authors describe Sigma as the first family of large-scale continuous diffusion language models, which means the architecture has no track record at scale.

Common mistakes

First, do not load the base model and expect a chatbot. It is a pretrained model, not an assistant, and it works best with few-shot or benchmark-style prompting. Use the instruct variant for conversation. Second, do not run it on a CPU. The sampler is numerically degraded there and the output is unusable, so a CUDA GPU is a requirement, not a recommendation. Third, do not use the DDPM sampler at low step counts. The card recommends DDIM for few-step generation, and the distilled variants for anything below roughly 16 steps. Fourth, do not treat the distilled variants as adjustable. Each one is trained for a single fixed step count, and the bundled code does not let you override it. Fifth, do not quote the 3.5 times throughput figure as a general speed claim. It is a single-request measurement on a GB200 against the model's own autoregressive baseline, and independent tests do not exist yet.

Frequently asked questions

Is Nemotron-Labs-Sigma free?

Yes. The weights are free to download under the NVIDIA Nemotron Open Model License, which permits commercial and non-commercial use, and NVIDIA hosts no paid service. The only cost is the GPU you run it on, plus electricity.

What hardware do I need to run Sigma 8B?

A CUDA-capable NVIDIA GPU in bfloat16, with PyTorch 2.4 or newer and Transformers 4.55 or newer, on Linux. The 8B weights take about 16 GB, so a 24 GB consumer card has room for the weights and the cache. CPU inference is not supported.

How is Sigma different from a normal language model?

A normal model writes one token at a time. Sigma denoises whole blocks of tokens in parallel, starting from noise in a continuous embedding space. NVIDIA's paper claims about 3.5 times the throughput of standard decoding on its own hardware, with quality close to the teacher model at a fraction of the steps.

Can I use Sigma for chat?

Use the instruct variant, which is fine-tuned for instruction following and has an 8,192-token context. The base model is a research checkpoint, not an assistant, and works best with few-shot or benchmark-style prompts.

Is there an API for Sigma?

No. NVIDIA hosts no service for this model, and it does not appear on build.nvidia.com. The only way to use it is to download the weights from Hugging Face and run them on your own hardware.

Should I replace my current model with Sigma?

Not for production work. Sigma is a research release with no independent benchmarks, no reinforcement-learning post-training, English-only output and a 4,096-token base context. It is worth trying if you care about diffusion decoding or cheap local inference, and worth watching if independent tests confirm the speed claims.

Nemotron-Labs-Sigma is a genuine new idea with an honest presentation: a model that writes in parallel, free to download, with its limits printed on its own model card. The speed claims are NVIDIA's own, measured on NVIDIA's hardware, and they have not been tested by anyone else. If independent tests confirm even part of the 3.5 times figure, diffusion decoding becomes a real option for cheaper local inference. Until then, Sigma is best read as a research release to try, not a model to build on. Download the instruct variant, run it on a GPU you already own, and judge the output yourself.

Sources

12 min read
30 views
edit_note Written by

Ratul Ahamed

Founder & Editor-in-Chief

Ratul Ahamed is the founder and editor of AI Core Updates, an independent site covering AI tools, models and pricing. He is a web developer based in Bangladesh and also runs All In One Tools, a free collection of browser-based utilities. Contact: [email protected]

Read all stories by Ratul Ahamed arrow_forward
verified_user Human-Reviewed Editorial

forum Comments (0)

Leave a Reply

chat_bubble_outline

No comments yet. Be the first to join the conversation!

Weekly AI Digest

Get our weekly AI tools digest

One email a week analyzing new AI tools, model updates, and practical workflows. Tested and summarized with zero fluff. Unsubscribe any time.

bolt
Hands-on AI Tool Reviews
Pricing math, limits, and alternatives.
verified_user
Strict Anti-Spam Policy
1-click unsubscribe at any time.
mark_email_unread

Subscribe to Digest

Sent weekly • No spam

person_outline
mail_outline

No spam ever. You can unsubscribe with a single click at any time.

cookie

Cookie Settings

We use cookies to enhance your browsing experience, serve personalized content, and analyze our traffic. By clicking "Accept All", you consent to our use of cookies. Learn more