⏱️ Lectura: 11 min
NVIDIA just put into production a 30-billion-parameter model that promises to complete agentic tasks up to 30% faster than its rivals in the same category. It’s called Nemotron 3.5 Lightning, and it arrives alongside NeMo Switchyard, an open source library that decides, step by step, which AI model should handle each task an agent performs.
📑 En este artículo
The launch targets a problem that’s already hitting any team running AI agents in production: using a single frontier model for everything drives up both cost and latency. NVIDIA proposes solving it with a specialized, lightweight model, plus a router that chooses between it, open models, and proprietary models depending on the task.
TL;DR
- NVIDIA launched Nemotron 3.5 Lightning, a 30-billion-parameter MoE model for high-volume AI agents.
- Lightning delivers up to 4x more output speed and 30% shorter completion times for agentic tasks compared to rivals in its class.
- NVIDIA also released NeMo Switchyard, an open source library that routes each step of an agent to the most suitable model.
- According to NVIDIA’s internal benchmarks, Switchyard maintains frontier-level accuracy while cutting cost to nearly a third of using Opus 4.8 alone.
- CrowdStrike, Harvey (with Trajectory), CodeRabbit (with Baseten), Lila Sciences, and Fastino Labs have already customized Lightning for their domains.
- Lightning runs on RTX PCs, DGX Spark, DGX Station, Jetson, RTX PRO workstations, data centers, and the cloud.
- NVIDIA also released Nemotron-RL-Agentic-Terminal-Pivot, an agentic reinforcement learning dataset for post-training coding capabilities.
- The launch follows Nemotron 3 Nano and builds on evaluation and inference contributions from the Nemotron Coalition.
Introduction
AI agents that run continuously, what NVIDIA calls always-on agents, no longer depend on a single model. Today they operate as model systems: a frontier reasoning model, such as Nemotron 3 Ultra or GPT-5.6, plans and orchestrates the workflow, while smaller, specialized models execute specific tasks like code review, tool use, security alert monitoring, or billing responses.
Nemotron 3.5 Lightning was built for that second role. NVIDIA describes it as the most efficient model in its category for long-running agentic workloads, designed for high volume and narrowly scoped tasks, not general reasoning.
What Happened
NVIDIA expanded its Nemotron 3 family with Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts (MoE) model, and direct successor to the Nemotron 3 Nano release. It’s fully customizable: any organization can post-train it with NVIDIA NeMo using its own data, tools, and workflows to improve accuracy on specific tasks.
Alongside Lightning, NVIDIA released NeMo Switchyard, an open source intelligent routing library for agent tooling. Switchyard automatically directs each request to the most capable and efficient model for that specific step, whether open, proprietary, or from NVIDIA, without the development team having to rewrite their application.
Lightning’s development benefited from contributions by the Nemotron Coalition, whose members contributed evaluation methodologies, inference software, and datasets. NVIDIA also published Nemotron-RL-Agentic-Terminal-Pivot, an agentic reinforcement learning dataset used to post-train the model’s coding agent capabilities.
Context and History
The Nemotron family didn’t start with Lightning. NVIDIA has been building it as its bet on open models against closed systems, publishing both the weights and, when licensing allows, the training data and techniques, something that enables traceability, auditing, and lets third parties train their own models based on that work.
This launch arrives at a moment when the industry has already accepted that no single model wins on every front: some models are better at code, others at reasoning, others are lightweight and built to run locally for privacy and efficiency. Defaulting to a single model means either overpaying or losing quality; manually routing between several models turns into integration work that slows down any deployment. Switchyard aims to take that decision out of the development team’s hands and automate it.
Technical Details and Performance
Nemotron 3.5 Lightning is a 30-billion-parameter mixture-of-experts (MoE) model. According to NVIDIA, it delivers up to 4 times more output speed, which translates into 30% faster completion of agentic tasks compared to other models in its class. PinchBench benchmarks, cited by NVIDIA, show Lightning completing agentic tasks faster than comparable rivals without losing frontier-level accuracy.
On the router side, NVIDIA reports that NeMo Switchyard maintains frontier-level accuracy while cutting the cost of completing a task to nearly a third of what it would cost to solve it using Opus 4.8 alone, according to NVIDIA’s internal benchmarks. That matters because an agent’s cost doesn’t depend only on the model’s per-token price: it depends on how many steps in the workflow end up going through the most expensive model when they didn’t need to.
| Model / Role | Function in the System | Where It Runs | When to Use It |
|---|---|---|---|
| Nemotron 3 Ultra / GPT-5.6 | Frontier orchestrator and planner | Data center / cloud | Plan the agent’s full workflow |
| Nemotron 3.5 Lightning | High-volume specialist (30B MoE) | RTX PC, DGX Spark, DGX Station, Jetson, RTX PRO, data center, cloud | Specific tasks: code review, tool use, alerts, billing |
| NeMo Switchyard | Router across open, proprietary, and NVIDIA models | Deployed alongside the agent app | When the agent combines several models and cost, latency, and quality need to be optimized |
Who’s Already Using It
Several teams have already customized Lightning for their own domains: CrowdStrike adapted it for cybersecurity, Harvey integrated it alongside Trajectory for legal services, and CodeRabbit tuned it with Baseten for code review. Lila Sciences is using it to improve reasoning in agentic tasks for physical and life sciences, and Fastino Labs reports the best accuracy it has achieved so far for software development, finance, and healthcare after customizing the model.
💭 Key point: Lightning doesn’t compete with frontier reasoning models. It’s built for volume: the repetitive task an agent executes thousands of times a day, where speed and cost per request matter more than planning capability.
How to Get Started
There are two practical paths to try Nemotron 3.5 Lightning today: consuming it through NVIDIA’s API catalog, or running it locally with a NIM container. The first is good for quick prototyping; the second for cases where latency or privacy require keeping inference inside your own infrastructure.
To try it via API without installing anything, all you need is a key from NVIDIA’s catalog and a call compatible with the OpenAI format:
curl https://integrate.api.nvidia.com/v1/chat/completions \
-H "Authorization: Bearer $NVIDIA_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "nvidia/nemotron-3.5-lightning",
"messages": [
{"role": "user", "content": "Summarize this support ticket in 3 bullet points."}
],
"max_tokens": 512
}'The model field in the response confirms which actual model handled the request; it’s the first check once you put Switchyard in the middle and want to verify it routed to the right place.
To run it locally (RTX PC, DGX Spark, or your own data center), the pattern is the same as with any NVIDIA NIM microservice: pull the container with your NGC API key and spin it up with GPU access. Windows (PowerShell with Docker Desktop), macOS, and Linux share the same command via Docker:
# Linux / macOS (bash) and Windows (PowerShell with Docker Desktop)
docker login nvcr.io --username '$oauthtoken' --password $NGC_API_KEY
docker run --rm --gpus all \
-e NGC_API_KEY=$NGC_API_KEY \
-p 8000:8000 \
nvcr.io/nim/nvidia/nemotron-3.5-lightning:latestWith the container up, the endpoint is exposed at http://localhost:8000/v1/chat/completions using the same format as the previous call, just changing the base URL.
💡 Tip: If your agent already calls a fixed model by name, you don’t need to rewrite that logic to add Switchyard: the router sits in front and decides which model to forward each request to, keeping the same interface.
Impact and Analysis
The central point of this launch isn’t Lightning on its own: it’s the combination of a model that’s cheap to run with a router that decides when to use it. That shifts the conversation from “which model do I choose” to “how do I build a system of models,” which is exactly the problem facing teams that already have agents running 24/7 in production.
The typical flow of a system with Switchyard in front of an agent looks like this:
flowchart TD
A["Agent / app"] --> B["NeMo Switchyard"]
B --> C["Nemotron 3.5 Lightning"]
B --> D["Frontier model (e.g. Opus 4.8)"]
B --> E["Other open or proprietary model"]
C --> F["High-volume task"]
D --> G["Task requiring deep reasoning"]
E --> H["Task with local privacy requirement"]An example Switchyard routing configuration, prioritizing cost unless the task is marked critical, could look like this:
routes:
- match:
task_type: "code_review"
target: "nvidia/nemotron-3.5-lightning"
priority: cost
- match:
task_type: "incident_triage"
severity: "critical"
target: "anthropic/opus-4.8"
priority: quality
- match:
task_type: "billing_faq"
target: "nvidia/nemotron-3.5-lightning"
priority: latencyThe logic is simple but the effect compounds: if 80% of an agent’s steps are low-risk repetitive tasks, pulling them off the most expensive model and sending them to a 30B model optimized for that reduces total cost without touching quality where it actually matters.
The other side of the coin is that a router adds one more piece to the system that can fail or make a bad call. If the routing rules are poorly calibrated, a task that needed deep reasoning can end up on the lightweight model and return a lower-quality answer without anyone noticing until the error has already propagated. That’s why NVIDIA insists the router is tuneable: each organization defines its own priorities for quality, latency, and cost instead of accepting generic routing.
⚠️ Heads up: Adding a router between the agent and the models isn’t free in terms of operational complexity: you have to monitor which model answered each request and audit when the router got the priority wrong, not just how much it saved.
What’s Next
NVIDIA says it’s working with ecosystem partners to bring intelligent routing to the tools and platforms developers already use. One of those partners, Boomi, evaluated Switchyard across five different routing capabilities as part of that early integration.
For teams already running their own agents, the logical next step is to evaluate how much of their current traffic to a frontier model could be resolved with a smaller specialist without losing measurable quality, and use that proportion to size the real savings before moving to production.
📖 Summary on Telegram: View summary
Try it yourself: request a free key from NVIDIA’s catalog and make your first call to Nemotron 3.5 Lightning with the curl command above in under five minutes.
Frequently Asked Questions
What is Nemotron 3.5 Lightning?
It’s a 30-billion-parameter mixture-of-experts (MoE) model from NVIDIA, designed for high-volume agentic tasks within continuously running AI agent systems.
What is NeMo Switchyard?
It’s an open source library from NVIDIA that automatically routes each agent step to the most suitable model, whether open, proprietary, or from NVIDIA, based on quality, latency, or cost priorities defined by each organization.
Is Nemotron 3.5 Lightning an open model?
Yes. It’s fully customizable and can be post-trained with NVIDIA NeMo using each organization’s own data, tools, and workflows.
Where can I run Lightning?
It runs on NVIDIA RTX PCs, DGX Spark, DGX Station, Jetson, RTX PRO workstations, data centers, and the cloud, depending on the deployment’s privacy, latency, and scale needs.
How much does using Switchyard save compared to a single model?
According to NVIDIA’s internal benchmarks, Switchyard maintains frontier-level accuracy while cutting the cost of completing a task to nearly a third of what it would cost to solve it with Opus 4.8 alone.
Which companies have already customized Lightning?
CrowdStrike, Harvey (with Trajectory), CodeRabbit (with Baseten), Lila Sciences, and Fastino Labs have already adapted it for their own domains: cybersecurity, legal services, code review, physical and life sciences, and software development, finance, and healthcare respectively.
References
- NVIDIA Blog: official announcement of Nemotron 3.5 Lightning and NeMo Switchyard.
- GitHub: NVIDIA/NeMo: repository for the framework used to post-train Nemotron models.
- NVIDIA API Catalog: catalog where Nemotron models can be tried via API.
- Wikipedia: Mixture of experts: explanation of the MoE architecture Lightning uses.
📱 Enjoying this content? Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day.
Imagen destacada: Foto de BoliviaInteligente en Unsplash
0 Comments