LATEST
AI Tools

Gemini Tool Retrieval: Fewer Tokens for Agents With 100+ Tools

Updated November 20, 2026
15 min read
Editor's Note: This article was human-reviewed for accuracy
Close-up of colorful programming code on a dark computer screen

Quick Answer & Key Takeaways

  • • Google launched Tool Retrieval in preview in the Gemini Interactions API on November 19, 2026.
  • • It keeps an agent's full tool library on standby and loads each tool's parameter schema only when the task needs it, so you stop paying input tokens for tools your agent never calls.
  • • The feature itself has no separate charge: you pay the standard per-token rate of the Gemini model you call.

Every tool you hand an AI agent costs tokens, whether the agent uses it or not. Connect fifty tools and the model reads fifty full parameter schemas on every single request, before it does any real work. That tax is the problem Google set out to fix on November 19, 2026, with Tool Retrieval, a preview feature in the Gemini Interactions API that keeps your whole tool library on standby and loads each tool's schema only when the task actually needs it.

The idea is simple enough that it is surprising it took this long. Your agent still knows every tool exists. It just stops paying to read the fine print of tools it never calls. For a support agent with dozens of tools, or a coding agent wired into an MCP server with hundreds, the token saving is real money. The catch is that Google has published no measured savings figures of its own, so the math below is our own, built from Google's published per-token rates and stated assumptions.

What Tool Retrieval changes

In normal function calling, you declare every tool up front, with its full parameter schema, and that whole block rides along in the input of every request. Google names three costs of doing it that way: inflated input token counts, added latency, and lower tool-selection accuracy as the candidate list grows. The last one is easy to miss. When a model picks from a hundred tools at once, it picks the wrong one more often. A shorter list of live options is a more accurate list.

Tool Retrieval changes the default. Instead of shipping every schema on every call, the system retrieves only the schemas the model asks for, when it asks for them. The model still sees the name and description of every deferred tool, so it knows what is available. Only the detailed parameter schemas stay out of context until they are needed. There is no index to build and no separate service to host. Gemini runs the search on the server, according to Google's announcement, so the only code changes are a new tool and a flag on the tools you want to hold back.

The feature lives in the Interactions API, Google's newer stateful interface for building agents. That API reached general availability in June 2026, according to TalkAndroid's reporting on the launch, and Google has been shipping new agent features there first ever since. It is not available on the older generateContent path, which matters if your code still uses the classic request shape.

How it works, step by step

Turning it on takes two changes to your request. First, add a tool_search tool. Second, set defer_loading: true on the tools you want to hold back. Google's own launch example uses a customer-service agent and the Gemini 4 Hydrogen model, and it shows the shape clearly:

from google import genai
client = genai.Client()

tools = [
    {"type": "tool_search"},
    {
        "type": "function",
        "name": "issue_refund",
        "description": "Refund an order to the original payment method.",
        "defer_loading": True,
        "parameters": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "amount": {"type": "number"},
            },
            "required": ["order_id", "amount"],
        },
    },
    # ... 50+ other deferred functions
    # (leave frequent tools undeferred)
]

interaction = client.interactions.create(
    model="gemini-4-hydrogen",
    input="Order A-1042 arrived damaged. Please refund 40 dollars.",
    tools=tools,
)

Inside that single API call, three steps happen. The model searches for issue_refund, the API loads its schema, and the model emits the function call. All three steps appear in interaction.steps as tool_search_call, tool_search_result and function_call, so you can inspect which tools the model considered when a call goes wrong. From there it is normal function calling: your code runs the refund and sends back a function_result.

Google's advice is to keep your most-used tools loaded up front and defer the long tail, the tools that are useful once in a while but not on most turns. That is the right default. A tool your agent calls on nine out of ten turns saves nothing by being deferred, and it costs a search step every time.

Server-side or client-side: two ways to search

Tool Retrieval comes in two modes, and the choice depends on how much control you want over tool discovery.

Server-side retrieval is the default described above. You declare your functions, flag the ones to defer, and Gemini runs the search itself. Nothing else to build. This is the mode most teams should start with, because it adds no infrastructure and no extra round trip.

Client-side retrieval is for teams that maintain their own tool registry, embedding index, or per-user permission filter. You declare tool_search with "execution": "client" and define your own search parameters. When the model wants a tool, it emits a function call for your search function, your app runs the lookup, and you return the matching schemas in a function_result. Client-side retrieval adds one round trip per search, because your app runs the lookup, and Google says so plainly in its announcement.

Client-side mode has one ability the server-side mode does not. Tools do not need to be declared up front, so you can expose new tools mid-conversation when a user connects an integration. If your product lets users link a new app halfway through a session and immediately use it, that is the mode you need.

What it costs: the token math

Tool Retrieval has no separate charge. Input and output tokens are billed at the standard rates for the model you call. Turning on tool_search adds a small, fixed block of input tokens, and each schema the model loads is billed as input for the rest of the conversation, exactly like a tool declared up front. You save tokens when the schemas you never load outweigh the ones you do, which Google says is the usual case for large libraries.

Google gives no token count for that fixed tool_search block, and it publishes no measured savings, latency, or accuracy figures. The worked example below is our own arithmetic, and every assumption is stated so you can re-run it with your own numbers.

Take a support agent with 50 tools. Assume each tool's parameter schema runs about 400 tokens, and each tool's name and short description runs about 40 tokens. Assume the agent loads one schema per request on average.

  • Without deferral: 50 tools at 400 tokens each is 20,000 input tokens per request, spent on schemas alone.
  • With deferral: 50 names and descriptions at 40 tokens each is 2,000 tokens, plus the one loaded schema at 400 tokens, for 2,400 tokens per request.
  • Saved: 17,600 input tokens per request, before the small fixed cost of the tool_search block, which Google does not quantify.

At 10,000 requests a day, that saving is 176 million input tokens a month. Here is what it is worth on each supported Gemini model, at the rates on Google's own pricing page, read November 20, 2026:

Model Input rate Saved per request Saved per day (10,000 requests) Saved per month (30 days)
Gemini 3.6 Flash $0.75 per 1M tokens $0.0132 $132 $3,960 (about £3,023, about C$5,672)
Gemini 4 Hydrogen $1.50 per 1M tokens $0.0264 $264 $7,920 (about £6,045, about C$11,343)
Gemini 4 Neon $3.00 per 1M tokens $0.0528 $528 $15,840 (about £12,091, about C$22,686)
Gemini 4 Carbon $2.00 per 1M tokens through Nov 30, 2026 $0.0352 $352 $10,560 (about £8,060, about C$15,124)

Currency conversions use November 20, 2026 market rates of 1 USD = 0.7633 GBP and 1 USD = 1.4322 CAD, and are approximate. Two details change the numbers. First, Carbon's input rate doubles to $4.00 per 1M tokens on December 1, 2026, which doubles the dollar value of every token saved on that model, to $21,120 a month in this example. Second, the Batch API cuts input and output rates in half on Neon and Hydrogen, so a batched workload saves half as many dollars per token, while still saving the same tokens.

The saving scales with library size and shrinks with nothing. A 10-tool agent saves little. A 200-tool agent saves far more than this example, because the schemas you never load are the whole point. The saving also only touches input tokens. Output tokens, including thinking tokens, are billed the same either way, and every token the model processes is counted and billed on these models, including thinking tokens and internal tool system instructions, according to Google's pricing page.

Which models support it

Tool Retrieval works with Gemini 3.6 and later models, including all Gemini 4 models, in the Interactions API. The table below lists the supported models and their standard paid-tier input and output rates, read from Google's pricing page on November 20, 2026. We covered Gemini 4 Neon at launch, and Gemini 4 Argon when its intro pricing was announced.

Model Input per 1M tokens Output per 1M tokens Free tier
Gemini 3.6 Flash $0.75 through Dec 31, 2026, then $1.50 $3.75 through Dec 31, 2026, then $7.50 Yes
Gemini 3.8 Flash $0.75 through Dec 31, 2026, then $1.50 $3.75 through Dec 31, 2026, then $7.50 Yes
Gemini 4 Hydrogen $1.50 $7.50 Yes
Gemini 4 Neon $3.00 $15.00 No
Gemini 4 Argon $2.00 through Nov 30, 2026, then $4.00 $10.00 through Nov 30, 2026, then $20.00 No
Gemini 4 Carbon $2.00 through Nov 30, 2026, then $4.00 $10.00 through Nov 30, 2026, then $20.00 No

Watch the dates. Three of these models get more expensive within weeks. Carbon and Argon double on December 1, 2026, and the two Flash models double on January 1, 2027. If you are costing a deployment, price it at the January rates, not the November ones. Also note that the pricing page does not say whether Tool Retrieval itself is available on the free tier, so treat paid-tier access as the safe assumption for production agents.

It plays well with caching and MCP

Two details in the announcement matter for teams already running Gemini agents in production.

First, Tool Retrieval works with prefix caching. Deferred schemas stay out of the start of the prompt, and the schemas the model loads are added later in the conversation, so the cached prefix stays the same from turn to turn. That is a quiet second saving. Without deferral, a long tool list sits inside the cached prefix, and any change to it can bust the cache. With deferral, the prefix is mostly your instructions and conversation, which change less often.

Second, Tool Retrieval works with remote MCP servers in the Gemini API. Set defer_loading: true on an mcp_server tool and every tool that server exposes is loaded on demand, so you can connect a large server without listing its tools one by one. That is the combination that matters most. MCP servers are how tool libraries reach the hundreds, and a single MCP connection can now bring a whole catalog without paying the schema tax for all of it. Google's own Data Agent Kit ships dozens of MCP tools for its data services, which shows how quickly a library grows once MCP is involved.

The catches

Tool Retrieval is a real improvement, but it is a preview, and previews come with fine print. Here is what to weigh before you rebuild around it.

There are no measured numbers. Google's announcement includes no token-savings figures, no latency measurements, and no tool-selection accuracy benchmarks, and independent write-ups of the launch note the same gap. Google's own guidance is to measure it against your own traffic. Our worked example above is arithmetic from stated assumptions, not a benchmark. Run it against your own logs before you quote a saving to anyone.

The fixed cost is unknown. Turning on tool_search adds what Google calls a small, fixed block of input tokens, with no number attached. For a small library, that block plus the loaded schemas could cost more than sending everything up front. The break-even point depends on your library size and your call volume, and only your own traffic can find it.

Loaded schemas stay billed. Once a schema is loaded, it is billed as input for the rest of the conversation, exactly like a tool declared up front. A long conversation that gradually loads half your library ends up paying for half your library anyway. Deferral saves the most on short, single-purpose turns and the least on long sessions that touch many tools.

Names and descriptions still ride along. The model sees every deferred tool's name and description on every request, so those tokens are never saved, and the description text is still part of the prompt. Write descriptions with that in mind. A long, chatty description on a tool nobody calls is a small tax on every request.

Client-side mode costs a round trip. If you run your own registry, every search adds one round trip, because your app runs the lookup. For latency-sensitive voice or realtime agents, that extra hop may matter more than the tokens saved.

The API surface can change. This is a preview in the Interactions API, and Google has said nothing about when it becomes generally available. Pin your model version and keep an eye on the shape of interaction.steps if you build tooling around it, because preview fields are the ones most likely to move.

Who should switch now, and who should wait

Switch now if your agent already carries a large tool library. Support agents, IT helpdesk agents, coding agents with MCP servers, and internal tools agents that expose a whole internal API catalog are the obvious cases. If you can count your tools on two hands, the saving is small and the added moving parts are not worth it yet.

Switch now if you are choosing between Gemini models for a new agent and token cost is the deciding factor. Tool Retrieval removes one of the hidden costs that made large tool libraries expensive, and it works on the cheapest supported model, Gemini 3.6 Flash at $0.75 per 1M input tokens through December 31, 2026, as well as on the flagship Carbon. Start on the cheap model, measure, and move up only if the quality gap shows up in your evals.

Wait if your agent runs on the classic generateContent path and a migration is not already planned. Tool Retrieval only exists in the Interactions API, so adopting it means adopting the newer interface too. That is a reasonable move, and Google has been steering new agent features there since its general availability in June 2026, but it is a migration, not a flag flip.

Wait if your tool library is small, stable, and already fast. A ten-tool agent with a short schema list will not see a meaningful saving, and the preview status adds a change risk you do not need. Check back at general availability.

How to try it today

Tool Retrieval is live in preview now, and trying it takes minutes if you already call the Gemini API.

  1. Move your agent call to the Interactions API, if it is not there already. Tool Retrieval is only available there.
  2. Pick a model from the supported list: Gemini 3.6 or later, including all Gemini 4 models.
  3. Add a tool_search tool to your request.
  4. Set defer_loading: true on the long tail of your library, and leave your most-used tools loaded up front.
  5. Run your normal traffic through it and compare input tokens per request against your current baseline, using the same prompts. Google's own advice is to measure it against your own traffic, and that is the only measurement that counts for your bill.
  6. Watch interaction.steps in your logs. It shows every search, load, and call, which is the fastest way to spot a bad tool description or a tool the model keeps failing to find.

Feedback goes to Google through the Gemini developer Discord, which the announcement links. Preview features change fastest when developers report what breaks, so if you hit a rough edge, that is the channel.

Frequently asked questions

What is Gemini Tool Retrieval?

Tool Retrieval is a preview feature in the Gemini Interactions API, launched November 19, 2026. It keeps an agent's full tool library on standby and loads each tool's parameter schema only when the task needs it, instead of sending every schema on every request. The model still sees every tool's name and description, so it knows what is available.

How much does Tool Retrieval cost?

There is no separate charge for the feature. You pay the standard input and output token rates of the Gemini model you call. Turning on tool_search adds a small, fixed block of input tokens, and each schema the model loads is billed as input for the rest of the conversation. The saving comes from the schemas you never load.

Which Gemini models support Tool Retrieval?

Gemini 3.6 and later models, including all Gemini 4 models: Gemini 3.6 Flash, Gemini 3.8 Flash, Gemini 4 Hydrogen, Gemini 4 Neon, Gemini 4 Argon, and Gemini 4 Carbon. It works only in the Interactions API, not on the older generateContent path.

Does Tool Retrieval work with MCP servers?

Yes. Set defer_loading: true on an mcp_server tool and every tool that server exposes is loaded on demand, without listing them one by one. That makes it practical to connect a large MCP catalog to a Gemini agent without paying the schema cost for all of it on every call.

How is Tool Retrieval different from normal function calling?

In normal function calling, every tool's full parameter schema is sent with every request. With Tool Retrieval, only names and descriptions are sent up front, and schemas load on demand inside the same request. The search, the schema load, and the function call all appear in interaction.steps, so the flow is inspectable in your logs.

Is Tool Retrieval ready for production?

It is in preview, so the API surface can change before general availability, and Google has published no measured savings, latency, or accuracy figures. It is fine to run in production if you pin your model version and measure the saving on your own traffic first. Teams that need a frozen API should wait for general availability.

Bottom line

Tool Retrieval fixes a cost that grows silently with every tool you add. It is free to turn on, it works with the models most teams already use, and the saving is largest exactly where agents are heading, toward MCP catalogs with hundreds of tools. The missing piece is measurement: Google has published no numbers of its own, so treat our worked example as a starting point and run your own traffic through it before you promise a saving. If your agent carries a big library, that test is worth an afternoon. The Gemini team has been shipping agent features into the Interactions API first all year, from computer use on the Mac to this, and Tool Retrieval looks like the one that will show up on the bill.

Sources

Currency conversions use November 20, 2026 market rates of 1 USD = 0.7633 GBP (Finnhub) and 1 USD = 1.4322 CAD (Currencyflow), and are approximate. Worked token-saving figures are the author's own arithmetic from the stated assumptions, not Google-published benchmarks.

15 min read
21 views
edit_note Written by

Ratul Ahamed

Founder & Editor-in-Chief

Ratul Ahamed is the founder and editor of AI Core Updates, an independent site covering AI tools, models and pricing. He is a web developer based in Bangladesh and also runs All In One Tools, a free collection of browser-based utilities. Contact: [email protected]

Read all stories by Ratul Ahamed arrow_forward
verified_user Human-Reviewed Editorial

forum Comments (0)

Leave a Reply

chat_bubble_outline

No comments yet. Be the first to join the conversation!

Weekly AI Digest

Get our weekly AI tools digest

One email a week analyzing new AI tools, model updates, and practical workflows. Tested and summarized with zero fluff. Unsubscribe any time.

bolt
Hands-on AI Tool Reviews
Pricing math, limits, and alternatives.
verified_user
Strict Anti-Spam Policy
1-click unsubscribe at any time.
mark_email_unread

Subscribe to Digest

Sent weekly • No spam

person_outline
mail_outline

No spam ever. You can unsubscribe with a single click at any time.

cookie

Cookie Settings

We use cookies to enhance your browsing experience, serve personalized content, and analyze our traffic. By clicking "Accept All", you consent to our use of cookies. Learn more