Billion-Context Review: 5x Fewer Tokens for Coding Agents

Long coding sessions with an AI agent usually end the same way: the context window fills up with test logs and file dumps, the agent forgets a decision you made an hour ago, and you start over in a fresh chat. Billion-context, a free open-source proxy from GitHub user ranxianglei, takes a different approach. It sits between your coding agent and the model API and lets the model fold old messages into summaries itself, so one session can keep running for days inside a window as small as 100,000 tokens. The project passed 6,000 GitHub stars on November 10, 2026, about three months after the repository was created, and an independent hands-on test published November 9 measured what actually happens when you run a real session through it.

This review covers what billion-context does, how to set it up, what it costs in tokens, and where its headline claim of 5x fewer tokens holds up and where it does not. The short version: the idea is sound, the measured savings are real but smaller than the headline, and the project is early enough that its own README warns you to expect rough edges.

What Billion-Context Is

Billion-context is a local proxy, run through a command called bili, that sits between a coding agent such as Claude Code, Codex, OpenCode or Gemini CLI and the model API it calls. It works with Anthropic, OpenAI and Google-native requests. Instead of truncating history when the window fills, it injects four tools the model can call on its own: compress folds a range of old messages into a detailed summary, decompress restores a folded range when exact details are needed again, search_context searches folded summaries and visible messages by keyword, and acp_status reports how full the context is and which ranges can still be compressed.

The key difference from the auto-compaction built into most coding agents is who decides what gets folded. With Claude Code, compaction happens automatically when the window fills, and the summary is written by the host. With billion-context, the model decides when and what to fold, nudged by the proxy as the window fills, and the originals stay in a local SQLite store so a fold can be undone. Summaries can themselves be folded again into higher-level summaries. The project says a healthy session keeps a 95 to 97 percent prefix-cache hit rate, and that compression itself costs 2 percent or less.

It is not a model, a memory database, or a coding agent of its own. Its only job is to shrink what gets sent upstream on each request. That distinction matters for the cost math below: billion-context is free software, but every token it adds or removes still shows up on your API bill.

How It Works: Four Tools Between Agent and Model

When your agent sends a request, the proxy forwards it to the real model API, rewriting the stream on the way. The four context tools ride along, so the model sees them as tools it can call like any other. When the model calls compress, the proxy runs the fold server-side and from then on sends the folded view upstream. User messages are protected from folding, which is why the design holds up well on decisions and constraints you typed yourself rather than on tool output.

The proxy supports more than 20 coding agents, with native plugin installs or launchers for Claude Code, Codex, OpenCode, Gemini CLI, Kimi Code, Aider, GitHub Copilot CLI, pi, omp, dsh, Hermes, Antigravity and others. For anything without a native hook, there is a universal path: start the proxy and prefix the upstream URL with /bili/. One notable gap: Cursor is not in the supported client list as of November 2, 2026.

For closed-source clients with no plugin seam, some setups use certificate interception, where the proxy presents its own local certificate so it can read the traffic. That is how tools like this work, and it is the part to think about before installing: your API keys and your code pass through a local proxy, and the traffic stays on your machine, but you are still adding a middleman between your agent and your model provider.

Setup in About Five Minutes

Installation is one npm command, and it needs Node.js 20 or newer. The published package has a single runtime dependency, according to an independent review of the project, and the current npm release is 0.1.213, verified on November 10, 2026:

npm install -g billion-context --prefix=~/.local

From there, the simplest path is a launcher for your agent, for example bili claude for Claude Code or bili codex for Codex, or a native plugin install such as bili plugin install pi. The README warns that native plugins and the standalone extensions are mutually exclusive: running both means double compression, and the installer replaces legacy entries and snapshots the original config to a .bili-bak file once.

Two settings deserve attention before a first real session. First, the proxy checks npm every three minutes by default and upgrades itself after a staggered delay. On a pinned setup or a shared machine you will want ACP_AUTO_UPDATE=0 and a pinned version instead. Second, the default growth gate only starts folding after about 50,000 tokens of growth, so a short task may never fold at all, while the injected tools still add overhead to every request. The independent test pinned the window to 40,000 tokens to force folds inside one afternoon.

What It Costs: Free Software, and the Token Math

Billion-context itself costs nothing. It is MIT-licensed with one extra term, covered below, and there is no paid tier, no account and no hosted version to subscribe to. The money question is tokens: the proxy adds about 4,000 tokens of tools and instructions to every request, and it saves tokens by shrinking the history it sends upstream. Whether that trade pays off depends on how long your sessions run.

The clearest measured numbers come from an independent hands-on test by Ishank Choudhary, published November 9, 2026, which ran a 41-request Gemini agent session through version 0.1.213 with the model gemini-3.5-flash-lite. The final request shrank from 38,816 tokens uncompressed to 13,431 through the proxy, 2.9 times smaller, and total input over the session fell 12.5 percent, from 758,108 tokens to 663,321. The proxy added a median of 10 milliseconds of local processing per request.

The cost rows cut against the headline. At Gemini 3.5 Flash-Lite list prices of $0.30 per million input tokens, $0.03 per million cached tokens and $2.50 per million output tokens, the compressed session cost about $0.067 while the uncompressed session with perfect prefix caching cost about $0.039. Compression lost on cost. Every fold rewrites the prompt prefix, which restarts the prefix cache from scratch on the next request, and the roughly 4,000-token tool overhead is charged on every call. With no caching on either side, compression won, $0.205 against $0.234, but that is not how most providers bill in practice.

For readers outside the US, those list prices convert to roughly £0.22 and C$0.42 per million input tokens at November 10, 2026 rates (1 USD = 0.7474 GBP, 1.4164 CAD), and the $0.067 test session works out to about £0.05 or C$0.09. These are approximations. The point is scale: on a short session the amounts are pennies, and the saving only matters when sessions run long enough for history to dominate the bill.

The 5x Claim Against an Independent Test

The README makes three headline claims: a 100,000-token context window is enough, 5x fewer tokens, and month-long single sessions running into billions of tokens. The project’s own preprint, shipped in the repository, reports a 4.5-month study with 174,327 model calls and 18.76 billion cumulative input tokens, zero window violations on 204,800-token models, and marathon sessions of 8,584 to 12,049 calls. A benchmarks folder in the repo, labeled as independent third-party results, reports first place on ContextBench at 58.3 percent lower cost than full context, 79.20 percent against 62.60 percent for the best retrieval baseline on LongMemEval-S at 71.1 percent lower cost, and 100 percent survival on a 1.48-million-token MemoryAgentBench run with 30x steady-state compression.

Those numbers are worth reporting and worth discounting at the same time, because they all live inside the project’s own repository. The hands-on reviewer put it plainly: both the paper and the benchmark folder are the project’s claims, not evidence. The independent test that does exist is a single 41-request session on one small model, and it measured a 2.9x smaller final prompt and 12.5 percent less total input, not 5x fewer tokens overall. The reviewer also tested whether the agent still remembered what it was told. On ten easy questions about decisions from the session, the folded agent scored 10 out of 10, the same as the full-history agent. On eight hard questions about tool output, it scored 4 out of 8 without retrieval, and 6 out of 8 once it was allowed to search and decompress, matching the full-history agent.

That is a fair result for the design: instructions and decisions survive folding well, because user messages are never folded, while fine details from tool output survive only if the agent goes back and retrieves them. It is not a confirmation of the 5x figure, which would need a session thousands of calls long. Until a longer independent test exists, treat 5x as the project’s target, 2.9x on the final prompt as the measured number on a short session, and the month-long claim as untested outside the project.

Billion-Context vs. Built-In Compaction

Most coding agents already compact context when the window fills, so the honest comparison is against what you already have. The differences come down to control, reversibility and cost.

There is also a neighboring idea worth distinguishing: memory tools such as claude-mem, which we covered separately, store what an agent did across sessions and re-inject relevant context later. Billion-context does not do that. It manages the context of one live session, not a memory of past ones. The two solve different halves of the same problem, and they are not substitutes.

The Catches Worth Knowing Before You Install

The first catch is maturity. The README’s status line reads, in full: early, protocol handling and compression work against mock tests with more than 500 passing, and real-model integration testing is the next milestone, so expect rough edges. The repository had about 730 open issues and pull requests on November 2, 2026, and shipped 565 commits in the week before the November 9 hands-on test. That pace is a sign of life and a reason to pin a version before depending on it.

The second catch is a real bug the independent test found. On Gemini’s OpenAI-compatible endpoint, the proxy dropped the thought_signature field that Gemini 3.x returns with each tool call, so every request after the first tool call failed with an HTTP 400. The same script worked when sent directly and when switched to Gemini’s native generateContent wire. If you run Gemini behind an OpenAI-style client, test multi-turn tool calls before routing real work through the proxy.

The third catch is the license. The code is MIT-licensed plus one additional term: any product or service whose users can see or interact with it, and which uses this software, must state that it uses billion-context with a link back to the repository, on its home page, documentation or About page. Pure server-side use can satisfy this through shipped documentation. GitHub shows the license as NOASSERTION because of the extra term. For personal use this changes nothing, but anything user-facing needs to read that term first.

The fourth catch is the security shape. The proxy reads your agent’s traffic by design, and for closed clients it may do so with a local certificate. That keeps everything on your machine, but a bug or misconfiguration in the proxy can see your API keys and your code. Treat it like any early open-source dependency: read the code path you rely on, keep it updated or pinned deliberately, and do not route credentials you cannot afford to expose.

Who Should Try It, and Who Should Wait

Billion-context fits a specific user: someone running long agent sessions on small-window or cheap models, where hitting the window is the actual failure, and who wants folds they can audit and undo. The export command dumps the folded view and every block summary, so you can check what the model actually kept. The median 10-millisecond overhead is small enough to ignore.

Wait if your tasks are short. Under roughly 50,000 tokens, the default settings may never fold at all, and the tool overhead only adds cost, which is exactly what the independent test showed. Wait if you use Cursor, which is not supported, or if you run Gemini through an OpenAI-compatible client until the signature bug is fixed. And wait if you need a stable dependency: hundreds of commits a week plus self-update by default means you should pin a version and upgrade on your own schedule.

For context on the wider tooling, our Claude Code explainer covers the agent most billion-context users will pair it with, our OpenCode review covers the open-source agent on the supported list, and our roundup of the 7 best AI coding agents puts the pricing of the underlying models in one place.

Frequently Asked Questions

Is billion-context free?

Yes. The software is free and open source under MIT plus an attribution term, with no paid tier or hosted version. You still pay your model provider for every token sent, including the roughly 4,000 tokens of tools the proxy adds to each request.

Does billion-context work with Cursor?

No, Cursor is not in the README’s supported client list as of November 2, 2026. The list covers more than 20 agents including Claude Code, Codex, OpenCode, Gemini CLI, Aider and GitHub Copilot CLI, plus a universal URL-prefix path for other clients.

Does billion-context store my code?

Folded originals are kept in a local SQLite store on your machine so folds can be reversed and searched. Nothing is sent to the project’s servers, because there is no hosted component. The proxy does see your agent’s traffic by design, since rewriting it is its job.

How much can it really save?

The project claims 5x fewer tokens. The one independent test measured a final prompt 2.9 times smaller and 12.5 percent less total input on a 41-request session, and the session cost more, not less, once prefix caching was counted. Savings grow with session length, so treat the headline as a target for marathon sessions, not a promise for short ones.

Is it safe to route API keys through it?

The traffic stays on your machine, but the proxy reads it by design and some closed clients need local certificate interception. That is normal for a proxy and still a trust decision. Pin a version, disable self-update on shared machines, and read the license term before using it in anything user-facing.

Billion-context solves a real annoyance in a clever way: instead of the host silently summarizing your session when the window fills, the model folds its own history, keeps the originals, and can fetch them back. The measured savings on a short session are smaller than the headline, the project is early, and one real bug is already documented. But the design held up where it matters, on the decisions and constraints the agent was told, and for long sessions on cheap models that is the part worth having. Try it on a throwaway project first, pin the version, and measure your own token bill before trusting the 5x figure.

Sources

• billion-context repository, GitHub — README, install, client list, status, license term, preprint and benchmark figures, verified November 10, 2026
• Billion-context Hands-On, Ishank Choudhary, Medium, November 9, 2026 — independent 41-request test, token and cost numbers, thought_signature bug
• Billion-Context Setup for Claude Code, Nishant Gupta — setup guide, auto-compact comparison, client coverage
• Billion-context Review, andrew.ooo, October 18, 2026 — star count and release pace at the time, Node 20 requirement, single runtime dependency
• ForexInvestors, GBP/USD rates, November 10, 2026 — 1 USD = 0.7474 GBP, 1.4164 CAD