{"id":103,"date":"2026-10-06T09:58:25","date_gmt":"2026-10-06T09:58:25","guid":{"rendered":"https:\/\/www.0xprathamesh.com\/blog\/?p=103"},"modified":"2026-10-06T10:01:36","modified_gmt":"2026-10-06T10:01:36","slug":"inference-cache-isolation","status":"publish","type":"post","link":"https:\/\/www.0xprathamesh.com\/blog\/inference-cache-isolation\/","title":{"rendered":"The Cache Key Was the Trust Boundary: Eight vLLM Advisories in Forty Eight Hours"},"content":{"rendered":"

Between the 5th and 6th of October 2026 the vLLM project published eight security advisories. The highest scored 6.5. One scored 3.1.<\/strong> Read individually they are a routine batch of medium severity issues. Read together they are one finding, and it is a serious one: in every case the cache key was doing the work of an access control decision.<\/p>\n

In every one of those eight advisories, something that behaves as an access control decision was made from untrusted input. Either the caller controlled the data behind that decision, or the decision was skipped on a code path nobody re-checked. That something is the cache key.<\/p>\n

This piece argues that inference cache isolation is now its own security property, distinct from authentication and distinct from authorisation. It proposes a four check procedure for testing it, and maps each check to the advisory that fails it.<\/p>\n

Why eight medium advisories matter more than one critical<\/h2>\n

Severity scoring measures the blast radius of a single instance. It does not measure how many places the same mistake appears.<\/p>\n

The same week produced fourteen vm2 advisories, several at CVSS 10.0, including full sandbox escape to host code execution. Those are more severe by any scoring system. They are also less interesting, because vm2 is already publicly understood to be unsuitable as a security boundary. Learning that a deprecated sandbox can be escaped again teaches nobody anything new.<\/p>\n

The vLLM batch is different. vLLM is the default serving engine for a large share of self hosted model deployments. It is the thing sitting behind an internal chat assistant, a retrieval pipeline, or an agent platform. It is frequently multi tenant, because that is the entire economic argument for batching requests onto one GPU.<\/p>\n

Eight advisories in forty eight hours, all touching the same boundary, is not eight bugs. It is one design assumption failing in eight places.<\/p>\n

The shared cache is the point<\/h2>\n

An inference engine is fast because it does not redo work. Prefix caching keeps the computed key value state for a prompt prefix, so a second request sharing that prefix skips the computation. Multimodal caches keep processed image and video tensors. Scoring paths cache query embeddings.<\/p>\n

Every one of those caches is shared across requests. That is deliberate. A cache that is not shared is not a cache, it is a buffer.<\/p>\n

The moment a cache is shared across requests from different tenants, the lookup key stops being a performance detail. It becomes the thing that decides whether tenant A can observe, influence or corrupt tenant B. The key is the boundary.<\/p>\n

Most infrastructure code does not treat it that way. Cache keys are written by performance engineers optimising hit rates, reviewed as performance code, and tested for correctness under success conditions. They are rarely reviewed as authorisation logic, because they do not look like authorisation logic.<\/p>\n

The Cache Boundary Test<\/h2>\n

Here is the procedure I now apply to any shared inference cache. Four checks, in order. Each one maps to a real failure in this batch.<\/p>\n

Check one: Derivation<\/h3>\n

Is the cache key derived by the server, or supplied by the caller?<\/strong><\/p>\n

If any part of the key comes from a client controlled field, the client can choose which cache entry to collide with.<\/p>\n

CVE-2026-105755 fails this check. On late interaction scoring deployments, the worker cached per request query embeddings under a key derived from the X-Request-Id<\/code> header. That header is set by the caller. A second concurrent request reusing the victim’s header value replaced the victim’s cached query embedding before document scoring, so the victim’s documents were scored against the attacker’s query. The victim receives results that look valid and are answers to somebody else’s question.<\/p>\n

CVE-2026-105754 fails the same check from a different direction. The disaggregated scale out transport splits a request into a trusted render step and a separate generate step. The generate route accepted a caller supplied features<\/code> object. That object contains serialised encoder tensors, multimodal hashes and placeholder ranges, and the route forwarded it into the engine as though it had come from the trusted renderer. The trust was in the shape of the data, not in its origin.<\/p>\n

This distinction matters because shape is cheap to forge. A schema check confirms that a payload looks like something the renderer would produce. It cannot confirm that the renderer produced it. Origin has to be established by the channel, not inferred from the contents.<\/p>\n

Check two: Completeness<\/h3>\n

Does the key include the tenant identifier on every code path?<\/strong><\/p>\n

Not on the main path. On every path. Retries, continuations, re-submissions and internal re-entry all build keys, and each one is a separate opportunity to forget.<\/p>\n

CVE-2026-105752 fails this check, and it is the most instructive advisory in the batch despite scoring only 3.1. vLLM offers cache_salt<\/code> precisely so that a tenant can isolate its prefix cache namespace. A caller that sets it has done the right thing. On the Harmony tool calling path, a request that invokes a tool runs as a multi turn loop. After each tool call vLLM re-renders the next turn prompt and re-submits it to the engine. Turn one correctly carried the salt. The continuation re-submission rebuilt the engine input without it.<\/p>\n

So the prefix landed in the global unsalted namespace. A second tenant could then test for membership of that prefix and learn whether somebody else had submitted it. The isolation control was present, configured, and correct on the path the author tested. It evaporated on turn two.<\/p>\n

This is the failure mode I keep returning to. A security control that works on the first call and not the second is worse than no control, because the operator has evidence it works.<\/p>\n

Check three: Agreement<\/h3>\n

When caches are mirrored across processes, do they still agree after a request fails?<\/strong><\/p>\n

CVE-2026-105753 fails this check. The default multimodal cache mirrors state across two processes. The frontend holds metadata only. The engine core holds the real payload. The design invariant is that both sides update in lockstep for every request, so the frontend can answer whether an item is cached in the engine without asking it.<\/p>\n

That invariant holds on the success path. It breaks when a request is rejected after the frontend has already rendered and hashed the media. The frontend believes the engine has an entry the engine never received. A later request reusing the same media hash then trips a receiver assertion in the engine core.<\/p>\n

Error paths are where cache invariants go to die. The success path gets the tests.<\/p>\n

Check four: Confinement<\/h3>\n

When a key is malformed, does the request fail, or does the engine fail?<\/strong><\/p>\n

CVE-2026-105756 fails this check. vLLM validated cache_salt<\/code> loosely, requiring only a non empty string, with no character or length restriction. It then passed that value unguarded into the scheduler cache lookup, where the downstream LMCache library applied a stricter check and raised an uncaught error. One request, one bad string, and the shared engine core dies for every tenant on it.<\/p>\n

CVE-2026-105757 is the same shape through three structured output paths. An ordinary request reaches a condition that raises an engine fatal exception rather than a per request validation error. The failure escapes the request boundary and triggers a fatal engine shutdown, denying service to all concurrent and subsequent tenants.<\/p>\n

The pattern is a validation gap between two components, each assuming the other was strict.<\/p>\n

Confinement is the check most likely to be missed in review, because the failure looks like a crash rather than a breach. A crash in a single tenant service is an availability bug affecting the caller who caused it. The same crash in a shared engine is a cross tenant denial of service that any authenticated caller can trigger at will. The code did not change. The tenancy model did.<\/p>\n

Why this class survives code review<\/h2>\n

Each of these advisories would survive a competent review, and it is worth being precise about why.<\/p>\n

A reviewer reading cache key construction is checking whether the key is correct, meaning whether it produces hits when it should and misses when it should. That is a performance question and the answer is usually yes. The security question is different: can a caller influence what this key resolves to? Those two questions look at the same three lines of code and come to different conclusions.<\/p>\n

The tenancy assumption also arrives late. Caching is built when the service has one consumer, which is when the hit rate work pays off. Multi tenancy is added afterwards, at the API boundary, with authentication and quotas. Nobody revisits the cache keys written months earlier, because nothing about them changed. What changed is that the data behind them now belongs to more than one party.<\/p>\n

Then there is the scoring problem. A membership oracle on a prefix cache scores 3.1, which in most triage queues is a low priority ticket. The oracle tells an attacker whether a given prompt has been submitted by somebody else on that engine. For a deployment serving legal, medical or security workloads, confirming that a specific document or query passed through the system is often the sensitive fact. Severity scores are computed on generic impact, and the sensitivity of a prefix is deployment specific.<\/p>\n

The ninth thing that is not a cache bug<\/h2>\n

Two further advisories in the batch, CVE-2026-105758 and CVE-2026-105760, concern video samplers bounded on request controlled frame counts, permitting CPU and memory exhaustion. They are not cache key failures. I include them because they share the root assumption: a value that arrived in a request was used to size shared infrastructure without the server imposing its own ceiling.<\/p>\n

Caller controlled data decided how much shared resource to consume. That is the same mistake at a different layer.<\/p>\n

What this means if you run an inference endpoint<\/h2>\n

If you serve more than one tenant from one engine, the following are now part of your threat model rather than your performance model.<\/p>\n\n\n\n\n\n\n\n\n\n\n
Question<\/th>\nWhat a bad answer looks like<\/th>\n<\/tr>\n<\/thead>\n
What goes into your cache key?<\/td>\nAny header, any client field, any request identifier<\/td>\n<\/tr>\n
Who sets the tenant identifier?<\/td>\nThe client, in a field the server copies<\/td>\n<\/tr>\n
Does every re-entry path rebuild the key correctly?<\/td>\nTool continuations and retries skip the salt<\/td>\n<\/tr>\n
What happens on a rejected request?<\/td>\nMirrored caches diverge and stay diverged<\/td>\n<\/tr>\n
What happens on a malformed key?<\/td>\nThe engine process exits<\/td>\n<\/tr>\n
Who reviewed the cache code?<\/td>\nPerformance reviewers, not security reviewers<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
\"Four
Figure 1: The Cache Boundary Test. Four checks applied in order to any shared inference cache, each mapped to the vLLM advisory that fails it.<\/figcaption><\/figure>\n

The practical first step is cheap. Find every place your serving stack builds a cache key, and for each one write down where each component of that key came from. If any component arrived in the request, you have a finding to investigate today.<\/p>\n

The fix is a version bump, the lesson is not<\/h2>\n

All eight are fixed. Upgrade to vLLM 0.30.0, except for CVE-2026-105753 which is fixed in 0.28.0. Affected versions are confirmed on 0.25.1 and the maintainers note the lower bound predates it.<\/p>\n

Patching closes these eight. It does not close the ninth, which has not been found yet, because the conditions that produced them are structural. Cache code is written for throughput. Cache keys are reviewed as performance artefacts. Multi tenancy is often added after the caching layer exists, and nobody goes back to re-derive which keys became security relevant when the second tenant arrived.<\/p>\n

Dependency scanning does not help here either. It tells you that your installed vLLM version has known CVEs. It does not tell you whether your cache keys are forgeable. There was no CVE for any of this before the 5th of October. The code was the same the week before.<\/p>\n

\n\"Two
Figure 2: Key provenance. A cache key component can only carry an isolation decision if its value cannot be chosen by the caller.<\/figcaption><\/figure>\n

Inference cache isolation as a named property<\/h2>\n

I think this needs a name, because unnamed properties do not get tested.<\/p>\n

Authentication asks who you are. Authorisation asks what you may do. Inference cache isolation asks a third question. Given that you are authenticated and authorised, can anything you submit alter, observe or destroy the computation performed for somebody else sharing the engine?<\/p>\n

A deployment can pass the first two and fail the third completely. Every tenant in the vLLM advisories above was authenticated. Every request was authorised. The membership oracle, the query swap, the engine kill all happen after both of those gates pass.<\/p>\n

Until that third question is on the review checklist, this class keeps producing advisories. Eight of them arrived in forty eight hours, and the only reason we are talking about them as a group is that they happened to be published together.<\/p>\n

Frequently asked questions<\/h2>\n

What is inference cache isolation?<\/h3>\n

It is the property that one tenant of a shared inference engine cannot observe, influence or destroy the computation performed for another tenant through a shared cache. It is distinct from authentication and authorisation, because every request in the vLLM advisories was both authenticated and authorised. The failures happened after those gates.<\/p>\n

Which vLLM versions are affected and what should I upgrade to?<\/h3>\n

The advisories confirm the issues on vLLM 0.25.1 at commit 752a3a504485, and the maintainers note the lower bound predates that version. Seven of the eight are fixed in 0.30.0. CVE-2026-105753, the mirrored multimodal cache desync, is fixed in 0.28.0.<\/p>\n

Why focus on eight medium advisories instead of the vm2 criticals from the same week?<\/h3>\n

The vm2 batch contains several CVSS 10.0 sandbox escapes, which are more severe. They are less informative because vm2 is already publicly understood to be unsuitable as a security boundary. The vLLM batch shows a live, widely deployed engine failing the same structural assumption in eight separate places.<\/p>\n

What is the cache_salt feature and why did it not protect tenants?<\/h3>\n

cache_salt lets a caller namespace its prefix cache entries so they cannot collide with another tenant. It works on the initial request. On the Harmony tool continuation path, vLLM rebuilt the engine input without carrying the salt forward, so continuation prefixes landed in the global unsalted namespace and a membership oracle returned.<\/p>\n

How do I test my own deployment for this class of problem?<\/h3>\n

Apply the four checks in order. Derivation asks whether the server or the caller computes the key. Completeness asks whether the tenant identifier is present on every code path including retries and continuations. Agreement asks whether mirrored caches stay consistent after a rejected request. Confinement asks whether a malformed key fails the request or the whole engine.<\/p>\n

Does dependency scanning catch inference cache isolation failures?<\/h3>\n

No. Scanning reports that an installed version matches a known advisory, which only works after the advisory exists. None of these eight had a CVE before the 5th of October 2026, although the code was unchanged the week before. Scanning tells you about disclosed versions, not about whether your cache keys are forgeable.<\/p>\n

Sources<\/h2>\n
    \n
  • GHSA-2phq-3phc-84px \/ CVE-2026-105755 – flash late interaction scoring caches query embeddings under a caller controlled request id (published 2026-10-05)<\/li>\n
  • GHSA-935w-9g4m-p28p \/ CVE-2026-105752 – Harmony tool continuations drop cache_salt, restoring a cross tenant prefix cache membership oracle (published 2026-10-06)<\/li>\n
  • GHSA-ph3r-5jfg-f84f \/ CVE-2026-105753 – mirrored multimodal IPC caches desync after a rejected request (published 2026-10-06)<\/li>\n
  • GHSA-2823-qmq8-rwvj \/ CVE-2026-105756 – loose cache_salt validation kills EngineCore on LMCache-MP deployments (published 2026-10-05)<\/li>\n
  • GHSA-85xf-c7hm-whqw \/ CVE-2026-105757 – structured output request errors escape the request boundary and terminate the shared EngineCore (published 2026-10-05)<\/li>\n
  • GHSA-ph72-cqr5-qpp7 \/ CVE-2026-105754 – disaggregated scale out multimodal transport trusts caller supplied features (published 2026-10-05)<\/li>\n
  • GHSA-x6mc-67gf-chw4 \/ CVE-2026-105758 and GHSA-58v5-2m8f-94pr \/ CVE-2026-105760 – video samplers bounded on request controlled frame counts (published 2026-10-05)<\/li>\n
  • vLLM releases 0.28.0 and 0.30.0<\/li>\n<\/ul>\n

    Limitations<\/h2>\n

    This analysis is based on the public GitHub Advisory Database records published between 5 and 6 October 2026 and the version ranges stated in them. I have not reproduced these issues against a live vLLM deployment, and no exploit code is provided. The advisories confirm behaviour on 0.25.1 at a specific commit and state that the lower bound of affected versions predates it, so operators should treat the affected range as open ended below that version. The Cache Boundary Test is a review procedure, not a guarantee of isolation.<\/p>\n

    <\/p>\n

    One next action<\/h2>\n

    Take the Cache Boundary Test to your own serving stack this week. Find every place it builds a cache key, and write down where each component of that key came from. If any component arrived in the request, you have a finding. The agent testing harness I maintain is at github.com\/prathamesh-ops-sudo\/AgentCrash<\/a>.<\/p>\n