Verifiable inference: everything we need is already here

September 18, 2026

If you force suspect responses through legitimate inference, you get strong agreement or disagreement quickly. If you force a random selection of untrusted responses through legitimate inference, you solve verifiable inference.

But first, let's ask DeepSeek V4.1 Flash to name an animal.

Surprisingly, it chose cat. The information content of that fact is -log₂ 0.047 = ~4.4 bits. Information content is a concept from information theory that measures how large a message would be if optimally compressed, sometimes also called surprisal (as more surprising messages are harder to compress).

The average response's information content is lower, at ~1.48 bits. The average response's information content is also called entropy. We can convert it out of bits if we want to: 0.5^(1.48) says the average response occurs 35.8% of the time, and 2^(1.48) (perplexity) says this distribution has the same uncertainty as a distribution with ~2.79 different equally likely options.

Now let's ask DeepSeek V4.1 Flash on a suspicious platform to name an animal.

We can quantify the obvious fact that this is from a different model by seeing how well the real model compresses the average suspicious response. This is called cross entropy, and here's the math:

• 0.317 * -log₂(0.045) +
• 0.192 * -log₂(0.047) +
• 0.15 * -log₂(0.004) +
• 0.091 * -log₂(0.054) +
• 0.091 * -log₂(0.167) +
• 0.159 * -log₂(0.004)

We get ~5.34 bits. Meanwhile, the suspicious model compresses its own average response into ~2.44 bits. So the suspicious model compresses its own responses 5.34 - 2.44 = ~2.9 bits better than the real model does. In other numbers, the suspicious model finds its average response 2^(2.9) = ~7.5 times more likely than the real model does. This is called KL divergence.

This scales to whole responses. You just add up the bits for each response, then subtract the responses from each other.

This "scales" to whole marketplaces. As long as you require sellers to include probabilities and to support forcing, you can compare sellers with each other and kick off any with too high KL divergences. From there, it's just a matter of dealing with low trust sellers (beyond the "can't detect when will be evaluated" KL divergence gives you for free: delaying payouts, sometimes forcing through an external provider instead of a peer to prevent collusion, speed minimums, bucketing KL divergences by request type, etc).

But what if you don't have probabilities? Information content is your friend then. You may not have enough data to verify, but you still have enough to falsify. Let's go back to one token to illustrate this.

G and all other tokens are off the charts because the legitimate model simply rarely generates them.

Let's try this with some real providers

After CrofAI went kablooey, a lot of people recommended me providers to test. Let's try a few.

Fusion Code

1 person submitted Fusion Code. Fusion Code is a Gonka wrapper, evident from their model selection. For a brief time, they were serving even more models on a third party OpenRouter-like platform - they were soon removed due to not fixing their inference to serve what they claim after plenty of advance warning. I captured a case before then: reasoning from "GLM 5.3" is better compressed by GLM 5.3 Flash.

Pareto Inference (affiliate link)

1 person submitted Pareto Inference. Pareto Inference advertises themselves as, and is for our purposes, a cheap ($0.03/0.1/0.006) GLM 5.3 Flash API. We can calculate KL divergence since Pareto includes logprobs, and it looks sound:

That's an average KL divergence of 0.045 bits per token, basically nothing. (Remember earlier we saw a 2.9 bit per token divergence between models.)

InferX

2 people submitted InferX. Their gateway is open source and can route both to their own platform and to an outside vLLM with a stored API key.

These 2 models run on separate vLLM. Both come back without the x-ttft-latency header InferX's own platform sets, and both are connected to the same Calgary ISP. InferX rents a thin slice of it: more than two or four parallel requests are blocked with all replicas at capacity.

Verifying GLM 5.3 Flash is easy, since Fireworks serves it and will force responses through it.

0.022 bits of divergence per token.

V4.1 Flash is odder than 5.3 Flash. It rejects malformed requests with a serde error rather than a pydantic one, because it runs vLLM's newer Rust server, which also hands out 8 character request ids, float32 logprobs, and the sampled token duplicated into top_logprobs so you get four real alternatives instead of five. Fireworks also won't echo it - Prompt logprobs (echo or echo_last with logprobs) are temporarily unsupported for this model. - a failure mode entirely unique to V4.1 Flash. But we can run it in reverse: InferX passes through prompt_logprobs to vLLM on every model, so we can ask InferX to score Fireworks' responses.

Also no significant divergence. (If we push the same Fireworks responses through InferX's V4 Flash 0731 instead of InferX's V4.1 Flash, we get 0.68 bits of divergence.)

Similarly to the first two, this is separate vLLM, but I didn't group it with them since I couldn't verify it was in Calgary. It's also the one model InferX charges above market for.

It is real though: flipped the same way as V4.1 Flash, Fireworks' responses diverge by 0.012 bits per token under InferX's hosting.

These seven are InferX's own platform, on CoreWeave: they set x-ttft-latency, and their image fetches arrive from CoreWeave with a stock vLLM/0.19.1 user agent. These were even harder to score, since they weren't available on Fireworks at all. Thankfully, OpenRouter allows us to get logprobs for a few of these models, and from there we can just run them in reverse as before. Qwen3.8 27B comes out at 0.022 bits per token from Parasail, 0.002 from CoreWeave and 0.001 from AkashML, against 0.009 for InferX's own samples, and swapping in Qwen3.6 35B would cost 0.86. Gemma 4 31B, the widest price gap of the lot, is 0.049 against a 0.023 floor.

All in all, InferX looks real.

Kitani

1 person submitted Kitani because they sell DeepSeek V4.1 Flash for $0.05/0.0008/0.10, cheaper than InferX. Their headers name the same upstream whether thinking is on or off: astra-ai, in cn-shenzhen.

Kitani has no /v1/completions and silently ignores echo, so it can't score anything itself, and Fireworks won't echo this model either. So we use InferX as the reference instead. Thinking on first, since that's how the model is meant to run.

Kitani's reasoning diverges by 0.033 bits per token, against 0.012 for Fireworks and 0.003 for InferX scoring its own samples. Now the thinking-off endpoint, where two more deployments can come along: DeepSeek's own API, and Sail Research, who advertise fp4.

(The two charts aren't on the same scale because disabling thinking creates more randomness.)

DeepSeek and Sail concentrate around 0.05 bits of divergence per token. Kitani sits around 0.12. I discussed this mild divergence with Kitani staff, and it turns out that they're experimenting with serving DeepSeek on Huawei chips. While this causes measurable divergence, they state this doesn't cause quality loss and that they still use 8 bit quantization.

Honorable mentions

Surplus Intelligence (affiliate link), InferHub, and Cheaper Inference (requested by 1 person) are each marketplaces (unlike the inference providers from the rest of this post). Right now, these all manage trust by limiting which providers are allowed on. Perhaps in the future they'll develop verification capabilities strong enough to let anyone who passes on.

Gonka, the inference Fusion Code wrapped, is interesting because they already try to verify inference themselves.

But Gonka does some abnormal stuff. They ban each LLM from stopping before 64 tokens via vLLM's min_tokens. And DeepSeek V4 Flash 0731 maps high to medium and max to xhigh in the Gonka chat template. (The other models have non-Gonka drawbacks: GLM 5.3 Flash only supports low, high, and max reasoning efforts officially (no none or medium), and MiniMax M2.7 is so old it doesn't allow disabling or controlling reasoning.)

Since one person requested Hyper and another requested b.ai, I ran light verification passes and found nothing amiss, but haven't done deeper verification.