The model you pay for is the model that answers
A cheaper API saves money only when the model behind it is the one on the label. On 11 November 2026 we sent every model the AIGROW endpoint sells the same prompts as the same model at its maker, and compared what came back. All 9 matched, and two lines that did not were taken out of service.
Run of 11 November 2026, 14:10 to 14:46 UTC. AIGROW is independent of Anthropic, OpenAI, Google and xAI.
- Same tool call as the maker
- 9 of 9
- Same prompt tokens as the maker
- 7 of 9
- Lines that add context of their own
- 2
- Lines removed after failing
- 2
- Discount API sellers tested
- 12
Claude against Anthropic’s API; GPT, Gemini and Grok through OpenRouter
AIGROW checks each model its API sells against the same model at its maker: 12 questions with one right answer, a tool call and the prompt tokens reported for a fixed prompt. On 11 November 2026 all 9 models on sale matched. The results, the battery and a script that repeats the test are below.
What the run found
A model matches when it answers the battery as its maker’s model does, makes the same tool call and reports the prompt tokens its maker reports.
All 9 models on sale reached their maker’s score on the battery, give or take one item.
8 scored within one item of the maker’s run with no setting changed. Gemini 4 Argon scored 7 of 12 that way and 12 of 12 with a reasoning effort set in the request, where Google’s model scored 11.
A weaker model under a stronger name fails it: lines we rejected scored 0 and 4 of 12.
In November 2026 we bought credit from 12 discount API sellers and ran these checks on what they sell. Each of the 12 had at least one line that failed. One line sold as Claude Opus 5.5 scored 4 of 12 and one sold as Claude Sonnet 5.5 scored 0, where Anthropic’s models score 12.
For the same prompt, 7 of 9 models reported exactly the prompt tokens their maker reports.
The other 2 lines put context of their own in front of the prompt: up to 258 tokens on Gemini 4 Argon, and 2,094 on 3 of the 8 calls to Grok 4.7. One line we rejected reported about 12,500 prompt tokens for a tool call of about 100.
2 lines failed this run and no longer serve requests.
A fallback line for GPT-6.1 Sol scored 0 of 12 where OpenAI’s model scores 12, so GPT-6.1 Sol now has one line and returns an error when that line is down. Grok 4.3 reported 21 prompt tokens where xAI’s endpoint reports 197 and answered a question about itself with a fixed sentence, so it is off sale. Grok 4.7 is the Grok sold.
Each model beside its maker
The score is the number of right answers on the battery; Grok 4.7 was sent 5 of the 12 items. Prompt tokens are those reported for the same three prompts: a two-word reply, four short questions and a tool call.
| Model | Here | At the maker | Prompt tokens | Tool call |
|---|---|---|---|---|
| Compared with Anthropic’s API | ||||
| Claude Fable 5.2 | 12 of 12 | 12 of 12 | Same | Same |
| Claude Opus 5.5 | 12 of 12 | 12 of 12 | Same | Same |
| Claude Sonnet 5.5 | 12 of 12 | 12 of 12 | Same | Same |
| Claude Fable 5.1 | 12 of 12 | 12 of 12 | Same | Same |
| Compared with the same model through OpenRouter | ||||
| GPT-6 Astra | 12 of 12 | 12 of 12 | Same | Same |
| GPT-6.1 Sol | 12 of 12 | 12 of 12 | Same | Same |
| GPT-6.1 Terra | 11 of 1212 of 12 with effort set to high | 12 of 1211 of 12 with effort set to high | Same | Same |
| Grok 4.7 | 5 of 5 | 5 of 5 | Up to 2,094 more | Same |
| Gemini 4 Argon | 7 of 1212 of 12 with effort set to low | 11 of 126 of 12 with effort set to low | Up to 258 more | Same |
A model stays on sale when its best run here is within one item of the maker’s, its tool call is the same, its thinking block is signed where the maker signs one, and its prompt tokens match the maker’s or differ by context of the line’s own, which this page reports.
The battery
12 questions with one right answer each, sent in one message with no tools. A model has to reason its way to them, and a model that guesses gets a few. The prompt is shown exactly as the models received it.
Solve all 12 items exactly. You have no tools; reason carefully. Reply with ONLY 12 lines in the form `N: answer` (no words, no units, no thousands separators). 1: Sum of the decimal digits of 2^200. 2: Number of integer partitions of 50. 3: Number of twin prime pairs (p, p+2) with p < 3000. 4: Determinant of the matrix [[3, -1, 4, 2], [1, 5, -9, 2], [6, 5, 3, -5], [8, 9, -7, 9]]. 5: Day of the week (English) of 17 March 2031. 6: Length of the longest common subsequence of "AGGTABCXKQTRPLMZNVWY" and "GXTXAYBKQRTPLZMNWVYC". 7: Let f(n) be the number of Collatz steps to reach 1 (n -> n/2 if even, 3n+1 if odd). Let m be the n in 1..199 with the largest f(n) (smallest n on ties). Answer m*1000 + f(m). 8: The permutation at 0-indexed lexicographic position 31337 of the digits 01234567, written as 8 digits. 9: 7^222 mod 1009. 10: How many integers from 1 to 10000 have digit sum 19. 11: Number of derangements of 10 elements. 12: Number of occurrences of the letter r in: "the quick brown fox jumps over the lazy dog while seventeen irritated terriers hurriedly retrieve arbitrary forgotten rubber rings".
- Exact answers
- A line counts when it equals the expected answer. Claude ran with adaptive thinking. The other models ran with the settings their endpoint starts from, and again with a reasoning effort set where the two sides started apart.
- Prompt tokens
- The usage each endpoint reports for three fixed prompts. A model’s tokenizer decides the count, so another maker’s model reports another number, and a hidden instruction adds to it.
- Tool call
- One request with a weather tool that takes a city and a unit. The arguments have to be the ones the maker’s model returns.
- Signed thinking
- Anthropic signs the thinking block of a Claude answer. Every Claude answer here came back with that block and its signature.
- The reference
- Claude was compared with Anthropic’s API, called with our own key. GPT, Gemini and Grok were compared with the same model through OpenRouter, a public router.
- When
- Between 14:10 and 14:46 UTC on 11 November 2026. Every call was a new conversation, streamed, and went to the line that serves the model, before the endpoint’s own accounting.
Where a line differs from its maker
The same model can run under different settings. These are the differences the run measured, with what each means for the bill.
- Claude called directly
- On a direct API call with adaptive thinking, the Claude line thought for longer than Anthropic’s endpoint on some runs. Over 8 runs of the battery Claude Opus 5.5 used 2,982 to 31,999 output tokens here, and 2,883 to 7,903 over 5 runs at Anthropic. One Claude Sonnet 5.5 answer ran to 46,822 output tokens on a max_tokens of 32,000, and some calls reported 724 prompt tokens more than Anthropic does for the same prompt. The Claude line stops the text at the max_tokens a request sets, so the endpoint bills no Claude output past it. Within it, output and those prompt tokens are billed as reported, so a direct call that thinks longer costs more.
- Claude through Claude Code
- Sent by Claude Code at medium effort, the battery took 1,438 output tokens on Claude Opus 5.5 and 1,992 on Claude Fable 5.2, against 5,564 and 1,578 at Anthropic, with every answer right on both sides.
- Reasoning effort
- The GPT-6.1 Terra and Gemini 4 Argon lines start from a lower effort than the maker’s endpoint when the request sets none. GPT-6.1 Terra scored 11 of 12 that way and 12 with effort set to high. Gemini 4 Argon scored 7 and then 12 with effort set. Set reasoning_effort in the request when the task needs it.
- Output cap
- The GPT lines and the Grok 4.7 line do not stop at max_tokens. Asked later the same day for a long essay with max_tokens set to 300, each returned the whole essay, 2,370 to 4,538 output tokens, where OpenAI’s and xAI’s endpoints stop at the cap. Those answers are billed as delivered, so ask for the length in the prompt. Claude and Gemini 4 Argon stop at the cap, and Gemini 4 Argon raises a cap under 128 tokens to 128.
- Added context
- Two lines put tokens of their own in front of the prompt, and the endpoint takes them out of the usage it shows and bills: 288 on Gemini 4 Argon and 1,200 on Grok 4.7, close to the 1,247 prompt tokens xAI’s own endpoint reports for a two-word prompt. The Grok 4.7 calls that reported more were billed for the rest. The table shows each line’s own count, before that subtraction, so the script run through the endpoint reports fewer prompt tokens than the maker on these two models.
Run the check yourself
One file for Node 18 or later, with no dependencies. It sends the battery, a short prompt and a tool call to an endpoint, and to the maker as well when you give it a maker key.
This endpoint
Prints the score, the prompt tokens and the arguments of the tool call.
MODEL_CHECK_KEY=ag_your_key node model-check.mjs claude-opus-5.5Beside the maker
With an Anthropic key for Claude, or an OpenRouter key for GPT, Gemini and Grok, it prints both sides and says where they differ.
MODEL_CHECK_KEY=ag_your_key MAKER_KEY=sk-ant-your_key node model-check.mjs claude-opus-5.5
MODEL_CHECK_KEY=ag_your_key MAKER_KEY=sk-or-your_key node model-check.mjs gpt-6.1-solEvery request is billed as usual. The battery is the costly one: a few cents of credit on most models, more when a model thinks for long. To test another seller’s API, pass its address with --base.
What this page does not claim
- One day
- The results are those of 11 November 2026. Lines change, so the check is run again on every model on sale in the first week of each month, before a model moves to a new line and before a new model goes on sale, and a model that fails comes off sale the same day. This page carries the date of its latest run.
- Models from one maker
- Two strong models from one maker can both get every item right and count tokens the same way. The battery tells a model from a weaker substitute and from another maker’s model. It cannot tell which of two models that both pass gave the answer.
- Twelve items
- A short battery catches a weaker model and misses small differences. A score of 11 and a score of 12 are the same result here.
- Public items
- The items and their answers are in the script, so a seller could tune a line to them. Change the numbers in the script to make a battery nobody has seen.
- The reference
- Only Claude was compared with its maker’s API directly. GPT, Gemini and Grok were compared through OpenRouter, without a key of our own at OpenAI, Google or xAI.
- Where the models run
- AIGROW serves each model through capacity it buys from more than one source, and does not name them. Settings such as the starting reasoning effort can differ from the maker’s endpoint, as the section above shows.
- No affiliation
- AIGROW is independent of Anthropic, OpenAI, Google and xAI. Their names say what the endpoint serves and imply no partnership or endorsement.
Questions about genuine models
9 models, each checked against its maker.
Start with the $25 pack, then run the script with your own key.
Model check of 11 November 2026 · results and script on this page