TL;DR
- It works for it: about 2.5× the model calls and 4× the tool calls per question of the other seven.
- It pays for it: $11.46 per question on v2 against Fable 5.1's $1.24.
- Nothing changed for Energent users. The agent is model-agnostic, so a model that arrived on a Thursday was measured by Friday, and the three corners of the trade-off are now: cheapest GPT-5.6 Terra, balanced Fable 5.1, strongest GPT-6 Astra.
Same test, one more model
The setup is the one from the seven-model post: OfficeQA v1 Hard, 133 questions over 697 Treasury Bulletins, and OfficeQA Pro V2, 90 questions over 1,435 scanned ledgers, the same tools, prompts and step budget, reasoning effort high for every model, one run per model scored by the official scorer. Astra ran on OpenAI's API directly, one run per edition, like every other model on the board.
Results
Accuracy, v1 to v2
The chart at the top of this post shows the eight models as arrows from their v1 score to their v2 score. The table has the numbers.
| Model | v1 (133 q) | v2 (90 q) | Change |
|---|---|---|---|
| GPT-6 Astra | 72.2% | 73.3% | +1.1 |
| Fable 5.1 | 72.2% | 70.0% | −2.2 |
| GPT-5.6 Sol | 71.4% | 68.9% | −2.5 |
| Opus 5 | 74.4% | 63.3% | −11.1 |
| Fable 5 | 72.2% | 57.8% | −14.4 |
| GPT-5.6 Terra | 60.9% | 54.4% | −6.5 |
| Gemini 3.1 Pro | 56.4% | 44.4% | −12.0 |
| Sonnet 5 | 67.7% | 38.9% | −28.8 |
Accuracy against cost
| Model | $ / question v2 | Minutes v2 | Model calls | Tool calls |
|---|---|---|---|---|
| GPT-6 Astra | $11.46 | 14.8 | 56.3 | 142.6 |
| Fable 5.1 | $1.24 | 3.2 | 18.1 | 17.6 |
| GPT-5.6 Sol | $2.12 | 6.9 | 24.3 | 61.9 |
| Opus 5 | $0.98 | 2.7 | 17.0 | 19.4 |
| Fable 5 | $2.00 | 4.0 | 14.8 | 19.5 |
| GPT-5.6 Terra | $0.63 | 3.8 | 20.7 | 43.4 |
| Gemini 3.1 Pro | $1.01 | 5.3 | 38.4 | 36.9 |
| Sonnet 5 | $0.83 | 7.0 | 25.9 | 33.0 |
Conclusion
The harder edition separated the field last week; Astra is the first model to close the gap from the other side, by working harder rather than by being cheaper. For Energent users that adds a third corner to the choice we make for you: Terra when cost per question matters most, Fable 5.1 when accuracy and cost both do, Astra when only the answer does.
Thanks
Again to the Databricks team for a benchmark that comes in two editions of the same task. Every model added to the board makes that continuity more valuable.
References
- The seven-model post, with the full method and the v1 and v2 tables
- GPT-6 Astra model page, OpenAI
- OpenAI API pricing
- Claude Fable 5.1 model page, Anthropic
- OfficeQA Pro V2 dataset, Databricks