Introducing Terminal X Bench

Producing a financial research answer is not one task performed by one model but rather a sequence of distinct prompt chaining steps: finding the right sources, extracting and structuring what's in them, and synthesizing that material into a written, cited answer. Each of these stages asks something different of a model. A model that identifies the correct filing to pull from a document set is doing a fundamentally different job than a model classifying and structuring the content of that filing, which is different again from a model writing the final answer a client reads.

Most model benchmarking treats this as one problem and produces one number, which tends to obscure critical performance information more than it reveals. Terminal X Bench is a new evaluation framework for grading models the way financial AI platforms actually use them: independently, at each stage of the answer process.

Three evaluation stages

Processing

• Extracting and structuring what's inside a source document. This includes classifying document types, pulling out relevant metadata, and converting raw content into a form that is retrievable downstream

Retrieval

• The process of selecting the right sources for a given query, and the right content from that source. Does the LLM select the correct filing, call transcript, correct price or fundamentals data out of everything available? Is the extracted content relevant to the user's question?

Answer Generation

• Synthesizing the processed material into a written, cited response that follows house style, is appropriate to the question asked, and accurately reflects its sources.

Why we created Terminal X Bench

A model can lead decisively at one of the aforementioned stages and lag at another. In our testing, models that perform well on document retrieval are not reliably the same models that perform well on structured extraction and classification, and the heavy-weight models best suited to writing a polished, well-grounded final answer are frequently not the cheapest or fastest options for the stages upstream of it. A single blended benchmark score would average these differences away and point teams toward the wrong model for the wrong job.

Terminal X Bench instead produces a scorecard per model, per stage. Rather than a single number for each new model release, we propose a Retrieval, Processing, and Answer score for each model we test, so the comparison is immediately visible rather than buried as an average.

Early findings

Common patterns emerging across our test sets:

• In retrieval steps, smaller and faster models are frequently competitive with, and sometimes exceed, larger frontier models on source selection tasks at a meaningfully lower cost and latency profile.
• In processing, a model that leads on document selection can underperform a smaller model on structured extraction, and vision-dependent processing tasks in particular separate tested models sharply.
• For answer generation, larger reasoning-oriented models tend to lead on synthesis quality and grounding, handling massive input token sizes while still adhering to user requests and internal prompting guidelines.

What's next

We intend to run Terminal X Bench as a standing practice rather than a one-time release. When a new frontier model ships, our aim is to have it scored across all stages within a day of general availability and to publish that scorecard alongside the models already in the benchmark. In this way, every model earns implementation objectively at whichever stage its scorecard supports. We will share our results for frontier models with the community as they release on our Blog and LinkedIn.