Blog
Product
Evaluating Opus 5.5, Sol 6 and Luna 6 on Terminal X Bench
September 25th, 2026 - New York, New York
Three major model releases rocked AI this week; first, Anthropic’s Opus 5.5, followed by GPT Luna 6 and GPT Sol 6 joining Astra in the new GPT-6 family (excluding Terra). Since we launched Terminal X Bench a couple of weeks ago, this is the first time we've had the chance to run a batch of new frontier releases through it side by side, so we wanted to share what we found.
As we shared in our original blog post from September 11th, we don't think a single benchmark score tells you much about how a model will behave inside a financial research product. Producing an answer for an analyst involves several distinct jobs that are chained together: reading and structuring documents when they come in, finding the right source and the right figures when a question is asked, and then writing a grounded, cited answer. We score each of those stages on its own, because in our experience the model that is best at one of them is often not the best at the others. In other words, a blended average hides the information you need to make a decision.
All three models were run with medium thinking on the same test sets and the same grading prompts. The costs quoted below are the average cost per query in each test set.
Retrieval is where Opus 5.5 separated itself most clearly. It had the highest score on all four of our retrieval test sets, and it averaged 80% across the stage, compared with 56% for Sol 6 and 51% for Luna 6.
The size of Opus’s lead depends on the task difficulty. For example, when the job is simply picking the correct SEC filing out of a candidate set, all three models do reasonably well: Opus 5.5 scored 88%, Sol 6 scored 81% and Luna 6 scored 75%. Factoring in latency considerations, there's an argument for using the smaller model on straightforward source selection after further prompt fine-tuning, which is consistent with what we've seen from earlier generations as well.
The picture changes once the model has to go into the source and pull out the right figures. For example, on our research retrieval set, where the model has to find specific numbers inside broker and sell-side research and assemble them correctly, Opus 5.5 scored 90% while Sol 6 scored 50% and Luna 6 scored 42%. Of course, for retrieval latency is also a large factor in our routing decisions. Opus 5.5 was the slowest of the three on these sets, averaging 18 seconds per query on research retrieval compared with 16 seconds for Sol 6 and 12 seconds for Luna 6.
Processing covers everything that happens to a document before a question asks about it. This includes working out what kind of document it is, pulling out the metadata that makes it findable, and turning its contents, including charts and tables, into something our retrieval layer can search. It's less visible than retrieval or the final answer, but if a document is tagged or described badly at this stage, no amount of model quality downstream will bring it back.
Averaged across our processing test sets, Opus 5.5 scored 76%, Sol 6 scored 61% and Luna 6 scored 50%. Opus 5.5's average comes with an asterisk however, as it only completed two of the three tasks. On the third it declined to respond to our prompt, so we've left that task out of its average. Across the tasks it did complete, it led the group, and the gap was widest on the work that involves reading pages visually rather than as text. However strong its processing performance for the tasks it did complete, answer refusal is a glaring issue that eliminates Opus 5.5 from production consideration for similar processing tasks.
Similar to processing refusals, we weren't able to give Opus 5.5 an answer generation score. On our 25-query set, roughly every other run failed because the model refused to respond on safety grounds, even though the queries are ordinary financial research questions. On the runs where it did produce an answer, our grader actually preferred its output to Sol 5.6, our current benchmark for this stage, so we think the underlying capability is there. Unfortunately models that refuse routine requests can't be deployed reliably; we'll rerun our evaluations as soon as this behavior is corrected.
Sol 6 did well on answer generation, scoring 79% at about $0.07 per answer, compared with $0.27 for Sol 5.6. Even so, our grader preferred Sol 5.6's answers in 68% of the head-to-head comparisons between the two, so for now Sol 5.6 remains the stronger answer writer. We're digging into the individual runs to understand where Sol 6 falls short.
Overall, Opus 5.5 is the strongest of the three on retrieval and processing, but answer refusals prevent us from implementing for now. Sol 6 and Luna 6 trail it on accuracy, but they are far cheaper and faster, which makes them a better fit for the simpler, high-volume steps in the pipeline. We'll continue to route each stage based on these results, and we'll publish updated scorecards here and on LinkedIn as new models are released.