Rendered at 22:04:25 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
notenchange 12 hours ago [-]
I’m trying to do some GPU unlocking work with some oldish mining cards (NVIDIA CMP 170HX 10G) to fully unlock them to be used in LLMs, and Sol 6.1 keeps refusing to do the work on the basis of “cyber concerns”. It’s highly frustrating. Sonnet 5.5 is even worst.
Luna 6.1 is fine with it, but the results aren’t good enough. I’m going to try Terra 6.1 (from the post, I should expect about the same results). And hope it doesn’t complain as much about these non existent cyber security risks that for some reason it keeps raising.
P.S. I also tried Deepseek Flash 4.1, but the results simply didn’t cut it.
P.P.S. This all investigation was a major eye opener on why we need to run local models or be forever tied to some artificial limits to AI for the masses.
micw 12 hours ago [-]
Try GLM for such things. Better than DS Flash in my observations but less restricted than GPT
Sammi 10 hours ago [-]
New hn post on deekseek and GLM models that are unrestricted.
Abliterated models have always struck me as severely brain damaged. Maybe that’s gotten better.
jeffy29 6 hours ago [-]
There is no Luna 6.1, yet.
throw20261022 6 hours ago [-]
Their model release pattern has been a joke. Any firm in their right mind would have released the large+mid+small tier models at the same time. OpenAI though can't seem to pull it off. It's like they either don't have the hardware for it, or they have to wait to collect substantial user data over which to distill the tiers. Either way it's disruptive to user workflows, and not sane to invest in.
basch 5 hours ago [-]
staggering has to be a better use of resources, and I would think most workflows can accommodate using a 6.0 of one model and a 6.1 of a completely different one without balking.
throw20261022 5 hours ago [-]
> most workflows can accommodate using a 6.0 of one model
No, they can't, considering Terra never existed for 6.0. It's a mess, and no self-respecting user would care for it.
senordevnyc 2 hours ago [-]
This seems like a really trivial thing to get so worked up over. Workflows work fine with a mix of different model versions. (I do it all the time, including open weight models, different reasoning levels, etc. Sounds like you need some evals.
Also, you can just…wait a little longer until all the models for the version number you want are available? You don’t lose anything, so how is not “sane” to do so?
krzyk 8 hours ago [-]
Nitpicking, but I had to chekc if they introduced it - there is no Luna 6.1, there is Luna 6.
minimaxir 1 days ago [-]
Similar 75% price cut trajectory as GPT-5.6 Sol -> GPT-6.1 Sol. Overall, Terra is still half the price of Sol.
But like the original release of Terra, it's still a weird middle child between the small-model-that-got-substantially-better (Luna) and the large-model-that-got-significiantly-cheaper (Sol). The TerminalBench 4.0 cost-per-task results are a bit confusing and I'm not sure why you would pick Terra over Sol in that case. Overall this release seems like a flex on Sonnet 5.5: a model which is having its own middle child problems between the godly Opus 5.5 and the surprisingly capable Haiku 5.5.
rockdoe 24 hours ago [-]
the surprisingly capable Haiku 5.5
Destroyed by Anthropics crazy decision to price it x5 after 100k context.
michaellee8 20 hours ago [-]
do people really use luna post 100k? small models generally seems to be hallucinating hard for me once you are over that threshold.
rockdoe 43 minutes ago [-]
Due the the size of the default system prompt you get over 100k really in easy in Claude, and because that is fixed 200k is really almost 3x more room to work with, not 2x.
200k also used to be the limit for Sonnet for a long time.
If Haiku 5.5 also had 200k, it'd actually be usable for us. As it is, it's Luna or Sol.
NewJazz 16 hours ago [-]
On purpose? I was using qwen3.8-27b the other day and very quickly ended up burning through half of the context (it has 128k total) pretty quickly. Not because of my directions, but simply because the codebase was very large.
tepmoc 13 hours ago [-]
256k seems sweet spot. Even large codebase can be handled you just has to map out areas for agent. Anything past 256k usually exception than norm to be.
At least this my discovery when I did evals on my reviewers skills. 10k LOC is high end for high effort or around 150k context using opus. You can push towards 15k but that’s on max effort with about 400-500k ctx.
Fast models like haiku do not even need 1M ctx in my view
NewJazz 4 hours ago [-]
Yeah codebase structure is very important. Admittedly, by large I mean average size but very poorly structured with a spaghetti (and sometimes cyclical) import graph.
zahlman 13 hours ago [-]
Is there a straightforward way to check the current context window usage in the web chat interfaces, btw?
rockdoe 44 minutes ago [-]
In Claude it used to be the circle on the lower bottom, but now it's a bit more hidden and you have to expand the sidebar.
NewJazz 5 hours ago [-]
I don't know, I usually use Pi.
toddmorey 21 hours ago [-]
Packaging & pricing seem to be two things Anthropic really struggles with
zozbot234 21 hours ago [-]
Artificial Analysis still has MiMo 2.6 Pro setting the Pareto frontier where it matters: it's as cheap as Terra High/Sol Low while being smarter than Terra Xhigh. Sol Medium beats MiMo in smarts but is priced a lot higher. Meanwhile Terra Max is dominated by Sol High in both price-per-task and smarts.
killingtime74 15 hours ago [-]
You wouldn't believe any of that if you've used MiMo 2.6 Pro. DeepSeek 4.1 flash 1017 is better in actual use in every regard.
spider-mario 12 hours ago [-]
I recently tried MiMo 2.6 Pro on a simple app and was positively impressed (modular code with sensible tests). I then tried it on a more difficult problem (which Opus 5.5 solved in minutes) and it looped for 12 hours without making any apparent progress before I finally pulled the plug.
bdangubic 20 hours ago [-]
I just imagined reading this comment like 2.5 years ago :)
aistripes 6 hours ago [-]
[dead]
frisbee6152 4 hours ago [-]
For technical work, I’m inclined to agree. I’m a fan of Luna and Sol, and Astra has its place, but I’ve always found Terra an unhappy medium.
If Luna is not good enough for a task, then the marginal improvement of Terra tends not to be worthwhile. If it is worthwhile, then the jump to Sol tends to also be worthwhile, so I tend not to use Terra.
While it does offer some meaningful improvements over 6-Luna for a few workflows, I suspect most of those benefits stem from 6.1-vs-6, as opposed to Terra-vs-Luna.
wyre 1 days ago [-]
I don't think software devs are the audience for 6.1 Terra, but instead computer work, automation, and technical writing.
Imustaskforhelp 1 days ago [-]
Honestly I'd much rather take this model then.
It is having improvements in other fields than coding while being more cost efficient.
If all I want is coding at a cheaper price, then deepseek is a good model for that.
I am more than okay to have a bifurcation of models good for coding and models good for everything other than coding. I strangely imagined it many years ago.
Everyone is suggesting OpenAI to release a model close to opus 5.5 yet it's releasing this. On one hand we can imagine that maybe their training cycle wasn't able to produce this whereas gpt 6.1 terra was made.
But on the other hand, I also now wish to see an OpenAI model which is good for just pure coding, like an absolute beast at it while being cheap (so the opposite of terra)
ironically, one of the few appreciations that I might give to OpenAI if they pushed this model to prod and are facing some pushback as developers like us want/only see coding models. They could've benchmaxed it and it would've been slightly irrelevant. I'd much rather see reality.
At the very least, this new model won't make things more expensive for anybody and would only make use-cases cheaper depending on what they are, so it's a fine deal for me.
I also wonder maybe this is why open source labs aren't targeting this space and why the only focus of AI labs is coding nowadays. Everything else just doesn't seem so worth it from investor, even end user, training perspective.
Maybe this is the reason why we have gotten almost an exponential growth in coding capabilities as that became the most (if not only) capability that truly really matters.
To not wish for discussion to go to a tangent, nonetheless this could also explain why our work as a software engineer is going through a slight crisis of meaning as these AI models have gotten impressive at coding.
As an (aspiring) developer (currently in 1st year college), I don't wish for AI to replace coding even more for me or to push it even more or to suck the joy out of it even further. So I am more than happy for these models to stay as they are and rather make advancements elseplace. Ironic because it is us developers who use AI as well and we want cheaper workflows and so we would also want models to be cheaper and better at coding. There are some fine lines of contradictions and irony in all of it. I'd much rather also have open-weight models of such terra-like models but I imagine that because of the reasons that I mentioned above, we are most likely gonna see it the most focused on coding but still open-weights is pretty close.
The more choices the better (though choice-fatigue is a real thing), I hope I was able to explain some of the nuance I feel about this particular launch.
SomeonesAccount 1 days ago [-]
Well put.
Someone1234 1 days ago [-]
I'm very confused about their model market position/naming schema.
First we had:
> Luna (Small), Terra (Medium), Sol (Big).
Which was fairly smooth after you got used to their planets.
But then they added Astra as the "Bigger," which was mostly still self-coherent:
> Luna (Small), Terra (Medium), Sol (Big), Astra (Bigger)
But then things go confusing, they released Sol-6.1 which sometimes is superior to Astra, and discontinued Terra which made it look like this:
> Luna (Small), Sol-6.1 == Astra
But with Sol-6.1 costing the old Terra-6.0 price, while, as I said performing as good as the old "Bigger" model in many non-3D/UI situations.
Now they're re-introduced Terra, as a 6.1 release, making this even more confusing.
Did Microsoft's chief branding officer move over to OpenAI?
tedsanders 1 days ago [-]
I work at OpenAI and I can try to explain our logic.
In general, LLM performance is incredibly multidimensional. There's no guarantee of a fixed ordering where the model that is best at math is also the model that's best at jokes. Despite that complexity, much of the variation in performance is explained by just a single variable: model size. In general, models that are bigger will (a) be more performant / reliable, (b) cost more per token, and (c) have better recall of esoteric facts. As you note, our models are ordered:
Luna < Terra < Sol < Astra
Even today, I'd recommend this as a general starting point for picking models. If you care about quality only, pick Astra. If you're a bit price sensitive, pick Sol. If you're very price sensitive, pick Terra or Luna.
Of course, model size doesn't capture all variation. Models that are trained later might have new data or new techniques or new architectures that earlier models lack. You can see this with models from all providers - Sonnet 5.5 outperforms Fable 5.1 in ways, and Gemini 3.8 Flash outperforms 3.1 Pro in ways.
In the case of GPT-6.1 Terra, I'd say Terra is probably better than Astra and Sol at technical communication, similar on vision and computer use (but much faster!), and a bit worse on most other things. For API developers, it's a nice midrange option if you want something cheaper than Sol but a lot smarter than Luna (especially for vision / computer use / communication). For subscribers, it's not a bombshell release - if you're hitting usage limits, maybe it helps you go a bit further, and if you want clearer communication or faster computer use, it can be a nice option to select.
If you have suggestions on how to name our products better, I'm all ears. Our intention is to make things clear and simple, even if we miss the mark sometimes.
Unfortunately, a lot of the complexity is somewhat irreducible and due to to the actual complexity of the products. Scaling model size is different from scaling reasoning effort which is different from scaling multi-agent count. So if we just swept all of those under a single name of "more IQ", it would make the naming simpler but the experience much worse, as important details would be obfuscated from end users.
atombender 1 days ago [-]
Do you then recommend Sol for technical documentation?
What's confusing to me in addition to the naming is that there are no published guidelines I can use, as far as I can see. Put yourself in the user's shoes. On what basis should one pick a model for a given task?
I can understand smaller models being dumber and faster. That aligns well with scenarios where you are willing to compromise; e.g. you're classifying content or asking a model to format text into a table.
But when you want "maximum intelligence, cost and time be damned", there's the implication that bigger models are smarter on a linear scale, and every provider's model release naming is currently conflicting with that expectation.
This isn't just a naming problem, but a communication problem. If I knew that Astra was the best at coding but a bit bad at writing, and Sol was almost as smart but better at technical writing, then that's the kind of thing I expect to be documented.
One possible way: A simple matrix with horizontal columns like "writing", "coding", "math", rows are the models, and each box is a number of stars from 1-5.
Instead, this sort of thing becomes lore, shared by word of mouth.
tedsanders 1 days ago [-]
> Do you then recommend Sol for technical documentation?
I don't know, to be honest. Technical documentation is such a multidimensional skill, and I also don't know your budget. I'd imagine Terra might have the most readable prose, but Astra might be more likely to catch bugs or mistakes to highlight. If the documentation is about a more esoteric language or situation, Astra might have a greater edge than if it's a common language or situation. If you have a long list of documentation guidelines, that might also give more of an edge to Astra. I wouldn't want to make a recommendation without doing a lot of testing, and I haven't done a lot of testing.
flockonus 22 hours ago [-]
Some food for thought, if someone from oAI can't make a direct recommendation for something as direct as "Technical documentation is such a multidimensional skill" - how could we know assess what model to use?
tedsanders 22 hours ago [-]
To elaborate, I can ask each model to write a doc about something I know well and tell you which output I like better.
But to assess whether my preference on my problem is going to generalize across other people, other programming languages, other documentation types/formats, etc., and then analyze statistical significance is a thornier question that I don't have the data to answer. I'd rather make no recommendation than a weakly informed recommendation.
mrandish 5 hours ago [-]
> I can ask each model to write a doc about something I know well and tell you which output I like better.
That's what users currently end up having to do on their own. Or they just give up in confusion, which presents in your site analytics as either bounce, buy later or buy less.
Why not have a model comparison site? Visitors choose from a dozen-ish broad categories of work (eg technical documentation, market communications, etc), then pick from a dozen sample prompts across typical tasks in that category, and see the model responses with avg time and approximate cost. Especially helpful would be having a toggle overlay with annotation highlighting relevant differences, then if they have (or create) an account, they can edit the annotations and add their own with a sharing link.
If my job was to actually help my co-workers and friends make better informed choices they feel more confident in, I'd be doing something like that. The extra credit version would help users narrow down to two models and then let them convert to a 'Test Drive' trial - same workflow but from their own prompts, side-by-side A/B response comparisons with annotation and sharing link.
As you said, the answer to "Which model is best for me?" is actually complicated, conditional and inevitably a bit confusing. When that's the reality, the right pattern isn't 'Communicate to Convert', it's 'Build credibility and confidence with maximum clarity and genuine helpfulness'.
People vote on pairs of model responses and they aggregate everyone's votes into a leaderboard.
One thing to be aware of that the distribution of prompts on arena.ai can be pretty different than real-world usage. The people who spend time on that website casting votes tend to be more interested in entertainment than professional productivity. And turn-based voting can punish useful behavior like asking clarifying questions instead of guessing at assumptions.
mrandish 5 hours ago [-]
Yes, I've seen Arena.ai and the reasons you cited are why I suggested something better that's more private, personalized and shareable internally without being skewed and gamed.
scarab92 22 hours ago [-]
I use Astra 6 Extra High for everything.
It’s overkill most of the time, but frankly all these models are “cheap”, and I’d rather waste compute than brain cycles trying to discern which dials are best for each individual as-hoc query.
selestify 21 hours ago [-]
How are they cheap? Does your organization offer unlimited spend, or does your subscription not run into the same limits as mine?
hsuduebc2 20 hours ago [-]
Maybe if you have subsidized 200/500 dollars subscription it may look that way.
junnan 14 hours ago [-]
But Astra is simply bad at writing. I have to use Argon to rewrite everything Astra wrote in an academic setting. Astra always tries to cover every small detail and engages in defensive writing that is unreadable.
thirtygeo 16 hours ago [-]
Totally agree. I'm working a lot faster when being just about everything I want on the first hop, without model switching and goofing around to find the sweet spot. When I need a janky answer I run a model locally with llama.cpp
noahmichael89 22 hours ago [-]
[flagged]
lazharichir 1 days ago [-]
> Put yourself in the user's shoes. On what basis should one pick a model for a given task?
Create a realistic eval suite for your use case using your actual data that you manually label, with easier, more complex, rarer samples, and run each model you are considering against it and decide based on the usual confusion matrix, cost, latency, or whatever metrics matter to you. There's really no other way to know – and you might be surprised that cheaper models can often perform your task(s) relatively well at a fraction of the cost.
awb 23 hours ago [-]
> Create a realistic eval suite for your use case using your actual data that you manually label, with easier, more complex, rarer samples, and run each model you are considering against it and decide based on the usual confusion matrix, cost, latency, or whatever metrics matter to you.
I’d imagine there’s only a tiny percentage of users that can or want to do this.
aacid 12 hours ago [-]
if you consider every single person that uses some LLM then sure, it is tiny percentage, but I would wage that large percentage don't write technical documentation as GP. large userbase probably don't need THE best solution for their use case and will be comfortable with A solution that is within their budget. so for them simple luna < terra < sol < astra would suffice.
but if you want to use some LLM agent for anything commercial and you want to get the best and at the same time you don't want to overpay the easiest and the surest would be to create your own benchmark for your specific use case.
CSMastermind 1 days ago [-]
> Do you then recommend Sol for technical documentation?
Every model I've tried is just abysmal at this, for what it's worth.
If any do it well please let me know and I'll give it a spin.
zenoprax 24 hours ago [-]
My thoughts exactly.
I recently experimented with some image generation models and I was surprised at how much context/prompting is needed to get something that doesn't look Lovecraftian. There was seemingly no downside to adding more and more detail up to the limit. Prose sits between code and images in terms of fuzziness and I wonder if promptmaxxing is the winning strategy for technical documentation.
I've been thinking that fine-tuning a model to write differently shouldn't be that hard (tedious, perhaps) and if I had the resources I would consider doing it myself.
ilaksh 1 days ago [-]
You can start with Artificial Analysis benchmarks but, then you need to test in the real tasks anyway.
no-name-here 14 hours ago [-]
> On what basis should one pick a model for a given task?
> If you're not sure where to start, use GPT-6 Astra, our flagship model for complex reasoning and coding. Choose GPT-6.1 Sol to balance intelligence and cost, or GPT-6 Luna for cost-sensitive, high-volume workloads.
That page also seems to be updated after each new release.
13 hours ago [-]
saagarjha 1 days ago [-]
I assume having a person write words for other people to read is not feasible?
atombender 1 days ago [-]
Humans cannot be in the loop for autonomous work. I have agents review code and design/architecture documents, write alert messages based on rules, write analyses (e.g. outage incidents), and many other things. "Having a person write words" would mean not using agents, defeating the whole point.
I do not use agents to write on behalf of myself. I write my won technical write-ups and my own slide presentations. A lot of people don't; and it shows. There's much that agents can do better than humans now, but not everything.
saagarjha 24 hours ago [-]
This isn't technical documentation though, it's just notes the agents are keeping for each other. I think that is a very different task
atombender 23 hours ago [-]
It's human-facing.
GodelNumbering 1 days ago [-]
> I'm all ears. Our intention to make things clear and simple, even if we miss the mark sometimes.
Thanks. I think the very response you wrote here is a better form of marketing for the software engineers than what openAI puts out there. When everything is groundbreaking and everything tops charts, nothing is.
I suspect a lot of target demographic (developers) prefer simpler and honest communication about not just the models high points but low points too. With the linked Terra post, if I saw 1 chart that clearly highlighted the usecases for which Terra is _worse_ than higher tier models, I would be more convinced overall.
Everything trying to occupy the top spot in my brain's limited "top model" representation space is not helpful.
vineyardmike 1 days ago [-]
While I generally agree that it'd be nice to just have "here is the Pareto optimal model for your use case. Here is the specifics we optimized each model for" so users don't have to do benchmarking and research as much, they're getting much better at explaining. It's basically what the commenter above said.
The website linked starts out with:
> We’re introducing GPT-6.1 Terra, a fast, efficient model that combines detailed visual reasoning and precise computer use alongside clearer, more natural writing. A significant step up from GPT‑5.6 Terra, it excels at the communication and computer-based tasks that fill the workday, from drafting proposals and interpreting technical drawings to filling out forms and editing videos.
If you go father down, it has graphs and details with specifics of this, including:
> We measured GPT-6.1 Terra’s computer-use capabilities... At medium reasoning effort, GPT-6.1 Terra scores 15 percentage points higher than GPT-5.6 Terra at about one-third the cost. It even performs on par with GPT-6 Astra at only one-tenth its price per token.
and
> On MMMU-Pro... It even nearly matches the scores of our most capable frontier model, GPT-6 Astra, and GPT-6.1 Sol at much lower cost.
elxr 1 days ago [-]
The 4-tier naming is honestly as clear as you can get. I have no idea what people are so confused about.
Terra and Luna are dirt cheap and good enough for 50% (or more) of the typical junior dev's workload, and great for routine work. Sol and Astra have so much written about them at this point that it should be obvious what those are good at. The large/gigantic models for things that need full intelligence and slightly more branching thinking.
margalabargala 1 days ago [-]
The obnoxious/confusing thing is that models are released individually rather than as a family.
Luna < Terra < Sol < Astra, fine. But does Terra 6.1 beat Sol 6? Does Fable 5.1 beat Opus 5.5? If I build something around this Terra 6.1 release, then when Luna 6.2 is released in a month, should I swap?
cdirkx 1 days ago [-]
I understand the confusion, but isn't this kind of similar to asking does the iPhone 14 Max beat the iPhone 16 Mini? At least with phones there are hard specs we can use to compare, but in a lot of cases the answer will be "it depends"
Closi 1 days ago [-]
At least there is a simple system - the words 'Mini' and 'Max' are intended to refer to the size of the item (that one is miniature, and the other is their maximum size).
If they were called iPhone 14 Luna and iPhone 16 Sol then this is less clear.
Or on the flipside, if the models were called GPT 6.1 Mini and GPT 6.1 Max, nobody would be arguing about the names being hard to grok. The words 'mini' or 'small' are more descriptive than 'Luna' or 'Terra'.
lukeschlather 24 hours ago [-]
The problem with that is then you would have GPT 6.1 Max with max thinking which is different from GPT 6.1 Small with max thinking.
If you look back the naming problem started around the point that OpenAI started doing reasoning models (o1, 4o) where they had to describe this fork in model behavior that let a smaller model think longer and be smarter.
You can come up with naming schemes that don't clash, but there's something to be said for this approach where the thinking level is like you describe and the model name requires you to learn that Astra is the largest model.
rpeden 21 hours ago [-]
I guess it never really occurred to me that people wouldn't immediately grok:
* Luna (Moon)
* Terra (Earth)
* Sol (Sun)
* Astra (Galaxy/Universe)
The relative sizing seems dead simple.
no-name-here 14 hours ago [-]
> dead simple.
1. Even more dead simple if it just used Moon instead of Luna, Sun instead of Sol, etc. - one less quick translation in the head.
2. Or even more dead simple if it just used "Small", "Large", etc. - removing another quick translation in the head. (On the other hand, all these suggestions are English - maybe Luna, etc. is more understandable for non-English speakers?)
Closi 3 hours ago [-]
100%! The other problem is that the next model release from OpenAI there will be another different 4 terms.
Closi 3 hours ago [-]
I mean Astra actually means star, but putting that aside, we also have GPT sunburst and GPT flare - how does the logic apply to these?
Is a sunburst supposed to be bigger than a flare? I mean you get solar flares too? Or is a solar flare bigger than a sun 'burst' which usually just means some sun coming through clouds.
margalabargala 20 hours ago [-]
You're misunderstanding the misunderstanding.
You're right! The relative sizing of models is dead simple. This doesn't help when there are improvements between release numbers, and models are released one at a time instead of as a family.
When Sol 6.4 comes out and the latest Astra is still 6.1, which do you use?
arcfour 18 hours ago [-]
I use Sol 6.4 because it's newer (and "big") unless there is a specific benchmark that 6.1 Astra scores better on that I think I need.
Or I might try both and pick whichever I prefer for a given task...
margalabargala 18 hours ago [-]
Okay, now how many generations until you use Terra over Astra?
arcfour 17 hours ago [-]
Probably never? Unless they get super far out of sync, a "medium" model just isn't the right choice for a "large" (or "extra large") model task.
margalabargala 24 hours ago [-]
> does the iPhone 14 Max beat the iPhone 16 Mini?
Sure, in a hypothetical world where the latest Max phone is two generations older than the latest Mini phone. In which case now you have to evaluate not only screen size but ALSO specs and component age and software support etc. How badly do you need the bigger screen? Etc
If you are comparing the iPhone 16 Max to the iPhone 16 Mini, everything is a lot easier.
saagarjha 1 days ago [-]
> iPhone 16 Mini
I wish
10 hours ago [-]
XCSme 1 days ago [-]
Yeah, but shouldn't then all 4 tiers be released at once? So we get 6.1 for all Luna/Terra/Sol/Astra?
nailer 1 days ago [-]
It's more like if Apple called the iPhone 16 the "iPhone 14 16 edition"
kridsdale1 1 days ago [-]
I’d rock an iPhone 1942 Edition.
someguyiguess 1 days ago [-]
Don’t forget to refill the kerosene tank!
breezybottom 20 hours ago [-]
I mean yes, phone manufacturers have the same problem.
someguyiguess 1 days ago [-]
No. It’s not similar at all.
elxr 1 days ago [-]
Run your own evals then if you have this much of a need to "correctly" choose. That'll make it clear.
I'll just stick to mostly Sol on low/medium/high and start experimenting with Terra tomorrow onward to see what it does noticeably better than Luna/Deepseek flash.
Any pareto frontier model, especially from the leading labs is guarnateed to be good enough for whatever your expectations are at this point. A quick few day test should be enough for anyone to make their own conclusions.
margalabargala 23 hours ago [-]
Right, having products so poorly described that a customer needs to run their own evals to choose well is exactly what is being complained about.
1 days ago [-]
solyin 1 days ago [-]
Or I'm going to give up and not use any. Consumers aren't perfectly rational. This isn't like toothpaste, where, despite being overwhelmed at the store, I still need to brush my teeth so ahhhhhh fuckit and just pick a toothpaste out of the 30 available. The alternative is I keep using Google search and think AI is stupid. Or just use Facebook's thing.
2III7 22 hours ago [-]
The frontier toothpaste is the one with bioglass 45S5. Especially if you have sensitive teeth or your teeth are prone to caries.
mc3301 20 hours ago [-]
Nano-HAp is good, too
imtringued 12 hours ago [-]
The irony with tooth paste is that out of those 30, probably 28 are exactly identical. If all you want is fluoride tooth paste then pick the cheapest tooth paste.
If you want to go optimize your mouth hygiene it is actually more important to use dental bristles (flossing is not as good), because most of the work is done by the mechanical cleaning action of the tooth brush and you need to reach the places where the tooth brush cant get into.
Once you have done that, the next step is alcohol-free xylitol based mouth wash. Caries bacteria have trouble digesting xylitol so this reduces the presence of them in your mouth.
Only after you have done that, are you ready for something better than fluoride. Fluoride is a remineralization agent. Your saliva also contains calcium that helps your teeth remineralize. So the next thing you need is a more potent remineralization agent. Basically your tooth paste acts as a more concentrated saliva. Because of that, it is actually better to use e.g. fluoride in the morning and a different agent in the evening.
The classic options are bioglass (novamin) or hydroxyapatite. The latter is also available as mouth wash.
nailer 1 days ago [-]
> > But does Terra 6.1 beat Sol 6?
> Run your own evals then if you have this much of a need to "correctly" choose.
No. OpenAI want my money, it's on them to communicate properly.
nharada 1 days ago [-]
What would you want to see instead? Not sure about you but my needs are specific enough that I don't think OAI has evals for them, and I wouldn't want to give them that data even if they wanted to run it for me.
nailer 1 days ago [-]
“ChatGPT 6.1 Max” would be simple.
austhrow743 22 hours ago [-]
Then you're going to have ChatGPT 6.1 Max Low and ChatGPT 6.1 Max Max.
nailer 2 hours ago [-]
No you're not.
austhrow743 38 minutes ago [-]
You're suggesting renaming the effort level as well?
ssl-3 1 days ago [-]
How does one more-properly articulate "it depends"?
nailer 2 hours ago [-]
"New model, middle version" is already communicated well by Apple.
0-_-0 1 days ago [-]
On what?
Majromax 19 hours ago [-]
> Luna < Terra < Sol < Astra, fine. But does Terra 6.1 beat Sol 6? Does Fable 5.1 beat Opus 5.5? If I build something around this Terra 6.1 release, then when Luna 6.2 is released in a month, should I swap?
I understand how it can be confusing, but I don't think that this is exactly a problem.
Luna through Astra offer qualitative descriptions of the model's size. Presumably OpenAI trains its biggest model (Astra-ish, maybe even larger) first, then distills it to smaller sizes.
Verison numbers reflect something about the architecture or training dataset. Again, they keep these details close to their chest, but we should expect that weight-for-weight higher versions should beat lower versions.
This offers us two axes of 'better', but these axes are also not directly comparable. An earlier but bigger model may or may not beat a newer but smaller model. This is where 'spiky' intelligence comes in.
This is also where calls to 'run your own evaluation' really do start to matter. If you care enough about this performance[1], then you also have specific problems that you care about. No generic benchmark can fully answer that question, hence the need for a specific evaluation.
[1] — And you don't have to! A casual user is probably served reasonably well by anything put out in the past six months. I think Microsoft is still serving GPT-4o under some Copilot branding.
margalabargala 19 hours ago [-]
I think it is a problem, for OpenAI. If their answer to "which of your models are better" is to throw their hands in the air and say "fuck if we know, go figure it out yourself" then that's going to go poorly as soon as they try to actually sell to some of the market they're predicting they can sell to.
Whether they train the biggest model first or not is not relevant; what matters is what they release. Right now we've got 6.1 Sol and 6.0 Astra. Over at Anthropic they've got Opus 5.5 and Fable 5.1. The existence of a hypothetical Fable 5.5 that I'm not allowed to use doesn't help me.
at1as 1 days ago [-]
> The 4-tier naming is honestly as clear as you can get. I have no idea what people are so confused about.
I wouldn't necessarily say that. Atmospheric mass may be obvious to us, but I suspect it's not nearly as universal knowledge as one may assume.
And I actually assumed at first that they were based on distance from us (terra < luna < sol), so I kept confusing terra and luna.
someguyiguess 1 days ago [-]
It’s totally clear. About as clear as mud.
torben-friis 1 days ago [-]
I think the main problem is surfacing two topics:
- Understanding how effort and class interact (Is a high effort terra above a low effort Sol? Are larger models mostly always better regardless of effort?)
- understanding when (if ever) a new model tends to be better than an older but higher class one. Is an old astra better than a new terra?
If increasing effort and increasing model class produce improved results in a different way, I miss making it explicit in which way. In other words: What kind of issue warrants going up in effort? what kind of issue warrants going up in model class? What kind of issue is unlikely to be solved by either?
tedd4u 1 days ago [-]
OpenAI itself cannot answer these questions. A prospective price-sensitive (or speed-sensitive) user has to experiment to find the cheapest (or fastest) model that is sufficient for each task.
someguyiguess 1 days ago [-]
Right. Because they don’t know. They’re just pumping out crap desperately trying to stop hemorrhaging billions.
Artificial Analysis has their own curves which are pretty similar.
irthomasthomas 1 days ago [-]
That's good, but they haven't updated it for a month. It is missing all of the current frontier models.
aftbit 1 days ago [-]
Seems to be missing all of the 6.0 and 6.1 models?
dist-epoch 1 days ago [-]
> Is a high effort terra above a low effort Sol
You can literally see this in the charts from the article
mrandish 6 hours ago [-]
> Luna < Terra < Sol < Astra
Ted,
I'm assuming you're not in the OAI marketing dept because leading with a simple thumbnail like "Luna < Terra < Sol < Astra" is helpful framing and dropping next-level bombs like:
> "Terra is probably better than Astra and Sol at technical communication, similar on vision and computer use (but much faster!), and a bit worse on most other things. For API developers, it's a nice midrange option if you want something cheaper than Sol but a lot smarter than Luna (especially for vision / computer use / communication). For subscribers, it's not a bombshell release - if you're hitting usage limits, maybe it helps you go a bit further, and if you want clearer communication or faster computer use, it can be a nice option to select."
Is just far too clear and useful! I'm kidding... but it's funny because it's a little bit true.
> Our intention is to make things clear and simple
Your post is the single best product marketing communication I've ever seen from OAI. Serious suggestion: maybe just say something like that on the website. Not buried in a blog post but above the fold.
mckirk 1 days ago [-]
Since internally you probably have a fairly good intuition for what these models do well, what about assembling a benchmark suite (with multiple dimensions) that produces results that show that intuition, but in a 'somewhat objective' way?
That could solve the 'more IQ' problem, because you'd actually have different dimensions (e.g. for 'raw intelligence', technical writing, emotional understanding, etc) that you could use to communicate the differences in models better.
(Of course people would instantly start complaining that the benchmarks numbers are probably fake etc, but it is what it is.)
Yajirobe 1 days ago [-]
I think different domain benchmarks already exist and are widely used.
mckirk 1 days ago [-]
True, but afaik none that an AI company itself uses to say 'this is what we think showcases the individual strengths of our models the best'.
Iolaum 1 days ago [-]
It's not gonna happen but:
> Despite that complexity, much of the variation in performance is explained by just a single variable: model size.
please release model size :p
__jl__ 1 days ago [-]
Internal evals are still running but GPT-6.1 Terra is an AWESOME release for anyone using the API in agentic production applications where cost matters.
Personally, I pretty much exclusively use claude for coding work (sorry!). Our application is running almost 100% on OpenAI API (except for some fallback models). It is the best API on the market. The models are excellent particularly for any application that is cost sensitive.
user_7832 7 hours ago [-]
> Of course, model size doesn't capture all variation. Models that are trained later might have new data or new techniques or new architectures that earlier models lack. You can see this with models from all providers - Sonnet 5.5 outperforms Fable 5.1 in ways, and Gemini 3.8 Flash outperforms 3.1 Pro in ways.
Is this really so/could you give a rough example please? I've not used a lot of Claude or ChatGPT, but 3.1 Pro always feels most grounded and intelligent. 3.8 Flash (in my very limited testing) isn't bad, but it feels less nuanced.
(Fwiw I don't use it for code, but ask maths and real world logic based questions.)
robswc 18 hours ago [-]
> If you have suggestions on how to name our products better, I'm all ears. Our intention is to make things clear and simple, even if we miss the mark sometimes.
I'm not the one you were talking with but catching you is godsend since I just ran into a lot of confusion on the naming myself. I think some of it has to do with how they are labeled in the app.
I think simple numbers in naming is great, even 6.1 is fine bc I understand that if I want the latest model, I just need to look for the biggest number. Where I got confused is in the "work" tab, I saw 'Astra' is set as the default. I see the label for it is:
"Frontier intelligence for your most demanding tasks."
To me that reads "this is the 'best' model." Then I see Sol-6.1 which is labeled:
"For coding and every day work."
Then Terra-6.1 is:
"Terra 6.1 Codex Model" (tbh this one might be the most confusing, I'm aware of codex the product but Sol is the one labeled good at coding)
From my POV as a basic plan-based user wanting to do something like...
> design a robust event-driven system, here's the details [...]
I honestly have no idea which one I should use.
Would Astra-6 produce a better design than Sol-6.1? Is the '.1' a big step up from Sol-6? (I'm guessing no but I truly have no idea). I would assume the answer is Astra-6 would produce the best one, even though the labels don't imply it is necessarily geared towards coding?
But tbh the "default" seems to always work for me... and after reading your comment, typing this one out, being forced to think about it... it doesn't seem too bad given your challenges in communicating the purposes of each model and wide audience of consumers. I'm also sure anyone could find a million problems with decisions I've made in software I've built so I try to not be too harsh :)
qhwudbebd 1 days ago [-]
Please can I ask about one parameter you didn't mention here? All of your models have both a reasoning effort setting and also a non-pro vs pro setting. I have some intuitive feel (from experimenting) for the difference a larger vs smaller model makes, and why that's different from adjusting reasoning levels, but I have no real feel for how pro vs non-pro is different in behaviour from adjusting effort.
mnicky 1 days ago [-]
I think you can think of Pro approximately as running multiple instances of a non-pro model, than giving all their outputs to a final judge model which combines them into a better one.
sgerenser 1 days ago [-]
I use OpenAI models all the time (via Codex) and I've never seen this pro toggle. Where are you seeing that?
qhwudbebd 1 hours ago [-]
It's reasoning.mode, next to reasoning.effort in the responses API. Set to the string "pro" for pro mode. Sorry, I don't use first-party harnesses so have no idea what it maps to there.
p0sixlang 9 hours ago [-]
I believe it is specific to chat mode.
phonon 20 hours ago [-]
[dead]
osigurdson 21 hours ago [-]
>> If you have suggestions on how to name our products better, I'm all ears
<version>-<size>
Where <version>: 6.0, 6.1, etc., and <size>: xs | sm | md | lg
E.g. 6.1 Terra = 6.1-sm
If you eventually make specializations like "math" and "jokes" then I would call it something like 6.1-jokes-sm.
6thbit 1 days ago [-]
Instead of benchkmarks it'd be useful to get a sort of Radar chart across measured capabilities like computer use, communication, coding, etc.
We're probably headed to further specialized agents where a single ordering or even model size won't mean a lot.
pietz 1 days ago [-]
This was lovely. Can we have more of this style of communication?
ceroxylon 1 days ago [-]
It would be interesting to know which model is internally viewed as being best at jokes, I have a personal humor benchmark and so far models will either steal jokes (from comedians like Maria Bamford), or write something very unfunny.
CompoundEyes 1 days ago [-]
When I was first learning to have agents interact I had a little script to orchestrate a model telling a joke, a critic collaborator helping develop material and an audience emoji rank them.
A favorite…
Prompt: “Tell a joke about mother in laws”
Gemma2 response: “What’s the best kind of mother in law? A dead one.” 0_o
ortonomy_ 15 hours ago [-]
Try not using putting two variables in the product name. Stagger the release and make sure you don’t release a more up to date version in the middle so that it looks like a “medium” model is more advanced than a “big” model. People see bigger numbers they think better.
Medium v10 vs Big v2 makes it look weird and confusing. Especially when there are a sea of influencers and twitters with vibe assessments who will tell you new models best olds models.
Can’t you just start at the top or bottom.
Release new big for paying customers -> release new medium for paying customers -> release new small for everyone
Rinse and repeat. We’d all be much happier
nish__ 18 hours ago [-]
> In general, LLM performance is incredibly multidimensional. There's no guarantee of a fixed ordering where the model that is best at math is also the model that's best at jokes.
Maybe stop using version numbers then and just name models like you'd name children.
ksec 22 hours ago [-]
Thank You for OP asking the question and thank you parents for some insight.
>If you have suggestions on how to name our products better......
Too late for that. I hope you guys can at least stick to these naming scheme for at least another 2-3 years going forward. Not everyone follows AI / LLM intensely, and it seems there is a new naming changes every year or so.
At least conceptually OpenAI's naming is easier to understand than Claude.
It terms of capability, it would be great to make it clear somewhere that this is 6.1 Model "+" technical communication, vision and computer use improvements. So that at least we know what we should try with these new release and choose a better model for certain workload.
Right now, other than pricing, it is confusing as hell. And in hindsight I wonder if there are fatigue in choosing one people just go with Open Model and stick with it if it is good enough.
mchusma 1 days ago [-]
I actually do have a recommendation, which is to do something like a new mode (i know i know):
Auto xlow
Auto low
Auto med
Auto high
Auto xhigh
Auto max
With an optional flash mode.
Then as you release models, you pick the model and thinking level that you believe is pareto optimal for cost/performance.
So this auto updates each release, like Terra was probably off this list entirely, but maybe this would appear now in Auto medium or something like that.
I know from looking at charts that each model seems to have sweet spots (like I always use this model in this thinking level and if i want to go cheaper i go here). If you picked those for us, it would be interesting. I do think you should keep the model families for those wanting more consistent experiences, but for many devs they may want to just pick auto something and have it update over time. I think nice if it actually just said in parenthesis what it was, something like:
Auto Medium (Terra 6.1 High)
I am not sure if everyone would agree here though. I'm obivously advocating for more naming schemes, and i understand the irony.
mchusma 1 days ago [-]
Just to be clear, I think the current mapping for OpenAI is something like:
xlow = Luna Low
low = Luna xhigh
medium = Terra medium
high = Sol medium
xhigh = Sol xhigh
Max = Astra something?
throw20261022 6 hours ago [-]
From an API pov, the reasoning effort is entirely orthogonal to the model. From a ChatGPT (Chat mode) pov, there might be something to what you say.
NikolaNovak 1 days ago [-]
Appreciate that these things have orthogonal measurements; with that in mind, given that the name signifies size, what does version number signify?
I.e. are all version numbers independent and Model X 7.5 is not necessarily of the same generation or cut off as Model Y 7.5? Or do all of your 6.1 models have something in common - cut off data, generation, baseline technology, approach, architecture, or something else?
Thanks! :)
tedsanders 1 days ago [-]
These days, version number signifies a few things at once: the order of release, the relationship to other releases, and the magnitude of improvement. So something like 6.1 would indicate it was launched after 5.6, may share some similarities to other 6.1 models in terms of training data / personality (which is why it's not 5.7), and isn't a giant enough leap to merit the headlines and expectations that would come with a 7.
Back in the day we used numbers to also sometimes indicate size, which could make it extra confusing: e.g., we launched GPT-4.1 (an incremental update on GPT-4) after GPT-4.5 (a new model that was a big jump over GPT-4).
jonplackett 1 days ago [-]
This makes a lot of sense.
It’s probably just unavoidable. Like when Apple release a new MacBook Air first and it is (in some ways) faster than the old MacBook Pro.
I think the planets thing is a bit confusing though. I hadn’t really thought about it being based on the size of the planet. I assumed it was based on how far away they were with terra therefore being the base one.
miki123211 1 days ago [-]
What makes this even more confusing is reasoning effort.
You have Luna, Terra, Sol and Astra, each with 4+ reasoning levels. That's ~20 options to choose from.
On a repetitive task like customer support or something, where you have decent benchmarks, you can plot cost versus performance, but you can't really do that on coding.
sin4ch 13 hours ago [-]
Any news on when Terra gets into the hands of Free and Go users, maybe replacing 5.6 Terra?
Also, I think the naming is great. Like you said, Anthropic faces the same issues you do. Opus is much better at Fable in a lot of stuff without being quite as huge or as smart.
Congrats on the launch.
krmmalik 1 days ago [-]
I like to use OpenAI models for ingesting a customer brief, usually a paragraph or two and then a read of a several customer files (eg spreadsheets and docs etc) and turning it into a system design blueprint. I'm currently using Astra for this phase but it's incredibly expensive relative to the other tasks that come after this. Would you recommend I stick to Astra or can I downgrade to say Sol or Terra for this job?
I'm just worried the system design won't be as good if I choose a lesser model so to speak.
Edit: typos/grammar
jasondigitized 1 days ago [-]
Use colloquial terms: Big, Fast, Genius, Affordable, etc.
curiousObject 21 hours ago [-]
This is legally dangerous for the company because those words have generally well understood meanings, so a lawyer could argue that they promise a certain level of service. It’s safer to use vague words to describe the product variants.
breezybottom 20 hours ago [-]
Trmendous, huge, bigly
ssk42 1 days ago [-]
Straight up - why not just name them Cheapest, Cheaper, Expensive and More Expensive? Cost seems to be the main driver of the intuitive difference between models.
tedsanders 1 days ago [-]
Same reason that Apple doesn't name its series of phones the iPhone cheap, iPhone regular, iPhone semi-expensive, and iPhone expensive. It's poor branding, isn't easily extended to future models, and gets confusing when comparing across model generations, where an iPhone 14 expensive might be cheaper than an iPhone 16 cheap.
pama 1 days ago [-]
I love the current naming scheme and I am excited to see the astronomy trend expand in the future. I hope to see asteroids, binary stars, and solar systems at some point soon, as well as neutron stars, black holes, and eventually galaxies, and clusters of galaxies.
jamiek88 21 hours ago [-]
GPT ‘visible universe’ is gonna be rad.
pavlus 1 days ago [-]
> Models that are trained later might have new data or new techniques or new architectures that earlier models lack.
This seems like one of those cases, where CalVer would be appropriate, and you likely have ideas what you aim for while training, so technical model names (ids?) could include aliases to most things you said. Like
2026q4-xlarge-researcher, 2026q4-medium-humanitarian, 2026q4-medium-coder, 2027q1-small-analyst etc.
switchbak 1 days ago [-]
This kind of heuristic is actually extremely useful - and mostly what I try to develop over a couple weeks as the new models come out. I would love if we could get a fairly standardized take like this on the regular.
Of course we have far too many benchmarks (of questionable utility), but it would be helpful to get OpenAI's take on where it slots in, and how the landscape changes as these get rolled out.
pizzafeelsright 19 hours ago [-]
I have gone deep in dealing with naming and names.
Versions are good for backend but when communicating I want 2 or 3 levels of detail.
Class_Generation_Modifier
Astra_8_Fast
That's it.
This means:
Class - Tells me what it can do, how well.
Generation X end of life is ~18 months.
Modifier: Variation of options.
And the modifier can be dropped if you add more classes.
Closi 1 days ago [-]
> I work at OpenAI and I can try to explain our logic [...] general, models that are bigger will be more performant.
> If you have suggestions on how to name our products better, I'm all ears. Our intention is to make things clear and simple, even if we miss the mark sometimes.
How about naming the models:
"Small", "Medium", "Large" and "Very Large"
Or refer to them by parameter count like other companies (e.g. this is 6.1 5T)
If I have a clothing store, and rather than name the clothes sizes Small, Medium and Large I decide to call them Urgon, Planfar and Fangor I shouldn't be surprised if people think my size naming is confusing or silly. And IMO it's not a great explanation to say "oh it's easy, if you are skinny then you should get Urgon, but then if you are a bit more of a medium build you should get a Planfar, but if you want something a bit baggier than Fangor will really work out for you"
tedsanders 1 days ago [-]
The problem with small, medium, large naming is that they don't last. In a few years our names would have to advance to labels like giant, behemoth, gargantua. This is different than clothing, where we know that in five years humans will be most the same size as today.
As a point of comparison, GPT-3 was launched in a series of small, medium, large, and XL. Back then, XL meant a mere 1.3 billion parameters!
Closi 1 days ago [-]
> In a few years our names would have to advance to labels like giant, behemoth, gargantua
You wouldn't - you would just put the version before it - 6.1 Small, 6.0 Medium etc. Not that complex.
Think of an iPhone 18 Pro - No need for 'iPhone 18 Even Better Super Pro' because the 'Pro' is relative with the version/generation.
Bluestein 1 days ago [-]
Reminds me of astronaut sizing charts for ... their waste management devices.-
pmcarlton 24 hours ago [-]
"In the case of GPT-6.1 Terra, I'd say Terra is probably better than Astra and Sol at technical communication" -- you're doing the thing here too, underspecifying your own models: 6.1 Terra is better than…which Sol, 6.1, 6, 5.6, just the latter two, all of them? This 2D bluriness is a big part of the confusion.
plaguuuuuu 23 hours ago [-]
it depends on the task and it's hard to say
Despite Opus 5.5 being newer and "better" if you read benchmarks, I've found Fable to be better for some things (subjective stuff)
1 days ago [-]
Ntrails 1 days ago [-]
> important details would be obfuscated from end users.
I don't expect the vast majority of your users care. I would guess they want a rough proxy for "cost" and will assume cost=>capability are roughly linked in a linear fashion.
it is very possible that the users weighted by spend do, and I'm therefore simply wrong. But, 2c
zahlman 13 hours ago [-]
Out of interest, are there things where Luna is noticeably better than it would otherwise be expected to be?
alcidesfonseca 1 days ago [-]
In your own response you mentioned GPT-6.1 Terra, but I've seen GPT Terra-6.1. I would start by the part that is the most powerful:
GPT 6.1 X > GPT 6.0 X
GPT Y Astra > GPT Y Sol > GPT Y Terra > GPT Y Luna
This order is very important.
mi_lk 1 days ago [-]
What prevents OAI (and Ant for that matter) from releasing new version for all models at once?
Instead, we got Opus 5.5, Sonnet 5.5 and Haiku 5.5 released weeks apart, and so are Sol 6.1 and Terra 6.1?
RussianCow 1 days ago [-]
My guess is nothing, and they're currently just releasing them individually as soon they're ready instead of all at once so that they hit the market more quickly. AI moves quickly enough that a delay of a couple weeks could result in significant lost revenue.
manquer 1 days ago [-]
Lower models typically use distilled outputs from higher model to improve .
all training runs have to at some level share the compute capacity ; while they have more than enough to run many large runs concurrently, they don’t have enough compute to run everything concurrently.
24 hours ago [-]
Sha1rholder 15 hours ago [-]
This is absolutely the most practical piece of information I learned today
deno 5 hours ago [-]
If it’s better at writing how about making it an option for Chat in ChatGPT?
I still go back to Gemini Pro 3.1 for any writing.
someguyiguess 1 days ago [-]
The length of your explanation shows just how bad of a naming scheme this is.
Haunt1000 16 hours ago [-]
I think what a lot of developers want is a kind of LTS release model that they can build a product over for 4 years and not worry about deprication.
avazhi 1 days ago [-]
> Our intention is to make things clear and simple, even if we miss the mark sometimes.
Well you definitely missed that mark extremely badly here.
tedsanders 1 days ago [-]
What would you have like to seen instead?
avazhi 1 days ago [-]
This is kind of like Apple with the iPhone. You need 3 model products max. 3 is easily understood and people can be bothered (if they care) to track the difference. More than 3 is a mix of being redolent of ADHD (viz bad PR) while also being annoying for consumers to track the difference.
I can’t tell you the difference between the 10 different versions of the current iPhone and I can’t you the difference between the 5-6 different models put out by the Anthropics/OpenAIs of the world. At this point I just try to track the flagship and then the cheapest one or worst case I just use the flagship.
I’m sure internally there’s data showing people want choice and options but decision fatigue is of course very real.
tedsanders 1 days ago [-]
Very fair. There's always a tradeoff in the number of products, and it's hard to make everyone happy. My suspicion is that the optimal number is probably a bit higher for developers on our API (who often build their own internal evals) than for individual users (who don't want to waste time re-optimizing every month). We're excited by Terra's uniquely fast computer use, but if it traction is poor, maybe we'll go back to 3. It all depends on the market signal - we only want to ship models people find uniquely useful.
1 days ago [-]
greenavocado 24 hours ago [-]
> If you have suggestions on how to name our products better, I'm all ears
Terra 2026.2
misiti3780 1 days ago [-]
Is there someone I can email directly about fine-tuning - all my models have been queued for 24/48 hours and your customer support seems non-existent
0xbadcafebee 1 days ago [-]
Fwiw, if you have to explain your product's name to the customers who just want to use it, that's a sign the products are confusing.
Tiny/2026-11. Medium/2026-6. Big/2026-10. XL/2026-01. It may not be sexy, but people know exactly what it is and what to use.
avaer 1 days ago [-]
If there's no fixed ordering, can we have model names that describe the capability?
Have a model alias called "math", another "jokes". Quality labels/date suffixes to these obvious labels. Repoint as needed.
Something for people who don't want to play the cute "which part of the solar system is trending this week" game.
tedsanders 1 days ago [-]
I promise you this would become a giant mess. No one wants to pick GPT-good-at-writing-but-not-in-documents-and-decent-at-instruction-following-but-has-some-known-quirks-to-be-aware-of-p.s.-good-at-knock-knock-jokes-MAY-2027.
Even "writing" as a single concept is very multidimensional. There's fiction writing, nonfiction for experts, nonfiction for learners, deep research, quick lookup Q&A, copyediting, writing in documents / artifacts, instruction following about style, and default quirks (e.g., em dashes, negation, loving the word "tapestry"). Some models are great writers but less diverse, meaning people prefer the first story they read but disprefer the tenth. How does one label such a model?
avaer 23 hours ago [-]
> No one wants to pick GPT-good-at-writing-but-not-in-documents
But this is exactly what we have! Except it's worse, because the model usage recommendation isn't in the API, it's in some random person's X advice that's no longer current this week.
I understand AI companies wash their hands of this problem because it's hard, and they themselves barely understand the true capability matrix of the models they're selling.
I just don't think some effort to categorize models and expose that to users would be messier than what we have -- model selection driven by influencer voodoo bandwagons and analogies to literal astrology.
SoKamil 1 days ago [-]
What do you understand by “technical communication”?
thejazzman 1 days ago [-]
naming suggestion: small, medium, large, x-large
it's almost like ya'll let the AI name it ;)
signatoremo 1 days ago [-]
What if GPT 6 small is bigger than GPT 5 large?
qlte 23 hours ago [-]
Why would that matter? They don't disclose info like parameter sizes to begin with, that's just going off rumors/"leaks".
nickv 23 hours ago [-]
That’s totally fine since it’s a different family. The new iPhone mini is much larger than the iPhone 4 but its mini in its current family line.
inopinatus 17 hours ago [-]
Disappointing absence of Galactica.
ur-whale 1 days ago [-]
> If you have suggestions on how to name our products better, I'm all ears.
Hire an actual marketing person instead of letting engineers / ML researchers do things they're clearly terrible at ?
throw20261022 6 hours ago [-]
> As you note, our models are ordered: Luna < Terra < Sol < Astra
That is false, considering Astra and Luna don't even exist for 6.1. Terra didn't exist for 6.1 until yesterday. You don't get to make up imaginary tiers. The inconsistency wrt their availability, starting with day 1 of a new model version, is precisely the problem. OpenAI is a user-hostile joke of a firm that users would do well to abandon.
jwpapi 1 days ago [-]
[dead]
nightshift1 24 hours ago [-]
[dead]
1 days ago [-]
6i56i6i6rirj 1 days ago [-]
[flagged]
6i56i6i6rirj 1 days ago [-]
[flagged]
BodyCulture 1 days ago [-]
Would you like to recommend a model that does not produce errors and can be relied on like a traditional computer, please.
You certainly know that it happened not by accident that traditional computers are producing reliable results, a lot of work went into this.
Computers must produce reliable results, they are simply not useful if not.
People will understand that at some point, but a lot of damage already has been done.
senordevnyc 1 days ago [-]
[flagged]
halfmatthalfcat 1 days ago [-]
> If you have suggestions on how to name our products better, I'm all ears.
For some reason, this is just hilarious to me. Idk if you're being serious or if this is passive aggressive but the thought of OAI actually taking advice from a random HNer and basing their naming scheme off that is peak.
jaggederest 1 days ago [-]
This is one of the last bastions of semi-serious discussion on the internet, and it's full of incredibly high level engineering talent.
Where else would you go? I'm honestly scratching my head thinking of someplace that wouldn't run you out on a rail, but that would give serious feedback. Kernel mailing lists are out, maybe there's something like that which I can't remember off the top of my head, but...
Also, nomenclature is incredibly hard. There are only two hard problems in computer science: cache expiration and naming things.
halfmatthalfcat 1 days ago [-]
That’s the issue - this isn’t an engineering problem, it’s a general consumer problem. It’s what focus groups full of regular people are for.
jaggederest 1 days ago [-]
But regular people are not the consumers of OpenAI's API - they consume "Chatgpt" and don't care about model nomenclature.
The primary users of any API are the designers and engineers of the applications that use it. How many laypeople know what Postgresql is? How many of the applications they use run on it?
halfmatthalfcat 22 hours ago [-]
We aren't talking about the API. We're talking about the model names. If someone fires up the OpenAI website to ask it a question, nobody knows what the names mean. That's what normal people do and are probably super confused at what is going on.
mediaman 1 days ago [-]
I don't understand what's so confusing about this.
The lineup of models is still the same.
There are four models: luna, terra, sol, and astra.
Pricing increases for each level. This new 6.1 Terra is 50% cheaper than Sol 6.1.
They are lowering prices for newer releases, so yes if you compare Sol 6.1 to 5.6 Terra it won't look right, but why are you doing that? Look at the same release version for comparable numbers.
Each model improves in successive releases but they maintain pricing and capability coherency within a release version.
Yes, Sol 6.1 is supposed to rival Astra for some work, which is what you'd expect from a later versioned release model against a prior release bigger model.
Assuming they release Astra 6.1 at some point, I expect it'll be better than Sol 6.1, at a higher price.
I don't understand what people are asking for here? No reductions in pricing? Different model names on every version release? Not releasing any models for a version number until they're all ready?
the__alchemist 1 days ago [-]
Hey! I think it's confusing because there is a mix of three components:
- Model name
- Version number
- Effort level
All 3 of these are a one-dimensional balance of price/speed, and capability (Loose terms). The confusion is in how these overlap, how they're announced/marketed etc. So you go from 1 dimensions to 3; you can visualize it as a box. For a given task, how do you choose where in this box you select? Which regions are your favorite? Can you map each region of this 3D space to specific tasks? Label them with intuitive names?
Given the operations of the model are opaque, can you see how someone (most people?) might be confused?
mediaman 1 days ago [-]
But what's the ask? This is the cleanest way I see of them doing it.
I see a ton of criticism and not a single person coming up with anything better, unless it's to take away options, like removing effort level.
What about buying Intel CPUs? A car? Electrical components?
If you're not just using this at a consumer level to ask about cooking recipes (and if that's your use case, you really don't care about API pricing, or any of this, and can just use the default), and are using it professionally, then this seems like a low bar of effort to understand what you're buying.
I do grant you the relationship between model and effort level has an unclear relationship, but there are also tradeoffs in most complex technical products that are not always clear; this is exacerbated by the fact that this product is stochastic.
Why are engineers using this as a professional tool expecting this to be bought like a consumer gadget and then blaming the maker?
bel_marinaio 1 days ago [-]
“The ask” is give me a clue how to choose the best settings for a specific task. Model releases happen so often that the calculation keeps changing. Maybe even OpenAI doesn’t always know the answer.
This morning I asked GPT which models and settings I should use to inspect my VM dashboard with computer control. It said:
- GPT-6 Astra at Medium
- GPT-6.1 Sol at Medium (alt)
- use Luna only when the route was already known.
But I had already completed the task twice with Luna and it was 5.6! It navigated through a few pages of my dashboard, pulled up the 30-day usage chart, analyzed my CPU and RAM, and explained why I didn’t need an upgrade. I’d give Luna 5.6 an A on this task.
Maybe OpenAI should give us a “Tiptronic” selector: recommend the model and reasoning level based on the prompt, while still letting users override it.
phoghed 1 days ago [-]
> “The ask” is give me a clue how to choose the best settings for a specific task.
What are you talking about? If it's for some slop you vibe check it. If it's for a serious effort you write an eval. Same as it has been for the last 3 years.
brianpan 1 days ago [-]
These names are not just for engineers.
Why is it ok for engineers to have confusing names and ways to interact with something?
How am I supposed to keep up with the changes every few months?
lukan 1 days ago [-]
"How am I supposed to keep up with the changes every few months?"
No worries, you can always ask a AI.
mediaman 1 days ago [-]
You are showing a selection from some third-party interface that mixes together models from six different suppliers: Mistral, Kimi, OpenAI, xAI, Anthropic and Google, with a bunch of old obsolete models.
Are you presenting this as evidence that OpenAI's naming scheme is bad?
If you don't want to spend the time to know the difference between all these different suppliers and their models, then why are you using whatever this program is that gives you so many choices, most of which are bad? Why is this OpenAI's fault?
unethical_ban 1 days ago [-]
The OpenAI employee above says that some smaller models of the same generation are better at certain tasks than larger models, regardless of cost difference.
They should have a guide saying "Terra is best at technical documentation and communication, Astra is best at large software projects and research, Sol is good for development" or something like that.
ipaddr 20 hours ago [-]
Then he qualified to say it depends on the unique details of the documentation. I don't think its possible. You have to try all models with effort and compare all or go on reputation or vibes.
zahlman 13 hours ago [-]
There's also this "Instant" thingy which doesn't seem to be on the model-name axis but I really can't understand exactly what it is, even after asking ChatGPT about it.
Y_Y 1 days ago [-]
Effort (as it's used in the context, how many "thinking" tokens to burn) is still a free variable. It's not always shown to the end user but you can set it independently for any of these models.
After that you just have model "strength" and version number. But of course strength is only comparable at equal version number and the two axes are almost parallel.
Anthropic also does this, perhaps to a lesser extent.
1 days ago [-]
margalabargala 1 days ago [-]
It's confusing because the models are released one at a time rather than as a family.
Let's say I have some task that Luna 6.1 cannot do, but Terra 6.1 can. So I build my product around that. In a month when Luna 6.2 is released, then I have to figure out whether it's worth to evaluate and swap or just wait for Terra 6.2.
senordevnyc 2 hours ago [-]
Yes, but is that really “confusing”? The way they could solve your dilemma would be to wait to release Luna 6.2 until Terra 6.2 is ready, which just means you’d have fewer options and you’d have to wait. Which you’re free to do now!
margalabargala 1 hours ago [-]
Yes, doing releases this way means that it is confusing to compare model capabilities without going to enormous effort to run my own tests to determine something that should be knowable at a glance.
Waiting is the way to solve this. And despite the outcome being the same, making me feel like I'm being forced to choose to wait is a worse consumer experience.
laurels-marts 23 hours ago [-]
I think a lot of confusion comes from the discombobulated rollout of these models as well.
They released gpt-6-sol (alongside astra and luna but no terra) and 7 days later released gpt-6.1-sol making gpt-6-sol basically DOA before any real adoption. Everyone knows it wasn’t just better RL/post-training, it was a completely different model underneath. Now they randomly reintroduce terra.
I’m sorry, but if you’re not even a tiny bit befuddled by this then you must be taking whatever OpenAI is having.
senordevnyc 2 hours ago [-]
Sorry, but what exactly are you confused about? Their reasoning?
jedberg 1 days ago [-]
Part of the confusion is that it's not linear capabilities. One model is not strictly "better" than the other. They have orthogonal capabilities.
I think the names are more of a reference to price. Luna is always cheaper than terra which is cheaper than Sol which is cheaper than Astra (for the same model number).
stymaar 1 days ago [-]
> I think the names are more of a reference to price. Luna is always cheaper than terra which is cheaper than Sol which is cheaper than Astra
Per token, yes. But per task it's not even necessarily the case depending on how much thinking is needed.
jedberg 1 days ago [-]
That's why "They have orthogonal capabilities" matters. You need to find the one that's best suited to your particular task.
But the model names are linear by price.
bananaflag 1 days ago [-]
I guess it's the same as Opus 5.5 being in a way better than any Fable.
Someone1234 1 days ago [-]
That's definitely a problem too.
But minor credit to Anthropic, at least they've mostly stuck to their four model names (Haiku, Sonnet, Opus, and Fable). You mostly know where a model is MEANT to be positioned from the name alone. OpenAI cannot keep it consistent from release to release.
But, yes, currently Opus Vs. Fable has the same issue as Sol-6.1 Vs. Astra.
laurels-marts 24 hours ago [-]
I don’t think opus-5.5 is better than fable-5.1. I think opus is better overall because it’s much faster and cheaper than fable but if speed and cost is not a factor fable it’s probably just as if not slightly better than opus, especially at front-end design (tho the difference is marginal and no one should be really using fable day to day due to this alone).
zahlman 13 hours ago [-]
> OpenAI cannot keep it consistent from release to release.
Hmm? They've been using the Luna/Terra/Sol/Astra scale for a few versions now and as far as I can tell it maps directly onto Haiku/Sonnet/Opus/Fable, in terms of the kind of thing it's meant to convey (if not in terms of the scale of differences).
senordevnyc 2 hours ago [-]
People are just desperate to complain about the big labs.
pants2 1 days ago [-]
I don't know why this is so hard to understand.
It's like iPhone 17 being faster than iPhone 16 Pro. Because it's newer and benefits from advances in chip architecture, despite being a lower model tier. Sometimes different iPhone models are released at different times.
bluerooibos 1 days ago [-]
Yeah this isn't intuitive at all.
Both companies have screwed up their naming royally. Maybe that's intentional - I'm sure it doesn't hurt Anthropic to have people using Fable a bit longer than they need to and racking up a huge bill in the process.
scottyah 22 hours ago [-]
They're always very clear about it when a new model comes out. You have to click away a banner to continue using the service. There's not much they can do to make it any more obvious without it being a major hassle for all the people who understand versioning.
cesarvarela 1 days ago [-]
It is: Fable 5.5 will be better than Opus 5.5. Opus 6 will be better than Fable 5.5, Fable 6 will be better than Opus 6, etc.
dumberquestions 1 days ago [-]
You're comparing different generations, any 5.5 models could be better than any 5.0 model.
SomeonesAccount 1 days ago [-]
Really? Haiku 5.5 is better than Fable 5?
dumberquestions 14 hours ago [-]
No, I said could as in it's not an apples to apples comparison.
mrighele 24 hours ago [-]
It's like Intel i3/i5/i7/i9, or nvidia 50/60/70/80.
It is meant to give a rating inside a generation. Different generations are not necessarily comparable. an i7 is better than an i5, but not if you compare an i7 7xxx with an i5 14xxx.
A nvidia xx80 card is better than a xx60 card, but not if you compare a 1080 to a 5060.
The only difference for OpenAI is that not every generation has every model.
lbreakjai 1 days ago [-]
> Did Microsoft's chief branding officer move over to OpenAI?
Obviously not, otherwise the four models would be called Copilot, as well as the harness.
InsideOutSanta 1 days ago [-]
You're just not paying attention. If you run copilot in copilot using copilot effort, it performs much better than copilot in copilot using copilot effort. This is clearly explained if you visit copilot.com using copilot and then search for copilot.
iAMkenough 1 days ago [-]
“Introducing our first mobile devices: Pocket Copilot Luna for on-the-go inference, and our new tablets, Copilot Sol+ and Copilot Astra Duo.”
elxr 1 days ago [-]
If you daily all these models (and have a modest budget, not some >$400 corporate allowance), then it's pretty obvious that Luna is small and fast, and Sol 6.1 is the big one for jobs that need significantly above average intelligence, comparatively to a human engineer. I use both constantly. Astra is really not relevant for the average dev except to do some moonshot task with a vague verification loop.
Sure Sol 6.1 on low/medium can be considered "cheap" considering its abilities, but it's clearly not meant for the same type of tasks as Luna. You just grow an intuition for when something is solvable with a Luna-sized model. I'm guessing Terra is good for the tasks that you'd typically use Sol 6.1-low for, or for Luna type tasks (or other not-novel problem) where you want to reduce the risks of suboptimal decision making. It's exactly what I'd go with for feature additions where the correct solution is already mostly designed/specced-out.
rao-v 1 days ago [-]
The way to think of this is that (current) inferencing hardware has natural cut points of efficient scale (assuming some optimal architecture which at this point is almost always an MOE variant), so you always have an incentive to provide 3-4 model sizes (which nicely enough often means diff speed to answer also).
You are also rolling the training of your models by size and the lat est small model can sometimes beat an older model.
For a while we thought you just train the big model first then distill smaller models from it but since we've learnt the importance of on policy RL (basically the way a small model figures out a problem is very diff from a big model so it's best to RL optimize each model distinctly)
redox99 1 days ago [-]
Name means size which means price
Yes, a more modern smaller model can beat an older bigger model at some tasks, that's what can be confusing.
thomasahle 1 days ago [-]
Training models is still not a precise science.
If you scale a model down, or switch out attention with linear attention, or introduce KV-compression, etc., you don't really know how it will affect model performance before you've trained the model.
And once you've spent millions training it, you might as well make it available if it's pareto optimal for some tasks. It also gives you more insight seeing how people use it in practice.
fallingbananna 1 days ago [-]
Flip it around. The number (generation) is the most important. Just like it's often with hardware.
And some variants (e.g., the medium one) being skipped or delayed for a certain generation doesn't really change the overall size branding.
kwinkunks 1 days ago [-]
Agree, it's confusing. I hear people discussing which model to use for what tasks and my head starts to spin because there are so many properties to account for and it's generally hard to reason about LLMs.
"Speed, quality, price: pick two". I think I'd rather see a choice like that than the current convolution of $-per-token, sub quota, latency, simulated reasoning level, model size, context size, etc, etc.
hadlock 1 days ago [-]
>But then things go confusing, they released Sol-6.1 which sometimes is superior to Astra, and discontinued Terra which made it look like this
I don't want to rush to their defense here, but, when model families are getting refreshed every 6 weeks, there is going to be significant overlap, particularly with these staggered releases
xena 1 days ago [-]
The moon is smaller than the earth. The earth is smaller than the sun. The sun is smaller than all of outer space.
dukeyukey 1 days ago [-]
Next AGI test, when model companies have consistently good version branding.
irthomasthomas 1 days ago [-]
And low/medium/high/xhigh/ultra reasoning. And pro mode and ultra-speed versions. There's at least 25 options to choose from.
baby 1 days ago [-]
Wait I always thought terra was the smaller model and the further away you went in space the bigger the model (luna is further away so it’s better than terra)
onlyrealcuzzo 1 days ago [-]
It's especially confusing because Sol 6.1 is mostly just Astra but 10x slower at 1/10th the price - unless you pay for it to be as fast as Astra - at which point it's the same speed and price...
It's nice that they have Terra which seems to fill a void that's been needed since Sol 6 focused on cost (at the expense of quality), Sol 6.1 got a huge boost in quality, but became unusably slow unless you paid Astra prices - at which point, why not just use Astra?
Now at least we have a viable other option again.
trollbridge 1 days ago [-]
Huh? I have not observed Sol 6.1 and Astra 6 to be the same.
meerita 1 days ago [-]
You're not alone on this. They have too many models, and too many effort levels, it starts to feel like what to chose.
oh_no 1 days ago [-]
the only confusing bit was dropping terra for a bit,
Luna (Small), Terra (Medium), Sol (Big), Astra (Bigger)
is perfectly clear, sol beats astra sometimes because Astra 6.1 was scrapped. and note that opus 5.5 beats fable 5.1 most of the time too without a skipped model.
dev_l1x_be 1 days ago [-]
No idea, i still try to figure out how to use the right one per task.
golergka 23 hours ago [-]
Different classes and different generations. Isn't it roughly the same as what car manufacturers do? Or even Macbooks? 2014 macbook pro has worse performance than 2026 macbook air, this doesn't confuse consumers.
QuesnayJr 1 days ago [-]
I suspect 6.1 Astra is the model that kept showing up in the news stories about an internal unreleased model trying to hack the Pentagon or whatever.
setgree 1 days ago [-]
also am I the only person who remembers Do Kwon and the Terra/Luna debacle?? how about Gaia/Selene (admittedly harder to non-native english speakers)
(For some reason they say "Cache read" instead of "Cached input" for 6.1 Terra)
6.1 Astra and 6.1 Luna must be coming up soon. Will they reduce their prices further?
agentdev001 1 days ago [-]
I guess 5x/95% is where we're landing with the 6.1 family, that's nice.
wellthatsinsa 23 hours ago [-]
[dead]
Archit3ch 1 days ago [-]
I rather enjoy OpenAI's focus on cost-per-task.
I'm optimistic about them tackling frontier intelligence next. Astra 6.1 please?
mapontosevenths 1 days ago [-]
They're doing its because they recently cut plan usage by about 50%, leaving Anthropic providing about 5x the value on their plans. [0]
The "cost per task" metric is just a weaselly way to sound like they didn't just double the cost for most users, and aren't having their proverbial lunch money stolen. They can no longer claim to provide more tokens, more capable models, or more usage in general. So they made up a new metric where "task" could be defined to mean whatever they need it to mean.
They had to become VERY specific in their wording to present any kind of value proposition that will entice people.
> leaving Anthropic providing about 5x the value on their plans
5x is calculated in terms of API costs, nobody pays those extreme Anthropic prices outside enterprise users.
> like they didn't just double the cost for most users
Most users aren't on their $200 plan. Most users (including on the $200 plan) continue being subsidized.
killingtime74 15 hours ago [-]
Enterprise users are the majority of the users in value. Subscribers are the loss leader hobbyists.
mapontosevenths 1 days ago [-]
> 5x is calculated in terms of API costs, nobody pays those extreme Anthropic prices outside enterprise users.
Ignore value then. Anthropic still provides about 3x as many raw tokens. The 20x\200 plans provide about 10B for openai versus almost 30B for Anthropic.
Do you really think that Opus uses 3x as many tokens as Sol per task? I don't.
> Most users aren't on their $200 plan.
Fair enough. I should have said most heavy users.
> Most users (including on the $200 plan) continue being subsidized.
They continue to be fifty percent less subsidized, I guess.
Look, I don't fault them for trying to become profitable. I fault them for the weird weaselly business speak about cost per task. It's BS game playing and we all know it.
I'm honestly rooting for them, and hope it all works out. For now though I'm pretty salty that my bill jumped by 150% in one month. Gaslighting about made up metrics just makes me more angry.
_aavaa_ 20 hours ago [-]
How do you know? Have you done the A/B test yourself?
> Do you really think that Opus uses 3x as many tokens as Sol per task? I don't.
You don’t have to speculate. Pull up artificial analysis, pick either opus or sol and scroll down to total tokens and to total price. Opus (and sonnet) use 4x the tokens and 7x the dollars.
That difference alone removes the difference that semi analysis found.
mapontosevenths 18 hours ago [-]
> How do you know? Have you done the A/B test yourself?
Fair point. I just tried it, and I stand corrected. Claude uses almost exactly 3x as many tokens.
In the end Claude produced 2.25x more code, and used 3.01x more tokens (ignoring the cached system prompt, etc). It's answer was also much more substantial and feature rich - including many features I did not ask for.
I gave them both the prompt "Without searching the internet please write and test a python script that accepts a year (in any common format) as input and provides the date that easter fell or will fall on in that year. The output should be in discordian date format. You may not use any libraries."
The thing I found most interesting were the unit tests. Claude implemented the core code using both the Meeus–Jones–Butcher algorithm AND Gauss, but only used Gauss as a sanity check to ensure that it's primary code worked. Sol just validated that it matched some dates it could remember. That explains a good chunk of the almost doubled output, and it was a reasonable thing to do, though neither answer was "wrong."
Token usage looked like this (based on parsing it out of the jsonl, neither provider seems to make it easy to get real billing information per project/conversation.
> neither provider seems to make it easy to get real billing information per project/conversation
I believe oh-my-pi will be able to ingest your Claude and codex conversations, count tokens, and then assign API-equivalent rates to them. May be worth looking into.
ComputerGuru 24 hours ago [-]
People in HN live in a bubble. Plenty of real world use cannot use subscription plans (not even talking about enterprise agentic use) and is absolutely at the mercy of the cost per task metric.
We agonize over cost per task to the extreme for services/functionality built atop of APIs and can absolutely guarantee you that it is the real world metric that counts most right after “is the model good enough for the task at hand,” and furthermore that, yes, in fact, it is quite often that we see certain models/providers easily using 3x or more tokens on average for the same tasks (or even the exact same task) than a supposedly similarly matched model from another provider. Once you become intimately familiar with the thinking traces from specific models, you can preempt this to some extent with manipulations to the system prompt (essentially heading off entire thinking branches) but the nuking of reasoning traces in the name of preventing against distillation attacks has made this harder.
_aavaa_ 1 days ago [-]
Cost per task, time per task, quality (good luck measuring objectively between models) are the metric that matter. I don't care how much a token costs, I'm not buying tokens, I'm buying solutions.
Once you are inserting models into non-creative (and software development is creative work) it is imperative to start creating your own small evals. Not just to check how different models perform in that equation, but to catch model drift over time.
agoodusername63 23 hours ago [-]
the point is that "task" is a dubiously defined unit that the frontier labs adjust to whatever makes them look good.
_aavaa_ 6 hours ago [-]
What is the point you're trying to make? You can define a task yourself and run two models and see which one costs more, takes longer, and produces better code.
mapontosevenths 1 days ago [-]
Are OpenAI's models delivering 50% (or more) completed tasks than they were last month? Are they delivering 50% more than Anthropic per token?
Nope to both. Not even close.
The only way it works is if you define "task" in a way that OpenAI is good at it and Anthropic isn't.
_aavaa_ 20 hours ago [-]
First of all, only the $200 plan got a cut.
Second of all, barring more cuts, yes I do think the $20 and $100 plans do deliver more tasks because the models have gotten cheaper in the meanwhile (like this very announcement).
> Are they delivering 50% more than Anthropic per token?
So say the public benchmarks. Pull up artificial analysis, “Cost (USD) to run all evaluations in the Artificial Analysis Intelligence Index”:
Sol 6.1 (max): $1,082
Opus 5.5 (max): $8,708
Sonnet 5.5 (max): $7,260
>7x difference in cost for the same tasks.
mapontosevenths 17 hours ago [-]
> First of all, only the $200 plan got a cut.
The original plan doesn't exist anymore. To get the same usage you now need a $500/month plan. The new plan that costs $200 is different in every way that counts. It was actually a 150% price increase for people who want or need what the old plan used to provide.
Don't get me wrong, I genuinely LIKE OpenAI. It's probably worth it even at the new price. I just hate the rhetoric designed to Goodhart past the math.
> the models have gotten cheaper in the meanwhile (like this very announcement).
That is not good news for plan users. Plan users now get less of something that also has less worth to begin with. It would only be good news if they got more of it.
For example, if I buy the "unlimited car wash" package for $19.99/month and the car wash lowers the price per wash from $10 to $1 my subscription becomes less valuable, not more. The breakeven has moved, so I now need to wash my car more than 20 times before the package is justified.
If the car wash then increased the price to $50/month would it be wise for me to keep the package? Would lowering the price per wash make the package more valuable to me somehow? No, it would do the opposite.
_aavaa_ 10 hours ago [-]
> It was actually a 150% price increase for people who want or need what the old plan used to provide.
I’m not sure I see how. Subsidy got cut by half. But in the meantime 6.1 came out which uses 25% fewer tokens than 5.6 and costs 1/3 as much per task.
So for 1/2 the budget you use to have with 5.6 on the old plan, you can do more with 6.1. I don’t see that as a problem.
4 hours ago [-]
oh_no 1 days ago [-]
I kind of expect them to go right to Astra 6.2, 6.1 was cancelled over the cybersecurity hubbub. I eagerly await either one.
Interesting that both "none" and "low" reasoning levels appeared not to spend any reasoning tokens at all, and in fact low ended up using less tokens and costing less than none here.
"medium" did use 496 reasoning tokens, but that's apparently not enough here to return a visible reasoning trace.
What's the point of using it without reasoning? Isn't that what Luna's for?
simonw 1 days ago [-]
It's neat that we can now compare the Terra without reasoning pelican (cost 0.64 cents) with the equivalently priced Luna with reasoning pelicans (Luna xhigh is 0.40 cents, Luna max is 0.91 cents). I think the Terra low one compares favorably to those, though I'd suggest also comparing them on tasks more representative of real work than generating pelican SVGs!
RussianCow 16 hours ago [-]
Terra without reasoning is almost certainly going to be faster than Luna on High (which is roughly equivalent in intelligence according to AA).
notatoad 1 days ago [-]
a bigger model still has more built-in knowledge to draw from, even without reasoning.
I appreciate that most of these pelicans are not holding the handlebars, i think that's more realistic than the alternative
kubb 1 days ago [-]
Do you have an agent scanning the front page waiting to post the pelicans?
simonw 1 days ago [-]
I should do but no, it's just a bad HN habit.
pietz 1 days ago [-]
I can literally see what the training data sample looked like that they included, so it performs well on this benchmark.
drewbitt 1 days ago [-]
12 seconds for a pelican on low. Pretty good.
bigcat12345678 1 days ago [-]
Seems Gemini is way ahead in pelican drawing
thimabi 1 days ago [-]
Apparently Google made it a point to train their models to output SVG, among other things, so that helps
chriddyp 17 hours ago [-]
In my tests [1], GPT-6.1 Terra joins the Pareto Frontier in between GPT-6 Luna and GPT-6.1 Sol. So, exactly as advertised!
It's also a very fast model, of all 33 models we've tested so far, only Haiku 5.5 and Nemotron 3.5 Super VL have passed the exam faster but both scored lower. It also wrote the _least_ output in its responses.
It is fast, but more verbose (a lot more verbose than Sol at higher efforts)
A_D_E_P_T 1 days ago [-]
Wow. If those charts are true, this effectively matches GPT-6-Astra at a fraction of the cost, and is neck-and-neck with 6.1-Sol.
user43928 1 days ago [-]
Looks great on the first chart, but terrible on the Terminal Bench 4.0 where it needs xhigh to come close to 6.1 Sol, and costs more for a slightly worse result.
fallingbananna 1 days ago [-]
I would guess only on the nitpicked benchmarks?
The pelicans place it nicely between Sol and Luna in terms of both pricing and results.
vdfs 1 days ago [-]
pelican is also nitpicked (and probably the most useless)
toxik 1 days ago [-]
Cherry-picked is the term, not nitpicked.
SomeonesAccount 1 days ago [-]
Yes, it may be dumb, but it isn't useless because it is both a) is a benchmark and b) is not picked by openai
vdfs 1 days ago [-]
IMO it's useless because we don't even know what a pelican on a bike is supposed to look like, It's very impressive that that models can make these directly using SVG, but we are way past the "wow" stage and test that can't be gamed and are easy to prove how good the model is in that specific arena are hard to find.
The only thing that pelican have is "models where not trained on this" maybe that true months (or years ago) now it's on every HN post and these AI labs are monitoring HN
tristanj 1 days ago [-]
Not necessarily, they may have selected benchmarks where Terra performs relatively well compared to those models. Pretty much every model release these days just cherry-picks & highlights the benchmarks where their latest model performs best. We'll have to wait for the complete benchmark suite to have a better picture.
xnx 1 days ago [-]
Why do Anthropic and OpenAI think it necessary to split their model into 4 versions (Opus, Sonnet, Fable, Haiku, Astra, Sol, Luna, Terra) vs. Google's 2 (Argon, Flash)?
kzrdude 1 days ago [-]
Google has flash-lite, too. I liked OpenAi's scheme better, as long as it is parallel and makes sense.
killingtime74 15 hours ago [-]
Make more money? Get 4x the press releases and marketing
nojvek 9 hours ago [-]
More choice is better?
Luna, Terra, Sol, Astra - are a good setting tweak on how much you want to spend vs what you want to get out of it.
Silagi 1 days ago [-]
Can anyone give me the compelling reason for Terra and Sonnet to exist?
Virtually every task I give to either is better served by going up a tier and down a reasoning level, or down a tier and up a reasoning level. I simply don't understand their use case. No testing I've done has shown them to have a niche where there isn't a cheaper-with-equal-results or more-performant-at-same-cost option.
h14h 1 days ago [-]
I think for the majority of the audience here who use these models inside of coding agent harnesses w/ subsidized monthly subscriptions, there likely isn't one. Depending on how Terra contributes to the rate limits, it may make sense to use it for some subset of a repeated workflow to get more mileage out of sub.
If you pay API prices for inference in your product stack & have specific use-cases w/ custom evals, these smaller models provide an additional cost/speed lever to adjust. Ultimately, I think the point is to compete on cost for use-cases where Sol on the lowest reasoning is overkill.
If you have a fairly straightforward usecase where Terra performs just as well as Sol, but is faster and cheaper, why not use it?
daynthelife 14 hours ago [-]
The real issue with coding harnesses is that you get a cache miss any time you switch models. I might have some easy task (e.g. "fix lints") that I could delegate to Terra, because my session has been using Sol up to this point, I'd pay a full uncached prefill if I routed it to Terra. It ends up being cheaper in practice to just stick with Sol.
Spinning up a fresh session would be a reasonable choice in the above case, but sometimes you still need the retained context.
fy20 22 hours ago [-]
Yes but you are missing the point. It's the case where Luna is not good enough, and Sol is too much. The problem is that on benchmarks Luna xhigh is typically comparable to Sol low in terms of capability. So where does Terra fit in?
dannyw 21 hours ago [-]
World knowledge, factual recall without grounding, more complex instruction following, etc.
Benchmarks don’t really tell you much. For API best will always be a model+effort sweep on your internal evals (and if you don’t have internal evals, do that first).
For individual coding harnesses, honestly I just go with vibes. The best workflow is the one that works for you.
foretop_yardarm 1 days ago [-]
Idk… I’ve written 100k sloc in ~2 weeks on the Claude £20 subscription with sonnet medium alone. Seems pretty good to me.
selcuka 22 hours ago [-]
I guess the question is whether you could have done the same using Haiku High/XHigh or Opus Low/Medium at roughly the same speed and cost.
eknkc 1 days ago [-]
On the API we use a lot of Luna (cost efficient) and sometimes Terra level models. I think we never use a larger model. These are models that our app runtimes depend on.
On subscription for coding and stuff I am basically on Astra Ultra all day long.
legulere 1 days ago [-]
They are the biggest models supported in Github Copilot Pro
You should look at model reasoning level curves on the cost/performance chart on artificial analysis. If you compare the curves for haiku/sonnet/opus I would agree with you that sonnet serves no purpose because it is priced too high. If you look at luna/terra/sol you will see that terra is priced such that it successfully fills a gap between luna and sol. And when you add speed into the equation it adds another dimension where terra makes sense. terra low falls between luna xhigh and max on both cost and intelligence, but it is way faster because it uses way fewer reasoning tokens. So I would say there are workloads where terra makes sense. But from my analysis I don't see any workloads where sonnet makes sense.
nfw2 1 days ago [-]
There is a chasm of performance difference between haiku and sonnet ime. Sonnet is cheaper and faster than Opus, so if you're providing a service via api where cost and speed is important, it's a big difference.
datadrivenangel 1 days ago [-]
Except Sonnet is slower, more expensive, and worse at reasoning levels above ~medium. Opus on low is faster, cheaper, and better at doing things.
nfw2 1 days ago [-]
1. In Anthropic's benchmarks, Opus's advantage over Sonnet diminishes at high reasoning levels. Sonnet max actually beats Opus max.
2. There are rote or simple tasks where higher reasoning isn't needed and one model vs the other wouldn't even be distinguishable.
3. How many tokens are used during reasoning depends as much on the task as the reasoning level.
4. Sonnet is faster and half the price per token, so Opus will only be cheaper if it can spend fewer tokens. This won't be the case for all tasks. Opus and Sonnet will also consume the same number of tokens to load context, so the relative price comparison depends a lot on where the task is context-heavy or reasoning-heavy.
datadrivenangel 1 days ago [-]
> Sonnet max actually beats Opus max.
What are you smoking and can I have some?
moecables 1 days ago [-]
Sonnet is way faster than Opus when doing everyday tasks, I think on the subscription plans it's a better option when you need something done faster than Opus' long thinking sessions.
hobofan 1 days ago [-]
For coding, no.
I have some business automation process for some customers that I maintain, where Terra with low reasoning ended up with the best price/performance & latency. With an equivalent Luna with high reasoning, price was usually better, but quality had a higher variance (very much depends on where the thinking takes the model) and latency for high reasoning was worse.
If you trust these benchmarks, I'd argue that there's almost never a reason to use anything other than Terra 6.1 xhigh.
qert12 1 days ago [-]
> Can anyone give me the compelling reason for Terra and Sonnet to exist?
To justify the pricing of Sol and Astra. Old business trick.
jhatemyjob 1 days ago [-]
This is the only reason that makes sense to me. Thank you
handzhiev 1 days ago [-]
If you use API in agent or your own integration with software, such models make a lot of sense. We use Terra 5.6 for "question to SQL" translation for example. It's good enough and affordable enough. Luna will probably not handle complex queries well while Sol is too expensive.
notatoad 1 days ago [-]
i use sonnet to generate answers to help-desk tickets, which it does for significantly cheaper and insignificantly worse than opus.
if you're just talking about use in codex or claude code, then sure, leave it on opus all the time. but for running routine api requests, sonnet is great.
datadrivenangel 1 days ago [-]
Sometimes this tier is faster, and getting a good enough result in a few seconds instead of a better result in ~15-30 seconds is a meaningful result.
I think it's also a fake lever to give to finops / it teams to try and keep costs down.
altcognito 1 days ago [-]
I've been refused to do cybersecurity (read-only investigative access against the recent brotli cve) with Opus 5.5 when I had full SSH access, so I guess you've got to have a fallback?
r0fl 1 days ago [-]
I cannot figure it out either.
I use Astra Ultra to plan and 6.1Sol High or X-High to build and even on huge projects with 10s of 1000s of pages it is fast and competent
rattynm 1 days ago [-]
I'm on Claude's default plan for personal use, and Sonnet is plenty enough for small projects while not hitting my usage limits too hard.
rfgplk 1 days ago [-]
Cheap and affordable. Dispatching proper tasks to these models can save you a huge % of your quota. Especially if you _know_ the task is simple (ie add something here or there, or archive a bunch of files and reformat kinda thing)
Silagi 1 days ago [-]
Maybe it's my tasks, but in the cases you're describing, I'm usually better off going down to Luna. I specifically don't see the gap where "It's too complex for Luna-high" and "It's not complex enough for Sol-low" exists.
rfgplk 1 days ago [-]
I've personally found that most models completely collapse at sub-medium effort. I generally run most of my stuff at medium/xhigh with heavy core tasks @ max
thefourthchime 19 hours ago [-]
PacBench results.
GPT-6.1 Terra scored 84, one point over Grok 4.7 (83), in 5.4 min for $0.19 vs Grok’s 36.5 min and $1.69. The cheap models keep getting better.
Too many models and too many knobs. There are like 5 models and 5 thinking levels, that is 25 different combinations.
keyle 19 hours ago [-]
I agree but innovation does involve jabbing in many directions. They're trying to innovate.
thewhitetulip 17 hours ago [-]
And every month there are new
I still don't know which gpt model to use! Anthropic one I know as its simple.
Here its gettung weird
ionwake 1 days ago [-]
cool update. Like super cool.
Just one thing about the advert, how can I be the only person here thinking the ad is weird - like for the marketing dep, its a bit weird the ad features the same tram that broke and tragically like ended some people - happened for like the first time ever in 100 years, like a year ago ( 14 months ago) just a bit weird.
Im honestly not trying to be negative, but its kinda weird I kinda feel I should mention it because noone else has?
Im sure literally anyone from southern europe must have thought this the second they saw the ad. It was big news last year.
Am I mad? Just a heads up for PR to always check things like this first when releasing a video? Maybe its just me? Just trying to help
dash2 23 hours ago [-]
It's a subliminal warning. "On the trolley problem, this model will try to kill you and make it look like an 'accident'."
matt3210 22 hours ago [-]
its just slop bro, welcome to the future
FeteCommuniste 21 hours ago [-]
New version of "it's not that deep" dropped.
bob1029 1 days ago [-]
The results for factual error rate on "difficult" prompts is staggering to me. Particularly the difference between 5.6-Terra and 6.1-Terra. These are wildly different Pareto fronts. Our custom enterprise agent is currently running on 5.6-Terra and it's already very good. I look forward to updating the configuration.
satnhak 23 hours ago [-]
the only things we've got to actually make a judgement on are vibes and token anxiety. vibes generally indicate that more expensive == better, but more expensive == more token anxiety. so you end up using the most expensive model x effort that keeps your anxiety in check. yeah, maybe terra is great at technical documents. i'm not going to use it because i'd need to spend a couple of days writing technical documents to work out if it was better than sol and then what if it isn't?
is opus 5.5 better than sol 6.1? hmm, i really can't tell. maybe claude is a bit more diligent and identifies more issues up front. but sol has this can do attitude and the codex harness just gets on and does it's job done.
agentdev001 1 days ago [-]
I'll wait for the runebench results!
On AA intelligence index, this fills in the cost per task frontier nicely between 6 Luna and 6.1 sol. So when 6.1 Luna inevitably releases...
The OAI model tango seems to not have an end in sight, for better or worse. Give me more API toys!
agentdev001 20 hours ago [-]
Cost v Performance pareto on Runebench, scoring the same as Opus 5.5 for 1/3 of the cost.
causal 1 days ago [-]
These "Pareto frontier" performance vs price graphs have become very popular and I find them mostly uninformative. Pricing is subjective. You chose the position on the X axis, and which measures to use on the Y Axis.
shepherdjerred 1 days ago [-]
Am I reading the Agents’ Last Exam v1 graph correctly where it's saying GPT Luna Medium performs better than High, but XHigh is better than Medium?
tedsanders 1 days ago [-]
The error bars on eval graphs are commonly like ±5%. It's fairly common to get out-of-order points when the gap between them is only a few percentage points. If we ran it again, or ran it with minor alterations, I'd expect the ordering to be expected. In general, we don't massage any of our eval numbers and for good statistical hygiene we'll report the first run we do, even if later runs show slightly smoother curves.
(I work at OpenAI.)
ttul 1 days ago [-]
That is correct. The gpt-6 models are strange.
mehdizare 1 days ago [-]
This tier is what I'd actually point opencode at for PR reviews. The top models are too slow and expensive to run on every diff.
gobwas 1 days ago [-]
I am wondering if there is a site with a perf/efficiency chart of the models over time? I am wondering is the pace still speeding up?
sehw 1 days ago [-]
Sell it to me. Why would I ever give money for this?
cesarvarela 1 days ago [-]
You are paying API prices, and 6.1 Terra gets you 90% of what 6.1 Sol gets you at half the price.
1 days ago [-]
mattfrommars 1 days ago [-]
My understanding is API prices are more expensive then monthly plan, is this true?
My plan for OpenAI recently expired and now I’m thinking to test Anthropic ecosystem. My primary use case is programming only.
Should I go with API use or subscription? For coding harness, I jump between codex, OpenRouter, pi, whatever coding harness is trending on that day
shepherdjerred 1 days ago [-]
Individuals are not paying API prices for coding. Everyone uses a subscription.
Anthropic does not allow you to use 3rd party harnesses -- you have to use Claude Code. Claude Code is an excellent harness, though.
If you want the freedom to move between harnesses you should use OpenAI or xAI
cesarvarela 1 days ago [-]
Subscriptions provide thousands of dollars' worth of tokens if measured at API prices. While the value was once absurd, it is now slightly lower but still much cheaper.
c0rruptbytes 1 days ago [-]
anthropic subscription does not allow 3rd party harnesses i believe
aiteamguide 3 hours ago [-]
[flagged]
ultramancode 17 hours ago [-]
How would it compare to Claude models in terms of performance?
czxck001 13 hours ago [-]
GPT 6/6.1 series underperform in UI/3D when compared with Opus 5.5. Does 6.1 Terra improve?
rayshen 18 hours ago [-]
Does it suitable for finance choice?
shnksi 23 hours ago [-]
These benchmarks made me want to try sonnet 5.5
_davide_ 16 hours ago [-]
6.0 re branded as terra?
snvzz 9 hours ago [-]
Intended to test it with openrouter, but no provider meets the ZDR requirement.
Unlike Sol, Luna and Astra. Odd.
ulfw 10 hours ago [-]
Do people STILL care about updated versions of LLMs? It's all a commodity now. This week this, that week that
totetsu 20 hours ago [-]
is it eligible for data sharing incentive free token quota?
mattbis 1 days ago [-]
Still can't really forgive any of them for essentially stealing most of this knowledge regardless...
I_am_tiberius 1 days ago [-]
And then talking about other people distilling from them...
ComputerGuru 24 hours ago [-]
I feel like OpenAI hasn’t been as (publicly) hypocritical and petty about their distillation spats as Anthropic has been?
e.g.: "You must not use OpenAI models or their outputs to generate, prepare, select, or supply training material or signals for models that are intended as alternatives to or substitutes for OpenAI models or services".
ComputerGuru 8 hours ago [-]
Of course they disallow it. In saying they’re not thick enough to not realize how it looks to publicize their anti-distillation efforts. You must not have seen the tirades and screeds Anthropic posts in comparison, then.
joquarky 1 days ago [-]
Anyone who has data that is so important to their livelihood should probably make backups.
1 days ago [-]
agluszak 21 hours ago [-]
Unfortunately it suffers from the same problem as Luna: it will stop at an earliest excuse and no amount of prompting not to stop until X is done will convince it otherwise.
desireco42 22 hours ago [-]
If you are confused, that is how this company was always been, remember GPT 4, then they did some weird versioning.
I personally like Terra, it kind of does what I would ask Sol or Astra, but dont have to count tokens as much.
And Luna has been amazing tbh, it takes designs from prototype and in record time is implementing it, so that was a huge win.
But yeah, they always had confusing way of naming products and if you are not following news, you will be confused about what it is.
According to this, Sol 6.1 x-high is both cheaper and smarter than Terra 6.1 max.
johnnyApplePRNG 1 days ago [-]
Is it stronger and cheaper than DeepSeek V4.1 Flash 1017?
If not, yawn...
oh_no 1 days ago [-]
AA has limited 1017 benchmarks so some terras versus 1017 Max:
max - 3 points smarter, 50% more expensive
xhigh - same score, 25% cheaper
high - 1 point dumber, 50% cheaper
mattfrommars 1 days ago [-]
Do people use deepseek for real world development or just for building experimental large projects?
wren6991 1 days ago [-]
Yes, V4.1 Flash 1017 is my daily driver cloud model. Copes fine with what I'd consider difficult brownfield development, like fixing bugs in vLLM prompt cache and coming up with small/sane overlay patches for them. Maybe it's disappointing for people who want to one-shot a game or website from a single prompt; I haven't tried that.
It's nice that their peak-rate hours end quite early in the day UK time.
dudisubekti 1 days ago [-]
I use it at work. It's dirt cheap, blazing fast, and is more than enough for most coding work.
If you live in a country with a weak currency, DeepSeek models are definitely a godsend.
deno 1 days ago [-]
You can do real world development with weaker models than Deepseek Flash.
I still use SOTA for the main model though, but also I'm perfectly aware it's mostly out of FOMO.
Deepseek is more than enough for default subagents though. Any well defined and verifiable task and there's absolutely zero difference between SOTA and good-enough models. Used to be just speed, but Deepseek is fast.
mattfrommars 19 hours ago [-]
I have same the same FOMO to use SOTA model for all planning and brainstorming session. However, I’m unsure how do you bring deepseek for subagent task. Could you describe your workflow?
For myself, using Codex, I always have to explicitly tell codex to use Luna with high reasoning for smaller, bit size tasks.
deno 16 hours ago [-]
I'm using OMP (oh-my-pi) and the minimum you need to do is just set the @task role to Deepseek:max.
Subagents can spawn with other profiles e.g. scout (uses @smol) and reviewer (@slow). You can set SOTA with higher thinking for @slow and Deepseek with lower thinking for @smol or something even faster/cheaper.
There's also weirdly named /vibe mode that gives explicit instructions to the main model to act as an orchestrator but I find this unnecessary if your main is SOTA, they are already super eager to spawn subagents by default.
deno 1 days ago [-]
It's only a little better than Deepseek with Max effort in benchmarks. Doesn't bode too well for 6.1 Luna.
simianwords 1 days ago [-]
The benchmarks are already saturated and the curves for Terra stick to the axes.
wellthatsinsa 23 hours ago [-]
not gonna lie they did really bad and this drained my usage SO FAST. that model used lots of unnecessary skills and read through way more memory files needed.
It literally burned through 2 subs.
I_am_tiberius 1 days ago [-]
Underwhelming
Starlevel004 1 days ago [-]
That terminal bench 4.0 result is... bad? Terra XHigh more expensive than Sol Low and still does worse.
It's very likely this is a next checkpoint of GPT-6 "Sol", over-optimised for the web chat and offered to Codex subscribers as a "please go away" offering.
If this was half the price, I'd be very interested, but at 1/5/0.05? Feels like another panic release because they can't get Astra 6.1 out of the door...
Really hate how they only show it vs Opus @ Max and not Low/Medium/High.
throw20261022 1 days ago [-]
OpenAI repeatedly keeps hurting user workflows by altering the model generations family for each new version in unpredictable ways. As such, it is not a sane ecosystem to develop in.
jshmrsn 21 hours ago [-]
2019: We have no idea if computers will ever able to talk like a human in natural language.
2026: We have multiple magical know-everything oracles that can speak any language better than most humans and produce pages of output for fractions of a cent, but the model names are a bit confusing so let's just not bother.
throw20261022 20 hours ago [-]
> the model names are a bit confusing so let's just not bother.
I said neither of these two statements or claims. The fact is that OpenAI's model family and release is not standardized. This means that if I have applications that are using the high, mid, and low tier models, e.g. at v5.6, it's not straightforward for me to update to a new version, e.g. v6 or v6.1 of these models because there isn't immediately a 1:1 mapping at the time of release of the version. Competitors to OpenAI don't have this problem, so I will bother with them.
agentdev001 20 hours ago [-]
What is your implication here, in the case of 6.1 Terra?
throw20261022 20 hours ago [-]
The implication is that 6.1 Terra should've been released together with 6.1 Sol and Luna on the same day, and that these tiers should stick across versions.
MichaelTestGrid 13 hours ago [-]
[flagged]
vinaygundeti18 15 hours ago [-]
[flagged]
evergreenokada 1 days ago [-]
[dead]
civvv 23 hours ago [-]
[dead]
RemingtonDavies 1 days ago [-]
[dead]
OhNoNotAgain_99 1 days ago [-]
[dead]
grd_dev 24 hours ago [-]
[dead]
throw20261022 1 days ago [-]
If they had any courage, they would release 6.1 Astra, but they don't.
Update: Look at the OpenAI employees downvoting me.
rfgplk 1 days ago [-]
The next model will likely be Nova
enraged_camel 1 days ago [-]
People keep asking OpenAI for a mode that's as good as Opus 5.5, and they release this instead. Interesting strategy. Let's see how it works out for them.
cloverich 1 days ago [-]
They don't have anything competitive with Opus 5.5 yet. There is a supposed model in the works, "Bel" last time I paid attention to gossip, that is thought to be better, later this year. Allegedly Anthropic caught OpenAI off guard and / or their different training approaches ended up working better than anticipated.
My expectation is Fable 5.5 is next overall improvement, and my bet is OpenAI's "Bel" will be the next competitive one (or, their lack of safety culture will bite them/us in the process).
laurels-marts 1 days ago [-]
There’s also a chance they surrender the dev market altogether to Ant and just focus on the “normie” segment with dots, computer use, excel etc. In which case they won’t have anything competitive to opus-5.5 anytime soon for SWE.
rockdoe 24 hours ago [-]
Luna and Sol-6.1 are crazy cost effective and all flows here have switched to them, so I'd say really well so far.
There's very few stuff that needs Astra/Opus level intelligence.
Luna 6.1 would be amazing.
user43928 1 days ago [-]
This release is interesting for enterprise at API rates.
At least unless the Terminal Bench 4.0 score is more indicative than the other ones, because that one shows it clearly worse than 6.1 Sol for both performance and cost.
babelfish 1 days ago [-]
Taking the benchmarks in the article at face value, this is ~on par with Opus 5.5 for ~1/4 the price!
mwigdahl 1 days ago [-]
Which is why I don't take the benchmarks at face value...
babelfish 1 days ago [-]
So you have decided the model is bad without having used it yet?
cbg0 1 days ago [-]
If they had a model on par with Opus 5.5 there would be no need to undercut it by that much, that's probably the logic.
rfgplk 1 days ago [-]
Anthropic models are comically overpriced. They also gobble up an obscene # of tokens to get the same result as others. The price delta is literally 5x.
mwigdahl 1 days ago [-]
No, I have decided I don't trust benchmarks without using the model first.
entrope 1 days ago [-]
Taking the benchmarks (as reported by OpenAI) at face value, why wouldn't they show Opus 5.5 on the OS world benchmark, and show no Anthropic models at all on the "factual error rate" benchmark
enraged_camel 1 days ago [-]
...no it's not? At max effort, it scores 50% on terminal bench vs. Opus 5.5's 70%.
babelfish 1 days ago [-]
ALE (top benchmark in post) is 60% for Terra vs 63% for Opus - I guess we'll have to wait for AA Index!
saretup 1 days ago [-]
48 for GPT-6.1 Terra Max vs 58 for Opus 5.5 Max on AA Index
hansonkd 1 days ago [-]
I hate Opus 5.5, constant overreach and over-engineering for any project
Silagi 1 days ago [-]
That's wild to say when comparing to the OpenAI family of models. I've never seen so many tests that validate tests that validate tests.
tamimio 1 days ago [-]
Right?! I think sol has some obsessive tendencies, i literally caught it the other day spawning a sub sub agent doing a final fourth test for a piece of code that isn’t even critical say in embedded or similar, now I have to instruct it to only test once!
cesarvarela 1 days ago [-]
Medium or High? Depending on the task, you might want to switch between them.
petesergeant 1 days ago [-]
You’re holding it wrong.
paulpauper 1 days ago [-]
"better writing"
let's see if it can pass the Pangram test. Many services purport to fool it, but fail.
_fw 1 days ago [-]
Because Pangram is snake oil.
Seattle3503 1 days ago [-]
2025 called and it wants it's take on AI detection back.
Note that while that page is from Pangram, the studies on it are not. If they are excluding studies that show Pangram performs poorly in realistic conditions, please share those studies!
BoorishBears 1 days ago [-]
If Pangram was snake oil (it's not but could be more mindfully marketed), it's definitely not now that every major lab has added watermarking.
You wouldn't even need to be able to read the information the watermark encodes to recognize it exists in a piece of text
ComputerGuru 24 hours ago [-]
This is incorrect. You cannot identify the presence of text watermarks if you do not have access to the models so that you can spot where expected tokens diverge from actual tokens based off the weights. That does not preclude Pangram from being able to reliably identify LLM-generated text from models whose outputs they’ve trained their own detection model upon.
BoorishBears 22 hours ago [-]
They have unwatermarked text from the same base models, human text, and watermarked text.
If they couldn't train a reliable detector with that they wouldn't exist.
petesergeant 1 days ago [-]
Fooling Pangram and writing well are not the same axes, even if there’s some correlation.
paulpauper 1 days ago [-]
Everyone defaults to checking pangram when AI is susepcted, so it's still an important test.
guybedo 1 days ago [-]
i'm not sure anyone cares about Terra though.
Previous Terra model was so good it was discontinued.
OpenAi needs a model to compete with Opus 5.5, and that's not Terra 6.1.
throw20261022 1 days ago [-]
It could have been Astra 6.1 but they don't have the courage to make it happen.
ckx 1 days ago [-]
GPT-6 only just came out! The AI landscape is always on the move!
NostraDavid 1 days ago [-]
2026-09-03 is almost 2 months ago already - that's basically half a lifetime in LLM land!
ckx 1 days ago [-]
True!
ape4 1 days ago [-]
They say it has "A significant step up on factuality and coding".
My browser is underlining "factuality" in red. AI hallucinations or hallucinatiality are still a big problem in my experience.
bahmboo 1 days ago [-]
First known use of factuality was in 1859. It's a "real" word. You can look it up.
Luna 6.1 is fine with it, but the results aren’t good enough. I’m going to try Terra 6.1 (from the post, I should expect about the same results). And hope it doesn’t complain as much about these non existent cyber security risks that for some reason it keeps raising.
P.S. I also tried Deepseek Flash 4.1, but the results simply didn’t cut it.
P.P.S. This all investigation was a major eye opener on why we need to run local models or be forever tied to some artificial limits to AI for the masses.
https://news.ycombinator.com/item?id=50254836
No, they can't, considering Terra never existed for 6.0. It's a mess, and no self-respecting user would care for it.
Also, you can just…wait a little longer until all the models for the version number you want are available? You don’t lose anything, so how is not “sane” to do so?
But like the original release of Terra, it's still a weird middle child between the small-model-that-got-substantially-better (Luna) and the large-model-that-got-significiantly-cheaper (Sol). The TerminalBench 4.0 cost-per-task results are a bit confusing and I'm not sure why you would pick Terra over Sol in that case. Overall this release seems like a flex on Sonnet 5.5: a model which is having its own middle child problems between the godly Opus 5.5 and the surprisingly capable Haiku 5.5.
Destroyed by Anthropics crazy decision to price it x5 after 100k context.
200k also used to be the limit for Sonnet for a long time.
If Haiku 5.5 also had 200k, it'd actually be usable for us. As it is, it's Luna or Sol.
At least this my discovery when I did evals on my reviewers skills. 10k LOC is high end for high effort or around 150k context using opus. You can push towards 15k but that’s on max effort with about 400-500k ctx.
Fast models like haiku do not even need 1M ctx in my view
While it does offer some meaningful improvements over 6-Luna for a few workflows, I suspect most of those benefits stem from 6.1-vs-6, as opposed to Terra-vs-Luna.
It is having improvements in other fields than coding while being more cost efficient.
If all I want is coding at a cheaper price, then deepseek is a good model for that.
I am more than okay to have a bifurcation of models good for coding and models good for everything other than coding. I strangely imagined it many years ago.
Everyone is suggesting OpenAI to release a model close to opus 5.5 yet it's releasing this. On one hand we can imagine that maybe their training cycle wasn't able to produce this whereas gpt 6.1 terra was made.
But on the other hand, I also now wish to see an OpenAI model which is good for just pure coding, like an absolute beast at it while being cheap (so the opposite of terra)
ironically, one of the few appreciations that I might give to OpenAI if they pushed this model to prod and are facing some pushback as developers like us want/only see coding models. They could've benchmaxed it and it would've been slightly irrelevant. I'd much rather see reality.
At the very least, this new model won't make things more expensive for anybody and would only make use-cases cheaper depending on what they are, so it's a fine deal for me.
I also wonder maybe this is why open source labs aren't targeting this space and why the only focus of AI labs is coding nowadays. Everything else just doesn't seem so worth it from investor, even end user, training perspective.
Maybe this is the reason why we have gotten almost an exponential growth in coding capabilities as that became the most (if not only) capability that truly really matters.
To not wish for discussion to go to a tangent, nonetheless this could also explain why our work as a software engineer is going through a slight crisis of meaning as these AI models have gotten impressive at coding.
As an (aspiring) developer (currently in 1st year college), I don't wish for AI to replace coding even more for me or to push it even more or to suck the joy out of it even further. So I am more than happy for these models to stay as they are and rather make advancements elseplace. Ironic because it is us developers who use AI as well and we want cheaper workflows and so we would also want models to be cheaper and better at coding. There are some fine lines of contradictions and irony in all of it. I'd much rather also have open-weight models of such terra-like models but I imagine that because of the reasons that I mentioned above, we are most likely gonna see it the most focused on coding but still open-weights is pretty close.
The more choices the better (though choice-fatigue is a real thing), I hope I was able to explain some of the nuance I feel about this particular launch.
First we had:
> Luna (Small), Terra (Medium), Sol (Big).
Which was fairly smooth after you got used to their planets.
But then they added Astra as the "Bigger," which was mostly still self-coherent:
> Luna (Small), Terra (Medium), Sol (Big), Astra (Bigger)
But then things go confusing, they released Sol-6.1 which sometimes is superior to Astra, and discontinued Terra which made it look like this:
> Luna (Small), Sol-6.1 == Astra
But with Sol-6.1 costing the old Terra-6.0 price, while, as I said performing as good as the old "Bigger" model in many non-3D/UI situations.
Now they're re-introduced Terra, as a 6.1 release, making this even more confusing.
Did Microsoft's chief branding officer move over to OpenAI?
In general, LLM performance is incredibly multidimensional. There's no guarantee of a fixed ordering where the model that is best at math is also the model that's best at jokes. Despite that complexity, much of the variation in performance is explained by just a single variable: model size. In general, models that are bigger will (a) be more performant / reliable, (b) cost more per token, and (c) have better recall of esoteric facts. As you note, our models are ordered:
Luna < Terra < Sol < Astra
Even today, I'd recommend this as a general starting point for picking models. If you care about quality only, pick Astra. If you're a bit price sensitive, pick Sol. If you're very price sensitive, pick Terra or Luna.
Of course, model size doesn't capture all variation. Models that are trained later might have new data or new techniques or new architectures that earlier models lack. You can see this with models from all providers - Sonnet 5.5 outperforms Fable 5.1 in ways, and Gemini 3.8 Flash outperforms 3.1 Pro in ways.
In the case of GPT-6.1 Terra, I'd say Terra is probably better than Astra and Sol at technical communication, similar on vision and computer use (but much faster!), and a bit worse on most other things. For API developers, it's a nice midrange option if you want something cheaper than Sol but a lot smarter than Luna (especially for vision / computer use / communication). For subscribers, it's not a bombshell release - if you're hitting usage limits, maybe it helps you go a bit further, and if you want clearer communication or faster computer use, it can be a nice option to select.
If you have suggestions on how to name our products better, I'm all ears. Our intention is to make things clear and simple, even if we miss the mark sometimes.
Unfortunately, a lot of the complexity is somewhat irreducible and due to to the actual complexity of the products. Scaling model size is different from scaling reasoning effort which is different from scaling multi-agent count. So if we just swept all of those under a single name of "more IQ", it would make the naming simpler but the experience much worse, as important details would be obfuscated from end users.
What's confusing to me in addition to the naming is that there are no published guidelines I can use, as far as I can see. Put yourself in the user's shoes. On what basis should one pick a model for a given task?
I can understand smaller models being dumber and faster. That aligns well with scenarios where you are willing to compromise; e.g. you're classifying content or asking a model to format text into a table.
But when you want "maximum intelligence, cost and time be damned", there's the implication that bigger models are smarter on a linear scale, and every provider's model release naming is currently conflicting with that expectation.
This isn't just a naming problem, but a communication problem. If I knew that Astra was the best at coding but a bit bad at writing, and Sol was almost as smart but better at technical writing, then that's the kind of thing I expect to be documented.
One possible way: A simple matrix with horizontal columns like "writing", "coding", "math", rows are the models, and each box is a number of stars from 1-5.
Instead, this sort of thing becomes lore, shared by word of mouth.
I don't know, to be honest. Technical documentation is such a multidimensional skill, and I also don't know your budget. I'd imagine Terra might have the most readable prose, but Astra might be more likely to catch bugs or mistakes to highlight. If the documentation is about a more esoteric language or situation, Astra might have a greater edge than if it's a common language or situation. If you have a long list of documentation guidelines, that might also give more of an edge to Astra. I wouldn't want to make a recommendation without doing a lot of testing, and I haven't done a lot of testing.
But to assess whether my preference on my problem is going to generalize across other people, other programming languages, other documentation types/formats, etc., and then analyze statistical significance is a thornier question that I don't have the data to answer. I'd rather make no recommendation than a weakly informed recommendation.
That's what users currently end up having to do on their own. Or they just give up in confusion, which presents in your site analytics as either bounce, buy later or buy less.
Why not have a model comparison site? Visitors choose from a dozen-ish broad categories of work (eg technical documentation, market communications, etc), then pick from a dozen sample prompts across typical tasks in that category, and see the model responses with avg time and approximate cost. Especially helpful would be having a toggle overlay with annotation highlighting relevant differences, then if they have (or create) an account, they can edit the annotations and add their own with a sharing link.
If my job was to actually help my co-workers and friends make better informed choices they feel more confident in, I'd be doing something like that. The extra credit version would help users narrow down to two models and then let them convert to a 'Test Drive' trial - same workflow but from their own prompts, side-by-side A/B response comparisons with annotation and sharing link.
As you said, the answer to "Which model is best for me?" is actually complicated, conditional and inevitably a bit confusing. When that's the reality, the right pattern isn't 'Communicate to Convert', it's 'Build credibility and confidence with maximum clarity and genuine helpfulness'.
People vote on pairs of model responses and they aggregate everyone's votes into a leaderboard.
One thing to be aware of that the distribution of prompts on arena.ai can be pretty different than real-world usage. The people who spend time on that website casting votes tend to be more interested in entertainment than professional productivity. And turn-based voting can punish useful behavior like asking clarifying questions instead of guessing at assumptions.
It’s overkill most of the time, but frankly all these models are “cheap”, and I’d rather waste compute than brain cycles trying to discern which dials are best for each individual as-hoc query.
Create a realistic eval suite for your use case using your actual data that you manually label, with easier, more complex, rarer samples, and run each model you are considering against it and decide based on the usual confusion matrix, cost, latency, or whatever metrics matter to you. There's really no other way to know – and you might be surprised that cheaper models can often perform your task(s) relatively well at a fraction of the cost.
I’d imagine there’s only a tiny percentage of users that can or want to do this.
but if you want to use some LLM agent for anything commercial and you want to get the best and at the same time you don't want to overpay the easiest and the surest would be to create your own benchmark for your specific use case.
Every model I've tried is just abysmal at this, for what it's worth.
If any do it well please let me know and I'll give it a spin.
I recently experimented with some image generation models and I was surprised at how much context/prompting is needed to get something that doesn't look Lovecraftian. There was seemingly no downside to adding more and more detail up to the limit. Prose sits between code and images in terms of fuzziness and I wonder if promptmaxxing is the winning strategy for technical documentation.
I've been thinking that fine-tuning a model to write differently shouldn't be that hard (tedious, perhaps) and if I had the resources I would consider doing it myself.
It's high-level (and doesn't break out coding/writing etc), but https://developers.openai.com/api/docs/models says:
> If you're not sure where to start, use GPT-6 Astra, our flagship model for complex reasoning and coding. Choose GPT-6.1 Sol to balance intelligence and cost, or GPT-6 Luna for cost-sensitive, high-volume workloads.
That page also seems to be updated after each new release.
I do not use agents to write on behalf of myself. I write my won technical write-ups and my own slide presentations. A lot of people don't; and it shows. There's much that agents can do better than humans now, but not everything.
Thanks. I think the very response you wrote here is a better form of marketing for the software engineers than what openAI puts out there. When everything is groundbreaking and everything tops charts, nothing is.
I suspect a lot of target demographic (developers) prefer simpler and honest communication about not just the models high points but low points too. With the linked Terra post, if I saw 1 chart that clearly highlighted the usecases for which Terra is _worse_ than higher tier models, I would be more convinced overall.
Everything trying to occupy the top spot in my brain's limited "top model" representation space is not helpful.
The website linked starts out with:
> We’re introducing GPT-6.1 Terra, a fast, efficient model that combines detailed visual reasoning and precise computer use alongside clearer, more natural writing. A significant step up from GPT‑5.6 Terra, it excels at the communication and computer-based tasks that fill the workday, from drafting proposals and interpreting technical drawings to filling out forms and editing videos.
If you go father down, it has graphs and details with specifics of this, including:
> We measured GPT-6.1 Terra’s computer-use capabilities... At medium reasoning effort, GPT-6.1 Terra scores 15 percentage points higher than GPT-5.6 Terra at about one-third the cost. It even performs on par with GPT-6 Astra at only one-tenth its price per token.
and
> On MMMU-Pro... It even nearly matches the scores of our most capable frontier model, GPT-6 Astra, and GPT-6.1 Sol at much lower cost.
Terra and Luna are dirt cheap and good enough for 50% (or more) of the typical junior dev's workload, and great for routine work. Sol and Astra have so much written about them at this point that it should be obvious what those are good at. The large/gigantic models for things that need full intelligence and slightly more branching thinking.
Luna < Terra < Sol < Astra, fine. But does Terra 6.1 beat Sol 6? Does Fable 5.1 beat Opus 5.5? If I build something around this Terra 6.1 release, then when Luna 6.2 is released in a month, should I swap?
If they were called iPhone 14 Luna and iPhone 16 Sol then this is less clear.
Or on the flipside, if the models were called GPT 6.1 Mini and GPT 6.1 Max, nobody would be arguing about the names being hard to grok. The words 'mini' or 'small' are more descriptive than 'Luna' or 'Terra'.
If you look back the naming problem started around the point that OpenAI started doing reasoning models (o1, 4o) where they had to describe this fork in model behavior that let a smaller model think longer and be smarter.
You can come up with naming schemes that don't clash, but there's something to be said for this approach where the thinking level is like you describe and the model name requires you to learn that Astra is the largest model.
* Luna (Moon)
* Terra (Earth)
* Sol (Sun)
* Astra (Galaxy/Universe)
The relative sizing seems dead simple.
1. Even more dead simple if it just used Moon instead of Luna, Sun instead of Sol, etc. - one less quick translation in the head.
2. Or even more dead simple if it just used "Small", "Large", etc. - removing another quick translation in the head. (On the other hand, all these suggestions are English - maybe Luna, etc. is more understandable for non-English speakers?)
Is a sunburst supposed to be bigger than a flare? I mean you get solar flares too? Or is a solar flare bigger than a sun 'burst' which usually just means some sun coming through clouds.
You're right! The relative sizing of models is dead simple. This doesn't help when there are improvements between release numbers, and models are released one at a time instead of as a family.
When Sol 6.4 comes out and the latest Astra is still 6.1, which do you use?
Or I might try both and pick whichever I prefer for a given task...
Sure, in a hypothetical world where the latest Max phone is two generations older than the latest Mini phone. In which case now you have to evaluate not only screen size but ALSO specs and component age and software support etc. How badly do you need the bigger screen? Etc
If you are comparing the iPhone 16 Max to the iPhone 16 Mini, everything is a lot easier.
I wish
I'll just stick to mostly Sol on low/medium/high and start experimenting with Terra tomorrow onward to see what it does noticeably better than Luna/Deepseek flash.
Any pareto frontier model, especially from the leading labs is guarnateed to be good enough for whatever your expectations are at this point. A quick few day test should be enough for anyone to make their own conclusions.
If you want to go optimize your mouth hygiene it is actually more important to use dental bristles (flossing is not as good), because most of the work is done by the mechanical cleaning action of the tooth brush and you need to reach the places where the tooth brush cant get into.
Once you have done that, the next step is alcohol-free xylitol based mouth wash. Caries bacteria have trouble digesting xylitol so this reduces the presence of them in your mouth.
Only after you have done that, are you ready for something better than fluoride. Fluoride is a remineralization agent. Your saliva also contains calcium that helps your teeth remineralize. So the next thing you need is a more potent remineralization agent. Basically your tooth paste acts as a more concentrated saliva. Because of that, it is actually better to use e.g. fluoride in the morning and a different agent in the evening.
The classic options are bioglass (novamin) or hydroxyapatite. The latter is also available as mouth wash.
> Run your own evals then if you have this much of a need to "correctly" choose.
No. OpenAI want my money, it's on them to communicate properly.
I understand how it can be confusing, but I don't think that this is exactly a problem.
Luna through Astra offer qualitative descriptions of the model's size. Presumably OpenAI trains its biggest model (Astra-ish, maybe even larger) first, then distills it to smaller sizes.
Verison numbers reflect something about the architecture or training dataset. Again, they keep these details close to their chest, but we should expect that weight-for-weight higher versions should beat lower versions.
This offers us two axes of 'better', but these axes are also not directly comparable. An earlier but bigger model may or may not beat a newer but smaller model. This is where 'spiky' intelligence comes in.
This is also where calls to 'run your own evaluation' really do start to matter. If you care enough about this performance[1], then you also have specific problems that you care about. No generic benchmark can fully answer that question, hence the need for a specific evaluation.
[1] — And you don't have to! A casual user is probably served reasonably well by anything put out in the past six months. I think Microsoft is still serving GPT-4o under some Copilot branding.
Whether they train the biggest model first or not is not relevant; what matters is what they release. Right now we've got 6.1 Sol and 6.0 Astra. Over at Anthropic they've got Opus 5.5 and Fable 5.1. The existence of a hypothetical Fable 5.5 that I'm not allowed to use doesn't help me.
I wouldn't necessarily say that. Atmospheric mass may be obvious to us, but I suspect it's not nearly as universal knowledge as one may assume.
And I actually assumed at first that they were based on distance from us (terra < luna < sol), so I kept confusing terra and luna.
- Understanding how effort and class interact (Is a high effort terra above a low effort Sol? Are larger models mostly always better regardless of effort?)
- understanding when (if ever) a new model tends to be better than an older but higher class one. Is an old astra better than a new terra?
If increasing effort and increasing model class produce improved results in a different way, I miss making it explicit in which way. In other words: What kind of issue warrants going up in effort? what kind of issue warrants going up in model class? What kind of issue is unlikely to be solved by either?
Oh, easy:
https://deepswe.datacurve.ai/
Artificial Analysis has their own curves which are pretty similar.
You can literally see this in the charts from the article
Ted,
I'm assuming you're not in the OAI marketing dept because leading with a simple thumbnail like "Luna < Terra < Sol < Astra" is helpful framing and dropping next-level bombs like:
> "Terra is probably better than Astra and Sol at technical communication, similar on vision and computer use (but much faster!), and a bit worse on most other things. For API developers, it's a nice midrange option if you want something cheaper than Sol but a lot smarter than Luna (especially for vision / computer use / communication). For subscribers, it's not a bombshell release - if you're hitting usage limits, maybe it helps you go a bit further, and if you want clearer communication or faster computer use, it can be a nice option to select."
Is just far too clear and useful! I'm kidding... but it's funny because it's a little bit true.
> Our intention is to make things clear and simple
Your post is the single best product marketing communication I've ever seen from OAI. Serious suggestion: maybe just say something like that on the website. Not buried in a blog post but above the fold.
That could solve the 'more IQ' problem, because you'd actually have different dimensions (e.g. for 'raw intelligence', technical writing, emotional understanding, etc) that you could use to communicate the differences in models better.
(Of course people would instantly start complaining that the benchmarks numbers are probably fake etc, but it is what it is.)
> Despite that complexity, much of the variation in performance is explained by just a single variable: model size.
please release model size :p
Personally, I pretty much exclusively use claude for coding work (sorry!). Our application is running almost 100% on OpenAI API (except for some fallback models). It is the best API on the market. The models are excellent particularly for any application that is cost sensitive.
Is this really so/could you give a rough example please? I've not used a lot of Claude or ChatGPT, but 3.1 Pro always feels most grounded and intelligent. 3.8 Flash (in my very limited testing) isn't bad, but it feels less nuanced.
(Fwiw I don't use it for code, but ask maths and real world logic based questions.)
I'm not the one you were talking with but catching you is godsend since I just ran into a lot of confusion on the naming myself. I think some of it has to do with how they are labeled in the app.
I think simple numbers in naming is great, even 6.1 is fine bc I understand that if I want the latest model, I just need to look for the biggest number. Where I got confused is in the "work" tab, I saw 'Astra' is set as the default. I see the label for it is:
"Frontier intelligence for your most demanding tasks."
To me that reads "this is the 'best' model." Then I see Sol-6.1 which is labeled:
"For coding and every day work."
Then Terra-6.1 is:
"Terra 6.1 Codex Model" (tbh this one might be the most confusing, I'm aware of codex the product but Sol is the one labeled good at coding)
From my POV as a basic plan-based user wanting to do something like...
> design a robust event-driven system, here's the details [...]
I honestly have no idea which one I should use.
Would Astra-6 produce a better design than Sol-6.1? Is the '.1' a big step up from Sol-6? (I'm guessing no but I truly have no idea). I would assume the answer is Astra-6 would produce the best one, even though the labels don't imply it is necessarily geared towards coding?
But tbh the "default" seems to always work for me... and after reading your comment, typing this one out, being forced to think about it... it doesn't seem too bad given your challenges in communicating the purposes of each model and wide audience of consumers. I'm also sure anyone could find a million problems with decisions I've made in software I've built so I try to not be too harsh :)
<version>-<size>
Where <version>: 6.0, 6.1, etc., and <size>: xs | sm | md | lg
E.g. 6.1 Terra = 6.1-sm
If you eventually make specializations like "math" and "jokes" then I would call it something like 6.1-jokes-sm.
We're probably headed to further specialized agents where a single ordering or even model size won't mean a lot.
A favorite…
Prompt: “Tell a joke about mother in laws”
Gemma2 response: “What’s the best kind of mother in law? A dead one.” 0_o
Medium v10 vs Big v2 makes it look weird and confusing. Especially when there are a sea of influencers and twitters with vibe assessments who will tell you new models best olds models.
Can’t you just start at the top or bottom.
Release new big for paying customers -> release new medium for paying customers -> release new small for everyone
Rinse and repeat. We’d all be much happier
Maybe stop using version numbers then and just name models like you'd name children.
>If you have suggestions on how to name our products better......
Too late for that. I hope you guys can at least stick to these naming scheme for at least another 2-3 years going forward. Not everyone follows AI / LLM intensely, and it seems there is a new naming changes every year or so.
At least conceptually OpenAI's naming is easier to understand than Claude.
It terms of capability, it would be great to make it clear somewhere that this is 6.1 Model "+" technical communication, vision and computer use improvements. So that at least we know what we should try with these new release and choose a better model for certain workload.
Right now, other than pricing, it is confusing as hell. And in hindsight I wonder if there are fatigue in choosing one people just go with Open Model and stick with it if it is good enough.
With an optional flash mode.
Then as you release models, you pick the model and thinking level that you believe is pareto optimal for cost/performance.
So this auto updates each release, like Terra was probably off this list entirely, but maybe this would appear now in Auto medium or something like that.
I know from looking at charts that each model seems to have sweet spots (like I always use this model in this thinking level and if i want to go cheaper i go here). If you picked those for us, it would be interesting. I do think you should keep the model families for those wanting more consistent experiences, but for many devs they may want to just pick auto something and have it update over time. I think nice if it actually just said in parenthesis what it was, something like: Auto Medium (Terra 6.1 High)
I am not sure if everyone would agree here though. I'm obivously advocating for more naming schemes, and i understand the irony.
xlow = Luna Low
low = Luna xhigh
medium = Terra medium
high = Sol medium
xhigh = Sol xhigh
Max = Astra something?
I.e. are all version numbers independent and Model X 7.5 is not necessarily of the same generation or cut off as Model Y 7.5? Or do all of your 6.1 models have something in common - cut off data, generation, baseline technology, approach, architecture, or something else?
Thanks! :)
Back in the day we used numbers to also sometimes indicate size, which could make it extra confusing: e.g., we launched GPT-4.1 (an incremental update on GPT-4) after GPT-4.5 (a new model that was a big jump over GPT-4).
It’s probably just unavoidable. Like when Apple release a new MacBook Air first and it is (in some ways) faster than the old MacBook Pro.
I think the planets thing is a bit confusing though. I hadn’t really thought about it being based on the size of the planet. I assumed it was based on how far away they were with terra therefore being the base one.
You have Luna, Terra, Sol and Astra, each with 4+ reasoning levels. That's ~20 options to choose from.
On a repetitive task like customer support or something, where you have decent benchmarks, you can plot cost versus performance, but you can't really do that on coding.
Also, I think the naming is great. Like you said, Anthropic faces the same issues you do. Opus is much better at Fable in a lot of stuff without being quite as huge or as smart.
Congrats on the launch.
I'm just worried the system design won't be as good if I choose a lesser model so to speak.
Edit: typos/grammar
This seems like one of those cases, where CalVer would be appropriate, and you likely have ideas what you aim for while training, so technical model names (ids?) could include aliases to most things you said. Like 2026q4-xlarge-researcher, 2026q4-medium-humanitarian, 2026q4-medium-coder, 2027q1-small-analyst etc.
Of course we have far too many benchmarks (of questionable utility), but it would be helpful to get OpenAI's take on where it slots in, and how the landscape changes as these get rolled out.
Versions are good for backend but when communicating I want 2 or 3 levels of detail.
Class_Generation_Modifier
Astra_8_Fast
That's it.
This means: Class - Tells me what it can do, how well. Generation X end of life is ~18 months. Modifier: Variation of options.
And the modifier can be dropped if you add more classes.
> If you have suggestions on how to name our products better, I'm all ears. Our intention is to make things clear and simple, even if we miss the mark sometimes.
How about naming the models:
"Small", "Medium", "Large" and "Very Large"
Or refer to them by parameter count like other companies (e.g. this is 6.1 5T)
If I have a clothing store, and rather than name the clothes sizes Small, Medium and Large I decide to call them Urgon, Planfar and Fangor I shouldn't be surprised if people think my size naming is confusing or silly. And IMO it's not a great explanation to say "oh it's easy, if you are skinny then you should get Urgon, but then if you are a bit more of a medium build you should get a Planfar, but if you want something a bit baggier than Fangor will really work out for you"
As a point of comparison, GPT-3 was launched in a series of small, medium, large, and XL. Back then, XL meant a mere 1.3 billion parameters!
You wouldn't - you would just put the version before it - 6.1 Small, 6.0 Medium etc. Not that complex.
Think of an iPhone 18 Pro - No need for 'iPhone 18 Even Better Super Pro' because the 'Pro' is relative with the version/generation.
Despite Opus 5.5 being newer and "better" if you read benchmarks, I've found Fable to be better for some things (subjective stuff)
I don't expect the vast majority of your users care. I would guess they want a rough proxy for "cost" and will assume cost=>capability are roughly linked in a linear fashion.
it is very possible that the users weighted by spend do, and I'm therefore simply wrong. But, 2cGPT 6.1 X > GPT 6.0 X GPT Y Astra > GPT Y Sol > GPT Y Terra > GPT Y Luna
This order is very important.
Instead, we got Opus 5.5, Sonnet 5.5 and Haiku 5.5 released weeks apart, and so are Sol 6.1 and Terra 6.1?
all training runs have to at some level share the compute capacity ; while they have more than enough to run many large runs concurrently, they don’t have enough compute to run everything concurrently.
Well you definitely missed that mark extremely badly here.
I can’t tell you the difference between the 10 different versions of the current iPhone and I can’t you the difference between the 5-6 different models put out by the Anthropics/OpenAIs of the world. At this point I just try to track the flagship and then the cheapest one or worst case I just use the flagship.
I’m sure internally there’s data showing people want choice and options but decision fatigue is of course very real.
Terra 2026.2
Tiny/2026-11. Medium/2026-6. Big/2026-10. XL/2026-01. It may not be sexy, but people know exactly what it is and what to use.
Have a model alias called "math", another "jokes". Quality labels/date suffixes to these obvious labels. Repoint as needed.
Something for people who don't want to play the cute "which part of the solar system is trending this week" game.
Even "writing" as a single concept is very multidimensional. There's fiction writing, nonfiction for experts, nonfiction for learners, deep research, quick lookup Q&A, copyediting, writing in documents / artifacts, instruction following about style, and default quirks (e.g., em dashes, negation, loving the word "tapestry"). Some models are great writers but less diverse, meaning people prefer the first story they read but disprefer the tenth. How does one label such a model?
But this is exactly what we have! Except it's worse, because the model usage recommendation isn't in the API, it's in some random person's X advice that's no longer current this week.
I understand AI companies wash their hands of this problem because it's hard, and they themselves barely understand the true capability matrix of the models they're selling.
I just don't think some effort to categorize models and expose that to users would be messier than what we have -- model selection driven by influencer voodoo bandwagons and analogies to literal astrology.
it's almost like ya'll let the AI name it ;)
Hire an actual marketing person instead of letting engineers / ML researchers do things they're clearly terrible at ?
That is false, considering Astra and Luna don't even exist for 6.1. Terra didn't exist for 6.1 until yesterday. You don't get to make up imaginary tiers. The inconsistency wrt their availability, starting with day 1 of a new model version, is precisely the problem. OpenAI is a user-hostile joke of a firm that users would do well to abandon.
You certainly know that it happened not by accident that traditional computers are producing reliable results, a lot of work went into this.
Computers must produce reliable results, they are simply not useful if not.
People will understand that at some point, but a lot of damage already has been done.
For some reason, this is just hilarious to me. Idk if you're being serious or if this is passive aggressive but the thought of OAI actually taking advice from a random HNer and basing their naming scheme off that is peak.
Where else would you go? I'm honestly scratching my head thinking of someplace that wouldn't run you out on a rail, but that would give serious feedback. Kernel mailing lists are out, maybe there's something like that which I can't remember off the top of my head, but...
Also, nomenclature is incredibly hard. There are only two hard problems in computer science: cache expiration and naming things.
The primary users of any API are the designers and engineers of the applications that use it. How many laypeople know what Postgresql is? How many of the applications they use run on it?
The lineup of models is still the same.
There are four models: luna, terra, sol, and astra.
Pricing increases for each level. This new 6.1 Terra is 50% cheaper than Sol 6.1.
They are lowering prices for newer releases, so yes if you compare Sol 6.1 to 5.6 Terra it won't look right, but why are you doing that? Look at the same release version for comparable numbers.
Each model improves in successive releases but they maintain pricing and capability coherency within a release version.
Yes, Sol 6.1 is supposed to rival Astra for some work, which is what you'd expect from a later versioned release model against a prior release bigger model.
Assuming they release Astra 6.1 at some point, I expect it'll be better than Sol 6.1, at a higher price.
I don't understand what people are asking for here? No reductions in pricing? Different model names on every version release? Not releasing any models for a version number until they're all ready?
Given the operations of the model are opaque, can you see how someone (most people?) might be confused?
I see a ton of criticism and not a single person coming up with anything better, unless it's to take away options, like removing effort level.
What about buying Intel CPUs? A car? Electrical components?
If you're not just using this at a consumer level to ask about cooking recipes (and if that's your use case, you really don't care about API pricing, or any of this, and can just use the default), and are using it professionally, then this seems like a low bar of effort to understand what you're buying.
I do grant you the relationship between model and effort level has an unclear relationship, but there are also tradeoffs in most complex technical products that are not always clear; this is exacerbated by the fact that this product is stochastic.
Why are engineers using this as a professional tool expecting this to be bought like a consumer gadget and then blaming the maker?
This morning I asked GPT which models and settings I should use to inspect my VM dashboard with computer control. It said: - GPT-6 Astra at Medium - GPT-6.1 Sol at Medium (alt) - use Luna only when the route was already known.
But I had already completed the task twice with Luna and it was 5.6! It navigated through a few pages of my dashboard, pulled up the 30-day usage chart, analyzed my CPU and RAM, and explained why I didn’t need an upgrade. I’d give Luna 5.6 an A on this task.
Maybe OpenAI should give us a “Tiptronic” selector: recommend the model and reasoning level based on the prompt, while still letting users override it.
What are you talking about? If it's for some slop you vibe check it. If it's for a serious effort you write an eval. Same as it has been for the last 3 years.
Why is it ok for engineers to have confusing names and ways to interact with something?
How am I supposed to decide between these options: https://imgur.com/a/bXHWowL
How am I supposed to keep up with the changes every few months?
No worries, you can always ask a AI.
Are you presenting this as evidence that OpenAI's naming scheme is bad?
If you don't want to spend the time to know the difference between all these different suppliers and their models, then why are you using whatever this program is that gives you so many choices, most of which are bad? Why is this OpenAI's fault?
They should have a guide saying "Terra is best at technical documentation and communication, Astra is best at large software projects and research, Sol is good for development" or something like that.
After that you just have model "strength" and version number. But of course strength is only comparable at equal version number and the two axes are almost parallel.
Anthropic also does this, perhaps to a lesser extent.
Let's say I have some task that Luna 6.1 cannot do, but Terra 6.1 can. So I build my product around that. In a month when Luna 6.2 is released, then I have to figure out whether it's worth to evaluate and swap or just wait for Terra 6.2.
Waiting is the way to solve this. And despite the outcome being the same, making me feel like I'm being forced to choose to wait is a worse consumer experience.
They released gpt-6-sol (alongside astra and luna but no terra) and 7 days later released gpt-6.1-sol making gpt-6-sol basically DOA before any real adoption. Everyone knows it wasn’t just better RL/post-training, it was a completely different model underneath. Now they randomly reintroduce terra.
I’m sorry, but if you’re not even a tiny bit befuddled by this then you must be taking whatever OpenAI is having.
I think the names are more of a reference to price. Luna is always cheaper than terra which is cheaper than Sol which is cheaper than Astra (for the same model number).
Per token, yes. But per task it's not even necessarily the case depending on how much thinking is needed.
But the model names are linear by price.
But minor credit to Anthropic, at least they've mostly stuck to their four model names (Haiku, Sonnet, Opus, and Fable). You mostly know where a model is MEANT to be positioned from the name alone. OpenAI cannot keep it consistent from release to release.
But, yes, currently Opus Vs. Fable has the same issue as Sol-6.1 Vs. Astra.
Hmm? They've been using the Luna/Terra/Sol/Astra scale for a few versions now and as far as I can tell it maps directly onto Haiku/Sonnet/Opus/Fable, in terms of the kind of thing it's meant to convey (if not in terms of the scale of differences).
It's like iPhone 17 being faster than iPhone 16 Pro. Because it's newer and benefits from advances in chip architecture, despite being a lower model tier. Sometimes different iPhone models are released at different times.
Both companies have screwed up their naming royally. Maybe that's intentional - I'm sure it doesn't hurt Anthropic to have people using Fable a bit longer than they need to and racking up a huge bill in the process.
It is meant to give a rating inside a generation. Different generations are not necessarily comparable. an i7 is better than an i5, but not if you compare an i7 7xxx with an i5 14xxx.
A nvidia xx80 card is better than a xx60 card, but not if you compare a 1080 to a 5060.
The only difference for OpenAI is that not every generation has every model.
Obviously not, otherwise the four models would be called Copilot, as well as the harness.
Sure Sol 6.1 on low/medium can be considered "cheap" considering its abilities, but it's clearly not meant for the same type of tasks as Luna. You just grow an intuition for when something is solvable with a Luna-sized model. I'm guessing Terra is good for the tasks that you'd typically use Sol 6.1-low for, or for Luna type tasks (or other not-novel problem) where you want to reduce the risks of suboptimal decision making. It's exactly what I'd go with for feature additions where the correct solution is already mostly designed/specced-out.
You are also rolling the training of your models by size and the lat est small model can sometimes beat an older model.
For a while we thought you just train the big model first then distill smaller models from it but since we've learnt the importance of on policy RL (basically the way a small model figures out a problem is very diff from a big model so it's best to RL optimize each model distinctly)
Yes, a more modern smaller model can beat an older bigger model at some tasks, that's what can be confusing.
If you scale a model down, or switch out attention with linear attention, or introduce KV-compression, etc., you don't really know how it will affect model performance before you've trained the model.
And once you've spent millions training it, you might as well make it available if it's pareto optimal for some tasks. It also gives you more insight seeing how people use it in practice.
And some variants (e.g., the medium one) being skipped or delayed for a certain generation doesn't really change the overall size branding.
"Speed, quality, price: pick two". I think I'd rather see a choice like that than the current convolution of $-per-token, sub quota, latency, simulated reasoning level, model size, context size, etc, etc.
I don't want to rush to their defense here, but, when model families are getting refreshed every 6 weeks, there is going to be significant overlap, particularly with these staggered releases
It's nice that they have Terra which seems to fill a void that's been needed since Sol 6 focused on cost (at the expense of quality), Sol 6.1 got a huge boost in quality, but became unusably slow unless you paid Astra prices - at which point, why not just use Astra?
Now at least we have a viable other option again.
Luna (Small), Terra (Medium), Sol (Big), Astra (Bigger)
is perfectly clear, sol beats astra sometimes because Astra 6.1 was scrapped. and note that opus 5.5 beats fable 5.1 most of the time too without a skipped model.
“Test them all”
(For some reason they say "Cache read" instead of "Cached input" for 6.1 Terra)
6.1 Astra and 6.1 Luna must be coming up soon. Will they reduce their prices further?
I'm optimistic about them tackling frontier intelligence next. Astra 6.1 please?
The "cost per task" metric is just a weaselly way to sound like they didn't just double the cost for most users, and aren't having their proverbial lunch money stolen. They can no longer claim to provide more tokens, more capable models, or more usage in general. So they made up a new metric where "task" could be defined to mean whatever they need it to mean.
They had to become VERY specific in their wording to present any kind of value proposition that will entice people.
[0] https://newsletter.semianalysis.com/p/anthropic-subscription...
5x is calculated in terms of API costs, nobody pays those extreme Anthropic prices outside enterprise users.
> like they didn't just double the cost for most users
Most users aren't on their $200 plan. Most users (including on the $200 plan) continue being subsidized.
Ignore value then. Anthropic still provides about 3x as many raw tokens. The 20x\200 plans provide about 10B for openai versus almost 30B for Anthropic.
Do you really think that Opus uses 3x as many tokens as Sol per task? I don't.
> Most users aren't on their $200 plan.
Fair enough. I should have said most heavy users.
> Most users (including on the $200 plan) continue being subsidized.
They continue to be fifty percent less subsidized, I guess.
Look, I don't fault them for trying to become profitable. I fault them for the weird weaselly business speak about cost per task. It's BS game playing and we all know it.
I'm honestly rooting for them, and hope it all works out. For now though I'm pretty salty that my bill jumped by 150% in one month. Gaslighting about made up metrics just makes me more angry.
> Do you really think that Opus uses 3x as many tokens as Sol per task? I don't.
You don’t have to speculate. Pull up artificial analysis, pick either opus or sol and scroll down to total tokens and to total price. Opus (and sonnet) use 4x the tokens and 7x the dollars.
That difference alone removes the difference that semi analysis found.
Fair point. I just tried it, and I stand corrected. Claude uses almost exactly 3x as many tokens.
In the end Claude produced 2.25x more code, and used 3.01x more tokens (ignoring the cached system prompt, etc). It's answer was also much more substantial and feature rich - including many features I did not ask for.
I gave them both the prompt "Without searching the internet please write and test a python script that accepts a year (in any common format) as input and provides the date that easter fell or will fall on in that year. The output should be in discordian date format. You may not use any libraries."
The thing I found most interesting were the unit tests. Claude implemented the core code using both the Meeus–Jones–Butcher algorithm AND Gauss, but only used Gauss as a sanity check to ensure that it's primary code worked. Sol just validated that it matched some dates it could remember. That explains a good chunk of the almost doubled output, and it was a reasonable thing to do, though neither answer was "wrong."
Token usage looked like this (based on parsing it out of the jsonl, neither provider seems to make it easy to get real billing information per project/conversation.
I believe oh-my-pi will be able to ingest your Claude and codex conversations, count tokens, and then assign API-equivalent rates to them. May be worth looking into.
We agonize over cost per task to the extreme for services/functionality built atop of APIs and can absolutely guarantee you that it is the real world metric that counts most right after “is the model good enough for the task at hand,” and furthermore that, yes, in fact, it is quite often that we see certain models/providers easily using 3x or more tokens on average for the same tasks (or even the exact same task) than a supposedly similarly matched model from another provider. Once you become intimately familiar with the thinking traces from specific models, you can preempt this to some extent with manipulations to the system prompt (essentially heading off entire thinking branches) but the nuking of reasoning traces in the name of preventing against distillation attacks has made this harder.
Once you are inserting models into non-creative (and software development is creative work) it is imperative to start creating your own small evals. Not just to check how different models perform in that equation, but to catch model drift over time.
Nope to both. Not even close.
The only way it works is if you define "task" in a way that OpenAI is good at it and Anthropic isn't.
Second of all, barring more cuts, yes I do think the $20 and $100 plans do deliver more tasks because the models have gotten cheaper in the meanwhile (like this very announcement).
> Are they delivering 50% more than Anthropic per token?
So say the public benchmarks. Pull up artificial analysis, “Cost (USD) to run all evaluations in the Artificial Analysis Intelligence Index”:
Sol 6.1 (max): $1,082 Opus 5.5 (max): $8,708 Sonnet 5.5 (max): $7,260
>7x difference in cost for the same tasks.
The original plan doesn't exist anymore. To get the same usage you now need a $500/month plan. The new plan that costs $200 is different in every way that counts. It was actually a 150% price increase for people who want or need what the old plan used to provide.
Don't get me wrong, I genuinely LIKE OpenAI. It's probably worth it even at the new price. I just hate the rhetoric designed to Goodhart past the math.
> the models have gotten cheaper in the meanwhile (like this very announcement).
That is not good news for plan users. Plan users now get less of something that also has less worth to begin with. It would only be good news if they got more of it.
For example, if I buy the "unlimited car wash" package for $19.99/month and the car wash lowers the price per wash from $10 to $1 my subscription becomes less valuable, not more. The breakeven has moved, so I now need to wash my car more than 20 times before the package is justified.
If the car wash then increased the price to $50/month would it be wise for me to keep the package? Would lowering the price per wash make the package more valuable to me somehow? No, it would do the opposite.
I’m not sure I see how. Subsidy got cut by half. But in the meantime 6.1 came out which uses 25% fewer tokens than 5.6 and costs 1/3 as much per task.
So for 1/2 the budget you use to have with 5.6 on the old plan, you can do more with 6.1. I don’t see that as a problem.
Interesting that both "none" and "low" reasoning levels appeared not to spend any reasoning tokens at all, and in fact low ended up using less tokens and costing less than none here.
"medium" did use 496 reasoning tokens, but that's apparently not enough here to return a visible reasoning trace.
OpenAI's current frontier model lineup is gpt-6-astra, gpt-6.1-sol, gpt-6.1-terra, gpt-6-luna, so here's a comparison grid of all of their pelicans at different reasoning levels: https://static.simonwillison.net/static/2026/openai-pelicans...
4 pelicans at once!
It's also a very fast model, of all 33 models we've tested so far, only Haiku 5.5 and Nemotron 3.5 Super VL have passed the exam faster but both scored lower. It also wrote the _least_ output in its responses.
[1] Benchmarks for data analytics tasks: https://plotly.com/blog/gpt-6-1-terra-plotly-data-analytics-...
Used on medium seems to be the best use-case[1] and cheaper than 6.1 Sol.
[0]: https://aibenchy.com/compare/openai-gpt-6-1-sol-max/openai-g...
[1]: https://aibenchy.com/compare/openai-gpt-6-1-sol-medium/opena...
The pelicans place it nicely between Sol and Luna in terms of both pricing and results.
The only thing that pelican have is "models where not trained on this" maybe that true months (or years ago) now it's on every HN post and these AI labs are monitoring HN
Luna, Terra, Sol, Astra - are a good setting tweak on how much you want to spend vs what you want to get out of it.
Virtually every task I give to either is better served by going up a tier and down a reasoning level, or down a tier and up a reasoning level. I simply don't understand their use case. No testing I've done has shown them to have a niche where there isn't a cheaper-with-equal-results or more-performant-at-same-cost option.
If you pay API prices for inference in your product stack & have specific use-cases w/ custom evals, these smaller models provide an additional cost/speed lever to adjust. Ultimately, I think the point is to compete on cost for use-cases where Sol on the lowest reasoning is overkill.
If you have a fairly straightforward usecase where Terra performs just as well as Sol, but is faster and cheaper, why not use it?
Spinning up a fresh session would be a reasonable choice in the above case, but sometimes you still need the retained context.
Benchmarks don’t really tell you much. For API best will always be a model+effort sweep on your internal evals (and if you don’t have internal evals, do that first).
For individual coding harnesses, honestly I just go with vibes. The best workflow is the one that works for you.
On subscription for coding and stuff I am basically on Astra Ultra all day long.
https://docs.github.com/en/copilot/reference/ai-models/suppo...
2. There are rote or simple tasks where higher reasoning isn't needed and one model vs the other wouldn't even be distinguishable.
3. How many tokens are used during reasoning depends as much on the task as the reasoning level.
4. Sonnet is faster and half the price per token, so Opus will only be cheaper if it can spend fewer tokens. This won't be the case for all tasks. Opus and Sonnet will also consume the same number of tokens to load context, so the relative price comparison depends a lot on where the task is context-heavy or reasoning-heavy.
What are you smoking and can I have some?
I have some business automation process for some customers that I maintain, where Terra with low reasoning ended up with the best price/performance & latency. With an equivalent Luna with high reasoning, price was usually better, but quality had a higher variance (very much depends on where the thinking takes the model) and latency for high reasoning was worse.
Take a look at this reply.
To justify the pricing of Sol and Astra. Old business trick.
if you're just talking about use in codex or claude code, then sure, leave it on opus all the time. but for running routine api requests, sonnet is great.
I think it's also a fake lever to give to finops / it teams to try and keep costs down.
GPT-6.1 Terra scored 84, one point over Grok 4.7 (83), in 5.4 min for $0.19 vs Grok’s 36.5 min and $1.69. The cheap models keep getting better.
GPT-6.1 Terra: https://jonclegg.github.io/pacman-bakeoff/#gpt-6.1-terra
Closest competitor (Grok 4.7): https://jonclegg.github.io/pacman-bakeoff/#grok-4.7
I still don't know which gpt model to use! Anthropic one I know as its simple.
Here its gettung weird
Just one thing about the advert, how can I be the only person here thinking the ad is weird - like for the marketing dep, its a bit weird the ad features the same tram that broke and tragically like ended some people - happened for like the first time ever in 100 years, like a year ago ( 14 months ago) just a bit weird.
Im honestly not trying to be negative, but its kinda weird I kinda feel I should mention it because noone else has?
Im sure literally anyone from southern europe must have thought this the second they saw the ad. It was big news last year.
Am I mad? Just a heads up for PR to always check things like this first when releasing a video? Maybe its just me? Just trying to help
is opus 5.5 better than sol 6.1? hmm, i really can't tell. maybe claude is a bit more diligent and identifies more issues up front. but sol has this can do attitude and the codex harness just gets on and does it's job done.
On AA intelligence index, this fills in the cost per task frontier nicely between 6 Luna and 6.1 sol. So when 6.1 Luna inevitably releases...
The OAI model tango seems to not have an end in sight, for better or worse. Give me more API toys!
(I work at OpenAI.)
My plan for OpenAI recently expired and now I’m thinking to test Anthropic ecosystem. My primary use case is programming only.
Should I go with API use or subscription? For coding harness, I jump between codex, OpenRouter, pi, whatever coding harness is trending on that day
Anthropic does not allow you to use 3rd party harnesses -- you have to use Claude Code. Claude Code is an excellent harness, though.
If you want the freedom to move between harnesses you should use OpenAI or xAI
Unlike Sol, Luna and Astra. Odd.
e.g.: "You must not use OpenAI models or their outputs to generate, prepare, select, or supply training material or signals for models that are intended as alternatives to or substitutes for OpenAI models or services".
I personally like Terra, it kind of does what I would ask Sol or Astra, but dont have to count tokens as much. And Luna has been amazing tbh, it takes designs from prototype and in record time is implementing it, so that was a huge win.
But yeah, they always had confusing way of naming products and if you are not following news, you will be confused about what it is.
According to this, Sol 6.1 x-high is both cheaper and smarter than Terra 6.1 max.
If not, yawn...
It's nice that their peak-rate hours end quite early in the day UK time.
If you live in a country with a weak currency, DeepSeek models are definitely a godsend.
I still use SOTA for the main model though, but also I'm perfectly aware it's mostly out of FOMO.
Deepseek is more than enough for default subagents though. Any well defined and verifiable task and there's absolutely zero difference between SOTA and good-enough models. Used to be just speed, but Deepseek is fast.
For myself, using Codex, I always have to explicitly tell codex to use Luna with high reasoning for smaller, bit size tasks.
Subagents can spawn with other profiles e.g. scout (uses @smol) and reviewer (@slow). You can set SOTA with higher thinking for @slow and Deepseek with lower thinking for @smol or something even faster/cheaper.
There's also weirdly named /vibe mode that gives explicit instructions to the main model to act as an orchestrator but I find this unnecessary if your main is SOTA, they are already super eager to spawn subagents by default.
It literally burned through 2 subs.
It's very likely this is a next checkpoint of GPT-6 "Sol", over-optimised for the web chat and offered to Codex subscribers as a "please go away" offering.
If this was half the price, I'd be very interested, but at 1/5/0.05? Feels like another panic release because they can't get Astra 6.1 out of the door...
I said neither of these two statements or claims. The fact is that OpenAI's model family and release is not standardized. This means that if I have applications that are using the high, mid, and low tier models, e.g. at v5.6, it's not straightforward for me to update to a new version, e.g. v6 or v6.1 of these models because there isn't immediately a 1:1 mapping at the time of release of the version. Competitors to OpenAI don't have this problem, so I will bother with them.
Update: Look at the OpenAI employees downvoting me.
My expectation is Fable 5.5 is next overall improvement, and my bet is OpenAI's "Bel" will be the next competitive one (or, their lack of safety culture will bite them/us in the process).
There's very few stuff that needs Astra/Opus level intelligence.
Luna 6.1 would be amazing.
At least unless the Terminal Bench 4.0 score is more indicative than the other ones, because that one shows it clearly worse than 6.1 Sol for both performance and cost.
let's see if it can pass the Pangram test. Many services purport to fool it, but fail.
https://www.pangram.com/blog/third-party-pangram-evals
Note that while that page is from Pangram, the studies on it are not. If they are excluding studies that show Pangram performs poorly in realistic conditions, please share those studies!
You wouldn't even need to be able to read the information the watermark encodes to recognize it exists in a piece of text
If they couldn't train a reliable detector with that they wouldn't exist.
Previous Terra model was so good it was discontinued.
OpenAi needs a model to compete with Opus 5.5, and that's not Terra 6.1.