OpenAI makes Decisions API widely available to all customers
The fast, low-cost model enters a market led by TypeSafe's Jev and other decision models.
Labs#05
Labs
Image: TechCrunch
The short version
- OpenAI made its Decisions API model available to all customers on Thursday, after announcing it at its developer day on September 29.
- The model is ChatGPT Luna, reconfigured to speed up inference and lower cost.
- In initial trials, customers including Box, Datadog, Ramp, and Speak used it in place of OpenAI's LLMs.
- Speak's CTO said the time between actions fell from 1.4 seconds to .13 seconds.
Why it matters
Decision models act as fast classifiers. They choose from a set of options and give a confidence rating, and developers have found they can replace LLMs in many workflows at much lower cost. The article notes that there is no clear standard for a good decision model, so teams need to test models on their own work.
“The big unlock with Decisions API has been the speed, shrinking things from one to three seconds to 100, 120, 180 milliseconds”