OpenAI has introduced a new feature called Structured Outputs for API to ensure model-generated responses conform to developer-supplied JSON Schemas. This improvement addresses a long-standing challenge for developers, where large language models (LLMs) often produce unstructured or inconsistent outputs that do not adhere to the desired format.
What are Structured Outputs?
Structured Outputs allow developers to ensure that AI model outputs match a specific structure, typically defined by a JSON Schema. This feature builds upon last year’s JSON mode, which improved the generation of valid JSON data but couldn’t guarantee adherence to complex schemas. With Structured Outputs, the models can now reliably produce data in the required format, eliminating the need for manual intervention, retries, or third-party tools.
One of the core applications of AI is generating structured data from unstructured inputs. For example, developers use OpenAI models to build assistants that fetch data, create reports, or execute complex workflows. Previously, ensuring outputs matched the correct format required tedious efforts, but Structured Outputs automates and guarantees this process.
How Structured Outputs Work
Structured Outputs are available in two forms:
- Function Calling: Developers can enforce strict adherence to tools with the strict: true parameter, which is supported by models like gpt-4-0613, gpt-3.5-turbo-0613, and the latest models gpt-4o-2024-08-06 and gpt-4o-mini-2024-07-18. This ensures that any model outputs will conform to the defined tool schema.
- Response Formats: Developers can supply a JSON Schema via the json_schema option for more general responses. Combined with strict: true, this guarantees the model’s response adheres to the format provided. This feature is exclusive to the newer models, GPT-4o versions, making it ideal for advanced workflows.
Safety and SDK Integration
OpenAI has incorporated its existing safety protocols into the new Structured Outputs feature. If a request violates safety guidelines, the model can refuse to generate output, and developers will receive a refusal string, enabling easy detection and handling of such cases. Additionally, OpenAI’s Python and Node SDKs now support Structured Outputs, simplifying implementation by allowing schemas to be defined using Pydantic or Zod objects.
Improved Reliability and Performance
Significant advancements in model training back the introduction of Structured Outputs. The newest model, gpt-4o-2024-08-06, has been trained to understand and produce outputs that match even the most complex schemas, achieving a perfect score in OpenAI’s JSON Schema evaluations. OpenAI uses constrained decoding to further enhance reliability, which restricts the model to only valid tokens based on the schema. This ensures outputs conform to the schema at every step, making the generation process robust and efficient.
Availability and Cost Savings
Structured Outputs are now generally available across the API, including the Chat Completions, Assistants, and Batch APIs. It is compatible with vision inputs and works with models supporting function calling. OpenAI has also announced cost savings for developers using the gpt-4o-2024-08-06 model, offering a 50% reduction in input costs and a 33% reduction in output costs compared to previous models.
This release marks a crucial step toward more reliable and structured AI outputs, streamlining the development of applications that rely on precise data handling.
