Structured Output with JSON Mode

1. The Problem: "Here is the JSON you asked for"

You ask the model for JSON. It replies: "Sure! Here is the JSON file: json* *{ "answer": 42 }* * Hope that helps!"

This breaks your app. Your JSON parser crashes because of the text "Sure! Here is..." Text is for humans. JSON is for systems.

2. Using json_mode

OpeAI (and many others) support a response_format parameter. It forces the model to output only valid JSON.

model = ChatOpenAI(model="gpt-4o", model_kwargs={"response_format": {"type": "json_object"}})

chain = prompt | model

# IMPORTANT: You MUST mention "json" in the prompt for this to work
prompt = ChatPromptTemplate.from_template(
  "Output a JSON object with a field 'joke' about {topic}"
)

res = chain.invoke({"topic": "bears"})
print(res.content)
# Output: { "joke": "Why did the bear..." } (No extra text)

3. Parsing to Dict (JsonOutputParser)

Getting a string that looks like JSON is good. Getting a Python dictionary is better.

from langchain_core.output_parsers import JsonOutputParser

chain = prompt | model | JsonOutputParser()

res = chain.invoke({"topic": "bears"})
print(type(res)) 
# Output: <class 'dict'> -> {"joke": "..."}

4. Summary

If your code needs to read the answer, use JSON mode. Never rely on "prompt begging" (e.g. "Please don't say anything else"). Enforce it at the API level.

Key Intuition: "If software consumes it, structure it."