Structured JSON output

When a program uses the model's answer, it needs data, not prose. Ask for JSON with a fixed shape: the field names, the type of each, which ones are required, and the allowed values for fields like a category.

Classify the customer message below. Reply with JSON only, in this shape:
{"category": "refund" | "delivery" | "other",
 "urgent": true | false,
 "order_id": string or null}
Use null for order_id if the message doesn't contain one.

Many APIs have a structured output or JSON mode. Some only promise the reply is valid JSON; others make it match a JSON Schema you supply. Check what yours guarantees. Either way, the values can still be wrong: a perfectly shaped answer can have the wrong category.

So your code checks every reply: it parses, the fields have the right types, categories are from the allowed list, and business rules hold (line items add up to the total, dates aren't in the future).

import json

reply = '{"category": "refund", "urgent": true, "order_id": "A-1042"}'
data = json.loads(reply)
allowed = {"refund", "delivery", "other"}
print(data["category"] in allowed, type(data["urgent"]).__name__)

Without a strict mode, replies sometimes come wrapped in a Markdown code fence or with a trailing comma. Parse carefully, and when it fails, retry once with the error message in the prompt, then hand the item to a person rather than guessing.

import json

reply = '{"category": "refund", "urgent": true,}'
try:
    json.loads(reply)
    print("ok")
except json.JSONDecodeError:
    print("invalid JSON: retry once, then send to a person")

Design the shape to make mistakes visible. Use null for "not in the text" so the model isn't pushed to invent a value. Use a fixed list of values rather than free text for categories. Ask for a short quote from the source next to an extracted value, so a reviewer can check it. Be wary of a self-reported confidence score: models are often confidently wrong.