Best practices

You have a business rule and want Clara to apply it reliably. This page shows how to split the rule into questions, word them, pick types and thresholds, check them against your own data, and keep cost and latency low once you are in production.

Everything here uses the request and answer shapes from the API reference. For complete end-to-end recipes, see Guides.

Decompose the policy into questions

Clara answers narrow questions about one record. Your code holds the rule itself: the amounts, the dates, who gets notified. Split every policy into those two parts.

This keeps comparisons with known numbers exact, lets you change a limit without retesting the model, and gives you an answer per fact that you can audit later.

Take an expense policy: “A meal over $75 needs manager approval when the receipt is unreadable or the description does not match the receipt.”

Ask all three questions in one request. The question ids you choose (receipt_readable, expense_type, description_match) come back as the keys of answers.

curl https://api.saanora.com/v1/decisions \
  -H "Authorization: Bearer $SAANORA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "clara-1",
  "state": {
    "employee_id": "E-1042",
    "category_entered": "Meals",
    "amount_usd": 92.40,
    "description": "Dinner with the Acme procurement team, 3 people",
    "receipt_text": "TRATTORIA SOLE  14 Apr  Table 12  Guests 3  ...  TOTAL 92.40"
  },
  "questions": {
    "receipt_readable": {
      "type": "noul",
      "instructions": "Can the merchant, date and total be read from the receipt text?",
      "criteria": {
        "true": "Merchant, date and total are all legible",
        "false": "At least one of merchant, date or total is missing or illegible"
      }
    },
    "expense_type": {
      "type": "choice",
      "instructions": "What kind of expense does the receipt show?",
      "criteria": {
        "meal": "Food or drink at a restaurant, cafe or bar",
        "travel": "Flights, trains, taxis, rideshares, parking or fuel",
        "lodging": "Hotels and other overnight stays",
        "supplies": "Office equipment, software or materials",
        "other": "Fits none of the options above"
      }
    },
    "description_match": {
      "type": "score",
      "instructions": "How well does the description match what the receipt shows?",
      "criteria": [
        "contradicts the receipt",
        "partly matches, or too vague to check",
        "fully matches the receipt"
      ]
    }
  }
}'

The amount is in the record so Clara can compare the description and the receipt, but the $75 comparison happens in your code.

Rules of thumb

• One fact or judgment per question. If a question joins two conditions with “and”, a yes cannot tell you which one was true. Split it.
• Keep arithmetic and lookups in code. Amounts, date ranges, account status and anything else you already hold as structured data are exact in code.
• Ask about the record, not your business. Clara sees only state and your questions. If an answer depends on context, such as the customer's plan or the order history, put that context in the record.
• Use readable question ids. Your code reads them, your logs show them, and they make the request self-documenting.

Write instructions Clara can answer

The instructions field is the question. Write it so a careful analyst who sees only the record could answer it, with no knowledge of your company or the conversation that led here.

For a yes/no question, the optional criteria object ({"true": "...", "false": "..."}, or yes and no keys) lets you define both sides. Use it whenever the boundary is not obvious, as receipt_readable does above.

Write option descriptions

A choice question takes a criteria object that maps each option id to a short description, with 2 to 255 options. Clara reads each option's id and description to decide. Your code reads the ids, which come back in choice and as the keys of probabilities. Keep ids short and stable, and put what belongs in each option in its description.

Aim for options that are mutually exclusive (a record fits at most one) and that together cover every record you will send. If several labels can apply at once, such as tags on a support ticket, ask one noul question per label instead of one choice.

Choose the question type

Prefer noul and choice. Score scales are the least reliable type. When you need one:

• Keep it to 3 to 5 levels.
• Describe each level instead of numbering it: "low: one user affected, a workaround exists" rather than "2".
• If you only act on one cut point, consider a noul question instead: “Is the business impact high or critical?”

Give Clara a way out

Answers are always one of your options, and the probabilities always sum to 1. With no way out, the probability for a record that fits nothing has to land on options that do not fit. When “none of these” or “not enough information” is a real outcome, make it an option:

"team": {
  "type": "choice",
  "instructions": "Which team should own this ticket?",
  "criteria": {
    "engineering": "Broken product functionality or errors",
    "billing": "Invoices, charges, refunds and plan changes",
    "success": "How-to questions and account guidance",
    "unclear": "The message does not say enough to choose a team"
  }
},
"wants_refund": {
  "type": "noul",
  "instructions": "Does the customer ask for their money back?",
  "criteria": {
    "true": "The customer explicitly asks for a refund, credit or chargeback",
    "false": "No refund is requested, or the message does not say"
  }
}

For choice, add an option such as unclear or none and treat it in code like any other outcome, usually by sending the record to a person. For noul, say in the false description that “not stated” counts as no, so a missing detail does not turn into a guess.

Ask about missing information explicitly

In agents, Clara is right only 59% of the time when it should ask the user for a missing detail. It tends to pick a lookup tool instead. Add an explicit yes/no question next to the tool choice:

"missing_detail": {
  "type": "noul",
  "instructions": "Is a detail needed to complete the request missing from the conversation?",
  "criteria": {
    "true": "The user still has to provide something, such as an order number, date or address",
    "false": "Everything the agent needs is in the conversation or in earlier tool results"
  }
}

Then check that question first, and route to the user above your threshold:

def next_step(answers):
    # Check the explicit question before reading the tool choice.
    if answers["missing_detail"]["noul"] >= 0.5:    # set the cut point from your labeled data
        return ("ask_user", None)

    tool = answers["next_tool"]                      # your choice question over the agent's tools
    if tool["confidence"] >= 0.8:
        return ("call", tool["choice"])
    return ("review", tool["choice"])                # a person approves the tool call

action, tool = next_step(r.json()["answers"])

Act, review or decline

Every answer is a probability, not just a label. Use it to decide how much to trust each answer. The common pattern has three bands: act automatically above a high threshold, send the middle band to a person, and decline or take the safe default below.

# Placeholder thresholds: set each one from your own labeled data.

def decide_refund(answer):
    """noul: three bands on the probability of yes."""
    p = answer["noul"]
    if p >= 0.90:
        return "approve"
    if p >= 0.5:                     # noul is never between 0.2 and 0.8, so review is 0.801 to 0.9
        return "review"
    return "decline"

def route_ticket(answer):
    """choice: act on the top option only when the model is sure of it."""
    if answer["confidence"] >= 0.80 and answer["choice"] != "unclear":
        return answer["choice"]
    return "triage_queue"            # a person picks the team

def is_urgent(answer):
    """score: sum the probabilities of the levels at or above your cut."""
    p = answer["probabilities"]      # levels: 0 none, 1 low, 2 high, 3 critical
    return p["2"] + p["3"] >= 0.70

a = r.json()["answers"]
print(decide_refund(a["refund_eligible"]), route_ticket(a["team"]), is_urgent(a["severity"]))

Why use confidence for choice questions

A top probability of 0.6 means different things for different numbers of options. With 2 options it is barely better than a coin flip: confidence is (0.6 - 0.5) / 0.5 = 0.2. With 10 options it is a strong signal: confidence is (0.6 - 0.1) / 0.9 = 0.56. Confidence puts both on the same scale, so one review threshold can work across questions.

Why not threshold the score value

score is the expected level index: the sum of each index times its probability. On a four-level scale, a record split evenly between levels 1 and 2 scores 1.5, and so does a record split evenly between levels 0 and 3. The second one deserves a person. Sum the probabilities of the levels you care about, as is_urgent does above, and send low-confidence scores to review.

Put the threshold in the question

If you only need one cut point, you can send it with the question. The answer then carries a boolean decision:

"refund_eligible": {
  "type": "noul",
  "instructions": "Does the customer ask for a refund for an order that arrived damaged or never arrived?",
  "threshold": 0.9
}

The answer:

"refund_eligible": { "type": "noul", "noul": 0.94, "threshold": 0.9, "decision": true }

This is useful when thresholds live in configuration next to the questions. For three bands, compare noul in code.

Calibrate on your own labeled data

Benchmarks describe other people's records. Before Clara acts on real traffic, measure it on yours and pick each threshold from the results.

• Collect a few hundred real records that look like production traffic, including the awkward ones.
• Label each one with the correct answer, by people who know the policy.
• Run them through Clara with the exact questions and mode you will use in production.
• Choose thresholds per question: the lowest approve threshold whose correct rate meets your target, and the highest decline threshold whose correct rate meets yours. Everything in between goes to review.
• Confirm on records you held back from the threshold choice, so the numbers are not tuned to one sample.

This script scores a labeled file and prints, for each candidate threshold, the share of records handled automatically and how often those decisions were correct:

import json, os, time, requests

URL = "https://api.saanora.com/v1/decisions"
HEADERS = {"Authorization": f"Bearer {os.environ['SAANORA_API_KEY']}"}
MODE = "fast"                      # calibrate in the mode you will run in production
QUESTIONS = {                      # the exact questions you will send in production
    "refund_eligible": {
        "type": "noul",
        "instructions": "Does the customer ask for a refund for an order that arrived damaged or never arrived?",
    },
}

def p_yes(record):
    for attempt in range(5):
        r = requests.post(URL, headers=HEADERS, timeout=120,
                          json={"model": "clara-1", "mode": MODE, "state": record, "questions": QUESTIONS})
        if r.status_code in (429, 502, 503) and r.json()["error"]["code"] != "project_spend_limit":
            time.sleep(float(r.headers.get("Retry-After", 2 ** attempt)))
            continue
        r.raise_for_status()
        return r.json()["answers"]["refund_eligible"]["noul"]
    r.raise_for_status()

# labeled.jsonl: one {"record": {...}, "label": true or false} per line
rows = [json.loads(line) for line in open("labeled.jsonl", encoding="utf-8")]
scored = [(p_yes(row["record"]), row["label"]) for row in rows]
n = len(scored)

print("approve if p >=   share  correct")
for t in (0.5, 0.85, 0.9, 0.95, 0.98):
    hits = [label for p, label in scored if p >= t]
    if hits:
        print(f"{t:>15}  {len(hits) / n:>6.0%}  {sum(hits) / len(hits):>7.1%}")

print("decline if p <    share  correct")
for t in (0.02, 0.05, 0.1, 0.15, 0.5):
    hits = [not label for p, label in scored if p < t]
    if hits:
        print(f"{t:>15}  {len(hits) / n:>6.0%}  {sum(hits) / len(hits):>7.1%}")

The share outside both thresholds is your review workload. If it is too large, improve the questions first (clearer instructions, better descriptions, a way-out option) before lowering the bar.

Calibration is cheap: 500 requests of about 400 billed tokens each (record and question) are about 200,000 tokens, or $0.10 in fast mode at $0.50 per 1M input tokens.

Fast or consistent mode

Both modes return the same response shape. They differ in how the record is read and billed.

Set the mode in the body, or as a query parameter:

# In the body
{ "model": "clara-1", "mode": "consistent", "state": "...", "questions": { ... } }

# Or as a query parameter, for clients that cannot change the body
curl "https://api.saanora.com/v1/decisions?mode=consistent" ...

Start with fast. Switch to consistent when:

• Answers feed fixed thresholds or audits, and must not move when someone adds a question to the request.
• You add and remove questions over time and do not want to recalibrate the others each time.
• Accuracy matters more than cost: consistent scores 83.6 on JevBench-mini against 81.8 for fast.

For a 1,500-token record with five questions of about 40 tokens each, fast bills about 1,700 tokens ($0.00085) and consistent about 7,700 tokens ($0.00385). usage.input_tokens always reports the exact billed figure.

Batch questions per record

Ask every question about one record in a single request. In fast mode the record is read and billed once. Clara answers five questions about one record in about 0.1 s.

Separate requests give the same answers as one fast request, because each question is answered on its own either way. They cost as much as consistent mode, though, and use five times the requests per minute and five round trips, so batch them.

• Question limits: up to 32 questions per request on Tier 1 and 64 from Tier 2. More returns 422 too_many_questions. If you need more, split them across two requests about the same record.
• One record per request: answers are keyed by question, not by record. To process many records, send them as separate requests in parallel, within your rate limits.

Keep latency and cost low

Clara costs $0.50 per 1M input tokens. Output is $0 and there is no per-decision fee, so the record you send is almost the whole bill.

• Send only what the questions need. Drop internal ids, unused fields, HTML markup, encoded attachments, quoted email threads and signatures. JSON objects are rendered compactly, so you can send structured data as is.
• Stay under 16K tokens. Longer records are shortened in the middle. If a record is long, put the fields your questions depend on near the start or the end, since the middle is what gets cut.
• Batch questions and use fast mode unless you need consistent, as described above.
• Keep instructions and descriptions short. They are billed too, once per question.
• Retry only what can succeed. Failed requests are not charged, and 502 model_unavailable never is, so retry 429 rate_limit_exceeded, 502 and 503 with backoff. On 429 rate_limit_exceeded, wait for the number of seconds in Retry-After. Fix the request or the account instead of retrying 400, 401, 402, 404, 413, 422 and 429 project_spend_limit.
• Know where time goes. latency_ms in the body is model time. The saanora-processing-ms header is total time in the platform. The rest of your round trip is network.

Monitor in production

Saanora processes state and questions in memory and never writes them to its database. For each request it keeps only metadata: time, organization, project, key, model, mode, status, number of questions, input tokens, latency, cost and request id. If you need to audit a decision later, store the record, the answers and the action you took in your own systems.

Log these for every request:

import json, logging, os, time, requests

log = logging.getLogger("saanora")
URL = "https://api.saanora.com/v1/decisions"
HEADERS = {"Authorization": f"Bearer {os.environ['SAANORA_API_KEY']}"}

def decide(payload):
    start = time.monotonic()
    r = requests.post(URL, headers=HEADERS, json=payload, timeout=120)
    round_trip_ms = round((time.monotonic() - start) * 1000)
    body = r.json()

    if not r.ok:
        err = body["error"]
        log.warning(json.dumps({
            "event": "decision_failed", "status": r.status_code, "code": err["code"],
            "message": err["message"], "request_id": body.get("request_id"),
        }))
        r.raise_for_status()

    usage = body["usage"]
    log.info(json.dumps({
        "event": "decision",
        "id": body["id"],
        "request_id": r.headers.get("x-request-id"),
        "mode": body["mode"],
        "questions": usage["decisions"],
        "input_tokens": usage["input_tokens"],
        "cost_usd": usage["cost_usd"],
        "model_ms": body["latency_ms"],
        "platform_ms": r.headers.get("saanora-processing-ms"),
        "round_trip_ms": round_trip_ms,
        "remaining_requests": r.headers.get("x-ratelimit-remaining-requests"),
        "remaining_tokens": r.headers.get("x-ratelimit-remaining-tokens"),
    }))
    return body

What to watch

• The review rate. If more answers land in the middle band than during calibration, your records have changed. Label a fresh sample and recalibrate.
• Accuracy on a sample. Have a person label a small share of automatic decisions on a regular schedule and compare.
• Cost per record. A rise usually means records are growing. Check what is being sent.
• 429 responses. rate_limit_exceeded means you are over your tier's requests or tokens per minute. project_spend_limit means the project reached its monthly spend limit.

To review spend and volume outside the console, export your usage as CSV from Console > Usage. Use a separate project for each environment or feature so usage, keys and spend limits stay apart, and give production projects a monthly spend limit (UTC month) as a safety cap.

Before you go live

• Each question asks about one fact, answerable from the record alone.
• Every choice option has a description, and options do not overlap.
• Questions where “none” or “not enough information” is possible have an option for it.
• Agents ask an explicit yes/no question about missing details.
• Thresholds come from a few hundred of your own labeled records, in the mode you run.
• Your client timeout is at least 120 s, and it retries 429 rate_limit_exceeded, 502 and 503 with backoff.
• You log request ids, usage and the action taken, and production projects have a spend limit.