TypeSafe's Jev Model: Using Structured AI Evaluation in the O2C Cycle

Quote-to-cash has dozens of small decision points that never look like "decisions" on a process map: is this contract clause standard or does it need legal review, is this invoice reply a dispute or a payment promise, does this order need manual approval before it hits billing. Most teams either hardcode rules for these (brittle, misses edge cases) or route everything to a human (accurate, doesn't scale).

A third option has become more common over the past year: send the text to an LLM and ask it to classify. The problem is that a normal chat completion returns free text. You ask "is this urgent," the model writes a paragraph, and now you're regex-parsing a sentence to get a boolean out of it. It works until the model phrases something differently, wraps the answer in a caveat, or you need a confidence level and there isn't one to extract.

TypeSafe's evaluation API (the model is called Jev) is built specifically to skip that step. You send a state (the text or structured data you want evaluated) and a map of typed questions, and you get back typed, probability-scored answers - no parsing required. It's a narrow tool, not a general chat model, but that narrowness is exactly what makes it useful for the kind of repeatable, auditable decisions that show up throughout O2C.

Flavio Copes described it well: Jev is less a chatbot and more "a smart if statement" you call from inside an application. It's also cheap and fast enough to run at the point a document arrives rather than in a nightly batch - $0.042 per million input tokens with free output, and roughly 100ms per call, which is fast enough to sit inline in a webhook that fires the moment a dispute email or an amended order lands.

What Jev Actually Returns

There are three question types, and all three return a structured answer instead of prose.

The distinction that matters for finance workflows: you're not getting a confident-sounding sentence you have to trust or second-guess. You're getting a number you can threshold. "Route to collections if dispute_probability > 0.8, otherwise auto-file" is a rule you can write, test, and audit. "Route to collections if the model sounds pretty sure it's a dispute" is not.

Where This Fits in Quote-to-Cash

The O2C cycle is full of exactly the kind of unstructured-text-to-structured-decision problems Jev is built for. Four places it's a natural fit:

1. Deal Desk: Does This Order Need Manual Review

Most CPQ setups already have approval rules based on discount thresholds or deal size. The gap is everything a rule can't see: unusual payment terms buried in a notes field, a non-standard SOW attached to an otherwise normal order, a customer email that changes the scope mid-negotiation. A Noul question against the order notes and attached documents catches what the discount-threshold rule misses.

Copy-paste the raw request and response:

{
  "state": {
    "order_notes": "Customer wants NET 90 instead of our standard NET 30, and asked to bill quarterly in arrears instead of upfront. Sales rep verbally agreed pending approval.",
    "deal_size": 84000,
    "standard_terms": "NET 30, billed upfront annually"
  },
  "model": "jev-latest",
  "questions": {
    "needs_manual_review": {
      "type": "noul",
      "instructions": "Does this order deviate from standard payment terms in a way that requires deal desk approval before it can be provisioned?",
      "criteria": {
        "true": "Payment terms, billing frequency, or timing differ materially from the standard_terms",
        "false": "Order matches standard terms or deviations are trivial"
      }
    }
  }
}

{
  "model": "jev-1.13.0",
  "answers": {
    "needs_manual_review": { "type": "noul", "noul": 0.97 }
  },
  "usage": { "input_tokens": 397, "output_tokens": 22 }
}

A 0.97 is an easy auto-route to deal desk. The useful part is what this doesn't show: an answer landing between roughly 0.3 and 0.8 is genuinely ambiguous and belongs in front of a human, while anything clearly below that threshold can clear without anyone reading the notes field at all.

2. Invoice Disputes: Classify the Reason Before a Human Reads It

When a customer replies to an invoice, someone on the AR team has to read the email and figure out what kind of response it is before they can act on it. A Choice question does that triage automatically, and because it's a fixed set of options with defined criteria, the routing logic downstream stays simple.

Copy-paste the raw request and response:

{
  "state": "We can't approve this invoice - the quantities on line 3 don't match what was actually delivered. Can you send a corrected version?",
  "model": "jev-latest",
  "questions": {
    "dispute_reason": {
      "type": "choice",
      "instructions": "What kind of response is this to an invoice?",
      "criteria": {
        "billing_error": "Customer is disputing a specific line item, quantity, or price as incorrect",
        "service_issue": "Customer is disputing the invoice because of a problem with the delivered product or service",
        "payment_promise": "Customer isn't disputing anything, just indicating when they'll pay",
        "cashflow_delay": "Customer isn't disputing the invoice but is asking for more time due to their own cash position",
        "no_response_needed": "General acknowledgment with no dispute or commitment"
      }
    }
  }
}

{
  "model": "jev-1.13.0",
  "answers": {
    "dispute_reason": {
      "type": "choice",
      "choice": "billing_error",
      "confidence": 1.0,
      "probabilities": {
        "billing_error": 1.0,
        "service_issue": 0.0,
        "payment_promise": 0.0,
        "cashflow_delay": 0.0,
        "no_response_needed": 0.0
      }
    }
  },
  "usage": { "input_tokens": 450, "output_tokens": 63 }
}

That answer alone tells you whether the invoice goes back to billing for correction (billing_error), to account management (service_issue), or just gets a payment plan note (payment_promise, cashflow_delay) with no billing team involvement at all. Confidence of 1.0 here isn't surprising - the email explicitly names a line-item quantity mismatch, which is about as unambiguous as a dispute reason gets.

3. Collections: Score Urgency Without Guessing From Day-Past-Due Alone

Day-past-due is a blunt instrument. A 45-day-late invoice from a customer who just wrote "sorry, processing delay on our end, paying Friday" doesn't need the same escalation as a 45-day-late invoice from a customer who's stopped responding entirely. A Score question against the full thread (not just the latest message) gives you a rubric-based urgency read that can sit alongside day-past-due as a second signal, feeding into something like this site's own collections email generator to decide tone before the next email goes out.

Copy-paste the raw request and response:

{
  "state": {
    "thread": [
      "Reminder: Invoice #4471 was due 15 days ago.",
      "Sorry, this one slipped through - our AP person is out this week, will process by Friday.",
      "Following up: it's now Friday and we haven't seen payment.",
      "Apologies again, having some internal delays, will update you next week."
    ]
  },
  "model": "jev-latest",
  "questions": {
    "collections_urgency": {
      "type": "score",
      "instructions": "How urgently should this account be escalated in the collections cadence?",
      "criteria": [
        "Low - genuine delay, customer is responsive and committing to specific dates",
        "Medium - repeated delays past committed dates, still responsive",
        "High - unresponsive, vague, or repeatedly broken commitments with no new date"
      ]
    }
  }
}

{
  "model": "jev-1.13.0",
  "answers": {
    "collections_urgency": {
      "type": "score",
      "score": 1.5,
      "confidence": 0.25,
      "legend": {
        "0": "Low - genuine delay, customer is responsive and committing to specific dates",
        "1": "Medium - repeated delays past committed dates, still responsive",
        "2": "High - unresponsive, vague, or repeatedly broken commitments with no new date"
      },
      "probabilities": { "0": 0.0, "1": 0.5, "2": 0.5 }
    }
  },
  "usage": { "input_tokens": 425, "output_tokens": 20 }
}

This one is the most instructive result of the three, precisely because it's not a confident answer. A 1.5 sitting exactly between "Medium" and "High," with confidence at only 0.25, is the model saying the thread genuinely splits between two readings: repeated apologies with a broken Friday commitment read as escalating, but the customer is still responsive and naming dates. That's the correct behavior, not a failure. A low-confidence score is exactly what should route to a human collections rep rather than auto-firing the next dunning stage - the number is doing its job by refusing to be more certain than the evidence supports.

4. Revenue Recognition: Flag Contract Modifications That Need Re-Assessment

Under ASC 606, not every contract change is treated the same way - a routine renewal at the same terms is not the same event as a material scope change that requires reassessing the performance obligations. A Choice question against the amendment language gives the revenue team a first-pass flag for which changes need a closer look before revenue schedules get updated automatically.

Where Jev Doesn't Belong in O2C

Copes' write-up is direct about what Jev is bad at, and the list matters for finance workflows specifically: it can't reliably do arithmetic, counting, or date and time comparisons, and it won't generate explanatory text. His summary of the model's own posture toward these is blunt - "pick a card from the deck, you don't ask it to name a card."

That maps onto a mistake that's easy to make in O2C: don't ask Jev to calculate days-past-due, don't ask it to sum line items, and don't ask it whether an invoice date has passed. Compute those in your own code from actual dates, the same way you already do, and pass the result into state as a fact the model reasons over - the collections urgency example above works because "due 15 days ago" was already computed before it reached the prompt, not because Jev figured out the date math itself. Use it for the judgment calls sitting on top of clean data, never for producing the clean data.

Why the Typed Response Matters More Than the Model

The reason this is worth a separate post from "just use an LLM for classification" is specifically the response shape, not the underlying model quality. Three things fall out of it:

• Thresholds instead of vibes. A confidence or probability value lets you set different automation levels for different risk tolerance - auto-file at 0.95, human review between 0.6 and 0.95, always-escalate below 0.6. You can't build that rule against a paragraph of prose.
• Stable schema for downstream systems. The answer for a choice question is always { choice, probabilities, confidence }. Nothing downstream breaks because the model phrased its reasoning differently this time.
• An audit trail. Finance workflows get audited. "The model returned dispute_reason: billing_error with 0.91 confidence, logged at this timestamp" is defensible in a way that "the AI said it looked like a billing error" is not.

None of that makes the underlying judgment infallible - a Score or Choice answer is still a probabilistic estimate, not a fact, and low-confidence answers should route to a human rather than auto-execute. Treat it as decision support with a clean interface, not a replacement for the judgment call. The value is that the interface is clean enough to actually build automation rules on top of, which free-text classification never quite gets you.

The Pattern, Not Just the Vendor

The specific API shape here belongs to TypeSafe, but the underlying pattern - typed questions in, typed probability-scored answers out, instead of prose you have to parse - is the right shape for almost any classification or scoring step inside O2C: dispute triage, approval routing, urgency scoring, contract change flagging. If you're evaluating tools for this, the API reference is the thing to read closely before building against it, since criteria wording directly shapes how the model splits probability across your options.

Worth borrowing regardless of vendor: Copes' rollout advice is to shadow-run the model alongside an existing manual process first, log its answers next to what a human actually decided, tune criteria and thresholds against that gap, and only then let the low-risk end of the confidence range run unattended. For something touching revenue recognition or payment terms, that's not caution for its own sake - it's how you get a defensible reason to trust a threshold before money moves on it.

If you're building this kind of triage into your own AR workflow and want to see the pattern applied to collections specifically, how Claude automates collections emails and dunning cadences covers the generation side once you know which cadence step an account belongs on.