cerno 0.1.2: Ollama's own System One endpoint as a host

A week ago I wrote about cerno , a small service that answers JEV-style typed questions — noul, choice, score — by reading one token’s probability distribution out of a local model. That post ended with “JEV is the polished, hosted version of this idea”. Since then Ollama 0.35 has shipped its own version of it: an endpoint, POST /v1/systemone, and two decision models built for it, nimble and tev1.

cerno 0.1.2 can use that endpoint as a host. CERNO_HOST=systemone sends every question to Ollama’s /v1/systemone instead of cerno’s own letter prompt. Nothing changes for callers: the API, the SDKs, the TUI, calibration and confidence all stay exactly as they were.

What Ollama shipped

The endpoint takes the same three primitives, with Ollama’s field names:

curl localhost:11434/v1/systemone -H 'content-type: application/json' -d '{
  "model": "tev1",
  "state": "Ticket: Serverraum-Klima ausgefallen, 31 Grad und steigend.",
  "questions": {
    "urgent": {"type": "noul", "instructions": "Ist das dringend?"},
    "team": {"type": "choice", "instructions": "Welches Team?",
             "criteria": {"IT": null, "Facility": null, "HR": null}},
    "sev": {"type": "score", "instructions": "Wie schwer?",
            "criteria": ["unkritisch", "gering", "mittel", "hoch", "kritisch"]}
  }}'

{"model": "tev1",
 "answers": {
   "urgent": {"type": "noul", "noul": 0.9558},
   "team":   {"type": "choice", "choice": "Facility",
              "probabilities": {"IT": 0.0297, "Facility": 0.9427, "HR": 0.0276},
              "confidence": 0.764},
   "sev":    {"type": "score", "score": 2.65,
              "legend": {"0": "unkritisch", "1": "gering", "2": "mittel",
                         "3": "hoch", "4": "kritisch"},
              "probabilities": {"0": 0.0079, "1": 0.1567, "2": 0.2126,
                                "3": 0.4216, "4": 0.2012},
              "confidence": 0.165}},
 "usage": {"input_tokens": 753, "output_tokens": 4}}

The important difference from what cerno does with a general model is who writes the prompt. Here Ollama does, and it hands back a probability for every offered answer rather than the top 20 tokens of a distribution I then have to sift. There is nothing to fold, and nothing can fall out of the window.

The models are small: tev1 is a 4.2B model at 4.5 GB, with a 0.8B variant at 810 MB, and nimble is 9B at 9.5 GB. All three are Qwen-family models, and Ollama lists them with a decision capability.

Why cerno stays in front

If Ollama answers the questions itself, why keep a service in between? Because the endpoint’s answer isn’t quite cerno’s answer, and the callers shouldn’t have to care which host is behind it.

• The pick is the argmax. Ollama’s score is a probability-weighted level on a 0-based scale (2.65 above, between “mittel” and “hoch”). cerno reports the most likely level as score and the weighted mean separately as expected_score, both 1-based, on every host.
• Confidence means the same thing everywhere. cerno ignores the endpoint’s confidence and computes its own, one minus the normalised entropy, so a 0.8 from systemone and a 0.8 from gemma4 through vLLM are the same statement.
• Calibration still applies. The per-request and per-model temperature works on the endpoint’s probabilities exactly as on logprobs.
• Validation still happens first. A malformed request is refused before any model is asked, whichever host is configured.

The cost is one host call per question: cerno asks the endpoint one question at a time, with up to CERNO_MAX_CONCURRENT_QUESTIONS (default 4) in flight.

Through cerno, the same ticket reads:

{"answers": {
   "urgent": {"type": "noul",   "noul": 0.8822, "truncated": false},
   "team":   {"type": "choice", "choice": "Facility", "index": 1,
              "confidence": 0.855, "truncated": false},
   "sev":    {"type": "score",  "score": 4, "expected_score": 3.83,
              "legend": "hoch", "confidence": 0.233, "truncated": false}},
 "model": "tev1",
 "usage": {"input_tokens": 411, "questions": 3},
 "timing_ms": {"total": 248}}

Three questions in 248 ms, answered by tev1 through Ollama’s endpoint; sev comes back as level 4, “hoch”, with an expected score of 3.83.

One seam, still

The first post described ModelHost as a trait that answers exactly one question: what is the distribution over the next token? The endpoint doesn’t produce tokens at all, and I wanted to keep that seam rather than grow a second code path in the engine.

So the new SystemOneHost answers the same question, honestly shaped: it returns a distribution whose tokens are exactly the offered labels A, B, C, each at ln p. Folding, calibration, normalising and confidence then run on the one engine path as for every other host, and truncated is always false, because every label is always there.

For that, the request a host receives now carries the question behind the prompt — its id, the state, the question text, its shape and the lettered options. The token hosts ignore it, and SystemOneHost ignores the rendered prompt instead.

Two details were less obvious than they look.

JSON has no minus infinity

A model that is certain reports a probability of exactly 0 for the losers, and ln 0 is -inf. Every answer carries its raw_logprobs in the response, and -inf is not a JSON number. So a reported zero is clamped:

/// `ln p`, clamped so a reported 0 stays finite: `raw_logprobs` is serialised to JSON, which has
/// no `-inf`. A softmax of these at T=1 gives back `p`.
fn ln(p: f64) -> f64 {
    p.max(1e-12).ln()
}

The order of the options is part of the question

The endpoint takes a choice’s options as a JSON object. A Rust map type would serialise them alphabetically, which changes what the model sees. The obvious fix, serde_json’s preserve_order feature, is a trap: Cargo unifies features across the workspace, so it would change the key order of every JSON document cerno writes. A dozen lines of hand-written serializer keep the order local instead:

/// Options as a JSON object in the order given, each mapped to `null` so the name describes
/// itself. Hand-written because a map type would alphabetise the options and change what the
/// model sees, and serde_json's `preserve_order` would reorder JSON across the whole workspace.
struct NullCriteria<'a>(&'a [(String, String)]);

impl Serialize for NullCriteria<'_> {
    fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
        let mut map = serializer.serialize_map(Some(self.0.len()))?;
        for (_, text) in self.0 {
            map.serialize_entry(text, &())?;
        }
        map.end()
    }
}

The endpoint also refuses an empty instruction, while cerno allows a choice or score without a question. Those get a neutral one (“Which option applies?”, “Which level applies?”).

What the measurements say

cerno-bench can now measure the decision models both ways: through cerno’s letter prompt like any other model, and with --systemone through Ollama’s endpoint, marked ². Same 36 cases as before:

The decision models clearly want their own prompt. Through the endpoint, nimble and tev1 answer all 36 cases correctly, against 97% and 94% on cerno’s letter prompt; they are better calibrated there and, on this machine, faster too. The 0.8B tev1 gains eight points the same way.

The default model doesn’t change, though. gemma4:e2b-it-qat still answers 97% of the cases in 28 ms, and the ² rows are never candidates for the benchmark’s verdict: the endpoint always answers with one of the offered options, so fidelity — the gate from the first post — can’t be measured there at all. Which one to use is now a real choice: the smaller, faster general model on any runtime, or the decision models on Ollama.

Errors

The error mapping is the same as for every other host. A model Ollama doesn’t have is a 422 unknown_model. A model Ollama has but the endpoint doesn’t support — anything that isn’t a decision model — is a 502 carrying Ollama’s own message, “not supported by System One”. So with CERNO_HOST=systemone, CERNO_DEFAULT_MODEL has to name a decision model.

Try it

You need Ollama 0.35 or later, a decision model and cerno 0.1.2:

ollama pull tev1                       # 4.5 GB; tev1:0.8b is 810 MB
cargo install cerno-server cerno-tui

CERNO_HOST=systemone CERNO_DEFAULT_MODEL=tev1 cerno-server
cerno-tui                              # second terminal

To compare the two paths on your own hardware:

cargo run -p cerno-bench -- --reference gemma4:26b-a4b-it-q4_K_M \
  --models gemma4:e2b-it-qat,nimble,tev1 --systemone nimble,tev1

The full numbers are in docs/model-selection.md, and the source is at github.com/cebor/cerno (opens in a new window) .

It is a nice turn of events for a side project: a week after I built the primitives on top of a runtime, the runtime shipped them natively, and the seam I’d drawn around “one token’s distribution” turned out to fit an endpoint that has no tokens at all.