OpenJev

Fast, calibrated, typed decisions from an open model. OpenJev is an open-source "System One" decision server. Send it a state and typed questions (yes/no, choice, score). It returns a probability and a confidence for each answer in tens of milliseconds. It reads the answers directly from the model's probabilities and parses no text, so an answer cannot go off-schema. Questions can also ask about images.

OpenJev uses the same wire API as TypeSafe's Jev, so their SDKs work with it unchanged. It also answers OpenAI's Decisions API, so the OpenAI SDKs' client.decisions.create works too. It runs DiffusionGemma 26B-A4B (Apache-2.0) through vLLM on an NVIDIA GPU, or through MLX on Apple silicon.

Hosted for free on Codiv, an inference platform for open System One models. Sign up and get 100M input tokens, no card required. https://api.codiv.ai/v1/systemone

OpenJev is an independent project. It is not affiliated with or endorsed by TypeSafe AI.

Models

diffusiongemma-26b is the same DiffusionGemma for text generation. All weights are Apache-2.0. Laya, Verdict, CLM, JevK5 and Clef are other people's models: see Small encoder models, CLM, JevK5 and Clef for details and credit.

Try it

pip install typesafe-sdk
export TYPESAFE_BASE_URL=https://api.codiv.ai   # or http://127.0.0.1:8080 for your own server
export TYPESAFE_API_KEY=sk-codiv-...

from typesafe_sdk import TypeSafeClient

client = TypeSafeClient()
r = client.system_one(
    "Everything is down and we have a demo with our biggest client at noon.",
    {
        "urgent": {"type": "noul", "instructions": "Does the customer need a reply within the hour?"},
        "team":   {"type": "choice", "instructions": "Which team should handle it?",
                   "criteria": {"outage": "service down", "billing": "charges, refunds", "feature": "requests, how-to"}},
        "tone":   {"type": "score", "instructions": "How upset is the customer?",
                   "criteria": ["calm", "annoyed", "furious"]},
    },
)
r.nouls["urgent"].noul        # 1.00
r.choices["team"].choice      # "outage", confidence 1.00
r.scores["tone"].score        # 2.00 (expected level, 0-indexed)

Or with curl:

curl https://api.codiv.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "openjev-latest", "state": "I was charged twice this month.",
       "questions": {"is_billing": {"type": "noul", "instructions": "Is this a billing issue?"}}}'

API

Question types:

• noul (yes/no): takes optional criteria: {true, false}. Returns {noul: P(yes)}.
• choice: takes criteria: {name: description}. Returns {choice, probabilities, confidence}.
• score: takes criteria: [level0, level1, …] (1–10 levels; a single level is answered directly, as for a choice with one option). Returns {score: Σ i·pᵢ, legend, probabilities, confidence}.

confidence is 1 − H(p)/ln K: 1 when the model is certain, 0 when the distribution is uniform. usage.input_tokens counts prompt tokens, image tokens included. usage.output_tokens is 0 unless you set think.

Each response has a Server-Timing header:

server-timing: model;dur=41.2, server;dur=2.8, total;dur=44.0

model is the time spent on the model, summed over the request's reads. It can be more than total when reads run in parallel. server is the remaining time: schema compile, tokenization, validation and serialization. Neither includes your network.

Errors use the same shapes as Jev, checked against the live API:

• 422 with a FastAPI validation list for a field of the wrong shape.
• 400 with a plain-text reason for a question the server cannot ask (no options, too many options, or too many score levels).
• 400 api_usage_error for an unknown model or question type.
• {"detail": {"error_type", "message"}} for auth errors (401/403).
• 429 for rate limits. 529 when the server is overloaded.

Differences from Jev:

• Model names are OpenJev's own. jev-latest and jev-preview are aliases. A pinned Jev version such as jev-1.13.0 gets 400 Unknown model.
• The server reads many questions in chunks of about 12 per read. The chunks run in parallel.

Extensions

These optional request fields are OpenJev additions. A request without them behaves exactly like Jev. TypeSafe's SDKs never send them. They come from the example server in vllm-project/vllm#57250.

curl https://api.codiv.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "openjev-latest", "state": "Look at the photo.",
       "images": ["data:image/jpeg;base64,/9j/4AAQ..."],
       "questions": {"hotdog": {"type": "noul", "instructions": "The photo shows a hot dog"}}}'

think and sequential need a text state. A request that combines them with images gets a 400.

OpenAI Decisions API

POST /v1/decisions takes OpenAI's Decisions API request, so the OpenAI SDKs work with it unchanged. Point them at the server and name an OpenJev model:

from openai import OpenAI

client = OpenAI(base_url="https://api.codiv.ai/v1", api_key="sk-codiv-...")
d = client.decisions.create(
    model="openjev-latest",
    input="I was charged twice this month.",
    questions=[
        {"type": "predicate", "name": "is_billing", "instructions": "Is this a billing issue?"},
        {"type": "choice", "name": "team", "instructions": "Which team should handle it?",
         "choices": [{"value": "billing", "description": "charges, refunds"},
                     {"value": "outage", "description": "service down"}]},
        {"type": "score", "name": "tone", "instructions": "How upset is the customer?",
         "levels": [{"label": "calm"}, {"label": "annoyed"}, {"label": "furious"}]},
    ],
)
d.answers[0].probability  # P(yes)

It is the same read as /v1/systemone, and the same questions get the same answers:

Answers come back in question order, with their name when the question has one. Images must be inline data: URLs, with the same limits as images. Choice values may be strings or booleans. OpenJev never refuses a question, so no answer is a refusal. usage.output_tokens_details.reasoning_tokens counts a think thought, and cached tokens are always 0. The extensions work here too. Errors use OpenAI's {"error": {message, type, param, code}}, all with status 400 for a request the server cannot answer. Auth errors keep Jev's shape. OpenJev serves no gpt-6-luna: name an OpenJev model.

Text generation

POST /v1/chat/completions generates text with the same model, OpenAI style. Use model diffusiongemma-26b. It supports streaming and tools. The server ignores sampling fields (temperature, seed, penalties and similar) because vLLM refuses them for diffusion models. max_tokens defaults to 1024, with a cap of 8192. At most 8 generations run at once, so reads always have room. The MLX backend ignores tools and logprobs and does not return the thought.

How it works

DiffusionGemma is a discrete diffusion model. It denoises a full canvas of tokens in each forward pass, instead of writing left to right. OpenJev uses this to read answers, not write them.

OpenJev builds a canvas in which only the answer slots are masked, one token per question:

canvas in                 one read-only pass         answer out
  q1: [?]        ──►      P(yes) 0.001        ──►    noul  0.001
  q2: [?]                 P(A) 0.000                 choice "billing"
                          P(B) 0.999                 confidence 0.997
                          P(C) 0.000
  q3: [?]                 P(0) 0.000                 score 1.00
                          P(1) 0.996
                          P(2) 0.004

Each label is one token: yes/no for a noul, A/B/C for a choice, 0/1/2 for a score. The model never writes into these slots. One read-only pass gives the probability distribution for each slot, and that distribution is the answer. The numbers above are a real read of "The invoice looks wrong again. Second time this quarter.": not urgent, billing, mildly annoyed.

The read scores only the label tokens, so an answer cannot go off-schema. The confidence comes from the model's own distribution, not from a number that the model reports about itself.

If a slot is uncertain (entropy > 0.1), OpenJev reads three more times with fresh noise and averages the four results. One uncertain question causes a re-read of all the questions in the request. These extra reads add no tokens to usage. Set samples: 1 to get one read only. Question ids never go to the model. It sees q1, q2, q3.

The vLLM part is vllm-project/vllm#57250, merged on 2026-09-22. It adds seeded canvases, read-only steps, step caps and pinned canvas positions for DiffusionGemma. openjev/engine.py adapts that PR's structured_server.py example, with async I/O, bounded concurrency and backpressure. The image pins upstream vLLM main at a3e0243b, which has every DiffusionGemma fix merged since; CHANGELOG.md lists them.

Run your own

Two backends serve the same /v1/systemone. Select one by hardware:

NVIDIA GPU

You need an NVIDIA GPU with at least 24 GB of memory for the NVFP4 checkpoint. We tested on an RTX PRO 6000 Blackwell (sm_120).

The prebuilt image razorback16/openjev runs vLLM and the API server in one container, on CUDA 13. It pins the upstream vLLM commit below, with the two changes in Caveats.

git clone https://github.com/razorback16/openjev && cd openjev
docker compose up -d          # OpenJev on 127.0.0.1:8080 when the model is loaded
curl localhost:8080/v1/models

Or without compose:

docker run -d --gpus all --ipc=host -p 127.0.0.1:8080:8080 \
  -v ~/.cache/huggingface:/root/.cache/huggingface razorback16/openjev:0.7.0

The first start downloads the weights (about 18 GB) into ~/.cache/huggingface. To build the images yourself, build the shared base first, then run docker compose build:

docker build -f docker/Dockerfile.base -t razorback16/openjev-base:cu130-torch2.13 .
docker compose build

Each image has its own Dockerfile in docker/, and all of them start from docker/Dockerfile.base. They share its CUDA, Python and PyTorch layers (about 8.7 GB on disk), so a server that runs DiffusionGemma, Laya and Verdict stores about 21 GB in total.

vLLM listens only inside the container. Set OPENJEV_UPSTREAM to use a vLLM server that you already run.

Without Docker:

git clone https://github.com/vllm-project/vllm && cd vllm
VLLM_COMMIT=a3e0243b1c3ea313b902a43b0f6559fb4148ff81   # the commit the image pins
git checkout $VLLM_COMMIT
# a choice of more than 128 options needs a larger cap, as in the image
sed -i 's/^MAX_LOGPROB_TOKEN_IDS = 128$/MAX_LOGPROB_TOKEN_IDS = 512/' vllm/sampling_params.py
# bidirectional attention inside each image, as in the image
python path/to/openjev/docker/patches/vision_prefix_lm.py vllm/model_executor/models/diffusion_gemma.py
VLLM_USE_PRECOMPILED=1 VLLM_PRECOMPILED_WHEEL_COMMIT=$VLLM_COMMIT pip install -e .
vllm serve nvidia/diffusiongemma-26B-A4B-it-NVFP4 --served-model-name dgemma \
  --diffusion-config '{"canvas_length": 64}' --max-logprobs 32 --enable-prefix-caching \
  --async-scheduling --attention-backend TRITON_ATTN \
  --limit-mm-per-prompt '{"image": 8, "video": 0}' \
  --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4 \
  --override-generation-config '{"max_new_tokens": null}'
pip install -e path/to/openjev && python -m openjev

Performance

A read prefills the whole prompt, so the state length is the main cost. Measured with scripts/bench.py on OpenJev 0.6.0 and an RTX PRO 6000 Blackwell, in codiv's production setup: this image's defaults plus OPENJEV_VLLM_ARGS="--kv-cache-memory-bytes 45097156608", a fixed 42 GiB KV cache. Each request asks the three questions of the example with automatic re-reads on, and its state starts with a random nonce, so no request reuses another's prefill: every read is a full prefill, the worst case. A request is billed its state plus about 150 tokens. codiv.ai's light production traffic (about 2 req/s) ran at the same time.

• Up to about 2K tokens, more concurrency buys throughput. Past that the GPU is busy prefilling (about 40K prompt tokens/s at 2K tokens, 31K at 8K, 13K at 64K) and more concurrency only queues.
• A read that waits more than 120 s for vLLM returns 503 with retry-after. With 64K-token states that was 4 of 32 requests at 32 at a time, and 37 of 64 at 64 at a time.
• To reproduce: python scripts/bench.py --url http://127.0.0.1:8080. Its docstring lists the options and the method.

Apple silicon

A Mac needs no vLLM and no Docker. OPENJEV_BACKEND=mlx runs DiffusionGemma inside the OpenJev process through MLX and mlx-vlm. The 4-bit weights need about 16 GB of memory to load, and more in service (see Memory below).

pip install -e '.[mlx]'
OPENJEV_BACKEND=mlx python -m openjev     # 127.0.0.1:8080

This backend uses the same prompts, canvases and seeds as vLLM. It supports images, samples, sequential, steps, think and the automatic re-reads, and it bills the same. Each step after the first reuses one prefill of the prompt. More steps cost GPU time, not prompt tokens. The re-reads and samples of one request share one vision pass.

Reads run one at a time, so this backend is for local use, not for serving. A 3-question request takes about 0.2–0.4 s on an M3 Ultra and about 0.39 s on an M4 Max, both with the 4-bit weights. 16 concurrent requests finish at about 4 req/s.

Requests are limited to 32,768 tokens by default, half the 65,536 that vLLM serves. Set OPENJEV_MLX_MAX_PROMPT=65536 for the full window, if the Mac has the memory for it: a longer prompt needs more KV cache and a larger prefill, and we have not measured that.

Memory. Loading the 4-bit weights costs about 16 GB. In service, MLX keeps freed GPU buffers in a pool that grows to the peak working set. On an M4 Pro with 48 GB, one workload grew from 16.6 GB to 36.2 GB. With OPENJEV_MLX_CACHE_LIMIT_GB=4 it stayed at 23.5 GB, with the same answers and speed. The limit is off by default, because with the 8-bit or bf16 weights one read can need more than 4 GB. If you set it, use a value above your working set.

Small encoder models

OpenJev also serves two small System One models that other people built and trained. The credit is theirs. OpenJev only puts them behind the same API. Each is a bidirectional encoder with a classification head, not a diffusion model. It reads each question in one forward pass, so an answer is still a distribution over your options.

• laya-1.0: Laya by Nandakishor M / Convai Innovations, the laya-typed-decisions checkpoint: ModernBERT-large, fine-tuned on the typed-decisions workflows. Its options share 256 tokens, so with many options each option is cut to a few tokens. Use about 20 at most, or split the question.
• verdict-1.4: Verdict by Heman10x, the rlcd-modernbert-151m checkpoint with Verdict's v1.4 inference engine: ModernBERT-base with a GLiClass head, calibrated per option count.

Each model runs in its own container (OPENJEV_BACKEND=laya or verdict). docker compose up -d starts both next to the vLLM container on the same GPU. The openjev container sends their requests to them (OPENJEV_MODEL_ROUTES), so :8080 serves all three models. /v1/models lists a routed model even when its container is stopped. A request for it then gets a 503. On a GPU both models use bf16 weights and need about 3.7 GB together. Set OPENJEV_GPU_UTIL to keep that memory free.

Measured on an RTX PRO 6000 with 16 questions per request. GPU memory is the nvidia-smi value, CUDA context included. A short state is about 50 tokens. A full state fills the model's limit.

FlashAttention 2 gave no improvement. Memory was the same, speed was within 6%, and short states were slower.

To run one model alone, on a GPU or on the CPU:

docker build -f docker/Dockerfile.laya -t openjev-laya .   # after the base, as above
docker run -d --gpus all -p 127.0.0.1:8081:8080 \
  -v ~/.cache/huggingface:/root/.cache/huggingface openjev-laya
# or without Docker:
pip install -e '.[laya]' && OPENJEV_BACKEND=laya python -m openjev

How the other models differ covers their limits and billing. Also:

• Verdict adds an "insufficient evidence" option to each question. OpenJev removes it and scales the other probabilities to a sum of 1, as Jev's answer shapes require. Verdict also ignores a noul's criteria.
• Laya rounds each probability to 4 decimal places.
• On a GPU the weights are bf16, not fp32. On 36 test answers per model, no top option changed. Probabilities moved by at most 0.021 (Laya) and 0.007 (Verdict).
• The Verdict prompt format and temperatures come from Verdict's inference engine (v1.4). For the same input, the probabilities are the same as that engine's output.

For benchmarks, training, fine-tuning and known limits, read the authors' repositories: Laya and Verdict. The laya container runs Laya's laya package. Verdict uses ModernBERT (Answer.AI, LightOn) and GLiClass (Knowledgator).

CLM

CLM (Contrastive Language Model) is Contrastive-LM's model; the credit is theirs. The CLM-v0.1-8B checkpoint is two small heads (a state head and an action head, 9.4M parameters each) on top of a frozen Qwen3-8B. Qwen3-8B turns the state (with the question appended) and each option into its last-token embedding. The heads project them to 512 dimensions, and an answer is the softmax over the scaled cosine of each option with the state.

The clm image (docker/Dockerfile.clm) runs vLLM's pooling runner for Qwen3-8B and the heads in one container, on the vLLM commit in Caveats. The prompt layout, the heads and the scoring come from Contrastive-LM's contrastive-lm package (0.1.0).

docker build -f docker/Dockerfile.clm -t openjev-clm .   # after the base, as above
docker run -d --gpus all --ipc host -p 127.0.0.1:8083:8080 \
  -v ~/.cache/huggingface:/root/.cache/huggingface openjev-clm

Next to DiffusionGemma, run docker compose --profile clm up -d and add clm-v0.1=http://clm:8080 to OPENJEV_MODEL_ROUTES. CLM runs its own vLLM, so lower the openjev service's OPENJEV_GPU_UTIL to leave it room; OPENJEV_CLM_GPU_UTIL (default 0.12) is its share of the GPU.

The default weights are Qwen/Qwen3-8B-FP8. On an RTX 3090 (Ampere, no FP8 compute) vLLM runs them as weight-only FP8 through Marlin. On 581 four-way SQuAD questions, FP8 and bf16 agreed on 98.5% of top options. The mean embedding cosine was 0.9992 and accuracy went from 89.7% to 88.8%. RedHatAI/Qwen3-8B-FP8-dynamic does not start on Ampere with this vLLM. Set OPENJEV_MODEL=Qwen/Qwen3-8B for bf16.

Measured on one RTX 3090 at OPENJEV_GPU_UTIL=0.85. Each request has a unique state (a SQuAD paragraph) and 3 questions, about 550 prompt tokens in all:

The GPU is the limit: prefill is compute-bound, and weight-only FP8 does not add compute on Ampere. What FP8 buys there is latency at low load and 2.4 times the prefix cache. --max-num-batched-tokens 8192 gave no gain.

Score questions can ignore the state. Upstream reports one level winning whatever the state says (CLM issue #3), and in our checks a thankful customer scored "annoyed". Choice and noul questions follow the state. Evaluate score questions on your own data before you rely on them.

JevK5

JevK5 is Alibi Serikbay's model; the credit is theirs. The JevK5 checkpoint (v0.3) is Qwen3.5-4B with a LoRA distilled from Qwen3.6-27B and GPT-6 Luna, merged. Each question becomes a JSON prompt with its options lettered A to P, and the answer is a softmax over those letters' next-token logits under one calibration temperature (1.22, from the checkpoint's jevk5_config.json). The readout is SemIf's. A question with more than 16 options takes several passes, combined as JevK5 combines them, with the knockout's own temperature from the same file (0.93).

The jevk5 image (docker/Dockerfile.jevk5) runs the checkpoint in bf16 on vLLM (the commit in Caveats) and asks it for the letters' logprobs. The prompt and the combining of passes come from JevK5's own jevk5 package (0.3.3). JevK5's own server reads one question at a time; here vLLM batches the questions of all requests together.

docker build -f docker/Dockerfile.jevk5 -t openjev-jevk5 .   # after the base, as above
docker run -d --gpus all --ipc host -p 127.0.0.1:8084:8080 \
  -v ~/.cache/huggingface:/root/.cache/huggingface openjev-jevk5

Next to DiffusionGemma, run docker compose --profile jevk5 up -d and add jevk5-0.3=http://jevk5:8080 to OPENJEV_MODEL_ROUTES, as for CLM. OPENJEV_JEVK5_GPU_UTIL (default 0.12) is its share of the GPU.

On JevBench's 231 public items, OpenJev and JevK5's own published v0.3 run gave the same input token count on all 231, so the prompts are identical, and the same top answer on 230. The other is an exact 50/50 tie, broken the other way. Probabilities differed by 0.0002 at the median and 0.027 at most (vLLM's kernels are not transformers'). OpenJev scored 87.4%, JevK5's run 87.9% (the tie). None of these items has more than 16 options.

Measured on one RTX 3090 at OPENJEV_GPU_UTIL=0.85 (7.9 GB of weights, 284k KV tokens), with the same requests as for CLM:

The GPU is the limit (100% busy at its 350 W cap). --max-num-batched-tokens 8192 gave no gain. Qwen3.5's linear-attention layers make vLLM cache prompts in blocks of 528 tokens, so questions about a state shorter than that do not share its prefill; longer states do.

The image sets VLLM_USE_FLASHINFER_SAMPLER=0. A read takes one greedy token and keeps only logprobs, and FlashInfer's sampler would need a CUDA compiler the image does not have.

Clef

Clef and Clef-flash are Cloudflare's models; the credit is theirs. Clef is Qwen3.8-27B and Clef-flash is Qwen3.5-9B, each with its vision encoder and a small transformer head over the final hidden states. The head scores every option of every question in one forward pass, so the questions of a request are read together and the state is read once. The prompt layout and the head are Cloudflare's own joint_schema_model.py, which ships with the weights.

OpenJev serves two community quantizations, NVFP4 with FP8 where it matters most. They need a Blackwell GPU, and keep the vision encoder and lm_head (which the head reads) in bf16:

Those are their authors' numbers. Measure on your own data. On 8 text and image cases (17 questions) OpenJev gave the same top answer as each checkpoint's own clef_vllm.py on all 17, with the same token counts, and probabilities within 0.03.

Measured on one RTX PRO 6000 (Blackwell):

The backbone runs on vLLM inside the API server, with the head in vLLM's pooler (openjev/clef_vllm.py, from kurcontko's plugin). The head needs each request's question and option spans, which vLLM's HTTP pooling API cannot carry. vLLM batches concurrent requests. Images go before the state, as Clef was trained.

The clef image (docker/Dockerfile.clef) has the same vLLM commit as the OpenJev image, without its DiffusionGemma patches, so Clef can move to a newer vLLM on its own. It serves Clef-flash by default; OPENJEV_BACKEND=clef serves the 27B.

docker build -f docker/Dockerfile.clef -t openjev-clef .   # after the base, as above
docker run -d --gpus all --ipc host -p 127.0.0.1:8085:8080 \
  -v ~/.cache/huggingface:/root/.cache/huggingface openjev-clef

Next to DiffusionGemma, run docker compose --profile clef up -d and add clef-flash-1.0=http://clef:8080 to OPENJEV_MODEL_ROUTES. OPENJEV_CLEF_GPU_UTIL (default 0.15) is its share of the GPU; OPENJEV_CLEF_BACKEND=clef serves the 27B instead.

How the other models differ

Laya, Verdict, CLM and JevK5 take text only: images, steps above 1, samples above 1, think and sequential get a 400. Clef takes images; the rest get a 400.

Settings

The server reads its settings from the environment. The settings for one backend only are in that backend's section: Apple silicon, Small encoder models, CLM, JevK5 and Clef.

Caveats

• The image pins upstream vLLM and makes two changes. A build fails if either change no longer applies.
  • It raises the limit of exact label ids per request from 128 to 512, for choices of up to 255 options.
  • docker/patches/vision_prefix_lm.py gives image tokens bidirectional attention, as the checkpoint config asks. Upstream vLLM does this for Gemma4 but not yet for DiffusionGemma.
• The clm and jevk5 images still pin 0.5's vLLM commit (1b3b88ec), with neither change.
• Answer quality is the quality of DiffusionGemma 26B-A4B in this mode. Evaluate it on your own tasks before you rely on it.

Development

pip install -e '.[test]' && pytest
OPENJEV_LIVE_URL=http://127.0.0.1:8080 pytest tests/test_live.py   # end to end against a running server
OPENJEV_MLX_TEST_MODEL=path/to/weights pytest tests/test_mlx_model.py   # MLX against the real model

Run the live checks after you build an image and before a cutover. They cover each read option, images, chat, and the encoder models that the server lists.

License

Apache-2.0. The DiffusionGemma weights are Apache-2.0 (NVIDIA / Google). Laya (Nandakishor M / Convai Innovations) and Verdict (Heman10x) are Apache-2.0, weights and code. openjev/encoders.py adapts Verdict's prompt format and calibration from its repository.