AnyJev

AnyJev: turn any LLM into a Jev-style decision model. Typed decisions, real probabilities, training-free or self-distilled. Qwen3-8B on BANKING77-20, raw readout to L0 with no labels: order-flip rate 0.230 to 0.073, accuracy 0.747 to 0.803, auto-decidable at 5% risk 7.7% to 46.3%.

English · 简体中文 · ⚡ Serve it · 📊 Results · 🏆 Leaderboard · 🧭 Roadmap · 📖 Levels

Jiamu Zhang¹ Tianze Yang¹ Yucheng Shi² Evan Chen¹
Zixiang Nie¹ Kelly Wan¹ Liangjie Hong¹ Ninghao Liu³ Liang Wu¹

_{¹ Nokia, Sunnyvale, CA, USA ² Tencent Hunyuan ³ The Hong Kong Polytechnic University}

_{Qwen3-8B on a real BANKING77 item. Every number is a model output.}

Tip

🆕 Tacit: AnyJev's self-distilled models, 1.7B to 9B, are on Hugging Face. One forward pass per decision. With adaptive=True, a capped share of low-confidence decisions goes to the model's own reasoning. They run on transformers, in-process vLLM or a stock vllm serve. pip install "anyjev[hf]", then Tacit.from_pretrained("morriszjm/Tacit-9B"). Start here ↓

Note

🏆 AnyJev Tacitus on the Decision Index 0.4.1. AnyJev Tacitus 31B, fine-tuned from Gemma-4-31B and read with AnyJev, has a Full score of 66.47, tied 3rd on the board on 2026-10-21. Details ↓

Note

🏆 No training, on a public leaderboard. The training-free Decider (L0, two rotations) on the unchanged Gemma-4-31B has a Full score of 60.22 on the Decision Index 0.3, 5th of 149 entries on 2026-10-12. Details ↓

⚡ Serve it

A Tacit model, in Python.

from anyjev import Tacit

tacit = Tacit.from_pretrained("morriszjm/Tacit-9B")          # transformers, one GPU
d = tacit.decide(state="Customer: my package was due last Monday and it still has not arrived.",
                 question="What does the customer want?",
                 options=["track_order", "cancel_order", "refund", "change_address"])
d["answer"], d["probs"], d["route"]    # an option, {option: probability}, "one_forward" or "cot"

kind is "choice" (default), "yes_no" or "score" (options are ordered levels, lowest first, numbered from 1; first_level=0 numbers them from 0). decide_batch([...]) takes a list of such dicts.

Send the hardest decisions to reasoning, with a cap.

tacit = Tacit.from_pretrained("morriszjm/Tacit-9B", adaptive=True, tau=0.5, max_cot_share=0.2, cot_window=1000)

A decision whose top two options are less than tau apart in log-probability is answered again with thinking on. After the reasoning, the answer is read as a label distribution, so it also has probabilities and cannot fail to parse. At most max_cot_share of the last cot_window decisions escalate, however hard the traffic gets. Every escalation is reported (route == "cot", with the first pass kept), and tacit.stats counts them.

On vLLM. A stock server; the client needs only the tokenizer.

pip install "anyjev[client]"
vllm serve morriszjm/Tacit-9B --host 127.0.0.1 --port 8000

tacit = Tacit.from_pretrained("morriszjm/Tacit-9B", engine="server",
                              base_url="http://127.0.0.1:8000", adaptive=True)

engine="vllm" (pip install "anyjev[vllm]") runs vLLM in the same process instead.

An HTTP endpoint for many clients sharing one cap.

python -m anyjev.serve --model morriszjm/Tacit-9B --upstream http://127.0.0.1:8000 --adaptive --port 8100
curl -s 127.0.0.1:8100/v1/decide -H 'Content-Type: application/json' \
  -d '{"state": "Customer: my package has not arrived.", "question": "What does the customer want?", "options": ["track_order", "refund"]}'

POST /v1/decide takes one decision or {"items": [...]}; GET /v1/stats reports the escalation share. The gateway listens on 127.0.0.1 and has no authentication of its own.

argument	default	meaning
`adaptive`	`False`	send low-confidence decisions to the model's own reasoning
`tau`	`0.5`	low confidence: the log-probability gap between the top two options is below `tau`
`max_cot_share`	`0.2`	at most this share of the last `cot_window` decisions escalates; `None` removes the cap
`cot_window`	`1000`	how many recent decisions the cap counts; `None` counts every decision since loading
`cot_max_tokens`	`8192`	reasoning budget of one decision
`engine`	`"transformers"`	`"vllm"` runs vLLM in this process; `"server"` uses a running `vllm serve` at `base_url`

Any other model, training-free. The Decider reads the same typed decisions from any open causal LM, with no training and no labels.

from anyjev import Decider, Question
from anyjev.backends.hf import HFBackend          # or VLLMBackend / SGLangBackend(url, model), LlamaCppBackend(gguf)

d = Decider(HFBackend("Qwen/Qwen3-8B"))           # level="L0" by default
route = Question.choice("Which team should handle this?",
                        ["billing", "technical", "sales", "other"], name="route")
r = d.decide("My card was charged twice for one order.", [route])["route"]
r.distribution, r.level                           # {option: probability}, "L0"

L0 reads a K-option choice once per rotation of the options. With the rotation budget, it reads only as many rotations as the decision needs:

d = Decider(backend, adaptive_shifts=True, canonical_order=True)
d.calibrate_adaptive(route, unlabelled_tickets, target=0.01)   # a few hundred states, no labels

The stopping threshold is calibrated against our own full-cycle answer, so it needs no labels. At a certified 1% disagreement it read 7.2 rotations instead of 18. That was 2.2× the decisions per second on vLLM and 2.3×–2.7× on transformers, with accuracy unchanged (docs/rotation_budget.md).

✨ What it is

Give the model a state and a typed question (a choice, a yes/no or a score). You get back a decision with a probability over the options, read from the model's next-token distribution. Nothing is parsed. There are two ways in:

AnyJev Training-Free (Decider) works on any open LLM, with no training. Raw label logits change their answer when the options are reordered and carry the model's preference for some labels. L0 removes both without a single label.
AnyJev Self-Distilled (Tacit) uses the Tacit checkpoints. They are trained by self-distillation: the model learns to give, in one forward pass, the answers it reaches when it reasons. No human labels and no other model are involved. One prefill per decision; with adaptive=True, a capped share of decisions goes to reasoning.

	⚪ raw logits	🔵 L0 _{zero labels}
Labels required	none	none
Answer flips when options are reversed	0.230	0.073
Accuracy	0.747	0.803
Calibration error (ECE)	0.240	0.184
Auto-decidable at ≤5% error	7.7%	46.3%

_{Qwen3-8B, BANKING77 20-way, 300 test items · bench/results_v01/2026-09-22/Qwen__Qwen3-8B.json}

The last row matters most. Accuracy moves six points, but the share of decisions confident enough to automate at ≤5% error goes from 7.7% to 46.3%, with no labels at all.

📊 Tacit results

model	base	JevBench public (231)	adaptive	bev-decision test (46,320)	adaptive
Tacit-9B	Qwen3.5-9B	0.823	0.887 _15.2%	0.727	0.755 _18.1%
Tacit-8B	Qwen3-8B	0.736	—	0.663	—
Tacit-4B	Qwen3-4B	0.723	0.758 _14.3%	0.663	0.704 _17.8%
Tacit-2B	Qwen3.5-2B	0.671	—	0.626	—
Tacit-1.7B	Qwen3-1.7B	0.619	—	0.598	—

_{Accuracy on every item of both sets (JevBench public
set; bev-decision test split), in the order they
are stored. First column: one forward pass per decision (adaptive=False). Adaptive: adaptive=True at
the defaults (tau=0.5, at most 20% of every 1,000 decisions), with the share of decisions sent to
reasoning in small print. Served by vllm serve (vLLM 0.17.1), run with scripts/eval_tacit.py · bench/results_tacit/2026-10-06/}

On average, escalation adds 567 (JevBench) and 458 (bev-decision) generated tokens per decision for Tacit-9B, and 305 and 187 for Tacit-4B. Results with escalation for Tacit-8B, Tacit-2B and Tacit-1.7B come next.

🏆 AnyJev Tacitus on the Decision Index

AnyJev Tacitus 31B is a decision model fine-tuned from google/gemma-4-31B-it, with an inference technique designed together with the training. It reads every decision from one prefill through AnyJev's readout, with no generated tokens. On the Decision Index 0.4.1 board it is tied 3rd by Full score (2026-10-21); from edition 0.4 the Full score counts the private parts only.

Full score	same skills, private _60%	new domains, private _40%	public _{not counted}	latency median / mean / p95
66.47	62.48	56.93	65.52	95 / 265 / 1,000 ms

_{The board's maintainers ran the two private parts and measured the latency (single process, one RTX PRO 6000).
Served by vLLM 0.29.0 with FP8 at load and the model's vLLM plugin, anyjev 0.3.0. Engine, plugin and run settings:
MorrisZJ/anyjev-tacitus; results: decision-index-results ·
bench/results_decision_index/2026-10-21/anyjev-tacitus-31b.json}

🏆 Training-free on the Decision Index

The Decider at L0 on the unchanged google/gemma-4-31B-it: no training, no labels, two rotations per decision. On the Decision Index 0.3 board it is 5th of 149 entries by Full score (2026-10-12).

Full score	public _20%	same skills, private _50%	new domains, private _30%	latency median / mean / p95
60.22	59.79	59.13	52.98	98 / 260 / 1,080 ms

_{The board's maintainers ran the two private parts and measured the latency (single process, one RTX PRO 6000).
Served by vllm serve --quantization fp8 (vLLM 0.29.0) with anyjev 0.3.0. Engine and run settings:
examples/decision_index (submitted as
anyjev-l0-gemma); model card:
AnyJev-L0-Gemma-4-31B ·
bench/results_decision_index/2026-10-12/anyjev-l0-gemma-4-31b.json}

🧠 How it works

	needs	does	does not
`raw`	nothing	one prompt, softmax restricted to the option labels	correct order or label bias
`L0`	nothing	averages position bias out over the K rotations, divides out the label prior	calibrate against labels
Tacit `one_forward`	a Tacit checkpoint	one prompt, read like `raw` from a model trained to answer in one pass	correct order bias
Tacit `cot`	`adaptive=True`	reasons first, then reads the label distribution after the thought	run beyond its cap

Every Decision carries its level, and require="L0" makes downstream code refuse a weaker one. Every Tacit decision carries its route. The contract in full →

_{Every number in this README is read from committed JSON. Not affiliated with TypeSafe AI or Jev.}

🧭 Roadmap

Dated plan and help-wanted files: ROADMAP.md. Known limitations: docs/limitations.md.

🤝 Contributing and citation

Backends are one file each, and several are help wanted (ROADMAP.md, CONTRIBUTING.md). Changes: CHANGELOG.md. Credits: CREDITS.md.

Technical Report:

@misc{zhang2026anyjevtechnicalreport,
      title={AnyJev Technical Report}, 
      author={Jiamu Zhang and Tianze Yang and Yucheng Shi and Evan Chen and Zixiang Nie and Kelly Wan and Liangjie Hong and Ninghao Liu and Liang Wu},
      year={2026},
      eprint={2610.00831},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2610.00831}, 
}

Apache-2.0, see LICENSE. The Tacit models carry their base models' Apache-2.0 license. Datasets keep their own licenses, see THIRD_PARTY.md.

inference

xorbitsai/inference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

Pushed today.Apache License 2.0PythonDeveloper tools9.7k+10

StartLux-Decision

StartLuxLabs/StartLux-Decision

Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.

Pushed 21 days ago.Apache License 2.0PythonDeveloper tools1.7k

LlamaFactory

hiyouga/LlamaFactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

Pushed today.Apache License 2.0PythonDeveloper tools76.0k+124

peft

huggingface/peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

Pushed today.Apache License 2.0PythonDeveloper tools21.9k

English · 简体中文 · ⚡ Serve it · 📊 Results · 🏆 Leaderboard · 🧭 Roadmap · 📖 Levels

Jiamu Zhang¹ Tianze Yang¹ Yucheng Shi² Evan Chen¹
Zixiang Nie¹ Kelly Wan¹ Liangjie Hong¹ Ninghao Liu³ Liang Wu¹

_{¹ Nokia, Sunnyvale, CA, USA ² Tencent Hunyuan ³ The Hong Kong Polytechnic University}

_{Qwen3-8B on a real BANKING77 item. Every number is a model output.}

Tip

Note

⚡ Serve it

A Tacit model, in Python.

from anyjev import Tacit

tacit = Tacit.from_pretrained("morriszjm/Tacit-9B")          # transformers, one GPU
d = tacit.decide(state="Customer: my package was due last Monday and it still has not arrived.",
                 question="What does the customer want?",
                 options=["track_order", "cancel_order", "refund", "change_address"])
d["answer"], d["probs"], d["route"]    # an option, {option: probability}, "one_forward" or "cot"

Send the hardest decisions to reasoning, with a cap.

tacit = Tacit.from_pretrained("morriszjm/Tacit-9B", adaptive=True, tau=0.5, max_cot_share=0.2, cot_window=1000)

On vLLM. A stock server; the client needs only the tokenizer.

pip install "anyjev[client]"
vllm serve morriszjm/Tacit-9B --host 127.0.0.1 --port 8000

tacit = Tacit.from_pretrained("morriszjm/Tacit-9B", engine="server",
                              base_url="http://127.0.0.1:8000", adaptive=True)

engine="vllm" (pip install "anyjev[vllm]") runs vLLM in the same process instead.

An HTTP endpoint for many clients sharing one cap.

python -m anyjev.serve --model morriszjm/Tacit-9B --upstream http://127.0.0.1:8000 --adaptive --port 8100
curl -s 127.0.0.1:8100/v1/decide -H 'Content-Type: application/json' \
  -d '{"state": "Customer: my package has not arrived.", "question": "What does the customer want?", "options": ["track_order", "refund"]}'

POST /v1/decide takes one decision or {"items": [...]}; GET /v1/stats reports the escalation share. The gateway listens on 127.0.0.1 and has no authentication of its own.

argument	default	meaning
`adaptive`	`False`	send low-confidence decisions to the model's own reasoning
`tau`	`0.5`	low confidence: the log-probability gap between the top two options is below `tau`
`max_cot_share`	`0.2`	at most this share of the last `cot_window` decisions escalates; `None` removes the cap
`cot_window`	`1000`	how many recent decisions the cap counts; `None` counts every decision since loading
`cot_max_tokens`	`8192`	reasoning budget of one decision
`engine`	`"transformers"`	`"vllm"` runs vLLM in this process; `"server"` uses a running `vllm serve` at `base_url`

Any other model, training-free. The Decider reads the same typed decisions from any open causal LM, with no training and no labels.

from anyjev import Decider, Question
from anyjev.backends.hf import HFBackend          # or VLLMBackend / SGLangBackend(url, model), LlamaCppBackend(gguf)

d = Decider(HFBackend("Qwen/Qwen3-8B"))           # level="L0" by default
route = Question.choice("Which team should handle this?",
                        ["billing", "technical", "sales", "other"], name="route")
r = d.decide("My card was charged twice for one order.", [route])["route"]
r.distribution, r.level                           # {option: probability}, "L0"

L0 reads a K-option choice once per rotation of the options. With the rotation budget, it reads only as many rotations as the decision needs:

d = Decider(backend, adaptive_shifts=True, canonical_order=True)
d.calibrate_adaptive(route, unlabelled_tickets, target=0.01)   # a few hundred states, no labels

✨ What it is

AnyJev Training-Free (Decider) works on any open LLM, with no training. Raw label logits change their answer when the options are reordered and carry the model's preference for some labels. L0 removes both without a single label.
AnyJev Self-Distilled (Tacit) uses the Tacit checkpoints. They are trained by self-distillation: the model learns to give, in one forward pass, the answers it reaches when it reasons. No human labels and no other model are involved. One prefill per decision; with adaptive=True, a capped share of decisions goes to reasoning.

	⚪ raw logits	🔵 L0 _{zero labels}
Labels required	none	none
Answer flips when options are reversed	0.230	0.073
Accuracy	0.747	0.803
Calibration error (ECE)	0.240	0.184
Auto-decidable at ≤5% error	7.7%	46.3%

_{Qwen3-8B, BANKING77 20-way, 300 test items · bench/results_v01/2026-09-22/Qwen__Qwen3-8B.json}

The last row matters most. Accuracy moves six points, but the share of decisions confident enough to automate at ≤5% error goes from 7.7% to 46.3%, with no labels at all.

📊 Tacit results

model	base	JevBench public (231)	adaptive	bev-decision test (46,320)	adaptive
Tacit-9B	Qwen3.5-9B	0.823	0.887 _15.2%	0.727	0.755 _18.1%
Tacit-8B	Qwen3-8B	0.736	—	0.663	—
Tacit-4B	Qwen3-4B	0.723	0.758 _14.3%	0.663	0.704 _17.8%
Tacit-2B	Qwen3.5-2B	0.671	—	0.626	—
Tacit-1.7B	Qwen3-1.7B	0.619	—	0.598	—

🏆 AnyJev Tacitus on the Decision Index

Full score	same skills, private _60%	new domains, private _40%	public _{not counted}	latency median / mean / p95
66.47	62.48	56.93	65.52	95 / 265 / 1,000 ms

🏆 Training-free on the Decision Index

The Decider at L0 on the unchanged google/gemma-4-31B-it: no training, no labels, two rotations per decision. On the Decision Index 0.3 board it is 5th of 149 entries by Full score (2026-10-12).

Full score	public _20%	same skills, private _50%	new domains, private _30%	latency median / mean / p95
60.22	59.79	59.13	52.98	98 / 260 / 1,080 ms

🧠 How it works

	needs	does	does not
`raw`	nothing	one prompt, softmax restricted to the option labels	correct order or label bias
`L0`	nothing	averages position bias out over the K rotations, divides out the label prior	calibrate against labels
Tacit `one_forward`	a Tacit checkpoint	one prompt, read like `raw` from a model trained to answer in one pass	correct order bias
Tacit `cot`	`adaptive=True`	reasons first, then reads the label distribution after the thought	run beyond its cap

Every Decision carries its level, and require="L0" makes downstream code refuse a weaker one. Every Tacit decision carries its route. The contract in full →

_{Every number in this README is read from committed JSON. Not affiliated with TypeSafe AI or Jev.}

🧭 Roadmap

Dated plan and help-wanted files: ROADMAP.md. Known limitations: docs/limitations.md.

🤝 Contributing and citation

Backends are one file each, and several are help wanted (ROADMAP.md, CONTRIBUTING.md). Changes: CHANGELOG.md. Credits: CREDITS.md.

Technical Report:

@misc{zhang2026anyjevtechnicalreport,
      title={AnyJev Technical Report}, 
      author={Jiamu Zhang and Tianze Yang and Yucheng Shi and Evan Chen and Zixiang Nie and Kelly Wan and Liangjie Hong and Ninghao Liu and Liang Wu},
      year={2026},
      eprint={2610.00831},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2610.00831}, 
}

Apache-2.0, see LICENSE. The Tacit models carry their base models' Apache-2.0 license. Datasets keep their own licenses, see THIRD_PARTY.md.

Understand it first

Install

What we checked

README

⚡ Serve it

✨ What it is

📊 Tacit results

🏆 AnyJev Tacitus on the Decision Index

🏆 Training-free on the Decision Index

🧠 How it works

🧭 Roadmap

🤝 Contributing and citation

More in Developer tools

inference

StartLux-Decision

LlamaFactory

peft

⚡ Serve it

✨ What it is

📊 Tacit results

🏆 AnyJev Tacitus on the Decision Index

🏆 Training-free on the Decision Index

🧠 How it works

🧭 Roadmap

🤝 Contributing and citation