Research
Tin is our own model: a decision layer and a world model, loaded together from one checkpoint. It answers with a probability for every option and never generates text. Everything below was measured on one laptop, an Intel Core i7-10510U with four cores and no GPU. Intervals are 95% bootstrap intervals on held-out rows.
Tin alone, beside the decision models
Tin's released checkpoint, on held-out rows of ten tasks rebuilt from LangWatch's sources.
The best in each row is in bold. Read this as a reference, not a ranking: Tin was trained on each task's training rows, while the other models' figures are zero-shot on LangWatch's own test sets. On Kev's transfer suite, where every model answers the same 764 questions, Tin alone scores 61.3, against Jev 85.7, Kev-27B 84.8 and Kev-9B 82.2.
Banking77 intents
91.9% accuracy (90.2 to 93.6)
Complaint routing
82.1% accuracy (79.7 to 84.4)
Commit type
68.7% accuracy (65.6 to 71.5)
The axis starts at 60, not 0.
- Median time per decision
- 0.2 to 2.6 milliseconds across ten tasks.
- Calibration error
- 0.01 to 0.08 across ten tasks. Lower is better.
- Sure and wrong
- On Banking77, Tin was at least 90% sure on 754 answers and wrong on 9.
Tin with a 9B open-weight model
A separate configuration for comparisons with larger models. The 9B model is an open-weight model used as published: we never train it, and it is not part of Tin. Tin answers first, and the 9B joins where Tin is least sure, with weights fitted on validation questions.
- World model
- Before Oeon runs an action on your files, Tin predicts whether it will succeed, what it will create, change or remove, and whether it can be undone. On held-out episodes its success prediction was right 95.6% of the time. It advises; Oeon's gate keeps the final word.
- When you have finished speaking
- Over 300 spoken requests with a thinking pause inside, Tin cut the speaker off 1.3% of the time, against 20.3% for a fixed 1.7 second wait, with the same median wait.
What Tin does not do yet
- It can't write, chat or summarise. It only chooses among the options it is given.
- A new kind of decision needs training examples.
- It is behind Jev on off-topic detection, moderation and Kev's transfer suite.
- Its own speech model cannot transcribe yet: intent from speech on MINDS-14 is 12.5, against 83.7 for Oruk's model.
- When a hard question goes to the 9B with thinking, an answer on a four-core laptop takes 15 to 35 minutes.
- Not yet measured: GPQA Diamond, Humanity's Last Exam, OSWorld, Terminal-Bench and ARC-AGI. These run when Tin trains on a GPU.
Every figure, with its run report and caveats, is on Tin's model card: huggingface.co/Aravindhan11/tin.