Paying a Full Second for a One-Token Answer

Main takeaways on single-pass classification:

• Classification doesn't need generation. Reading the label directly from an LLM's logits in one forward pass is 8–40x faster than generative classification. It takes under 100ms per sample on a 72B model (compared to roughly a second in the traditional pipeline), and is far faster than a reasoning approach, which can run well beyond one second per sample.
• There's no accuracy tax. SALSA, Single-pass Autoregressive LLM Structured Classification, matches or beats purpose-built encoder models across diverse benchmarks: state-of-the-art on GLUE tasks, on sentiment analysis, medical QA, and hate-speech detection. The same pipeline later earned a Kaggle gold medal in content moderation and held strong out-of-distribution performance in code authorship classification at SemEval-2026 Task 13. All it requires is a calibration set, so the model can be aligned to the efficient task solution.
• It's open-weights-only. The method needs full logit access and a label-masked training objective, which no closed API provides. In air-gapped, resource-constrained environments, that constraint was already the architecture.

Here is how a typical LLM classification call goes:

1. The model reads the prompt.
2. It reasons, visibly or in a suppressed chain-of-thought.
3. It generates a preamble, a justification, a formatted answer.
4. Somewhere in that output is the one token needed (spam or not, fraud or not, misconfigured or not).

At scale, that overhead dooms the applicability of delegating a task to an LLM.

Generative classification on a large open model runs roughly a second per sample. When reasoning is enabled, that climbs. Score millions of emails, firewall rules, or code snippets a day, and "a second per sample" means "not feasible."

Dream researchers Ruslan Berdichevsky, Shai Nahum-Gefen, and Elad Ben-Zaken published one fix: stop letting the model talk, and use it more efficiently.

Their method, SALSA (Single-pass Autoregressive LLM Structured Classification), reads the answer directly from the model's output distribution in a single forward pass. The paper benchmarks it across a spectrum of classification tasks (GLUE, sentiment, medical QA, hate speech), and that is where the headline numbers below come from. The method has since been validated well beyond the paper: a gold-medal finish in a Kaggle content-moderation competition, and a strong out-of-distribution showing at SemEval-2026 Task 13.

Chain-of-thought doesn't help classification. Pipelines act like it does.

There is a common practice of using chain-of-thought for classification tasks. But look at how classification ships in production: a generative call, reasoning left on by default, a parser scraping the label out of whatever the model printed.

Berdichevsky frames the counterargument in one line:

"Reasoning doesn't introduce any new information. The information is already there. It hides in its weights."

The need for reasoning is just a symptom that the model isn't aligned to the task. Classification is a constrained next-token prediction problem wearing a generation costume. The logit distribution at the label position already contains the verdict. If you've ever watched someone's face change just before they answer a question, and realized you no longer needed the answer, you understand the mechanism. Decoding is just an expensive way of printing what the logits already said.

Single-pass LLM classification: how SALSA works

The idea of scoring labels from the output distribution isn't new. SALSA's contribution is operationalizing it end to end.

Each class maps to a single token under the tokenizer ("0" human, "1" machine), with the prompt structured so the label lands at a fixed position and chain-of-thought suppressed. Inference is one forward pass: project the logits onto the class tokens, softmax, argmax. No decoding loop, no output parsing. Fine-tuning supervises only the label position, a causal LM objective with the loss masked to the label token, so gradients push the class distribution and little else.

The payoff: an 8x to 40x speedup, under 100 milliseconds per sample on a 72B model with the LoRA adapter merged and logprobs served through vLLM.

Does skipping the reasoning cost accuracy?

The paper's answer is no, across the board. SALSA reaches state-of-the-art on GLUE tasks and, on the held-out test server, beats the top three GLUE leaderboard systems: 94.8 on RTE, 97.9 on SST-2. It carries the same result into topic and sentiment classification (95.9 on AG News, 97.6 on IMDb, 74.2 on Yelp-5) and into specialized domains, edging past prior SOTA on MedNLI (91.3 vs 90.2), MedMCQA (74.1 vs 73.6), and HateXplain (71.8 vs 70.4). Extended to regression via a discrete class ensemble, it even posts a new best of 93.8/93.6 Pearson/Spearman on STS-B. Built for generation, the decoder-only model matches or exceeds purpose-built classifiers, without generating a word.

Fine-tuning for OOD generalization: the recipe is restraint

The method's real-world outings sharpen the picture. The same single-pass formulation took a gold medal in Kaggle's Jigsaw Agile Community Rules content-moderation competition, under a writeup titled "Instruct LLM is all you need." And at SemEval-2026 Task 13, machine-generated code detection scored on out-of-distribution generalization, SALSA-tuned Qwen2.5-72B-Instruct scored 0.789 OOD macro-F1 against the official CodeBERT baseline of 0.305.

Un-tuned, the 72B scores 0.910 precision but 0.035 recall on the machine-generated class; it near-categorically refuses to call code machine-written. The knowledge is present, the task alignment isn't.

The winning code-detection fine-tune was defined by what the team refused to do. The training set was 91.5% Python; they discarded most of it, balancing across language-label groups, because majority-language dominance traded task alignment for in-domain pattern-matching. They trained one epoch at a 5×10⁻⁶ learning rate with LoRA (r=16, α=32) and stopped; every more aggressive configuration improved in-distribution metrics and degraded OOD performance.

"I'm not sure we can call it training," Berdichevsky says. "It's more like calibrating the model to extract its own knowledge for a given task."

And for anyone whose reflex in a constrained environment is "just use a smaller model": the fine-tuned 8B barely moved (0.450) while the 72B hit 0.789. Capability lives in the parameters. Efficiency should come from how you query them.

Because the formulation is task-agnostic, it travels. Other research teams have also applied SALSA to SemEval-2026 Task 9, detecting multilingual online polarization across 22 languages, the same pipeline with a new label space.

Why this requires open-weight models

SALSA needs access to logits. Closed APIs expose token trajectories alone, out of fear that exposing full logits could leak the model weights. And SALSA needs more than logits: a label-masked fine-tuning loss, merged LoRA adapters, and vLLM-grade serving. The systems above cannot exist behind an API. They require open weights you host yourself.

As Berdichevsky puts it: "It became an enabler. You cannot do large-scale classification with an open model without this kind of approach."

For most teams, this is a deployment detail.

But consider infrastructure that runs air-gapped by design: national cyber-defense, classified networks, critical systems.

Sovereign, self-hosted AI is usually framed as trading capability for control. This is a small, concrete counterexample: the architecture the constraint forces is the one that produced the result.

---

SALSA: Single-pass Autoregressive LLM Structured Classification (Berdichevsky, Nahum-Gefen, Ben-Zaken) is the method paper. Its applications appear at SemEval-2026 Task 13 (machine-generated code detection) and Task 9 (online polarization), and in a gold-medal Kaggle writeup. Code: github.com/dreamgroupai-ai/SALSA.