Logprobs: How to Classify Images with an LLM in a Single Token

⏱️ Lectura: 12 min

An open source script just showed that you don’t need to train an image classifier to label a webcam feed in real time: it’s enough to ask an LLM to answer with a single letter and read how confident it was in that letter. Developer Allan Riordan Boll published a wrapper on his personal blog on September 25, 2026, inspired by Jev, that extends this technique, based on logprobs, to vision models. With Gemma 4 12B running locally on an RTX 3090, it reaches close to 1 frame per second with three questions per frame; the same test against OpenAI’s GPT-6-luna drops to around 0.2 fps.

📑 En este artículo

1. TL;DR
2. What happened
3. Context and history
4. Technical details: logprobs and real-world performance
5. How to get started or try it
6. Impact and analysis
7. What’s next
8. Frequently Asked Questions
   1. What is a logprob and why is it useful for classification?
   2. Do you need to retrain the model to use this technique?
   3. Does it work with models that don’t expose logprobs in their API?
   4. Why is the limit 20 options per question?
   5. Is it worth using this instead of a specialized vision model?
   6. Can it run entirely locally, without sending anything to an external API?
9. References

The technique itself isn’t new: OpenAI has documented it for years in its logprobs cookbook, and projects like Jev, OpenJev, and SemIf use it to route tickets or classify text without calling a separate model. What this experiment contributes is applying it to images, adding an attachments field that Jev’s format doesn’t officially support yet.

TL;DR

• Allan Riordan Boll published a Jev-style wrapper on September 25, 2026 that adds image support via logprobs.
• The trick forces the LLM to answer with a single letter and reads the log probability of each possible option.
• With a local Gemma 4 12B on an RTX 3090, the script processes close to 1 frame per second with three questions per frame.
• Against OpenAI’s API with GPT-6-luna, performance drops to around 0.2 fps because each question opens a separate connection.
• The key request parameters are max_completion_tokens=1, logprobs=true, and top_logprobs=20.
• The format supports up to 20 criteria per question, one for each available letter from A to T.
• The code is a standalone Python script that uses OpenCV only to capture the webcam feed, without any dedicated computer vision.
• Jev, OpenJev, and SemIf were already using logprobs to classify text before this experiment extended it to video.

What happened

Boll read about Jev and the projects that sprang up around it, like OpenJev and SemIf, and came across a trick that’s new to many backend developers: reading the log probabilities an LLM assigns to each possible token in its response, instead of just keeping the generated text. The idea is simple. You build a multiple-choice-style prompt, for example Status: the order arrived broken. Question: which team should handle this? A) billing B) shipping C) returns, and ask the model to answer with a single letter. By adding two parameters to the request, max_completion_tokens set to 1 and logprobs set to true, the LLM returns the chosen letter along with the probabilities of the other options it considered.

Repeating that query question by question lets you build a complete classification without relying on a separate model or fine-tuning. Boll went a step further: Jev’s format currently only documents text or JSON states, so he added an attachments field to pass images in base64. With that, he built a script that captures webcam frames, encodes them, and asks the model whether a person is visible, whether the scene is indoors or outdoors, and how bright it is, all with a single response token per question.

Context and history

Reading logprobs for classification isn’t a discovery of Boll’s. OpenAI documents the pattern in its cookbook as a way to get calibrated confidence in classification tasks without paying to generate a long response. The advantage over asking the model for a paragraph of text is twofold: the response format is deterministic, always a single letter, and the associated probability works as a confidence score that can be thresholded.

Jev popularized this approach as a lightweight alternative to training a dedicated classifier for routing support tickets, moderating content, or labeling data. Derivative projects like OpenJev and SemIf replicated the idea in a self-hostable way, aimed at teams that don’t want to depend on an external API for simple, repetitive decisions. What none of them officially documented, until this experiment, was extending the same multiple-choice prompt to visual input: Jev’s spec covers text or JSON state, not binary attachments.

That’s Boll’s concrete contribution: he didn’t invent the logprobs trick, but he proves it works just as well when the state is an image instead of a sentence, as long as the underlying model is multimodal.

Technical details: logprobs and real-world performance

The HTTP request is a standard Chat Completions call with three extra parameters. max_completion_tokens set to 1 forces the model to return a single token, preventing it from rambling into a long response. logprobs set to true makes the API return the chosen token’s log probability along with the token itself. top_logprobs, with a number between 0 and 20, additionally requests the most likely alternatives the model considered, even if it didn’t choose them.

import requests

pregunta = (
    "Status: the customer says the order arrived broken.\n"
    "Question: which team should handle this?\n"
    "[A] billing\n[B] shipping\n[C] returns\n"
    "Answer with only the letter."
)

payload = {
    "model": "gemma-4-12b",
    "messages": [{"role": "user", "content": pregunta}],
    "max_completion_tokens": 1,
    "logprobs": True,
    "top_logprobs": 3,
}

resp = requests.post("http://localhost:8080/v1/chat/completions", json=payload, timeout=30)
top = resp.json()["choices"][0]["logprobs"]["content"][0]["top_logprobs"]
print(top)

The response includes a list of candidate tokens with their log probability. Taking the one with the highest value among A, B, and C gives you the classification, without ever having generated a long response.

For images, Boll adds the content as an image_url block with a base64 data URL, following the same multimodal format already supported by OpenAI-compatible APIs. The rest of the logic doesn’t change: the question is built, the options are listed as letters, and whichever had the highest log probability is read off.

import base64, json, pathlib, urllib.request

def clasificar_frame(ruta_imagen, pregunta, opciones, url, modelo):
    imagen_b64 = base64.b64encode(pathlib.Path(ruta_imagen).read_bytes()).decode()
    letras = "ABCDEFGHIJKLMNOPQRST"[: len(opciones)]
    texto_opciones = "\n".join(f"[{l}] {o}" for l, o in zip(letras, opciones))
    body = {
        "model": modelo,
        "messages": [{
            "role": "user",
            "content": [
                {"type": "text", "text": f"Question: {pregunta}\nOptions:\n{texto_opciones}\nAnswer with only the letter."},
                {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{imagen_b64}"}},
            ],
        }],
        "max_completion_tokens": 1,
        "logprobs": True,
        "top_logprobs": len(opciones),
    }
    data = json.dumps(body).encode()
    req = urllib.request.Request(url, data=data, headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=60) as resp:
        respuesta = json.loads(resp.read())
    top = respuesta["choices"][0]["logprobs"]["content"][0]["top_logprobs"]
    return sorted(top, key=lambda t: t["logprob"], reverse=True)[0]["token"]

letra = clasificar_frame(
    "frame_webcam.jpg",
    "Is there a person visible in the image?",
    ["yes", "no"],
    "http://localhost:8080/v1/chat/completions",
    "gemma-4-12b",
)
print(letra)

sequenceDiagram
    participant W as Webcam
    participant S as Python Script
    participant L as LLM Server
    W->>S: captures JPEG frame
    S->>S: encodes to base64
    S->>L: requests classification with 1 token and logprobs enabled
    L-->>S: chosen letter and top_logprobs
    S->>S: takes the letter with the highest log probability
    Note over S,L: repeated once for each question in the frame

In his tests, with Gemma 4 12B running on llama.cpp over an RTX 3090, the script sustains close to 1 frame per second while evaluating three questions per frame: person visible, indoors or outdoors, and brightness level. Against OpenAI’s API with GPT-6-luna, performance drops to around 0.2 fps. Boll attributes the difference to not having optimized the connection: each question opens a separate connection to OpenAI’s infrastructure instead of reusing an already open channel.

Option	When to use it	Advantage	Limitation
Local model (llama.cpp + Gemma 4 12B)	Prototypes, webcam privacy, no per-call cost	~1 fps with three questions, never leaves the local network	Depends on having a decent GPU on the machine
Remote API (OpenAI GPT-6-luna)	When no local GPU is available	Doesn’t require your own infrastructure	Drops to ~0.2 fps and each question is billed separately
Specialized vision model (trained CNN)	Large-scale production with a fixed task	More efficient and cheaper per inference	Requires training and retraining if the condition changes

💡 Tip: if your backend supports prefix caching (KV cache), the state shared across the three questions for the same frame can be cached once, avoiding reprocessing the image for every question.

How to get started or try it

To reproduce the experiment you need a backend that speaks the Chat Completions protocol with logprobs support, such as llama.cpp server or directly OpenAI’s API, plus a vision model if you’re going to classify images.

Linux and macOS, with uv:

curl -LsSf https://astral.sh/uv/install.sh | sh
uv run webcam.py

Windows, in PowerShell:

powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
uv run webcam.py

Alternative with pip on all three platforms, changing only the command to activate the environment:

python -m venv venv
source venv/bin/activate
pip install opencv-python requests
python webcam.py

On Windows, the activation step is venv\Scripts\activate instead of source venv/bin/activate.

To confirm your backend actually returns logprobs, before writing the full script, a single standalone call is enough:

curl -s http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gemma-4-12b","messages":[{"role":"user","content":"Answer A or B"}],"max_completion_tokens":1,"logprobs":true,"top_logprobs":5}' \
  | jq '.choices[0].logprobs.content[0].top_logprobs'

If the array jq returns isn’t empty, the backend supports logprobs and the trick works exactly as described above.

⚠️ Watch out: if you use a remote API, each question is a billable call with the full image in base64 inside it; three questions per frame multiply the cost and latency by three, they don’t divide anything.

Impact and analysis

A specialized vision model, trained for one specific task, is likely faster and cheaper at large-scale production. The advantage of this approach isn’t speed: it’s the flexibility to change a condition by rewriting a phrase in the prompt, without retraining anything. For a prototype, a one-off alert, or a pipeline with few frames per second, that flexibility can matter more than raw efficiency.

The real cost lies in the number of calls. Classifying a frame with three questions means three HTTP requests, even though each one returns just a single token. Against a local backend that’s nearly free; against a remote API, the cost multiplies with every question and every frame you want to process per second.

What’s next

Jev’s format still doesn’t officially document attachments, so the field Boll added remains a personal extension, not a standard. If the community around Jev, OpenJev, and SemIf adopts this idea, it would make sense to see it show up in the project’s spec at some point, with explicit support for images and maybe audio.

In the meantime, anyone with a Chat Completions-compatible backend can try the same pattern today: it doesn’t depend on Jev standardizing it, it just needs the API to expose logprobs and top_logprobs.

📖 Summary on Telegram: View summary

Try it yourself: spin up a local server with llama.cpp and a multimodal model, and send it an image with logprobs: true to see the real confidence behind the response.

Frequently Asked Questions

What is a logprob and why is it useful for classification?

It’s the logarithm of the probability the model assigns to a token given the context. It’s useful because it turns the LLM’s choice into a number you can compare across options, instead of just having the text it generated.

Do you need to retrain the model to use this technique?

No. The trick works with any model that already knows how to follow instructions and whose API exposes logprobs; no weights are adjusted and no new layer is added.

Does it work with models that don’t expose logprobs in their API?

Not directly. The backend, whether llama.cpp, vLLM, or OpenAI’s API, needs to implement the logprobs and top_logprobs parameters on the chat completions endpoint.

Why is the limit 20 options per question?

Because Boll’s script maps each option to a letter of the alphabet between A and T, and top_logprobs in most OpenAI-compatible APIs accepts a maximum of 20 values.

Is it worth using this instead of a specialized vision model?

It depends on the volume. For a few frames per second and conditions that change often, the flexibility of rewriting the prompt wins out. For large-scale production, a model trained for that specific task tends to be more efficient.

Can it run entirely locally, without sending anything to an external API?

Yes. The script speaks the same Chat Completions protocol against a local server like llama.cpp, so there’s no need to share the webcam feed with any external provider.

References

• Allan’s Blog: original post where Allan Riordan Boll describes the wrapper and publishes the full script.
• OpenAI Cookbook: documentation with examples of using logprobs for classification and quality control.
• OpenAI API Reference, Chat Completions: reference for the logprobs and top_logprobs parameters used in the request.
• Log probability, Wikipedia: mathematical context on what a log probability represents.

📱 Like this content? Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day.

Imagen destacada: Foto de Rahul Mishra en Unsplash