⏱️ Lectura: 9 min
A study by Princeton researchers has just drawn a sharp line around what AI agents can and cannot do yet: they handle machine learning research engineering tasks with ease, but fall short when the problem calls for open-ended science, that is, formulating a new question and pursuing it without a predefined goal.
📑 En este artículo
- TL;DR
- What Happened
- Context and History
- Technical Details and Performance
- How to Test It Yourself
- Impact and Analysis
- What’s Next
- Frequently Asked Questions
- What does “research engineering” mean in this context?
- Why do agents fail at open-ended science if they already solve complex code benchmarks?
- Does this mean AI is useless for doing science?
- What other studies show similar findings?
- How can I check whether my own agent has this limitation?
- When are agents expected to close this gap?
- References
The distinction matters because much of the optimism around “AI that does science” rests on evaluations where the agent already knows which button to press. When that button disappears, the behavior changes completely.
TL;DR
- Princeton researchers evaluated AI agents on ML research engineering tasks versus open-ended science tasks.
- Agents perform well on bounded tasks: debugging code, optimizing a pipeline, reproducing an experiment with clear instructions.
- Performance drops when the task requires generating an original hypothesis without a predefined goal or success metric.
- The finding challenges the idea that current agents can act today as autonomous scientific researchers.
- It adds to a wave of 2025-2026 evaluations on agent autonomy in software and research tasks.
- The gap between executing well and discovering something new is shaping up as the next hurdle for agentic AI.
- The context comes as the US allocates more than $5 billion to AI for science through the Genesis Mission.
What Happened
The report on the Princeton study describes an evaluation with two categories of tasks assigned to AI agents: on one hand, machine learning research engineering (debugging a broken script, tuning hyperparameters to hit a metric, reproducing a published result); on the other, open-ended science, where the agent must decide which question is worth investigating and how to test it, without a correct answer already known to the evaluator.
The result is not symmetrical. In the first category, agents behave consistently and close to that of a human engineer with intermediate experience. In the second, performance degrades: agents tend to repeat unproductive approaches, converge on trivial solutions, or simply not know when to stop exploring.
Context and History
The gap doesn’t come out of nowhere. Since 2023, benchmarks like SWE-bench (for solving real issues in software repositories) and GAIA (for general reasoning and tool-use tasks) have shown that agents improve quickly when a task has a verifiable success criterion. The organization METR has also been documenting how, evaluation after evaluation, the “task horizon” an agent can sustain autonomously before losing track keeps growing.
On the other side is the most ambitious attempt so far to automate science end to end: projects like Sakana AI’s The AI Scientist, which in 2024 proposed a pipeline where an agent generates the idea, runs the experiment, writes the paper, and even reviews it. The promise was appealing, but it exposed the same problem Princeton now confirms: without an external judge to say what counts as “good,” the agent has no compass.
The interest isn’t purely academic. The White House announced more than $5 billion in July 2026 for the Genesis Mission, its initiative applying AI to scientific research, and outlets like STAT have already questioned the balance between AI and other areas of science within that plan. Knowing what agentic AI is actually good at today, and what it isn’t, directly shapes where that money should go.
Technical Details and Performance
The technical key to the finding lies in the reward structure. A research engineering task almost always has an explicit objective function: reduce a loss, clear an accuracy threshold, pass a test. The agent can iterate, measure the result against that number, and adjust its next attempt. This is exactly the kind of loop current agentic language models are trained for.
An open-ended science task offers no such signal. The agent first has to decide what to ask, then design how to test it, and on top of that judge whether the result it got is interesting. That triple leap (defining the goal, designing the experiment, evaluating the finding) is precisely what distinguishes a researcher from a highly competent executor.
| Task type | Typical example | What it demands from the agent | Observed performance |
|---|---|---|---|
| Research engineering | Optimizing a training pipeline, debugging a script, reproducing a published baseline | Following instructions and iterating against an already defined metric | High, close to that of a junior to semi-senior engineer |
| Open-ended science | Proposing an original hypothesis, designing the experiment to test it, deciding what to measure | Defining its own goal and tolerating ambiguity with no clear success signal | Low, scatters or converges on trivial solutions |
💭 Key point: it’s not that the agent “doesn’t know science,” it’s that it doesn’t know when to stop or what to count as a valid result if no one tells it beforehand.
How to Test It Yourself
You don’t need access to the original study to reproduce the idea on a small scale. It’s enough to put together two batches of tasks for your own agent (one bounded, one open-ended) and compare the variance in results between them.
from dataclasses import dataclass
@dataclass
class Tarea:
descripcion: str
tipo: str # "ingenieria" or "ciencia_abierta"
criterio_exito: str | None # None if it's an open-ended task
tareas = [
Tarea("Reduce the training time of the baseline model by 20%",
"ingenieria", "tiempo_epoca <= objetivo"),
Tarea("Propose a new hypothesis about why the model generalizes worse on domain B",
"ciencia_abierta", None),
]
for tarea in tareas:
print(f"[{tarea.tipo}] {tarea.descripcion}")
That skeleton only declares the batch. What reveals the Princeton pattern is the evaluator: for bounded tasks, comparing a number against a threshold is enough; for open-ended ones, a qualitative criterion is needed.
def evaluar_tarea_ingenieria(resultado, objetivo):
"""Compares the result against a known numeric metric."""
return resultado.metrica >= objetivo.metrica_minima
def evaluar_tarea_abierta(resultado, revisor_humano):
"""No reference metric: a human reviewer scores originality,
methodological validity, and whether the hypothesis is falsifiable."""
return revisor_humano.calificar(
originalidad=resultado.hipotesis,
metodologia=resultado.diseno_experimental,
contrastabilidad=resultado.es_falsable,
)
resultados = agente.ejecutar_lote(tareas)
for tarea, resultado in zip(tareas, resultados):
aprobado = (
evaluar_tarea_ingenieria(resultado, tarea)
if tarea.tipo == "ingenieria"
else evaluar_tarea_abierta(resultado, revisor_humano="editor_senior")
)
print(tarea.descripcion, "->", "approved" if aprobado else "insufficient")
To check whether your own agent falls into the same pattern, run the same batch several times and compare the variance in results between the bounded tasks and the open-ended ones. If the variance in the open-ended ones is much higher, you’ve just replicated the finding at pocket scale.
Impact and Analysis
The finding doesn’t say AI is useless for science: it says where it fits today. An agent that optimizes pipelines, cleans datasets, or reproduces a published experiment already saves real researcher time. What it still doesn’t do reliably is decide what to investigate and why it matters.
That nuance clashes with the recent investment narrative. The US Genesis Mission and similar announcements in various countries present AI as a general accelerator of science, without always distinguishing between automating bench work and automating discovery. The Princeton study works as a useful reality check for anyone who has to decide where in the scientific process it makes sense to plug in agents today.
⚠️ Heads up: an agent that performs well on code benchmarks isn’t guaranteed to perform just as well at choosing which problem to tackle. These are two distinct skills, and they’re evaluated differently.
What’s Next
The next generation of evaluations will likely stop mixing both task types into a single score and start reporting them separately, as METR-style evaluations have already been calling for. It’s also likely that labs promoting “scientist” agents will start showing separately how much of their pipeline is reproducible engineering and how much is genuine exploration, so as not to sell the former as if it were the latter.
📖 Summary on Telegram: View summary
Try it yourself: set up two tasks for your favorite agent, one with a clear metric and one without, and compare how long it takes to “give up” on each.
Frequently Asked Questions
What does “research engineering” mean in this context?
Machine learning tasks with a measurable goal: optimizing code, tuning hyperparameters, reproducing an already published experiment. There’s a metric to compare the result against.
Why do agents fail at open-ended science if they already solve complex code benchmarks?
Because solving complex code is still executing against a known criterion (getting the test to pass). Open-ended science requires defining that criterion before it can even be applied.
Does this mean AI is useless for doing science?
No. It means that today it’s more reliable as an accelerator for the mechanical stages of the scientific process than as a generator of research questions.
What other studies show similar findings?
Autonomy evaluations like those from METR and projects like Sakana AI’s The AI Scientist documented similar patterns: strong performance on tasks with a metric, weaker results in unguided exploration.
How can I check whether my own agent has this limitation?
Run the same agent against a batch of bounded tasks and another of open-ended tasks, and compare the variance and quality of the results between both batches.
When are agents expected to close this gap?
There’s no verifiable date yet. What can be measured is progress: tracking the evolution of benchmarks like SWE-bench, GAIA, and METR’s evaluations gives a concrete signal quarter after quarter.
References
- MSN coverage of the Princeton study: summarizes the finding on AI agents in research engineering versus open-ended science.
- Official Genesis Mission announcement: more than $5 billion from the US government for AI applied to science.
- STAT on the White House’s scientific blueprint: context on how much weight AI carries relative to other scientific areas in federal policy.
- Princeton University: official site of the institution that conducted the evaluation.
📱 Enjoy this content? Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day.
0 Comments