For most of computational science, computers did what scientists told them to do, leaving scientists the difficult task of figuring out what they actually should have told them.

Still, the division of labor was legible. Scientists formed questions, selected methods, designed models and experiments, interpreted evidence, wrote dubious code, and decided what conclusions the evidence could bear. Computers computed.

This arrangement is changing, rapidamente.

AI systems increasingly span activities once treated as separate parts of scientific work: searching literature, proposing hypotheses, writing software, constructing models, analyzing data, invoking specialized scientific tools, and interpreting results (Wang et al., 2023; Lu et al., 2024; Gottweis et al., 2026). Recent systems have begun connecting these activities into iterative workflows, including experimental biology, where agents can move from literature search to hypothesis generation, experimental proposal, data analysis, interpretation, and revised hypotheses (Ghareeb et al., 2026).

Connecting the steps is not the same as doing them well. Independent evaluation of early autonomous-science systems has found basic failures in areas such as literature review and novelty assessment (Beel et al., 2025).

Anthropic’s recently released Claude Science workbench also offers a glimpse of what this might look like in practice. A coordinating agent can draw on dozens of scientific tools, send work from a researcher’s laptop to remote computing systems, and preserve the resulting code, environments, conversations, and artifacts. Another agent checks citations, calculations, and whether the figures bear any recognizable relationship to the code that supposedly produced them, a task traditionally reserved for Reviewer 2. This is a beta product described by the company that built it, not evidence that autonomous science has arrived. But the architecture is worth noticing. We are beginning to move from computers as scientific instruments toward computers as participants in the scientific process. One agent does the work. Another checks the first agent. Eventually, presumably, we will need an agent to check the checking agent. At some point a human will be asked to click Approve. Whether any of this produces better science is an empirical question that we hopefully still have the capacity to address.

This is usually described as greater capability. It is also greater delegated scientific judgment.

The distinction matters. The central problem of autonomous science may not be whether machines can perform scientific work. It may be whether we can delegate that work without quietly delegating the authority to decide what counts as good science.

The loss need not arrive as a handoff anyone remembers approving. Each researcher saves time. Each journal accelerates review. Each funder triages more proposals. The local decisions may be reasonable; together they can reorganize science around what machines can cheaply produce and score.

No machine needs to want this. Incentives can do the wanting for it.

A Cluster Is Not a Methodology

Much of the architecture of AI agents revolves around tools. Give the agent Python, a browser, scientific libraries, a cluster, and another agent to consult when the first becomes lonely or wrong. Tool access expands capability. Methodology governs how that capability should be used.

A computational model can be implemented perfectly and answer the wrong question. A sensitivity analysis can execute correctly while varying irrelevant parameters. A model can fit observations beautifully because the same evidence was used for calibration and validation. A simulation can complete ten million runs across a parameter space that should never have been defined that way. The software worked but the science did not.

These are methodological failures. Good computational modeling requires choices about purpose, abstraction, system boundaries, evidence, uncertainty, evaluation, and interpretation. Each must be justified relative to the scientific question.

An agent that can call a sensitivity-analysis library knows something useful. An agent that knows when sensitivity analysis is appropriate, which assumptions make it meaningful, which alternatives deserve consideration, and what conclusions it can support knows something more important.

The first is tool use. The second begins to resemble scientific judgment.

Scientific Error Can Exit Successfully

Software engineers have spent decades making computational failures conspicuous. Programs throw exceptions, tests fail, type checkers complain. Continuous integration goes red and sends the ops team a message during a nice dinner.

Scientific reasoning has a more troublesome failure mode: it can fail silently.

A badly specified model may produce attractive figures. An unjustified assumption may survive thousands of simulations without causing one warning. A weak evaluation protocol may culminate in a polished report declaring success. Every file opens. Every chart has a legend. The conclusion remains unsupported.

Generative AI intensifies this problem because its characteristic failure is often not an exception. It is a plausible answer. An agent can turn one plausible methodological mistake into an entire plausible workflow:

  1. A weak assumption becomes code.
  2. The code generates experiments.
  3. The experiments generate data.
  4. The data generate figures.
  5. The figures support an interpretation.
  6. The interpretation becomes a conclusion.

Every artifact may look reasonable. The causal chain is still wrong.

Scientific agents therefore need more than conventional software verification. We need to evaluate the reasoning structures that generate their workflows. A result should arrive with the consequential assumptions, methodological choices, uncertainties, and evidence that made it possible.

The green check mark is welcome. It is not peer review.

From Reproducible Computation to Reviewable Reasoning

Open science has built increasingly mature practices for preserving computational research. We version software, archive releases, record provenance, describe environments, publish datasets, package workflows, assign persistent identifiers, and make research objects FAIR (Wilkinson et al., 2016).

All of that remains essential. Autonomous workflows add another research object worth preserving: the reasoning that connects the question to the computation.

This does not require preserving an agent’s private internal monologue. What science needs is a durable record of the consequential decisions it made, the evidence and assumptions available when it made them, the alternatives considered, and the justification connecting those choices to the resulting computation.

Why was this model chosen? Which processes were represented or omitted? Which assumptions came from evidence, theory, or judgment? Why were these parameters and evaluation criteria selected? Why does the evidence support this conclusion rather than a stronger and more exciting one?

Papers answer some of these questions, imperfectly and after the fact. Autonomous science creates a stricter requirement. If an agent makes consequential methodological decisions while constructing a workflow, those decisions must become inspectable while the workflow is being constructed.

Scientific reasoning needs durable artifacts that outlast an agent’s context.

Intermediate Artifacts Are Scientific Memory

The final model is not enough. The source code is not enough. The paper is not enough.

A reviewable workflow may also need explicit records of its question, conceptual model, assumptions, evidence, uncertainty, experimental design, evaluation criteria, limitations, and provenance.

These records belong to the science itself.

Computational modeling moves backward as often as forward. Evaluation exposes trouble in calibration; calibration exposes trouble in data; data expose trouble in assumptions. A credible workflow preserves enough reasoning to retrace those steps without relying on institutional folklore or the memory of the postdoc who has since moved to Finland.

Humans have notebooks, meetings, arguments, disciplinary norms, and reviewers who ask unpleasant questions six months later. Agents have context windows. A context window is a working surface, not a scientific archive.

Reviewable artifacts provide persistent scientific memory. They also create places where people can intervene before one questionable decision acquires a large and well-formatted family.

Governing Scientific Authority

As software moves from tool to delegate, the design question changes from What can the system do? to What may the system decide? Human-centered design for autonomous systems faces the same inversion. The interface no longer waits, so interruption must be designed into the work.

Scientific agents make this unusually difficult. We can prevent an agent from deleting a database. It is harder to prevent it from choosing an inappropriate conceptual abstraction. We can require approval before spending $10,000 because money comes with units; epistemic overreach has resisted standardization, so no equivalent threshold tells us when a model has outrun the evidence supporting its intended use.

Permissions govern access to tools and data. Scientific authority also requires epistemic checkpoints:

  • Before implementation, is the conceptual model explicit and reviewable?
  • Before calibration, are parameter meanings and evidence sources clear?
  • Before experimentation, is there an analysis plan?
  • Before predictive claims, has uncertainty been characterized?
  • Before a model informs decisions, has evaluation established credibility for that purpose?

Calibration, verification, validation, sensitivity analysis, and robustness provide different kinds of evidence. Successful calibration does not establish universal validity. Agreement with observations does not prove that a model is correct. Credibility remains conditional on purpose, evidence, assumptions, and scope (NASA-STD-7009B, 2024).

A capable agent can perform all these activities. Scientific agency means retaining the ability to inspect why it performed them and whether the claims that followed are justified.

Uncertainty Is Information

Autonomous systems are often imagined as machines for removing uncertainty. Give the agent an objective and enough information; receive an answer. The answer may even arrive in a bordered panel.

Science is less accommodating.

Some uncertainty comes from limited data. Some comes from parameters, model structure, competing explanations, or assumptions about futures that have not happened. Some comes from disagreement about what the problem is.

Good modeling practice treats uncertainty as information to characterize and communicate, not an embarrassment to hide. Claims should remain proportional to the evidence supporting them.

Under deep uncertainty, defensible probabilities may not exist. People may disagree about objectives, system dynamics, or which futures matter. The responsible response may be to explore plausible futures and identify robust or adaptive strategies rather than predict one optimal path (Marchau et al., 2019).

This creates a useful test for scientific agents. A weak system produces an answer. A stronger one may produce a distribution. A scientifically mature system must sometimes say: the available evidence does not justify collapsing these possibilities into one answer.

Knowing when not to resolve uncertainty may matter as much as knowing how to calculate through it.

Scientific Judgment Is Social

There is a tempting answer to all this: train sufficiently capable models on enough papers, methods, and expert demonstrations, and they will learn what good science looks like.

Perhaps. Scientific judgment lives partly in literature and partly in practice, institutions, and communities.

Models used for policy or socio-environmental decisions involve questions about who defines the problem, whose knowledge counts, which outcomes matter, what harms are acceptable, and which uncertainties deserve attention. Those concerns shape the technical work from its beginning.

Participatory modeling makes this explicit. Stakeholders contribute observations, local knowledge, expertise, values, objectives, assumptions, and competing interpretations. Good practice records how those contributions changed consequential modeling decisions and where disagreement remained (Voinov and Bousquet, 2010).

Averaging disagreement can hide it beneath a synthetic consensus. Generating imaginary stakeholder perspectives creates the appearance of participation while leaving actual stakeholders absent. That is autocomplete wearing a name badge.

Some scientific questions are inseparable from legitimacy and authority. Scientific agency is therefore distributed across a system of scientists, stakeholders, institutions, software, data, models, agents, and reviewers.

The important question is not whether the AI “did the science.” It is whether the whole system preserves meaningful opportunities to understand, challenge, revise, and take responsibility for the judgments being made.

Methodology Can Become Infrastructure

Agent skills are usually described as reusable packages of capability. A skill teaches an agent to use a framework, query a database, deploy an application, or generate a document.

Scientific skills can encode something else: methodological constraints on action.

A modeling skill might require a clear purpose before method selection. A conceptual-modeling skill might require explicit system boundaries and abstraction choices. An uncertainty skill might prevent unsupported certainty claims. An evaluation skill might distinguish calibration from validation. A participatory-modeling skill might keep unresolved disagreement visible rather than blending it into synthetic harmony.

Such skills are computational artifacts containing pieces of scientific practice. Their quality is consequential.

Who wrote the methodology? What supports it? Which disciplinary assumptions does it encode? What does it omit? When should it trigger or defer? What happens when it is wrong?

Once methodology becomes software, disciplinary assumptions can ship as defaults (a class of decisions with a distinguished record of surviving long after everyone has forgotten choosing them).

Scientific communities already ask these questions about datasets, software, models, workflows, and standards. Skills extend the stewardship problem by creating another research object for communities to evaluate and maintain.

Science Has an Objective Function Too

Science is not one optimizer. It contains a disconcerting number of them.

Universities count papers. Journals ration attention. Funders rank proposals. Researchers learn what survives these systems and produce more of it. None of this began with AI. We were reward-hacking publication systems while computers were still making dial-up noises.

AI changes the scale, speed, and closure of the loop. If models help write manuscripts, evaluate manuscripts, and learn from the manuscripts that survive evaluation, scientific language can become increasingly optimized for machine-mediated judgment. Michael Hochberg and Peter Thrall call the danger “epistemic autophagy”: science consuming its own processed output and mistaking internal consistency for contact with the world.

Every citation may resolve. Reality may still decline to participate.

The seductive story of autonomous science is that scientific knowledge becomes cheap. Agents read the literature, write the code, run the simulations, analyze the results, and draft the paper. What becomes cheap is the production of scientific-looking artifacts. Determining which artifacts deserve trust remains expensive.

There is a darker objective function. Policy scholars call it policy-based evidence making: the preferred conclusion arrives first, and research is commissioned, selected, or arranged to support it (Straßheim and Kettunen, 2014). Humans have traditionally done this by hand, which at least imposed some limits on throughput. An autonomous system can search across model structures, exclusions, priors, time horizons, and framings until one produces the requested answer, then generate the review, figures, and provenance that make the route look inevitable. No hallucination is required. Every calculation can be correct. Every citation can exist. The machine need only be extremely diligent about answering the wrong question. This is progress of a sort. Guarding against it requires preserving the paths not taken: rejected alternatives, changing objectives, conflicts of interest, and the inconvenient analyses that failed to produce the desired result. It also requires genuinely independent evaluation, preferably by someone who was not told what the answer was supposed to be.

This concern predates the machinery. Karl Popper located scientific objectivity in intersubjective testing and mutual criticism. Robert Merton described organized skepticism as an institutional norm of science. Helen Longino later argued that objectivity depends on communities capable of transformative criticism, including criticism from differently situated researchers. These accounts disagree about plenty, as philosophers reliably do, but they share an important premise: scientific knowledge depends on social arrangements that expose claims, assumptions, and methods to challenge. A reviewer agent can participate in such an arrangement. Its presence does not create one.

That is why independent paths of evaluation matter. A human-only review track is not a purity ritual. Humans bring bias, fatigue, politics, and a long record of optimizing whatever the dean happened to count that year. Its value is that it preserves a different failure mode against which AI-mediated review can be compared.

Provenance markers establish who or what touched an artifact, which leaves a bracing possibility: a complete audit trail can tell us exactly how we arrived at the wrong answer. Science also needs replication, disconfirmation, competing methods, explicit uncertainty, and researchers trained to understand why its methods exist rather than merely how to invoke them.

Open science already struggles to curate datasets, maintain software, preserve workflows, keep standards current, and find reviewers. Autonomous science may give us vastly more things to curate. The printer has become faster. The library has not hired anyone.

Progress will depend not only on scientific capability, but on institutions, standards, communities, and technical infrastructure able to steward it.

From Human Presence to Human Agency

It is tempting to conclude that humans should remain “in the loop.” The phrase has achieved the dignified vagueness of a mission statement.

A researcher approving hundreds of agent-generated decisions without time to inspect them is technically in the loop. A principal investigator signing a workflow whose methodological decisions were made invisibly by agents is in the loop. A reviewer receiving a polished paper without access to the reasoning that produced its model is in the traditional scientific loop.

The important property is not human presence. It is human agency.

Can someone understand consequential decisions, inspect the evidence, challenge an assumption, and reject a methodological choice? Can they trace a conclusion backward, revise an upstream decision, identify stale downstream claims, or refuse to proceed?

These capabilities matter more than whether a workflow diagram contains a rectangle labeled HUMAN REVIEW.

The Scientific Autonomy Slider

Scientific autonomy spans a large territory between calculator and autonomous scientist.

An agent might:

  • observe a scientific workflow
  • suggest methodological choices
  • prepare analyses for inspection
  • execute approved analyses
  • interpret results within explicit boundaries
  • revise plans when evidence contradicts assumptions
  • initiate new scientific work independently

Activities belong at different positions on this slider. Running a reviewed simulation may require little supervision. Changing the conceptual model requires more. Formatting a figure exercises a different class of judgment from redefining the research question. The same distinction separates generating another replicate from deciding that the evidence supports a policy.

The slider applies to institutions too. Drafting a review, allocating attention, and setting a research agenda carry progressively more institutional authority. Proposal triage can become agenda-setting when its rankings determine what receives consideration. Combining production, judgment, and incorporation into future training data would leave one machine examining its own paperwork at every stage.

Autonomy should attach to classes of scientific judgment. Access to tools is a poor proxy. It should expand when evidence shows that the system can exercise that judgment reliably, visibly, and within a recoverable scope. This is the same principle that should govern agentic work more broadly: autonomy must earn its keep.

What Scientific Agency Means

The goal is to use autonomous systems where they expand what people and communities can investigate while keeping the epistemic foundations of that work visible and contestable. Human effort can move away from routine execution without surrendering authority over purpose and method.

Call this scientific agency: the capacity to understand, direct, inspect, challenge, and take responsibility for the purposes and methodological judgments through which scientific knowledge is produced.

Humans need it and autonomous systems need an executable form of it. Most importantly, the scientific community must retain it across the whole system.

That requires methodology, provenance, explicit uncertainty, reviewable artifacts, evaluation, reversibility, participation where it matters, and communities capable of curating and critically reviewing what has been delegated. It also requires boundaries around the judgments we will delegate.

Machines may perform extraordinary amounts of scientific work in the future. Good. Hopefully it was the busy work. There are more worthy questions than we have time to investigate, but science has also always depended on somebody being willing to say, “I do not think this follows.” Autonomous systems will greatly increase the number of times that sentence needs to be said.

But scientific progress has never consisted merely of producing more answers, it depends on knowing why some answers deserve belief.

Machines can absolutely help but the scientific community must retain authority over what counts as knowing, including when delegation occurs through many reasonable local decisions and no one remembers approving the handoff.

Sources