⏱️ Reading time: 9 min
A cybersecurity team from Indiana University has just joined a front that until recently went unnoticed: protecting the artificial intelligence models and agents that already write lab code, process genomic data, and suggest scientific hypotheses. The news comes days after Nature published an analysis on which scientific roles face the greatest exposure as AI enters the laboratory, and it leaves an uncomfortable idea: nobody yet audits AI cybersecurity with the same rigor applied to an experiment.
📑 En este artículo
IU’s case isn’t an isolated incident, but a shift in posture: treating every downloaded model, every agent connected to an instrument, and every training dataset as part of a supply chain that can fail.
TL;DR
- Experts from Indiana University are forming a team dedicated to protecting the AI systems used in scientific research.
- Nature published an analysis on which scientific jobs face the greatest exposure to AI automation.
- The most cited attack vectors are insecure model deserialization, data poisoning, and prompt injection.
- Formats like pickle allow arbitrary code execution when loading a model; safetensors avoids that risk by design.
- Tools like Sigstore and cosign make it possible to sign and verify a model’s origin before using it in a laboratory.
- The NIST AI Risk Management Framework is today the most cited reference for governing these risks, though it isn’t mandatory.
- No regulatory body yet requires a single verification standard for AI models used in science.
What scientific AI cybersecurity is
Scientific AI cybersecurity is the set of practices that protect the models, data, and pipelines a laboratory uses to analyze experiments, generate hypotheses, or automate research tasks, against manipulations that could alter their results or leak sensitive information.
It crosses two worlds that rarely talked to each other: IT teams that audit software against lists of known vulnerabilities, and scientists who just wanted the model to work. Companies like Anthropic, OpenAI, and Google have documented real attacks against their own production systems, but academic laboratories are only beginning to apply those same lessons to microscopes, sequencers, and Jupyter notebooks connected to AI agents.
What happened
According to IU Newsroom, cybersecurity specialists at Indiana University joined research projects to identify the hidden risks that emerge when AI is integrated into scientific workflows: from agents that execute automatically generated code to models that process clinical or genomic data without going through a traditional security review.
The move coincides with an article published by Nature that examines which specific scientific jobs face the greatest exposure to AI automation. Both pieces describe the same phenomenon from different angles: one analyzes the labor risk, the other the technical risk of the systems that make that automation possible.
Context and history
AI adoption in science grew fast: language models that summarize papers, agents that design experiments, and vision systems that classify samples now coexist in laboratories that, just a few years ago, didn’t even have a dedicated information security team. That leap left blind spots already known in the software industry but new to science: almost nobody signs a Hugging Face checkpoint before loading it onto the lab server, and few verify what a .pkl file contains before deserializing it.
This isn’t a theoretical problem. The security community has documented for years that Python’s pickle format executes any code embedded in the file as soon as pickle.load() is called, a behavior Hugging Face openly acknowledges in its own Hub security documentation. The industry’s response was to create safetensors, a format that only stores numerical tensors and doesn’t allow code execution.
That pattern has already been seen before in the broader software industry: security communities have repeatedly identified checkpoints with malicious code uploaded to public repositories, disguised as legitimate versions of popular projects. Science isn’t immune to that vector just because the person using it is a researcher rather than a software engineer; if anything, it’s more vulnerable, because few laboratories have a dependency review process equivalent to what a mature DevOps team already requires.
Technical details and performance
The table below summarizes the most cited attack vectors against AI systems in research environments, along with the practical mitigation and reference tool for each.
| Attack vector | How it works | Mitigation | Reference tool |
|---|---|---|---|
| Insecure deserialization (pickle) | The model file executes arbitrary code when loaded with torch.load | Migrate to formats that don’t execute code | safetensors |
| Data poisoning | Manipulated examples are injected into the training or fine-tuning set | Validate provenance and hash of each dataset | SHA-256 checksums / DVC |
| Prompt injection | A document or lab result contains hidden instructions for the AI agent | Sanitize inputs and limit the agent’s permissions | Sandboxing / allowlists |
| Pretrained model without provenance | A checkpoint is downloaded from an unverified repository | Sign and verify the origin before deploying | Sigstore / cosign |
None of these vectors are exclusive to science: they’re the same ones faced by any engineering team deploying models in production. What’s different is the context, because a laboratory usually connects the AI agent directly to instruments, clinical records, or genetic sequences, so a failure doesn’t just compromise a server, it compromises the experiment. That’s why AI cybersecurity still doesn’t have a single verification standard in academic settings, unlike what’s already required, for example, of a bank or a certified cloud provider.
flowchart TD
A["Researcher"] --> B["AI agent in the lab"]
B --> C["Pretrained model"]
C --> D[("Experimental data")]
E["Attacker"] -.->|"malicious pickle"| C
E -.->|"hidden prompt"| B
subgraph Lab
B
C
D
end⚠️ Watch out: Loading a model withpickleortorch.loadwithout checking its origin is equivalent to running a script downloaded from the internet with administrator permissions over the lab’s data.
How to start auditing your AI pipeline
You don’t need a full security team to take the first steps. Here are two checks any laboratory can run today before loading an external model.
First, scan any pickle file before deserializing it:
pip install picklescan
picklescan --path modelo_experimento.pklpicklescan checks the instructions embedded in the file and warns if it detects calls capable of executing code or opening a network connection, without needing to deserialize anything first.
Second, if your project already uses Hugging Face models, migrate to safetensors and verify the file’s signature before deploying it on the lab server:
pip install safetensors huggingface_hub
python -c "from safetensors.torch import load_file; pesos = load_file('modelo_bioinformatica.safetensors'); print(list(pesos.keys())[:5])"
cosign verify-blob --signature modelo.sig --key cosign.pub modelo_bioinformatica.safetensorsThe first command confirms the file loads without executing code, listing the first keys of its tensors. The second confirms the binary matches the signature published by whoever trained it, the same principle used by software packages signed with Sigstore.
💡 Tip: If your lab already uses Hugging Face, enable the automatic malware and pickle scanning the platform offers before downloading any new checkpoint.
Impact and analysis
Indiana University’s shift matters because it moves the conversation from “AI is going to take our jobs” to “the AI we use could be compromised,” a more immediate risk that’s also easier to mitigate with known software security practices. It also marks an institutional shift: universities are starting to treat their AI models as part of the software supply chain, with the same provenance obligations already required of a code library.
The real limitation is that scanning a file or signing a model doesn’t solve the governance problem: someone has to decide which models are authorized, who can deploy them, and what happens when an AI agent with access to an instrument receives an instruction it shouldn’t execute. That part still has no technical solution, only internal policy, which is why automatic scanning gives a false sense of security if it isn’t paired with clear rules about who approves what.
What’s next
The most cited framework for organizing this discussion is the NIST AI Risk Management Framework, which proposes a cycle of governing, mapping, measuring, and managing AI risks, but remains voluntary. Until a specific regulatory requirement exists for science, adoption will depend on more universities following IU’s example and adding security staff dedicated to their own AI labs, instead of delegating that responsibility entirely to model providers.
Try it yourself: run pip install picklescan safetensors today and scan the latest model you downloaded for your research project.
Frequently Asked Questions
What does a lab gain from protecting AI models?
It reduces the chance that a compromised model alters results, leaks patient data, or executes unauthorized code on a server connected to instruments, without slowing down the everyday use of AI in research.
Why is it risky to load a model with pickle?
Because the format allows executable instructions to be embedded within the file itself: when deserialized, Python runs that code with the same permissions as the process loading it, without any prior warning.
How do I verify a model’s provenance before using it?
By confirming it comes in safetensors format, checking its hash against the one published by the author, and, if available, verifying its signature with tools like cosign before deploying it.
Does AI defense in science replace traditional cybersecurity?
No: it adds to it. Labs still need firewalls, access controls, and operating system patches; what’s new is adding the models’ own supply chain to that same discipline.
Is there an official standard for the security of AI systems in science?
Not yet a mandatory one. The NIST AI Risk Management Framework is the most widely adopted voluntary reference, but no agency currently requires specific certification for models used in research.
References
- IU Newsroom: report on Indiana University’s cybersecurity team dedicated to protecting AI in research.
- Nature: analysis of which scientific jobs are most exposed to AI automation.
- safetensors (GitHub): official repository for the format that replaces pickle for distributing models without executing code.
- NIST AI Risk Management Framework: official reference framework for managing AI system risks.
- Sigstore: open source project for signing and verifying the origin of software artifacts, including models.
📱 Enjoying this content? Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day.
Featured image: Foto de Albert Stoynov en Unsplash
Did it work for you? Got a different error? Say so below: questions get answered and help the next reader.
Leave a comment
0 Comments