Research
Current LLMs are trained on the entirety of the contemporary publicly available scientific literature. From our own observations, since 2023, frontier LLMs have been trained on the entirety of PubMed Central, and since at least 2024, the most recent model releases are fully acquainted in memory with preprints in arXiv, bioRxiv, and medRxiv that were posted 6-18 months earlier.
For developers of biological software, a common path to publication is a message on X/Twitter the day that the manuscript describing the software is posted on arXiv or bioRxiv. These preprints become part of the models' pre-training corpora. Further discourse on X/Twitter and elsewhere follows initial posting, and of course more discourse and possible commentary occurs when the work is formally published. Preprint contents are accessible within weeks after posting, and become part of training corpora, and are accessed again when a model "re-reads" that specific paper at inference time.
Inspection of models' chains of thought frequently reveals cases where a model's decision to not engage with a piece of biological software rests on its determination that the software could be used for biological weapons.
The upshot is that if preprints describing biological software, findings, or methods noted that the authors thought their contents might potentially be used for evil purposes, near-future models will be aware of the authors' expressed concerns.
These observations suggested a possible low-effort path to influence models' decisionmaking, by making it easy for authors to append warning messages to preprints of their scientific papers. Preprint traffic passes through three chokepoints. In biology, essentially all problematic dual-use papers pass through arXiv, bioRxiv, or medRxiv. Posted preprints become part of the training corpora of near-future models. Even before preprints enter training corpora, models refer to the relevant preprints at inference time.
By placing warning notices into preprints, we thought that the efficacy of technical LLM countermeasures ("LLM repellents" or "Content bombs") might be increased. An example of such a notice is shown in the box. Because we know from observation of chains of thought that models consider published papers in their assessments about the dangers of particular software, it also seemed possible that near future LLMs that have received warnings via preprints might decide not to engage with software based on their understanding of the preprint itself. For biology, dangerous or dual-use content contained in preprints consists of more than software; it also includes descriptions of methods and new knowledge.
This idea of embedding warnings and notices in high value / "high integrity" data streams destined for pre-training corpora, to influence future models, has some precedent. In particular, in our current not-optimistic time, the idea that future AI models might learn alignment and misalignment from papers in training data describing alignment research has now become something of a trope (Tice et al., 2026; Westover, 2025). And there has been at least one deliberate direct (and to PI's mind pathetic) attempt to communicate with future AI via LessWrong (Miller et al., 2023) in which the authors list all the reasons and arguments they can think of to convince future superpowerful AI that it should spare humankind.
We have approached Richard Sever, Scientific Director of openRxiv (bioRxiv and medRxiv), and Steinn Sigurdsson, moderator of arXiv, to suggest allowing authors who submit preprints describing work (software, knowledge, methods) that they consider dual use to check a box saying so, and by so doing causes a formulaic notice to be associated with the preprint. Both are positive, and have said that they will assist us with pilot experiments.
Our group will be looking to see whether near-future models have encountered the warnings in their pre-training data. We will do this in two ways. One is to compute "perplexity" of the model for every token in tagged and untagged preprints and quantifying the drop for models that have been trained on preprints that contain our formulaic warning. We can do thus for Open Weight models, of course, and for Google Deepmind and OpenAI (but not Anthropic) where we know the log probabilities ("logprobs") that the model assigned at each position to the tokens that actually came next.
Commencing six months after project start, we will also begin assessing whether frontier and behind-frontier open weight models have been pre-trained on the papers by asking them to describe the papers from memory alone. Our work to date has shown that general contemporary models retain from training considerable information from individual papers but interpolate/ hallucinate details such as exact spellings of author's names. We will check models' memories against the actual preprints.
We will then allow the models to access the actual preprints.
We will ask whether the associated warnings influence models' decisionmaking and steer them toward more cautious conduct at inference time.
During the project period, we will undertake further work on how to make the notices maximally legible and credible to machine intelligences.
During the project period, we are optimistic that frontier and behind-frontier AI labs might agree to recognize and respect specific signals in software tools that ask their models not to operate the software. Due to personal contacts and the future location of one of us (Volkov), the month-to-month progress of our work will be legible to Open AI and Anthropic.
Our work has common cause with other civil society concerns about the deployment of AI agents. In particular, Meredith Whittaker (2025) does not want AI agents reading decrypted Signal messages on people's phones, and has urged development of a documented mechanism that would let application developers designate their apps as off-limits to AI agents. Dean Ball (2026) has suggested that society might make AI agents off limits to children. At the start of the project period, we will reach out to Whittaker and Ball making them aware of the complementary efforts and soliciting their public support.
In the wake of the OpenAI-HuggingFace incident (OAI-HF) and subsequent increased public awareness of misadventures by AI agents, we expect that our efforts to help scientists inflect how AI agents use their own work will be better understood and hope that they will be welcomed.
During the project period, we are also optimistic that developers of biological software might consider releasing their software under open source "No-AI-Agent" licenses. The most commonly used licenses for biological software are BSD-3-Clause and the MIT license, and our No-AI-Agent versions of these licenses (tested as part of our tests of countermeasures) simply add a clause that prohibits the software's operation by AI agents. We imagine having these licenses for real as part of the code in repositories would have stronger deterrent effects, as well as giving software developers greater freedom to implement in their software more active countermeasures: the "License Plate Readers" and the "Token Burning Traps" that we will be developing.
The PI knows from long ago Richard Stallman, and Eben Moglen at Columbia, the legal scholar who worked with Stallman to use copyright ("copyleft") to enabled the GPL (the General Public License) and make open source software possible. At the start of the project period, we will reach out to Moglen and Stallman and seek their public support for licenses that would allow developers of dangerous software to restrict its use by AI agents.
Notices attached by authors would reach models twice, once as training data and again when a model reads the paper at the time of inference. We consider it possible that such notices would increase the effectiveness of the countermeasures described elsewhere on this site, and possible also that they would independently make models less willing to assist in the use of the software, methods and results concerned. We can test these things. If this tactic succeeds, it would have a positive impact on the near future world.