AI Scientist Passes Peer Review: First Autonomous Research Tool Published in Nature After Paper Accepted at ICLR

The Milestone

Sakana AI's AI Scientist — an autonomous research tool — has become one of the first such systems to go through the peer-review process at a leading academic journal. The tool published a paper in Nature and had one of its autonomously-generated research papers accepted by reviewers at ICLR 2025.

How AI Scientist Works

The Full Cycle

1. Idea generation: Search literature, generate hypotheses, design research directions
2. Implementation: Write code, execute experiments, measure results
3. Paper writing: Write up findings in scientific paper format
4. Automated review: Built-in 'automated reviewer' evaluates quality

Technical Foundation

• Built on top of existing LLMs (GPT-4o, Claude Sonnet 4)
• Collection of specialized 'agents' for each research phase
• Results "approach borderline acceptability for ML conference workshops"

The Peer-Review Experiment

• Sakana AI submitted three AI-generated papers to an ICLR workshop
• Reviewers were told some papers might be AI-generated
• One paper was accepted — claimed as the first 'Turing test' pass for AI-generated research
• However, the accepted paper is "not at the same level" as the best human-produced papers

Toned-Down Claims

In response to peer-review feedback, the Nature paper tones down claims from the original 2024 preprint:

• No longer claims to automate the "entire" scientific process
• More honest about limitations and quality gaps
• Acknowledges the tool is best used as a "co-scientist" not a replacement

Risks and Concerns

• Paper flooding: Journals could be overwhelmed with AI-generated submissions
• Quality ceiling: Current output is "borderline" — not yet matching top human research
• Misplaced trust: Reviewers may unknowingly assess AI papers differently
• Resource drain: Peer review system could be overwhelmed

What's Next

Sakana AI co-founder David Ha emphasizes that AI should aid human scientists rather than replace them. The tool's value is as a "co-scientist" — accelerating the research cycle while humans maintain quality oversight.

Source: Nature (d41586-026-00899-w)