recAMPLIFY: Recursive Pretraining Shrinks Protein Language Models While Boosting Accuracy
Start here
Researchers have introduced recAMPLIFY, a protein language model that adapts recursive pretraining to masked language model architectures. Trained on the same 2 trillion tokens as AMPLIFY-350M, the model uses 155 million parameters — 56% fewer than its predecessor — yet performs better on several benchmarks. It improves contact recovery on CASP targets and wins on three of four supervised tasks, including a 22.5% gain in fold accuracy. The result suggests recursive pretraining can substantially cut model size without sacrificing, and indeed improving, predictive performance in protein modeling. Open questions include how the approach transfers to other model families and larger scales, and whether gains hold across broader downstream applications.
Written by a model from the reports · updated 3 hours ago
Coverage timeline
Follow the coverage to see the different angles.
- arxiv.orgPickrecAMPLIFY: Large-Scale Pretraining of Recursive Protein Language Models
The authors adapt recursive pretraining to masked protein language models with recAMPLIFY, training on the same 2T tokens as AMPLIFY-350M. The model has 155M parameters, 56% fewer than AMPLIFY-350M, and improves CASP contact recovery and three of four supervised tasks, raising fold accuracy by 22.5%.
Coverage and heat for this story
Reports so far 1·Heat now 5·Peak heat 10 (Oct 19 20:00)·Reports, 24h 0
Two panels, same clock (pointer stays in sync): top is cumulative editorial reports — only rises; last point = All reports; slope = new reports that hour. Bottom is Live-board heatIndex — 7-day participants with a 24-hour half-life, so it can fall when talk cools.