AI × 生物医药· arXiv · cs.LG·· 1 day agoPickScore63
recAMPLIFY: Large-Scale Pretraining of Recursive Protein Language Models
In brief
The authors adapt recursive pretraining to masked protein language models with recAMPLIFY, training on the same 2T tokens as AMPLIFY-350M. The model has 155M parameters, 56% fewer than AMPLIFY-350M, and improves CASP contact recovery and three of four supervised tasks, raising fold accuracy by 22.5%.
Why it matters
Trained on the same 2T tokens, recAMPLIFY improves CASP contact recovery and three of four supervised tasks with 56% fewer parameters, giving a parameter-efficiency comparison.
Source: AI × 生物医药 · arxiv.org