University of Southern California

Trends

• Alternatives to Transformers start winning at equal compute

Items

1. May 12

   USC's Attractor Models

   A Transformer backbone proposes an output that a small weight-tied module refines to a fixed point at constant training memory; a 770M model matches a 1.3B Transformer trained on 2x tokens, and a 27M model solves 91.4% of Sudoku-Extreme where frontier LLMs score 0%.