Research from our team and academic partners

Based: Simple Linear Attention Language Models Balance the Recall-Throughput Tradeoff

Simran Arora, Sabri Eyuboglu, Michael Zhang, Aman Timalsina, Silas Alberti, Dylan Zinsley, James Zou, Atri Rudra, Christopher Ré

arXiv, 2024

Zoology: Measuring and Improving Recall in Efficient Language Models

Simran Arora, Sabri Eyuboglu, Aman Timalsina, Isys Johnson, Michael Poli, James Zou, Atri Rudra, Christopher Ré

arXiv, 2024

TART: A Plug-and-Play Transformer Module for Task-Agnostic Reasoning

Kush Bhatia, Avanika Narayan, Christopher De Sa, Christopher Ré

arXiv, 2023

Mamba-3B-SlimPJ: State-space models rivaling the best Transformer architecture

Albert Gu*, Tri Dao*

Cartesia Blog, 2023

How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections

Albert Gu, Isys Johnson, Aman Timalsina, Atri Rudra, Christopher Ré

NeurIPS, 2022

S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spaces

Eric Nguyen, Karan Goel, Albert Gu, Gordon W. Downs, Preey Shah, Tri Dao, Stephen A. Baccus, Christopher Ré

NeurIPS, 2022

It's Raw! Audio Generation with State-Space Models

Karan Goel, Albert Gu, Chris Donahue, Christopher Ré

ICML, 2022

Efficiently Modeling Long Sequences with Structured State Spaces

Albert Gu, Karan Goel, Christopher Ré

ICLR, 2022

Domino: Discovering Systematic Errors with Cross-Modal Embeddings

Sabri Eyuboglu, Maya Varma, Khaled Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, Christopher Ré

ICLR, 2022

Model Patching: Closing the Subgroup Performance Gap with Data Augmentation

Karan Goel, Albert Gu, Yixuan Li, Christopher Ré

ICLR, 2021

HiPPO: Recurrent Memory with Optimal Polynomial Projections

Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, Christopher Ré

NeurIPS, 2020