About

I am an AI researcher working on efficient neural network architectures and training methods. I am currently a Member of Technical Staff at Abridge. Previously, I was a Research Scientist at Meta, where I worked on developing on-device LLMs and was technical lead on pretraining text encoder foundation models for recommendation and retrieval systems.

My research has explored ways to leverage structure and invariances in data to build more efficient and effective machine learning models: for instance, by exploiting the structure of language with Tree-LSTMs, the various symmetries in images with Equivariant Transformers, and the underlying physical constraints of the Earth for ML-driven earthquake detection methods.

I hold a Ph.D. in Computer Science from Stanford University, where I was advised by Peter Bailis and Gregory Valiant. While at Stanford, I was a member of the Future Data Systems group and the DAWN project. During my time as an MS student, I was also affiliated with the Stanford NLP Group.

News

Selected Publications

Full list on Google Scholar.

  1. The Path Not Taken: RLVR Provably Learns Off the Principals

    Hanqing Zhu, Zhenyu Zhang, Hanxian Huang, DiJia Su, Zechun Liu, Jiawei Zhao, Igor Fedorov, Hamed Pirsiavash, Zhizhou Sha, Jinwon Lee, David Z. Pan, Zhangyang Wang, Yuandong Tian, and Kai Sheng Tai

    NeurIPS 2026

    Why reinforcement learning on reasoning tasks changes so few of a model’s weights and how its updates differ from supervised fine-tuning.

  2. MobileLLM-Pro Technical Report

    Patrick Huber, Ernie Chang, Wei Wen, Igor Fedorov, Tarek Elgamal, Hanxian Huang, Naveen Suda, Chinnadhurai Sankar, Vish Vogeti, Yanghan Wang, Alex Gladkov, Kai Sheng Tai, Abdelrahman Elogeel, Tarek Hefny, Vikas Chandra, Ahmed Aly, Anuj Kumar, Raghuraman Krishnamoorthi, and Adithya Sagar

    arXiv 2025

    A 1B-parameter language model built to run on phones and wearables with a 128K-token context window and 4-bit quantization.

  3. uCAP: An Unsupervised Prompting Method for Vision-Language Models

    A. Tuan Nguyen, Kai Sheng Tai, Sirius Chen, Satya Narayan Shukla, Hanchao Yu, Philip Torr, Taipeng Tian, and Ser-Nam Lim

    ECCV 2024Oral

    Learns the prompt for zero-shot image classification with vision-language models from unlabeled images instead of tuning it by hand.

  4. Spartan: Differentiable Sparsity via Regularized Transportation

    Kai Sheng Tai, Taipeng Tian, and Ser-Nam Lim

    NeurIPS 2022

    Trains neural networks to a target sparsity by gradually sharpening a soft, differentiable choice of which weights to keep.

  5. An End-to-End Earthquake Detection Method for Joint Phase Picking and Association using Deep Learning

    Weiqiang Zhu*, Kai Sheng Tai*, S. Mostafa Mousavi, Peter Bailis, and Gregory C. Beroza

    * Equal contribution

    Journal of Geophysical Research: Solid Earth, 2022

    A single neural network that detects earthquakes and picks seismic phases jointly across an entire network of stations.

  6. Sinkhorn Label Allocation: Semi-Supervised Classification via Annealed Self-Training

    Kai Sheng Tai, Peter Bailis, and Gregory Valiant

    ICML 2021

    Semi-supervised learning that treats assigning pseudo-labels to unlabeled examples as an optimal transport problem.

  7. Equivariant Transformer Networks

    Kai Sheng Tai, Peter Bailis, and Gregory Valiant

    ICML 2019

    A drop-in layer that undoes rotation, scaling, shear and perspective changes in an image before it is classified.

  8. Compressed Factorization: Fast and Accurate Low-Rank Factorization of Compressively-Sensed Data

    Vatsal Sharan*, Kai Sheng Tai*, Peter Bailis, and Gregory Valiant

    * Equal contribution

    ICML 2019

    Shows that low-rank factorizations like NMF can be computed directly on randomly compressed data and still recover the original factors.

  9. Moment-Based Quantile Sketches for Efficient High Cardinality Aggregation Queries

    Edward Gan, Jialin Ding, Kai Sheng Tai, Vatsal Sharan, and Peter Bailis

    VLDB 2018

    A 200-byte quantile summary that merges in nanoseconds to speed up percentile queries across many slices of a dataset.

  10. Sketching Linear Classifiers over Data Streams

    Kai Sheng Tai, Vatsal Sharan, Peter Bailis, and Gregory Valiant

    SIGMOD 2018

    A compact sketch that learns a linear classifier over a data stream while keeping track of its most important features.

  11. Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

    Kai Sheng Tai, Richard Socher, and Christopher D. Manning

    ACL 2015

    Extends LSTMs from word sequences to parse trees for better sentence similarity and sentiment classification.

Open Source Projects