projects
Roughly ordered by how much they hurt to build. Everything has source code, and the live demos are all real, so feel free to poke at them.
latent diffusion from scratch
Rectified-flow latent diffusion in an MMDiT-style architecture, written from the paper up in PyTorch. This is the fifth iteration over eight months; the earlier attempts wouldn't train stably until I rebuilt the data pipeline. The demo samples 256px images from my trained checkpoint.
NeuroZip (3rd place, QBI Hackathon @ UCSF)
A task-aware neural codec for brain recordings. Instead of compressing EEG to preserve the waveform, it compresses to preserve meaning: what the person was looking at. At 144x compression the result is still searchable by free text. Built with Avinash Senthil; won $1,000 plus $10,000 in grants.
homegrown LLM
A transformer language model pretrained from scratch, then instruction-tuned. Third in an eight-month line of from-scratch LMs, after an autocomplete model and a first chat model. You can chat with it in your browser.
AWS Trainium Frontier
A competition to train the best language model you can in 30 minutes on a single AWS Trainium chip. Over a month and 106 submission versions I took the baseline from 1.259 to 0.959 bits/byte, with custom NKI kernels for most of the model. Finished in the top 15 of 100 teams.
Want more? See other projects here: other projects