Saurab Mishra IISER THIRUVANANTHAPURAM
DESKTOP
_TtenTion · ZERO CACHE · WORKING AS INTENDED

Everyone has an opinion about transformers. Almost nobody has touched one.

So here is one you can touch. Six levels: you do the tokenising, you fight the softmax for attention weight, you break layer norm and put it back together. The maths under every control is the real maths. No magical boxes, no “it just learns”. By the end you will know exactly how simple this thing is — which is the part nobody sells you.

WHO WROTE THIS

The machine, one piece at a time

LEVEL 1 OF 6
Saurab Mishra SAURAB MISHRA
BS-MS · COMPUTATIONAL & APPLIED MATHEMATICS
THE CONSTRAINT

No GPU cluster. Two free T4s and a twelve-hour session limit. It turns out that picks better research questions than ambition does.

Who put you through that

I work on multi-agent language models and the mathematical structure inside their representations, under Prof. Kripabandhu Ghosh. The short version: I want to know whether interpretability can be a mathematics problem instead of a visualisation problem — whether representation space has geometry you can describe, and whether sudden jumps in capability are phase transitions rather than surprises.

The longer version is six manuscripts. That is not modesty. That is a timeline.

THE NULL RESULT

One paper exists because I spent 7,752 API calls proving my own idea was decorative. Every ablation came back insignificant. I could have buried it in an appendix and nobody would have checked. It is the finding instead.

THE CLUB

Founded the Coding Club at IISER TVM in 2024, grew it to 200 members, ran an all-India hackathon for about a thousand people, then handed it over. Still on the advisory committee, which is a polite word for “occasionally consulted”.

THE SIX, WITH THEIR ACTUAL STATUS