Isabelle Lee
I’m a 2nd year PhD student at USC, working with Dani Yogatama. I’m interested in interpretability - how we make sense of machine learning models, and how interpretability might uncover the underlying science of large-scale models. Right now, I’m exploring how large models learn during pretraining, particularly analogous to Emergence/Self-Organization, where component-level interactions lead to complex, system-wide patterns. I am specifically interested in pretraining, and broadly interested in training dynamics and methodologies. By understanding (pre-)training dynamics of LLMs, I hope to better parse and possibly develop methods to “debug” how LLMs acquire reasoning capabilities.
In my past life, I used to study Physics and Complex Systems, so I borrow some approaches from that toolbox.