Isabelle Lee

Isabelle Lee

PhD Student

University of Southern California

I’m a 2nd year PhD student at USC, working with Dani Yogatama. I’m interested in interpretability - how we make sense of machine learning models, and how interpretability might uncover the underlying science of large-scale models. Right now, I’m exploring how large models learn during pretraining, particularly analogous to Emergence/Self-Organization, where component-level interactions lead to complex, system-wide patterns. I am specifically interested in pretraining, and broadly interested in training dynamics and methodologies. By understanding (pre-)training dynamics of LLMs, I hope to better parse and possibly develop methods to “debug” how LLMs acquire reasoning capabilities.

In my past life, I used to study Physics and Complex Systems, so I borrow some approaches from that toolbox.

Recent Publications

Quickly discover relevant content by filtering publications.
(2023). On Retrieval Augmentation and the Limitations of Language Model Training.

PDF Cite Link

(2023). SCORE: A framework for Self-Contradictory Reasoning Evaluation.

PDF Cite Link

Recent Posts

Contact