I am a research scientist at Google DeepMind working on AI alignment. My goal is to develop interpretable and trustworthy AI systems that learn from human feedback.

Previously, I was a doctoral student in the Learning & Adaptive Systems Group at ETH Zurich supervised by Prof. Andreas Krause and Dr. Katja Hofmann. My dissertation focused on developing “Algorithmic Foundations for Safe and Efficient Reinforcement Learning from Human Feedback”. Before that, I received a master’s degree in Data Science from ETH Zurich and a bachelor’s degree in physics from University of Cologne in Germany.

Research

My research aims to build safe, robust, and interpretable artificial intelligence (AI). Currently, I am primarily working on Reinforcement Learning from Human Feedback (RLHF) which I think is a key ingredient for building safe AI. My work in this context has two goals: first, making RLHF more sample efficient via active learning, and second, using constraint models in addition to reward models to specify tasks, particularly in contexts where safety is a critical concern. Recently, my interests have expanded to other areas including interpretability, specifically mechanistically understanding neural network models, and red-teaming models before and during deployment. Through my work, I strive to ensure AI models we deploy today and in the coming years are safe, robust, and transparent.

News

• December 2023: I’ll be at NeurIPS 2023 in New Orleans presenting Tracr at the main conference and our work on VLM-RMs at the FMDM Workshop.
• October 2023: New preprint on arXiv: “Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning”.
• September 2023: Our paper “Tracr: Compiled Transformers as a Laboratory for Interpretability” was accepted to NeurIPS 2023 as a Spotlight.
• September 2023: I joined Google DeepMind’s Alignment team in London.
• August 2023: I successfully defended my doctoral thesis titled “Algorithmic Foundations for Safe and Efficient Reinforcement Learning from Human Feedback”. Many thanks to my advisors Andreas Krause and Katja Hofmann, and to all my brilliant collaborators!
• July 2023: New preprint on arXiv: “Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback”
• July 2023: We presented RLHF-Blender at the ICML workshop on Interactive Learning with Implicit Human Feedback.
• May 2023: New preprint on arXiv: “Learning Safety Constraints from Demonstrations with Unknown Rewards”
• January 2023: New preprint on arXiv: “Tracr: Compiled Transformers as a Laboratory for Interpretability”
• December 2022: Our paper “Red-Teaming the Stable Diffusion Safety Filter” won a best paper award at the NeurIPS ML Safety workshop 2022.