I am an assistant professor in Natural Language Processing at the University of Edinburgh.
If you are a prospective PhD student or postdoc please visit the FAQ page for more information.
I am currently leading two main projects:
- Adaptive Tokenization and Memory in Foundation Models for Efficient and Long-Horizon AI (AToM ⚛︎) funded by an ERC Starting Grant. This project aims at designing the next generation of foundation model architectures to reduce energy demand and carbon emissions, while enabling permanent memories, inference time hyper-scaling, and long-horizon world modelling.
- Tracking Evolving AI and Systems (TEAS ☕︎), which aims at benchmarking cost–accuracy–performance trade-offs in existing hardware and devising analytical tools to extrapolate performance to new hardware.
My research focuses on:
-
adaptive memory and tokenization in foundation models (see my NeurIPS 2024 tutorial on dynamic sparsity): I aim to redefine the units of computation of foundation models by adaptively compressing the sequences of their hidden representations and memory. This allows models to tokenize raw, modality-agnostic data end-to-end, learning hierarchical abstractions. Simultaneously, it provides the foundations for permanent model memories and inference-time hyper-scaling.
-
modular deep learning: I am interested in designing neural architectures that route information to specialised modules (e.g., sparse subnetworks). This facilitates systematic generalisation and conditional computation.
-
computational typology: I wish to understand how languages vary, across the world and its cultures, within a computational framework. Multimodal models in particular give us an powerful tool to study how form depends on grounded, embodied representations of meaning and function.
I spent the last year as a visiting professor at NVIDIA. Previously, I was a visiting postdoctoral scholar at Stanford University and a postdoctoral fellow in computer science at Mila - Quebec AI Institute in Montreal. In 2021, I obtained a PhD from the University of Cambridge, St John’s College. Once upon a time I studied modern literature at the University of Pavia. Deep in my heart, I am still a humanist: some of my favourite writers are Italo Calvino, Ursula Le Guin, and Titus Lucretius Carus.
My research has been featured on the Economist and Scientific American, among others. it is currently supported by ERC, ARIA, and various gifts/compute from Google DeepMind, NVIDIA, and NatWest. I also received a Google Research Faculty Award, and Best Paper / SAC Highlight Awards at ACL, EMNLP, and RepL4NLP. I am a board member of SIGTYP, the ACL special interest group for computational typology, a Scholar of the European Lab for Learning and Intelligent Systems (ELLIS), and part of the TACL journal editorial team.