I'm a Director and Principal Research Scientist at Google DeepMind working on artificial intelligence. Currently, I contribute to Gemini's search, agentic and reasoning capabilities. I also help with Gemini research strategy, working primarily with the tech leads and in particular Oriol Vinyals.
Prior to working on Gemini, I led the DL:X team at DeepMind (mostly generative models, self-supervised learning, multi-modal large language models). I also led the Quantum Chemistry and Materials team in Science (mostly DFT and a few downstream applications).
See our ICML tutorial for a technical overview of some of that pre-Gemini work. See articles and videos on Quanta, Ars Technica, Stephen Colbert, TEDx and TANK for a non-technical overview of that work. See here and here for a flavour of the team's on-going quantum physics and chemistry research.
Prior to DeepMind, I was a post-doctoral researcher at Microsoft Research Cambridge working with John Winn. I did my PhD at the University of Edinburgh, where I was a Carnegie scholar working with Christopher Williams. During my PhD I was also a visiting researcher at Oxford University working with Andrew Zisserman.
Highlights
-
LLMs -
Gemini 1.5: Unlocking Multimodal Understanding Across Millions of Tokens of Context May 2024 -
Multimodal Few-Shot Learning with Frozen Language Models Neural Information Processing Systems (NeurIPS), 2021. -
Computer vision -
Representation Learning Without Labels International Conference on Machine Learning (ICML) tutorial, 2020. -
Self-supervised Video Pretraining Yields Human-Aligned Visual Representations Neural Information Processing Systems (NeurIPS), New Orleans, USA, 2023. -
Data-Efficient Image Recognition with Contrastive Predictive Coding International Conference on Machine Learning (ICML), Vienna, Austria, 2020. -
3D Understanding -
Neural Scene Representation and Rendering Science, 2018.
Extensions of the Generative Query Network: -
Few-shot learning -
Neural Processes and Family International Conference on Machine Learning (ICML), Stockholm, Sweden, 2018.
Overview of the Neural Process family:- Survey (Jha et al.)
-
Creativity -
Unsupervised Doodling and Painting with Improved SPIRAL
-
Robotics -
From Motor Control to Team Play in Simulated Humanoid Football Science Robotics, 2022.
Deployed on real robots (2022):
Precursor work (2017):
All Publications
-
Collaboration Between Clinicians and Vision–Language Models in Radiology Report Generation doi.org/10.1038/s41591-024-03302-1 (Nature Medicine), NM. -
Gemini 1.5: Unlocking Multimodal Understanding Across Millions of Tokens of Context -
Advancing Multimodal Medical Capabilities of Gemini -
Capabilities of Gemini Models in Medicine -
Consensus, Dissensus and Synergy Between Clinicians and Specialist Foundation models in Radiology Report Generation -
Self-supervised Video Pretraining Yields Human-Aligned Visual Representations Neural Information Processing Systems (NeurIPS), New Orleans, USA. -
From Data to Functa: Your Data Point is a Function and You Can Treat It Like One International Conference on Machine Learning (ICML), Baltimore, USA. -
Towards Real Robot Learning in the Wild: A Case Study in Bipedal Locomotion Conference on Robot Learning (CoRL), Auckland, New Zealand. -
Inferring a Continuous Distribution of Atom Coordinates from Cryo-EM Images Using VAEs Neural Information Processing Systems (NeurIPS), Workshop on Machine Learning in Structural Biology. -
Multimodal Few-Shot Learning with Frozen Language Models Neural Information Processing Systems (NeurIPS), Online. -
From Motor Control to Team Play in Simulated Humanoid Football doi:10.1126/scirobotics.abo0235 (Science Robotics), SR. -
Generative Art Using Neural Visual Grammars and Dual Encoders -
Game Plan: What AI can do for Football, and What Football can do for AI Journal of Artificial Intelligence (JAIR), ACM. -
Contrastive Training for Improved Out-of-Distribution Detection -
PolyGen: An Autoregressive Generative Model of 3D Meshes International Conference on Machine Learning (ICML), Vienna, Austria. -
Unsupervised Doodling and Painting with Improved SPIRAL -
A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguities -
Data-Efficient Image Recognition with Contrastive Predictive Coding International Conference on Machine Learning (ICML), Vienna, Austria. -
Meta-Learning surrogate models for sequential decision making -
Attentive Neural Processes Neural Information Processing Systems (NeurIPS), Workshop on Bayesian Deep Learning, Montréal, Canada. -
Neural Processes International Conference on Machine Learning (ICML), Workshop on Theoretical Foundations and Applications of Deep Generative Models, Stockholm, Sweden. -
Conditional Neural Processes International Conference on Machine Learning (ICML), Stockholm, Sweden. -
Encoding Spatial Relations from Natural Language -
Learning models for visual 3D localization with implicit mapping Neural Information Processing Systems (NeurIPS), Workshop on Bayesian Deep Learning, Montréal, Canada. -
Consistent Jumpy Predictions for Videos and Scenes Neural Information Processing Systems (NeurIPS), Workshop on Bayesian Deep Learning, Montréal, Canada. -
A Probabilistic U-Net for Segmentation of Ambiguous Images Neural Information Processing Systems (NeurIPS), Montréal, Canada. -
Neural Scene Representation and Rendering Science, doi:10.1126/science.aar6170 -
Generative Temporal Models with Spatial Memory for Partially Observed Environments International Conference on Machine Learning (ICML), Stockholm, Sweden. -
Synthesizing Programs for Images using Reinforced Adversarial Learning International Conference on Machine Learning (ICML), Stockholm, Sweden. -
Machine Theory of Mind International Conference on Machine Learning (ICML), Stockholm, Sweden. -
Kickstarting Deep Reinforcement Learning -
Learning and Querying Fast Generative Models for Reinforcement Learning -
The Multi-Entity Variational Autoencoder Neural Information Processing Systems (NIPS), Workshop on Learning Disentangled Features, Long Beach, California, USA. -
Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions International Conference on Learning Representations (ICLR), Vancouver, Canada. -
Emergence of Locomotion Behaviours in Rich Environments -
Unsupervised Learning of 3D Structure from Images Neural Information Processing Systems (NIPS), Barcelona, Spain. -
Attend, Infer, Repeat: Fast Scene Understanding with Generative Models Neural Information Processing Systems (NIPS), Barcelona, Spain. -
Reinforced Variational Inference Neural Information Processing Systems (NIPS), Workshop on Advances in Approximate Bayesian Inference, Montréal, Canada. Oral presentation. -
Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages Uncertainty in Artificial Intelligence (UAI), Amsterdam, The Netherlands. -
Just-In-Time Kernel Regression for Expectation Propagation International Conference on Machine Learning (ICML), Workshop on Large Scale Kernel Learning, Lille, France. -
Consensus Message Passing for Layered Graphical Models International Conference on Artificial Intelligence and Statistics (AISTATS), San Diego, California, USA. -
Just-In-Time Learning for Fast and Flexible Inference Neural Information Processing Systems (NIPS), Montreal, Quebec, Canada. -
The PASCAL Visual Object Classes Challenge — a Retrospective International Journal of Computer Vision (IJCV), Springer. -
Assessing the Significance of Performance Differences on the PASCAL VOC Challenges via Bootstrapping Technical note. -
The Shape Boltzmann Machine: a Strong Model of Object Shape International Journal of Computer Vision (IJCV), Springer. CVPR special issue. -
A Generative Model for Parts-based Object Segmentation Neural Information Processing Systems (NIPS), Lake Tahoe, California, USA. -
The Shape Boltzmann Machine: a Strong Model of Object Shape Computer Vision and Pattern Recognition (CVPR), Rhode Island, USA. Oral presentation. -
Factored Shapes and Appearances for Parts-based Object Understanding British Machine Vision Conference (BMVC), Dundee, UK. Oral presentation.
Theses
-
Generative Probabilistic Models for Object Segmentation The University of Edinburgh, PhD thesis. -
Evolving Robust Control Strategies for Simulated Animats The University of Edinburgh, BEng dissertation.
Selected talks
-
AI in the Era of Large Language Models - TANK magazine (Apr 2023)
-
Neural Priors, Neural Encoders, Neural Decoders - Royal Society Workshop on New Approaches to 3D Vision (Nov 2021)
-
Representation Learning Without Labels - ICML 2020 tutorial (July 2020)
-
Making Machines Creative - TEDx Warwick (February 2020)
-
Neural Scene Representation and Rendering - Eurographics Symposium on Rendering, Strasbourg, France (July 2019)
- CVPR Workshop on Generative Models for 3D Understanding, Long Beach (June 2019)
- Sharif University of Technology, Tehran, Iran (December 2018)
- ICVGIP, Hyderabad, India (December 2018)
- The post binary conference, Frankfurt, Germany (November 2018)
- Re:work conference, London, UK (October 2018)
- Research@Google, Mountain View, California, USA (August 2018)
-
Attend, Infer, Repeat: Fast Scene Understanding with Generative Models - Beyond Supervised Learning Workshop, ICCV, Venice, Italy (October 2017)
- Sharif University of Technology, Tehran, Iran (December 2016)
- University of Tehran, Tehran, Iran (December 2016)
- Data Learning and Inference (DALI) Meeting, RNN Workshop, Sestri Levante, Italy (March 2016)
- Robotics Research Seminar, Oxford Universty, Oxford, UK (November 2016)
-
Modern Artificial Intelligence - Invited Lecture, Digikala, Tehran, Iran (December 2016)
- Machine Learning Tutorial Series, Imperial College London, London, UK (October 2016)
- Sharif University of Technology, Tehran, Iran (December 2015)
- Shahid Beheshti University, Tehran, Iran (December 2015)
- Tehran University, Tehran, Iran (December 2015)
- Campus London, London, UK (August 2015)
- Robotic's Society Lecture Series, King's College London, London, UK (March 2015)
- COMPM041 Guest Lecture, University College London, London, UK (March 2015)
-
Artificial Intelligence and Computer Aided Design - Alan Turing Institute, London, UK (November 2019)
- ETH Zurich, Zurich, Switzerland (July 2019)
- Foster + Partners Architects, London, UK (July 2019)
- Cartwright Picard Architects, London, UK (January 2019)
- Grimshaw Architects, London, UK (January 2015)
- Colloquium Seminar Series, University College London, London, UK (January 2015)
- Sharif University of Technology, Kish, Iran (December 2014)
- Faculty of Fine Arts, University of Tehran, Tehran, Iran (December 2014)
-
Extensions to Message Passing Inference - Sharif University of Technology, Kish, Iran (December 2014)
- Microsoft Research, Cambridge, UK (September 2014)
-
Boltzmann Machines and their Extensions - Heriot-Watt University, Edinburgh, UK (March 2013)
-
Generative Models of Images of Objects - Sharif University of Technology, Tehran, Iran (January 2013)
- Redwood Institute at the University of California at Berkeley, USA (December 2012)
- Bosch Research and Technology North America, Palo Alto, USA (December 2012)
- Toyota Technological Institute, University of Chicago, Chicago, USA (June 2012)
- New York University, New York, USA (June 2012)
-
The Shape Boltzmann Machine: a Strong Model of Object Shape - Computer Vision and Pattern Recognition (CVPR), Providence, USA (June 2012)
- Rank Prize Symposium on Machine Learning and Vision, Windermere, UK (March 2012)
-
Factored Shapes and Appearances for Parts-based Object Understanding - British Machine Vision Conference, Dundee, UK (August 2011)
- ACM Seminar, University of Tehran, Tehran, Iran (September 2011)
-
Hierarchical Probabilistic Models for Object Segmentation - Canadian Institute for Advanced Research Summer School, Toronto, Canada (August 2010)
-
High Dynamic Range Lighting on the Xbox 360 - TechMeetup, Edinburgh, UK (April 2010)
Other things
-
6 Things Everyone Should Know About Visualizing Research Results -
15 Jan 2015
- Guide
-
The Zeroth Rule of Presentations -
14 Oct 2012
- Guide
-
Patterns for Research in Machine Learning -
18 Jul 2012
- Guide
-
Thoughts on the Role of Supervision in Computer Vision -
18 May 2011
- Slides
Service
I co-organised the NeurIPS 2020 workshop on Muslims in Machine Learning, that was held virtually on December 8, 2020.
I co-organised the NIPS 2017 workshop on Machine Learning for Creativity and Design, that was held in Long Beach, California on December 8, 2017.
I co-organised the NIPS 2015 workshop on Black Box Learning and Inference, that was held in Montréal, Canada on December 12, 2015.
I regularly AC and review for NeurIPS, ICML, ICLR.
