profile_picture
Karsten Kreis
Senior Research Scientist, NVIDIA
kkreis [at] nvidia [dot] com

I am a Senior Research Scientist at NVIDIA’s Toronto AI Lab. My primary research interests revolve around deep generative learning. I am interested both in fundamental algorithm development and in applying generative models on relevant problems in areas such as representation learning, computer vision, graphics and digital artistry. I am also broadly interested in research that takes inspirations from physics to improve machine learning techniques as well as in applying state-of-the-art deep learning methods to problems in the natural sciences.

I am trained as a physicist and completed my master’s in quantum information theory. For my Ph.D. in computational and statistical physics, I developed multiscale models and sampling algorithms for molecular dynamics simulations of complex chemical and biological systems. After I finished my Ph.D., I switched to deep learning. Before joining NVIDIA, I worked on deep generative modeling at D-Wave Systems, a quantum computation company, and I co-founded Variational AI, a startup focusing on generative modeling for drug discovery.

Publications

*,† indicate equal contribution / co-first authorship.
Magic3D: High-Resolution Text-to-3D Content Creation
arXiv preprint, 2022
Chen-Hsuan Lin* , Jun Gao* , Luming Tang* , Towaki Takikawa* , Xiaohui Zeng* , Xun Huang , Karsten Kreis , Sanja Fidler , Ming-Yu Liu , Tsung-Yi Lin

eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers
arXiv preprint, 2022
Yogesh Balaji , Seungjun Nah , Xun Huang , Arash Vahdat , Jiaming Song , Karsten Kreis , Miika Aittala , Timo Aila , Samuli Laine , Bryan Catanzaro , Tero Karras , Ming-Yu Liu

Differentially Private Diffusion Models
arXiv preprint, 2022
Tim Dockhorn , Tianshi Cao , Arash Vahdat , Karsten Kreis

Latent Space Diffusion Models of Cryo-EM Structures
Machine Learning for Structural Biology Workshop, NeurIPS, 2022 (Oral Presentation)
Karsten Kreis* , Tim Dockhorn* , Zihao Li , Ellen Zhong

GENIE: Higher-Order Denoising Diffusion Solvers
Neural Information Processing Systems (NeurIPS), 2022
Tim Dockhorn , Arash Vahdat , Karsten Kreis

LION: Latent Point Diffusion Models for 3D Shape Generation
Neural Information Processing Systems (NeurIPS), 2022
Xiaohui Zeng , Arash Vahdat , Francis Williams , Zan Gojcic , Or Litany , Sanja Fidler , Karsten Kreis

Polymorphic-GAN: Generating Aligned Samples across Multiple Domains with Learned Morph Maps
Computer Vision and Pattern Recognition (CVPR), 2022 (Oral Presentation)
Seung Wook Kim , Karsten Kreis , Daiqing Li , Antonio Torralba , Sanja Fidler

BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations
Computer Vision and Pattern Recognition (CVPR), 2022
Daiqing Li , Huan Ling , Seung Wook Kim , Karsten Kreis , Adela Barriuso , Sanja Fidler , Antonio Torralba

Score-Based Generative Modeling with Critically-Damped Langevin Diffusion
International Conference on Learning Representations (ICLR), 2022 (Spotlight Presentation)
Tim Dockhorn , Arash Vahdat , Karsten Kreis

Tackling the Generative Learning Trilemma with Denoising Diffusion GANs
International Conference on Learning Representations (ICLR), 2022 (Spotlight Presentation)
Zhisheng Xiao , Karsten Kreis , Arash Vahdat

Score-based Generative Modeling in Latent Space
Neural Information Processing Systems (NeurIPS), 2021
Arash Vahdat* , Karsten Kreis* , Jan Kautz

Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence
Neural Information Processing Systems (NeurIPS), 2021
Tianshi Cao , Alex Bie , Arash Vahdat , Sanja Fidler , Karsten Kreis

EditGAN: High-Precision Semantic Image Editing
Neural Information Processing Systems (NeurIPS), 2021
Huan Ling* , Karsten Kreis* , Daiqing Li , Seung Wook Kim , Antonio Torralba , Sanja Fidler

ATISS: Autoregressive Transformers for Indoor Scene Synthesis
Neural Information Processing Systems (NeurIPS), 2021
Despoina Paschalidou , Amlan Kar , Maria Shugrina , Karsten Kreis , Andreas Geiger , Sanja Fidler

Causal Scene BERT: Improving object detection by searching for challenging groups of data
International Conference on Computer Vision (ICCV), 2nd AVVision Workshop, 2021
Cinjon Resnick , Or Litany , Amlan Kar , Karsten Kreis , James Lucas , Kyunghyun Cho , Sanja Fidler

Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization
Computer Vision and Pattern Recognition (CVPR), 2021
Daiqing Li , Junlin Yang , Karsten Kreis , Antonio Torralba , Sanja Fidler

Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes
Computer Vision and Pattern Recognition (CVPR), 2021 (Oral Presentation)
Towaki Takikawa* , Joey Litalien* , Kangxue Yin , Karsten Kreis , Charles Loop , Derek Nowrouzezahrai , Alec Jacobson , Morgan McGuire , Sanja Fidler

VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models
International Conference on Learning Representations (ICLR), 2021 (Spotlight Presentation)
Zhisheng Xiao , Karsten Kreis , Jan Kautz , Arash Vahdat

Variational Amodal Object Completion
Neural Information Processing Systems (NeurIPS), 2020
Huan Ling , David Acuna , Karsten Kreis , Seung Wook Kim , Sanja Fidler

ESPResSo++ 2.0: Advanced methods for multiscale molecular simulation
Computer Physics Communications (CPC), 2019
Horacio V. Guzman , Nikita Tretyakov , Hideki Kobayashi , Aoife C. Fogarty , Karsten Kreis , Jakub Krajniak , Christoph Junghans , Kurt Kremer , Torsten Stuehn

From classical to quantum and back: Hamiltonian adaptive resolution path integral, ring polymer, and centroid molecular dynamics
Journal of Chemical Physics (JCP), 2017
Karsten Kreis , Kurt Kremer , Raffaello Potestio , Mark E. Tuckerman

The relative entropy is fundamental to adaptive resolution simulations
Journal of Chemical Physics (JCP), 2016
Karsten Kreis , Raffaello Potestio

Adaptive Resolution Simulations with Self-Adjusting High-Resolution Regions
Journal of Chemical Theory and Computation (JCTC), 2016 (Featured on the Journal Cover)
Karsten Kreis , Raffaello Potestio , Kurt Kremer , Aoife C. Fogarty

From Classical to Quantum and Back: A Hamiltonian Scheme for Adaptive Multiresolution Classical/Path-Integral Simulations
Journal of Chemical Theory and Computation (JCTC), 2016
Karsten Kreis , Mark E. Tuckerman , Davide Donadio , Kurt Kremer , Raffaello Potestio

Advantages and challenges in coupling an ideal gas to atomistic models in adaptive resolution simulations
The European Physical Journal Special Topics (EPJ ST), 2015
Karsten Kreis , Aoife C. Fogarty , Kurt Kremer , Raffaello Potestio

A unified framework for force-based and energy-based adaptive resolution simulations
Europhysics Letters (EPL), 2014
Karsten Kreis , Davide Donadio , Kurt Kremer , Raffaello Potestio

Classifying, quantifying, and witnessing qudit-qumode hybrid entanglement
Physical Review A (PRA), 2012
Karsten Kreis , Peter van Loock


Theses

Advanced Adaptive Resolution Methods for Molecular Simulation
Ph.D. Thesis, 2018
Carried out at the Max Planck Institute for Polymer Research and at New York University
Characterizing And Exploiting Hybrid Entanglement
Master Thesis ("Diplomarbeit"), 2011
Carried out at the Max Planck Institute for the Science of Light

Prizes, Awards and Scholarships

  • Highlighted Reviewer at the International Conference on Learning Representations (ICLR), 2022 --- Oustanding Reviewer (Top 10%) at the International Conference on Machine Learning (ICML), 2022
  • Prize of the Department of Physics, Mathematics and Computer Science of the Johannes Gutenberg University Mainz for an outstanding dissertation, 2019
  • Ph.D. thesis awarded highest grade "summa cum laude", 2018
  • Moore/Sloan & Washington Research Foundation Innovation in Data Science Postdoctoral Fellowship at the University of Washington, Seattle (declined), 2017
  • Ph.D. Scholarship of the "Graduate School of Excellence Materials Science in Mainz", 2013-2016
  • Poster prize for the best poster at the “Meet-Your-Collegue-Day 2013” of the Max Planck Institute for Polymer Research, 2013
  • "Ohm Prize" of the Department for Physics of the Friedrich-Alexander University Erlangen-Nuremberg for an outstanding diploma thesis, 2011
  • "Erasmus Grant" to study abroad at Imperial College London, 2007-2008
  • "Book Prize" of the German Physical Society for excellent achievements in the subject of physics in high school, 2005

  • Miscellaneous

    Tutorial on Denoising Diffusion-based Generative Modeling

    I have presented the tutorial “Denoising Diffusion-based Generative Modeling: Foundations and Applications” at the Conference on Computer Vision and Pattern Recognition (CVPR) 2022, together with Ruiqi Gao and Arash Vahdat.

    Blog Posts about Three Recent Works on Score-Based Generative Models

    We have written two blog posts (part 1 and part 2) about our three recent works on Latent Score-based Generative Models, Denoising Diffusion GANs and Critically-Damped Langevin Diffusion.

    Invited Talks

  • Jan 2023: “Latent Space Diffusion Models of Cryo-EM Structures” at Variational AI.
  • Dec 2022: “Advanced Diffusion Models: Accelerated Sampling, Smooth Diffusion, and 3D Shape Generation” at the Computer Vision Group at the University of Bern.
  • Dec 2022: “Accelerated Sampling and Improved Synthesis in Diffusion Models” at NeurIPS 2022 Workshop on Score-Based Methods.
  • Feb 2022: “Score-Based Generative Modeling with Critically-Damped Langevin Diffusion” at Deep Learning: Classics and Trends by ML Collective.
  • Sep 2015: “Adaptive Resolution: From Atomistic and Coarse-Grained Hybrid Simulations to Quantum-Classical Coupling” at D. E. Shaw Research.
  • Aug 2015: “Adaptive Resolution: From Atomistic and Coarse-Grained Hybrid Simulations to Quantum-Classical Coupling” at the Noid Lab at Pennsylvania State University.
  • Supervision and Mentoring

    I have had the opportunity to supervise and mentor several talented interns at NVIDIA:
  • Katja Schwarz
  • Andreas Blattmann
  • Robin Rombach
  • Tim Dockhorn
  • Xiaohui Zeng
  • Tianshi Cao
  • Zhisheng Xiao (as co-mentor)
  • Reviewing

    I have been serving as a reviewer for the following conferences:
  • NeurIPS 2021, 2022
  • ICLR 2022, 2023
  • ICML 2022, 2023
  • CVPR 2022
  • SIGGRAPH 2022
  • Software Development

    I used to be a software developer (between 2013-2019) of ESPResSo++, which stands for “Extensible Simulation Package for Research on Soft Matter”. I developed and implemented novel molecular dynamics algorithms and multiscale models for efficient and accurate computer simulations of relevant chemical and biological systems. I also performed code reviews, wrote tests, prepared documentation and tutorials, and taught new users in seminars and courses. Coding languages were C++ and Python with MPI-based parallelization.