profile_picture
Karsten Kreis
Senior Research Scientist, NVIDIA
kkreis [at] nvidia [dot] com

I am a Senior Research Scientist at NVIDIA’s Toronto AI Lab. My primary research interests revolve around deep generative learning. I am interested both in fundamental algorithm development and in applying generative models on relevant problems in areas such as representation learning, computer vision, graphics and digital artistry. I am also broadly interested in research that takes inspirations from physics to improve machine learning techniques as well as in applying state-of-the-art deep learning methods to problems in the natural sciences.

I am trained as a physicist and completed my master’s in quantum information theory. For my Ph.D. in computational and statistical physics, I developed multiscale models and sampling algorithms for molecular dynamics simulations of complex chemical and biological systems. After I finished my Ph.D., I switched to deep learning. Before joining NVIDIA, I worked on deep generative modeling at D-Wave Systems, a quantum computation company, and I co-founded Variational AI, a startup focusing on generative modeling for drug discovery.

Publications

* denotes equal contribution / co-first authorship.
Polymorphic-GAN: Generating Aligned Samples across Multiple Domains with Learned Morph Maps
Computer Vision and Pattern Recognition (CVPR), 2022 (Oral Presentation)
Seung Wook Kim , Karsten Kreis , Daiqing Li , Antonio Torralba , Sanja Fidler

BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations
Computer Vision and Pattern Recognition (CVPR), 2022
Daiqing Li , Huan Ling , Seung Wook Kim , Karsten Kreis , Adela Barriuso , Sanja Fidler , Antonio Torralba

Score-Based Generative Modeling with Critically-Damped Langevin Diffusion
International Conference on Learning Representations (ICLR), 2022 (Spotlight Presentation)
Tim Dockhorn , Arash Vahdat , Karsten Kreis

Tackling the Generative Learning Trilemma with Denoising Diffusion GANs
International Conference on Learning Representations (ICLR), 2022 (Spotlight Presentation)
Zhisheng Xiao , Karsten Kreis , Arash Vahdat

Score-based Generative Modeling in Latent Space
Neural Information Processing Systems (NeurIPS), 2021
Arash Vahdat* , Karsten Kreis* , Jan Kautz

Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence
Neural Information Processing Systems (NeurIPS), 2021
Tianshi Cao , Alex Bie , Arash Vahdat , Sanja Fidler , Karsten Kreis

EditGAN: High-Precision Semantic Image Editing
Neural Information Processing Systems (NeurIPS), 2021
Huan Ling* , Karsten Kreis* , Daiqing Li , Seung Wook Kim , Antonio Torralba , Sanja Fidler

ATISS: Autoregressive Transformers for Indoor Scene Synthesis
Neural Information Processing Systems (NeurIPS), 2021
Despoina Paschalidou , Amlan Kar , Maria Shugrina , Karsten Kreis , Andreas Geiger , Sanja Fidler

Causal Scene BERT: Improving object detection by searching for challenging groups of data
International Conference on Computer Vision (ICCV), 2nd AVVision Workshop, 2021
Cinjon Resnick , Or Litany , Amlan Kar , Karsten Kreis , James Lucas , Kyunghyun Cho , Sanja Fidler

Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization
Computer Vision and Pattern Recognition (CVPR), 2021
Daiqing Li , Junlin Yang , Karsten Kreis , Antonio Torralba , Sanja Fidler

Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes
Computer Vision and Pattern Recognition (CVPR), 2021 (Oral Presentation)
Towaki Takikawa* , Joey Litalien* , Kangxue Yin , Karsten Kreis , Charles Loop , Derek Nowrouzezahrai , Alec Jacobson , Morgan McGuire , Sanja Fidler

VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models
International Conference on Learning Representations (ICLR), 2021 (Spotlight Presentation)
Zhisheng Xiao , Karsten Kreis , Jan Kautz , Arash Vahdat

Variational Amodal Object Completion
Neural Information Processing Systems (NeurIPS), 2020
Huan Ling , David Acuna , Karsten Kreis , Seung Wook Kim , Sanja Fidler

ESPResSo++ 2.0: Advanced methods for multiscale molecular simulation
Computer Physics Communications (CPC), 2019
Horacio V. Guzman , Nikita Tretyakov , Hideki Kobayashi , Aoife C. Fogarty , Karsten Kreis , Jakub Krajniak , Christoph Junghans , Kurt Kremer , Torsten Stuehn

From classical to quantum and back: Hamiltonian adaptive resolution path integral, ring polymer, and centroid molecular dynamics
Journal of Chemical Physics (JCP), 2017
Karsten Kreis , Kurt Kremer , Raffaello Potestio , Mark E. Tuckerman

The relative entropy is fundamental to adaptive resolution simulations
Journal of Chemical Physics (JCP), 2016
Karsten Kreis , Raffaello Potestio

Adaptive Resolution Simulations with Self-Adjusting High-Resolution Regions
Journal of Chemical Theory and Computation (JCTC), 2016 (Featured on the Journal Cover)
Karsten Kreis , Raffaello Potestio , Kurt Kremer , Aoife C. Fogarty

From Classical to Quantum and Back: A Hamiltonian Scheme for Adaptive Multiresolution Classical/Path-Integral Simulations
Journal of Chemical Theory and Computation (JCTC), 2016
Karsten Kreis , Mark E. Tuckerman , Davide Donadio , Kurt Kremer , Raffaello Potestio

Advantages and challenges in coupling an ideal gas to atomistic models in adaptive resolution simulations
The European Physical Journal Special Topics (EPJ ST), 2015
Karsten Kreis , Aoife C. Fogarty , Kurt Kremer , Raffaello Potestio

A unified framework for force-based and energy-based adaptive resolution simulations
Europhysics Letters (EPL), 2014
Karsten Kreis , Davide Donadio , Kurt Kremer , Raffaello Potestio

Classifying, quantifying, and witnessing qudit-qumode hybrid entanglement
Physical Review A (PRA), 2012
Karsten Kreis , Peter van Loock


Theses

Advanced Adaptive Resolution Methods for Molecular Simulation
Ph.D. Thesis, 2018
Carried out at the Max Planck Institute for Polymer Research and at New York University
Characterizing And Exploiting Hybrid Entanglement
Master Thesis ("Diplomarbeit"), 2011
Carried out at the Max Planck Institute for the Science of Light

Prizes, Awards and Scholarships

  • Highlighted Reviewer at the International Conference on Learning Representations (ICLR), 2022
  • Prize of the Department of Physics, Mathematics and Computer Science of the Johannes Gutenberg University Mainz for an outstanding dissertation, 2019
  • Ph.D. thesis awarded highest grade "summa cum laude", 2018
  • Moore/Sloan & Washington Research Foundation Innovation in Data Science Postdoctoral Fellowship at the University of Washington, Seattle (declined), 2017
  • Ph.D. Scholarship of the "Graduate School of Excellence Materials Science in Mainz", 2013-2016
  • Poster prize for the best poster at the “Meet-Your-Collegue-Day 2013” of the Max Planck Institute for Polymer Research, 2013
  • "Ohm Prize" of the Department for Physics of the Friedrich-Alexander University Erlangen-Nuremberg for an outstanding diploma thesis, 2011
  • "Erasmus Grant" to study abroad at Imperial College London, 2007-2008
  • "Book Prize" of the German Physical Society for excellent achievements in the subject of physics in high school, 2005

  • Miscellaneous

    Tutorial on Denoising Diffusion-based Generative Modeling

    I will be presenting the tutorial “Denoising Diffusion-based Generative Modeling: Foundations and Applications” at the Conference on Computer Vision and Pattern Recognition (CVPR) 2022, together with Ruiqi Gao and Arash Vahdat.

    Blog Posts about Three Recent Works on Score-Based Generative Models

    We have written two blog posts (part 1 and part 2) about our three recent works on Latent Score-based Generative Models, Denoising Diffusion GANs and Critically-Damped Langevin Diffusion.

    Invited Talks

  • Feb 2022: “Score-Based Generative Modeling with Critically-Damped Langevin Diffusion” at Deep Learning: Classics and Trends by ML Collective.
  • Sep 2015: “Adaptive Resolution: From Atomistic and Coarse-Grained Hybrid Simulations to Quantum-Classical Coupling” at D. E. Shaw Research.
  • Aug 2015: “Adaptive Resolution: From Atomistic and Coarse-Grained Hybrid Simulations to Quantum-Classical Coupling” at the Noid Lab at Pennsylvania State University.
  • Supervision and Mentoring

    I have had the opportunity to supervise and mentor several talented interns at NVIDIA:
  • Robin Rombach
  • Tim Dockhorn
  • Xiaohui Zeng
  • Tianshi Cao
  • Zhisheng Xiao (as co-mentor)
  • Reviewing

    I have been serving as a reviewer for the following conferences:
  • NeurIPS 2021, 2022
  • ICLR 2022
  • ICML 2022
  • CVPR 2022
  • SIGGRAPH 2022
  • Software Development

    I used to be a software developer (between 2013-2019) of ESPResSo++, which stands for “Extensible Simulation Package for Research on Soft Matter”. I developed and implemented novel molecular dynamics algorithms and multiscale models for efficient and accurate computer simulations of relevant chemical and biological systems. I also performed code reviews, wrote tests, prepared documentation and tutorials, and taught new users in seminars and courses. Coding languages were C++ and Python with MPI-based parallelization.