Jacob R. Gardner

jacobrg@seas.upenn.edu

I am an assistant professor in the Computer and Information Science department at the University of Pennsylvania. I do research in machine learning, with a focus scalable probabilistic machine learning methods and Bayesian machine learning. Recently, I've also been interested in how machine learning techniques can be applied to large-scale, high dimensional optimization problems in the natural sciences. In 2022, I received an NSF CAREER award funding work on these kinds of optimization problems.

Before I joined Penn, I was a research scientist at Uber AI Labs. Before this, I was a postdoctoral associate in Operations Research and Information Engineering at Cornell University. I received my Ph.D. in Computer Science from Cornell University, where I was advised by Kilian Weinberger.

Students

Current students:

Publications

See also my Google scholar page.

The Behavior and Convergence of Local Bayesian Optimization[Paper]
Kaiwen Wu, Kyurae Kim, Roman Garnett, Jacob R. Gardner
Neural Information Processing Systems (NeurIPS 2023, to appear).

Variational Gaussian Processes with Decoupled Conditionals[Paper]
Xinran Zhu, Kaiwen Wu, Natalie Maus, Jacob R. Gardner, David Bindel
Neural Information Processing Systems (NeurIPS 2023, to appear).

On the Convergence of Black-Box Variational Inference[Paper]
Kyurae Kim, Jisu Oh, Kaiwen Wu, Yian Ma, Jacob R. Gardner
Neural Information Processing Systems (NeurIPS 2023, to appear).

Practical and Matching Gradient Variance Bounds for Black-Box Variational Bayesian Inference.[Paper]
Kyurae Kim, Kaiwen Wu, Jisu Oh, Jacob R. Gardner
International Conference on Machine Learning (ICML 2023). Oral.

Adversarial Prompting for Black Box Foundation Models[Paper]
Natalie Maus, Patrick Chao, Eric Wong, Jacob Gardner
(Preprint).

Discovering Many Diverse Solutions with Bayesian Optimization[Paper]
Natalie Maus, Kaiwen Wu, David Eriksson, Jacob Gardner
Artificial Intelligence and Statistics (AISTATS 2023). Notable paper.

Local Latent Space Bayesian Optimization over Structured Inputs[Paper]
Natalie Maus, Haydn T Jones, Juston S Moore, Matt J Kusner, John Bradshaw, Jacob R Gardner
Neural Information Processing Systems (NeurIPS 2022).

Local Bayesian optimization via maximizing probability of descent[Paper]
Quan Nguyen, Kaiwen Wu, Jacob R Gardner, Roman Garnett
Neural Information Processing Systems (NeurIPS 2022). Oral.

Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradients[Paper]
Kyurae Kim, Jisu Oh, Jacob R Gardner, Adji Bousso Dieng, Hongseok Kim
Neural Information Processing Systems (NeurIPS 2022).

Preconditioning for Scalable Gaussian Process Hyperparameter Optimization[Paper]
Jonathan Wenger, Geoff Pleiss, Philipp Hennig, John P Cunningham, Jacob R Gardner
International Conference on Machine Learning (ICML 2022). Long talk.

Extracting or Guessing? Improving Faithfulness of Event Temporal Relation Extraction[Paper]
Haoyu Wang, Hongming Zhang, Yuqian Deng, Jacob R Gardner, Muhao Chen, Dan Roth
(Preprint.)

Scaling gaussian processes with derivative information using variational inference[Paper]
Misha Padidar, Xinran Zhu, Leo Huang, Jacob Gardner, David Bindel
Neural Information Processing Systems (NeurIPS 2021).

Determining subpopulation methylation profiles from bisulfite sequencing data of heterogeneous samples using DXM[Paper]
Jerry Fong, Jacob R Gardner, Jared M Andrews, Amanda F Cashen, Jacqueline E Payton, Kilian Q Weinberger, John R Edwards
Nucleic Acids Research (2021).

Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Trees[Paper]
Shali Jiang, Daniel R Jiang, Maximilian Balandat, Brian Karrer, Jacob R Gardner, Roman Garnett
Neural Information Processing Systems (NeurIPS 2020).

Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization[Paper]
Geoff Pleiss, Martin Jankowiak, David Eriksson, Anil Damle, Jacob R. Gardner
Neural Information Processing Systems (NeurIPS 2020).

Deep Sigma Point Processes[Paper]
Martin Jankowiak, Geoff Pleiss, Jacob R. Gardner
Conference on Uncertainty in Artifical Intelligence (UAI 2020).

Parametric Gaussian Process Regressors [Paper]
Martin Jankowiak, Geoff Pleiss, Jacob R. Gardner
International Conference on Machine Learning (ICML 2020).

Scalable Global Optimization via Local Bayesian Optimization [Paper]
David Eriksson, Michael Pearce, Jacob R. Gardner, Ryan Turner, Matthias Poloczek
Neurial Information Processing Systems (NeurIPS 2019) Spotlight.

Exact Gaussian Processes on a Million Data Points [Paper]
Ke Alexander Wang, Geoff Pleiss, Jacob R. Gardner, Stephen Tyree, Kilian Q. Weinberger, Andrew G. Wilson
Neurial Information Processing Systems (NeurIPS 2019)

Simple Blackbox Adversarial Attacks [Paper]
Chuan Guo, Jacob R. Gardner, Yurong You, Andrew G. Wilson, Kilian Q. Weinberger
International Conference on Machine Learning (ICML 2019).

GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration [Paper]
Jacob R. Gardner, Geoff Pleiss, Kilian Q. Weinberger, David Bindel, Andrew G. Wilson
Neurial Information Processing Systems (NeurIPS 2018). Spotlight.

Constant Time Predictive Distributions for Gaussian Processes [Paper]
Geoff Pleiss, Jacob R. Gardner, Kilian Q. Weinberger, Andrew G. Wilson
International Conference on Machine Learning (ICML 2018).

Product Kernel Interpolation for Scalable Gaussian Processes [Paper]
Jacob R. Gardner, Geoff Pleiss, Ruihan Wu, Kilian Q. Weinberger, Andrew G. Wilson
Artificial Intelligence and Statistics (AISTATS 2018)

Discovering and Exploiting Additive Structure for Bayesian Optimization [Paper]
Jacob R. Gardner, Chuan Guo, Kilian Q. Weinberger, Roman Garnett, Roger Grosse
Artificial Intelligence and Statistics (AISTATS 2017)

Bayesian Active Model Selection with an Application to Automated Audiometry [Paper]
Jacob R. Gardner, Gustavo Malkomes, Roman Garnett, Kilian Q. Weinberger, Dennis Barbour, John P. Cunningham
Neural Information Processing Systems (NeurIPS 2015)

Psychophysical Detection Testing with Bayesian Active Learning [Paper]
Jacob R. Gardner, Xinyu Song, Kilian Q. Weinberger, Dennis Barbour, John P. Cunningham
Uncertainty in Artifical Intelligence (UAI 2015)

Deep feature interpolation for image content changes [Paper]
Paul Upchurch*, Jacob R. Gardner*, Kavita Bala, Robert Pless, Noah Snavely, Kilian Q. Weinberger
Computer Vision and Pattern Recognition (CVPR 2016)
* authors contributed equally

Deep manifold traversal: Changing labels with convolutional features [Paper]
Jacob R. Gardner*, Paul Upchurch*, Matt J. Kusner, Yixuan Li, Kilian Q. Weinberger, Kavita Bala, John E. Hopcroft
* authors contributed equally

Differentially Private Bayesian Optimization [Paper]
Matthew J. Kusner, Jacob R. Gardner, Roman Garnett, Kilian Q. Weinberger
International Conference on Machine Learning (ICML 2015)

A Reduction of the Elastic Net to Support Vector Machines with an Application to GPU Computing. [Paper]
Quan Zhou, Wenlin Chen, Shiji Song, Jacob R. Gardner, Kilian Q. Weinberger, Yixin Chen
Association for the Advancement of Artificial Intelligence (AAAI 2015)

Bayesian Optimization with Inequality Constraints. [Paper]
Jacob R. Gardner, Matt J. Kusner, Zhixiang Xu, Kilian Q. Weinberger, John P. Cunningham
International Conference on Machine Learning (ICML 2014)

Software

Geoff and I founded the GPyTorch project, which aims to implement Gaussian processes in a modular package with strong GPU acceleration. It is deeply embedded in the PyTorch ecosystem, and makes designing complicated models like deep kernel learning both easy to implement and highly efficient. It departs significantly from existing Gaussian process libraries, in that it makes use of modern numerical linear algebra techniques like linear conjugate gradients to perform the fundamental operations required for inference, rather than standard Cholesky based approaches.