Resources

These are some of the very recent and fairly advanced works in some of the leading research areas in theoretical deep learning.  For some of the more introductory and application oriented resources check out our Computer Vision summer school

Resources on Theoretical Deep Learning

Deep Learning Theory, Talks, Tutorials, Blogs and Courses

•  Summer School on Deeplearning July 2016
•  Short Cource on Deeplearning December 2016
• Generalization and Equilibrium in Generative Adversarial Nets (GANs)
• Variational Inference: Foundations and Modern Methods by David Blei, Rajesh Ranganath, Shakir Mohamed, NIPS 2016 Tutorial December 5, 2016.
• Bayesian Reasoning and Deep Learning in Agent-based Systems by Shakir Mohamed, NIPS 2016 .
• Tutorial on Variational Autoencoders by CARL DOERSCH, Carnegie Mellon, UC Berkeley, August 16, 2016.
• UCL Course on RL by David Silver.

Deep Learning Theoretical Papers

Theory of neural networks

• Provable Bounds for Learning Some Deep Representations by Sanjeev Arora, Aditya Bhaskara, Rong Ge, Tengyu Ma, ICML 2014

Generative models

• Improved Variational Inference with Inverse Autoregressive Flow Diederik P. Kingma, Tim SalimansTim Salimans, Rafal Jozefowicz, Xi Chen, Max Welling, NIPS 2016.
• Improved Techniques for Training GANs Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, arxiv 2016.

Bayesian deep learning

• Semi-supervised deep kernel learning by Neal Jean, Michael Xie, Stefano Ermon, NIPS 2016.
• Using Bayesian Deep Learningfor Transfer Learning in Optimisation by Jonas Langhabel, JannikWolff, Raphael Holca-Lamarre, NIPS Workshop 2016.
• Neural Variational InferenceFor Topic Models by Akash Srivastava, Charles Sutton, NIPS Workshop 2016.

Reinforcement Learning

• Variational Information MaximizingExploration Rein Houthooft, Xi Chen, Yan Duan John Schulman , Filip De Turck Pieter Abbeel, arXiv 2017.
• Reinforcement learning of motor skills with policy gradients Jan Peters, Stefan Schaal, Neural Networks 2008.
• Continuous Control With Deep Reinforcement Learning Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, ICLR 2016.

Optimisation

• Differentiable Optimization as a Layer in Neural Networks Brandon Amos, J. Zico Kolter, arXiv 2017.
• Optimisation As A Model Forfew-Short Learning Sachin Ravi and Hugo Larochelle, ICLR 2017.
• Unrolled Generative Adversarial Networks Luke Metz, Ben Poole, David Pfau, Jascha Sohl-Dickstein, ICLR 2017.
• Input Convex Neural Networks by Brandon AmosLei Xu, J. Zico Kolter, arXiv 2016.