RESOURCES

These are some of the very recent and fairly advanced works in some of the leading research areas in theoretical deep learning. For some of the more introductory and application oriented resources check out our Computer Vision summer school

Resources on Theoretical Deep Learning

Deep Learning Theory, Talks, Tutorials, Blogs and Courses

•  Nuts and Bolts of applying Deep Learning by Andrew Ng at NIPS 2016
•  Distill - a modern medium for publishing ML research
•  Summer School on Machine Learning July 2017
•  Summer School on Computer Vision July 2017
• Summer School on Deeplearning July 2016
• Short Cource on Deeplearning December 2016
• Generalization and Equilibrium in Generative Adversarial Nets (GANs)
• Variational Inference: Foundations and Modern Methods by David Blei, Rajesh Ranganath, Shakir Mohamed, NIPS 2016 Tutorial December 5, 2016.
• Bayesian Reasoning and Deep Learning in Agent-based Systemsby Shakir Mohamed, NIPS 2016 .
• Tutorial on Variational Autoencoders by CARL DOERSCH, Carnegie Mellon, UC Berkeley, August 16, 2016.
• UCL Course on RL by David Silver.

Deep Learning Theoretical Papers

Architectures

• Dynamic Routing via Capsules, NIPS 2017.
• Capsule Networks, explanation by Aurelien Geron.

Theory of neural networks

• Provable Bounds for Learning Some Deep Representations by Sanjeev Arora, Aditya Bhaskara, Rong Ge, Tengyu Ma, ICML 2014.

Generative models

• Numerics of GANs, NIPS 2017
• GANs are Broken in More than One Way, explained by Ferenc Huszar
• Generalisation and Equilibrium in GANs, by Sanjeev Arora
• Improved Variational Inference with Inverse Autoregressive Flow Diederik P. Kingma, Tim SalimansTim Salimans, Rafal Jozefowicz, Xi Chen, Max Welling, NIPS 2016.
• Improved Techniques for Training GANs Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, arxiv 2016.

Bayesian deep learning

• Semi-supervised deep kernel learning by Neal Jean, Michael Xie, Stefano Ermon, NIPS 2016.
• Using Bayesian Deep Learningfor Transfer Learning in Optimisation by Jonas Langhabel, JannikWolff, Raphael Holca-Lamarre, NIPS Workshop 2016.
• Neural Variational InferenceFor Topic Models by Akash Srivastava, Charles Sutton, NIPS Workshop 2016.

Reinforcement Learning

• Variational Information MaximizingExploration Rein Houthooft, Xi Chen, Yan Duan John Schulman , Filip De Turck Pieter Abbeel, arXiv 2017.
• Reinforcement learning of motor skills with policy gradientsJan Peters, Stefan Schaal, Neural Networks 2008.
• Continuous Control With Deep Reinforcement Learning Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, ICLR 2016.

Optimisation

• Differentiable Optimization as a Layer in Neural Networks Brandon Amos, J. Zico Kolter, arXiv 2017.
• Optimisation As A Model Forfew-Short LearningSachin Ravi and Hugo Larochelle, ICLR 2017.
• Unrolled Generative Adversarial Networks Luke Metz, Ben Poole, David Pfau, Jascha Sohl-Dickstein, ICLR 2017.
• Input Convex Neural Networksby Brandon AmosLei Xu, J. Zico Kolter, arXiv 2016.