Resources

There are many online resources that can be helpful for learning the content of this class. Here are a few. If you find others, please post on Piazza and we can add them here.

• Michael Nielsen’s online book
• Deep Learning textbook by Ian Goodfellow, Yoshua Bengio, and Aaron Courville
• Unsupervised Feature Learning and Deep Learning Tutorial from Stanford
• CS231n: Convolutional Neural Networks for Visual Recognition lecture notes by Andrej Karpathy
• CS294: Deep Reinforcement Learning Course on reinforcement learning by Sergey Levine
• Neural Networks Appendix E of Principles of Neural Science, 5th ed. Eds. E. R. Kandel et al. by S. Seung and R. Yuste

Python tutorial: PicScie.

Further reading:

• Part I
  • An overview of gradient descent optimization algorithms
  • Convolutional Neural Networks backpropagation: from intuition to derivation
  • A guide to receptive field arithmetic for Convolutional Neural Networks
• Part II
  • Autoencoders
  • Vector Representations of Words
  • Backpropagation Through Time and Vanishing Gradients
  • Understanding LSTM Networks
  • David Silver, Google DeepMind: Deep Reinforcement Learning

Extra readings on the history of neural networks and their recent revival:

• Nature article by Yann LeCun, Yoshua Bengio, and Geoffrey Hinton
• Technical report by Jurgen Schmidhuber