Size: 1.36 GB
GRU, LSTM, + more modern deep learning, machine learning, and data science for sequences
What you’ll learn
- Understand the simple recurrent unit (Elman unit)
Understand the GRU (gated recurrent unit)
Understand the LSTM (long short-term memory unit)
- Write various recurrent networks in Theano
- Understand backpropagation through time
- Understand how to mitigate the vanishing gradient problem
- Solve the XOR and parity problems using a recurrent neural network
- Use recurrent neural networks for language modeling
- Use RNNs for generating text, like poetry
- Visualize word embeddings and look for patterns in word vector representations
- Linear algebra
- Python, Numpy, Matplotlib
- Write a neural network in Theano
- Understand backpropagation
- Probability (conditional and joint distributions)
- Write a neural network in Tensorflow
DescriptionLike the course I just released on Hidden Markov Models, Recurrent Neural Networks are all about learning sequences – but whereas Markov Models are limited by the Markov assumption, Recurrent Neural Networks are not – and as a result, they are more expressive, and more powerful than anything we’ve seen on tasks that we haven’t made progress on in decades. So what’s going to be in this course and how will it build on the previous neural network courses and Hidden Markov Models?
- linear algebra
- probability (conditional and joint distributions)
- Python coding: if/else, loops, lists, dicts, sets
- Numpy coding: matrix and vector operations, loading a CSV file
- Deep learning: backpropagation, XOR problem
- Can write a neural network in Theano and Tensorflow
- Watch it at 2x.
- Take handwritten notes. This will drastically increase your ability to retain the information.
- Write down the equations. If you don’t, I guarantee it will just look like gibberish.
- Ask lots of questions on the discussion board. The more the better!
- Realize that most exercises will take you days or weeks to complete.
- Write code yourself, don’t just sit there and look at my code.
- Check out the lecture “What order should I take your courses in?” (available in the Appendix of any of my courses, including the free Numpy course)
- If you want to level up with deep learning, take this course.
- If you are a student or professional who wants to apply deep learning to time series or sequence data, take this course.
- If you want to learn about word embeddings and language modeling, take this course.
- If you want to improve the performance you got with Hidden Markov Models, take this course.
- If you’re interested the techniques that led to new developments in machine translation, take this course.
- If you have no idea about deep learning, don’t take this course, take the prerequisites