Learning Goal: I’m working on a data engineering discussion question and need the explanation and answer to help me learn.
In section 10.7 The Challenge of Long-Term Dependencies, from “Goodfellow, I., Bengio, Y., Courville, A. (2016). Deep Learning. United Kingdom: MIT Press,” authors discuss the mathematical challenge of learning long-term dependencies in recurrent networks. The basic problem is that gradients propagated over many stages tend to either vanish or explode. Discuss the following:
- why the gradients propagated over many stages tend to either vanish or explode
- possible ways to deal with the gradients vanishing or exploding
To participate in the discussion, respond to the discussion promptly by Thursday at 11:59PM EST. Then, read a selection of your colleagues’ postings. Finally, respond to at least two classmates by Sunday at 11:59PM EST in one or more of the following ways:
- Complete the discussion by the assigned due date.
- Do not claim credit for the words, ideas, and concepts of others.
- Do not copy and paste information or concepts from the Internet and claim that it is your work. It will be considered Plagiarism and you will receive a zero for your work.