Deep Learning Fundamentals — Book Notes

Deep Learning Fundamentals¶

The book moves from shallow to deep — explaining basic neural networks, then walking through case studies into CNNs and RNNs, and finally reinforcement learning, all paired with TensorFlow usage examples. A solid entry point for deep learning.

Neural Networks (NN) work somewhat like neurons: input → output. Add hidden layers, and the network searches for the weight (influence) of each input parameter. The key questions are when to stop searching (determined by the step size at each iteration) and how to know whether you've found a minimum, maximum, or saddle point. As dimensionality increases, identifying saddle points becomes significantly more complex and impossible to visualize.

Convolutional Neural Networks (CNN) convert multiple parameters into depth by extracting features (filters). For example, multiple horizontal points are read as a horizontal line; based on features, specific lengths of horizontal lines are read as eyebrows — avoiding the explosive growth in compute demand that comes with larger input sizes.

Max Pooling is like pixelating: compressing non-essential detail.

At this point, the overall pattern of a convolutional network is established: an input layer → compression layers (multiple) → fully connected layer (traditional NN) → softmax (finding local minima/maxima). The compression layers are subdivided into convolutional layers (which can be stacked) and pooling layers.

Recurrent Neural Networks (RNN): For language prediction where inputs can be variable-length sentences or passages, the better approach is to parse segments into subjects, objects, prepositions, etc., then predict their relationships. The downside is inability to handle longer, interdependent sentences. If the result of each segment's hidden layer is fed back into the next segment's parsing — repeated continuously — the network can account for meaning across most of the sentence (regression), allowing the output to handle long sentences.

RNN's limitation with long sentences: each pass is an entirely fresh computation. Long Short-Term Memory (LSTM) introduces the concept of memory — the model used to produce the current output is reused in the next computation and modified. Conceptually, RNN is like processing short-term memory; LSTM incorporates something like long-term memory (experience).

LSTM's structure is roughly divided into three gates: retain gate, write gate, output gate.

Deep Reinforcement Learning uses an agent to try different strategies, while a trainer defines what states are good or bad. On reward strategy: using raw future reward can make training lack urgency or time-sensitivity. Discounted future reward reduces the weight of far-future outcomes, increasing the agent's urgency.

The exploration-exploitation tradeoff: Over-relying on known strategies risks getting stuck in local optima; too much exploration makes training inefficient. The epsilon-greedy strategy addresses this: at each step the agent can either pick the recommended action or a random one, with the exploration weight decaying over time (later in training, focus shifts from exploring to exploiting the current best solution). #programming