Fundamentals of Deep Learning: Designing Next-Generation Machine Intelligence Algorithms
Publisher: O'Reilly Media
Published: Jun 29, 2017
Number of pages: 298
ISBN-13: 9781491925614
Table of contents: title: Preface
pagenum: ix; label: 1
title: The Neural Network
pagenum: 1; title: Building Intelligent Machines
pagenum: 1; title: The Limits of Traditional Computer Programs
pagenum: 2; title: The Mechanics of Machine Learning
pagenum: 3; title: The Neuron
pagenum: 7; title: Expressing Linear Perceptrons as Neurons
pagenum: 8; title: Feed-Forward Neural Networks
pagenum: 9; title: Linear Neurons and Their Limitations
pagenum: 12; title: Sigmoid, Tanh, and ReLU Neurons
pagenum: 13; title: Softmax Output Layers
pagenum: 15; title: Looking Forward
pagenum: 15; label: 2
title: Training Feed-Forward Neural Networks
pagenum: 17; title: The Fast-Food Problem
pagenum: 17; title: Gradient Descent
pagenum: 19; title: The Delta Rule and Learning Rates
pagenum: 21; title: Gradient Descent with Sigmoidal Neurons
pagenum: 22; title: The Backpropagation Algorithm
pagenum: 23; title: Stochastic and Minibatch Gradient Descent
pagenum: 25; title: Test Sets, Validation Sets, and Overfitting
pagenum: 27; title: Preventing Overfitting in Deep Neural Networks
pagenum: 34; title: Summary
pagenum: 37; label: 3
title: Implementing Neural Networks in TensorFlow
pagenum: 39; title: What Is TensorFlow?
pagenum: 39; title: How Does TensorFlow Compare to Alternatives?
pagenum: 40; title: Installing TensorFlow
pagenum: 41; title: Creating and Manipulating TensorFlow Variables
pagenum: 43; title: TensorFlow Operations
pagenum: 45; title: Placeholder Tensors
pagenum: 45; title: Sessions in TensorFlow
pagenum: 46; title: Navigating Variable Scopes and Sharing Variables
pagenum: 48; title: Managing Models over the CPU and GPU
pagenum: 51; title: Specifying the Logistic Regression Model in TensorFlow
pagenum: 52; title: Logging and Training the Logistic Regression Model
pagenum: 55; title: Leveraging TensorBoard to Visualize Computation Graphs and Learning
pagenum: 58; title: Building a Multilayer Model for MNIST in TensorFlow
pagenum: 59; title: Summary
pagenum: 62; label: 4
title: Beyond Gradient Descent
pagenum: 63; title: The Challenges with Gradient Descent
pagenum: 63; title: Local Minima in the Error Surfaces of Deep Networks
pagenum: 64; title: Model Identifiability
pagenum: 65; title: How Pesky Are Spurious Local Minima in Deep Networks?
pagenum: 66; title: Flat Regions in the Error Surface
pagenum: 69; title: When the Gradient Points in the Wrong Direction
pagenum: 71; title: Momentum-Based Optimization
pagenum: 74; title: A Brief View of Second-Order Methods
pagenum: 77; title: Learning Rate Adaptation
pagenum: 78; title: AdaGrad — Accumulating Historical Gradients
pagenum: 79; title: RMSProp — Exponentially Weighted Moving Average of Gradients
pagenum: 80; title: Adam — Combining Momentum and RMSProp
pagenum: 81; title: The Philosophy Behind Optimizer Selection
pagenum: 83; title: Summary
pagenum: 83; label: 5
title: Convolutional Neural Networks
pagenum: 85; title: Neurons in Human Vision
pagenum: 85; title: The Shortcomings of Feature Selection
pagenum: 86; title: Vanilla Deep Neural Networks Don't Scale
pagenum: 89; title: Filters and Feature Maps
pagenum: 90; title: Full Description of the Convolutional Layer
pagenum: 95; title: Max Pooling
pagenum: 98; title: Full Architectural Description of Convolution Networks
pagenum: 99; title: Closing the Loop on MNIST with Convolutional Networks
pagenum: 101; title: Image Preprocessing Pipelines Enable More Robust Models
pagenum: 103; title: Accelerating Training with Batch Normalization
pagenum: 104; title: Building a Convolutional Network for CIFAR-10
pagenum: 107; title: Visualizing Learning in Convolutional Networks
pagenum: 109; title: Leveraging Convolutional Filters to Replicate Artistic Styles
pagenum: 113; title: Learning Convolutional Filters for Other Problem Domains
pagenum: 114; title: Summary
pagenum: 115; label: 6
title: Embedding and Representation Learning
pagenum: 117; title: Learning Lower-Dimensional Representations
pagenum: 117; title: Principal Component Analysis
pagenum: 118; title: Motivating the Autoencoder Architecture
pagenum: 120; title: Implementing an Autoencoder in TensorFlow
pagenum: 121; title: Denoising to Force Robust Representations
pagenum: 134; title: Sparsity in Autoencoders
pagenum: 137; title: When Context Is More Informative than the Input Vector
pagenum: 140; title: The Word2Vec Framework
pagenum: 143; title: Implementing the Skip-Gram Architecture
pagenum: 146; title: Summary
pagenum: 152; label: 7
title: Models for Sequence Analysis
pagenum: 153; title: Analyzing Variable-Length Inputs
pagenum: 153; title: Tackling seq2seq with Neural N-Grams
pagenum: 155; title: Implementing a Part-of-Speech Tagger
pagenum: 156; title: Dependency Parsing and SyntaxNet
pagenum: 164; title: Beam Search and Global Normalization
pagenum: 168; title: A Case for Stateful Deep Learning Models
pagenum: 172; title: Recurrent Neural Networks
pagenum: 173; title: The Challenges with Vanishing Gradients
pagenum: 176; title: Long Short-Term Memory (LSTM) Units
pagenum: 178; title: TensorFlow Primitives for RNN Models
pagenum: 183; title: Implementing a Sentiment Analysis Model
pagenum: 185; title: Solving seq2seq Tasks with Recurrent Neural Networks
pagenum: 189; title: Augmenting Recurrent Networks with Attention
pagenum: 191; title: Dissecting a Neural Translation Network
pagenum: 194; title: Summary
pagenum: 217; label: 8
title: Memory Augmented Neural Networks
pagenum: 219; title: Neural Turing Machines
pagenum: 219; title: Attention-Based Memory Access
pagenum: 221; title: NTM Memory Addressing Mechanisms
pagenum: 223; title: Differentiable Neural Computers
pagenum: 226; title: Interference-Free Writing in DNCs
pagenum: 229; title: DNC Memory Reuse
pagenum: 230; title: Temporal Linking of DNC Writes
pagenum: 231; title: Understanding the DNC Read Head
pagenum: 232; title: The DNC Controller Network
pagenum: 232; title: Visualizing the DNC in Action
pagenum: 234; title: Implementing the DNC in TensorFlow
pagenum: 237; title: Teaching a DNC to Read and Comprehend
pagenum: 242; title: Summary
pagenum: 244; label: 9
title: Deep Reinforcement Learning
pagenum: 245; title: Deep Reinforcement Learning Masters Atari Games
pagenum: 245; title: What Is Reinforcement Learning?
pagenum: 247; title: Markov Decision Processes (MDP)
pagenum: 248; title: Policy
pagenum: 249; title: Future Return
pagenum: 250; title: Discounted Future Return
pagenum: 251; title: Explore Versus Exploit
pagenum: 251; title: Policy Versus Value Learning
pagenum: 253; title: Policy Learning via Policy Gradients
pagenum: 254; title: Pole-Cart with Policy Gradients
pagenum: 254; title: OpenAI Gym
pagenum: 254; title: Creating an Agent
pagenum: 255; title: Building the Model and Optimizer
pagenum: 257; title: Sampling Actions
pagenum: 257; title: Keeping Track of History
pagenum: 257; title: Policy Gradient Main Function
pagenum: 258; title: PGAgent Performance on Pole-Cart
pagenum: 260; title: Q-Learning and Deep Q-Networks
pagenum: 261; title: The Bellman Equation
pagenum: 261; title: Issues with Value Iteration
pagenum: 262; title: Approximating the Q-Function
pagenum: 262; title: Deep Q-Network (DQN)
pagenum: 263; title: Training DQN
pagenum: 263; title: Learning Stability
pagenum: 263; title: Target Q-Network
pagenum: 264; title: Experience Replay
pagenum: 264; title: From Q-Function to Policy
pagenum: 264; title: DQN and the Markov Assumption
pagenum: 265; title: DQN's Solution to the Markov Assumption
pagenum: 265; title: Playing Breakout with DQN
pagenum: 265; title: Building Our Architecture
pagenum: 268; title: Stacking Frames
pagenum: 268; title: Setting Up Training Operations
pagenum: 268; title: Updating Our Target Q-Network
pagenum: 269; title: Implementing Experience Replay
pagenum: 269; title: DQN Main Loop
pagenum: 270; title: DQNAgent Results on Breakout
pagenum: 272; title: Improving and Moving Beyond DQN
pagenum: 273; title: Deep Recurrent Q-Networks (DRQN)
pagenum: 273; title: Asynchronous Advantage Actor-Critic Agent (A3C)
pagenum: 274; title: Unsupervised Reinforcement and Auxiliary Learning (UNREAL)
pagenum: 275; title: Summary
pagenum: 276; title: Index
pagenum: 277