PyTorch, No Tears

Preface

This book is to teach students how program in PyTorch. To follow along and execute the code samples, you will need Docker installed. The Docker container is located on Docker Hub. After you have installed Docker, you may run the container as follows.

docker run -it \
 -p 8888:8888 \
 -p 6006:6006 \
 --gpus all \
 oneoffcoder/book-pytorch-intro-gpu

Note that this Docker container has Jupyter Lab running on port 8888. You may access Jupyter Lab at http://localhost:8888 when the Docker container is running.

Contents

• 1. Environment
  • 1.1. nvidia-smi
  • 1.2. nvcc
  • 1.3. PyTorch
• 2. Tensor
  • 2.1. Creation
  • 2.2. Numpy bridge
  • 2.3. Indexing, slicing, joining and mutating
  • 2.4. Random sampling
  • 2.5. Operations
  • 2.6. Basic math
  • 2.7. Broadcasting
  • 2.8. Device-agnostic tensors
  • 2.9. Serialization
• 3. Autograd
• 4. Data
  • 4.1. Dataset
  • 4.2. ImageFolder
  • 4.3. DataLoader
  • 4.4. Datasets
• 5. Transformation
  • 5.1. PIL transforms
  • 5.2. Tensor transforms
  • 5.3. Conversion transforms
• 6. Loss function
  • 6.1. L1
  • 6.2. Smooth L1
  • 6.3. Mean-Squared Error
  • 6.4. Cross-Entropy
  • 6.5. Negative Log-Likelihood
  • 6.6. Poisson Negative Log-Likelihood
  • 6.7. Kullback-Leibler Divergence
  • 6.8. Binary Cross-Entropy
  • 6.9. Binary Cross-Entropy with Logits
  • 6.10. Margin Ranking
  • 6.11. Soft Margin
• 7. Optimizer
  • 7.1. Stochastic gradient descent
  • 7.2. AdaDelta
  • 7.3. AdaGrad
  • 7.4. Adam
  • 7.5. AdamW
  • 7.6. Adamax
  • 7.7. ASGD
  • 7.8. RMSprop
  • 7.9. Rprop
• 8. Scheduler
  • 8.1. LambdaLR
  • 8.2. StepLR
  • 8.3. MultiStepLR
  • 8.4. ExponentialLR
  • 8.5. CyclicLR
  • 8.6. CosineAnnealingWarmRestarts
• 9. Models, Convolutional Neural Networks
  • 9.1. ResNet18
  • 9.2. ResNet152
  • 9.3. AlexNet
  • 9.4. VGG19_BN
  • 9.5. SqueezeNet1_1
  • 9.6. DenseNet201
  • 9.7. GoogleNet
  • 9.8. ShuffleNet_v2_x1_0
  • 9.9. ResNext101_32x8d
  • 9.10. Wide_ResNet101_2
  • 9.11. MNASNet1_0
• 10. Saving and Loading Models
  • 10.1. Saving
  • 10.2. Loading
• 11. Learning
  • 11.1. Training and testing
  • 11.2. Validation
• 12. Recurrent Neural Network
  • 12.1. Data
  • 12.2. Model
  • 12.3. Training
  • 12.4. Evaluation
• 13. Generative Adversarial Network
  • 13.1. Parameters
  • 13.2. Data
  • 13.3. Generator network
  • 13.4. Discriminator
  • 13.5. Loss function and optimizer
  • 13.6. Training
  • 13.7. Visualization
  • 13.8. Saving
• 14. Neural Transfer
  • 14.1. Input images
  • 14.2. Loss functions
  • 14.3. Model
  • 14.4. Normalization
  • 14.5. Loss
  • 14.6. Optimizer
  • 14.7. Transfer
  • 14.8. Results
  • 14.9. Visualize
• 15. Object Detection
  • 15.1. Data
  • 15.2. Model
  • 15.3. Optimizer and scheduler
  • 15.4. Helper code
  • 15.5. Training and testing
  • 15.6. Predictions
• 16. Tensorboard
  • 16.1. Add image
  • 16.2. Add model
  • 16.3. Add embedding
  • 16.4. Add training loss
  • 16.5. Add precision-recall curve

About

One-Off Coder is an educational, service and product company. Please visit us online to discover how we may help you achieve life-long success in your personal coding career or with your company’s business goals and objectives.

• Website

• Facebook

• Twitter

• Instagram

• YouTube

• LinkedIn

Copyright

This work is licensed under a Creative Commons Attribution 4.0 International License by One-Off Coder.



The full source code is available on GitHub.

Cite this book as follows.:

@misc{oneoffcoder_pytorch_intro_2019,
title={PyTorch, No Tears},
url={https://learn-pytorch.oneoffcoder.com},
author={One-Off Coder},
year={2019},
month={Oct}}

Author

Jee Vang, Ph.D.

• Patreon