Logo
0.4.0
version selector ▼

Notes

  • Autograd mechanics
    • Excluding subgraphs from backward
      • requires_grad
    • How autograd encodes the history
    • In-place operations with autograd
    • In-place correctness checks
  • Broadcasting semantics
    • General semantics
    • In-place semantics
    • Backwards compatibility
  • CUDA semantics
    • Asynchronous execution
      • CUDA streams
    • Memory management
    • Best practices
      • Device-agnostic code
      • Use pinned memory buffers
      • Use nn.DataParallel instead of multiprocessing
  • Extending PyTorch
    • Extending torch.autograd
    • Extending torch.nn
      • Adding a Module
    • Writing custom C extensions
  • Frequently Asked Questions
    • My model reports “cuda runtime error(2): out of memory”
    • My GPU memory isn’t freed properly
    • My data loader workers return identical random numbers
    • My recurrent network doesn’t work with data parallelism
  • Multiprocessing best practices
    • Sharing CUDA tensors
    • Best practices and tips
      • Avoiding and fighting deadlocks
      • Reuse buffers passed through a Queue
      • Asynchronous multiprocess training (e.g. Hogwild)
        • Hogwild
  • Serialization semantics
    • Best practices
      • Recommended approach for saving a model
  • Windows FAQ
    • Building from source
      • Include optional components
      • Speeding CUDA build for Windows
      • One key install script
    • Extension
      • CFFI Extension
      • Cpp Extension
    • Installation
      • Package not found in win-32 channel.
      • Why are there no Python 2 packages for Windows?
      • Import error
    • Usage (multiprocessing)
      • Multiprocessing error without if-clause protection
      • Multiprocessing error “Broken pipe”
      • Multiprocessing error “driver shut down”
      • CUDA IPC operations

Package Reference

  • torch
    • Tensors
      • Creation Ops
      • Indexing, Slicing, Joining, Mutating Ops
    • Random sampling
      • In-place random sampling
    • Serialization
    • Parallelism
    • Locally disabling gradient computation
    • Math operations
      • Pointwise Ops
      • Reduction Ops
      • Comparison Ops
      • Spectral Ops
      • Other Operations
      • BLAS and LAPACK Operations
  • torch.Tensor
  • Tensor Attributes
    • torch.dtype
    • torch.device
    • torch.layout
  • torch.sparse
  • torch.cuda
    • Random Number Generator
    • Communication collectives
    • Streams and events
    • Memory management
    • NVIDIA Tools Extension (NVTX)
  • torch.Storage
  • torch.nn
    • Parameters
    • Containers
      • Module
      • Sequential
      • ModuleList
      • ParameterList
    • Convolution layers
      • Conv1d
      • Conv2d
      • Conv3d
      • ConvTranspose1d
      • ConvTranspose2d
      • ConvTranspose3d
    • Pooling layers
      • MaxPool1d
      • MaxPool2d
      • MaxPool3d
      • MaxUnpool1d
      • MaxUnpool2d
      • MaxUnpool3d
      • AvgPool1d
      • AvgPool2d
      • AvgPool3d
      • FractionalMaxPool2d
      • LPPool1d
      • LPPool2d
      • AdaptiveMaxPool1d
      • AdaptiveMaxPool2d
      • AdaptiveMaxPool3d
      • AdaptiveAvgPool1d
      • AdaptiveAvgPool2d
      • AdaptiveAvgPool3d
    • Padding layers
      • ReflectionPad1d
      • ReflectionPad2d
      • ReplicationPad1d
      • ReplicationPad2d
      • ReplicationPad3d
      • ZeroPad2d
      • ConstantPad1d
      • ConstantPad2d
      • ConstantPad3d
    • Non-linear activations (weighted sum, nonlinearity)
      • ELU
      • Hardshrink
      • Hardtanh
      • LeakyReLU
      • LogSigmoid
      • PReLU
      • ReLU
      • ReLU6
      • RReLU
      • SELU
      • Sigmoid
      • Softplus
      • Softshrink
      • Softsign
      • Tanh
      • Tanhshrink
      • Threshold
    • Non-linear activations (other)
      • Softmin
      • Softmax
      • Softmax2d
      • LogSoftmax
    • Normalization layers
      • BatchNorm1d
      • BatchNorm2d
      • BatchNorm3d
      • InstanceNorm1d
      • InstanceNorm2d
      • InstanceNorm3d
      • LayerNorm
      • LocalResponseNorm
    • Recurrent layers
      • RNN
      • LSTM
      • GRU
      • RNNCell
      • LSTMCell
      • GRUCell
    • Linear layers
      • Linear
      • Bilinear
    • Dropout layers
      • Dropout
      • Dropout2d
      • Dropout3d
      • AlphaDropout
    • Sparse layers
      • Embedding
      • EmbeddingBag
    • Distance functions
      • CosineSimilarity
      • PairwiseDistance
    • Loss functions
      • L1Loss
      • MSELoss
      • CrossEntropyLoss
      • NLLLoss
      • PoissonNLLLoss
      • KLDivLoss
      • BCELoss
      • BCEWithLogitsLoss
      • MarginRankingLoss
      • HingeEmbeddingLoss
      • MultiLabelMarginLoss
      • SmoothL1Loss
      • SoftMarginLoss
      • MultiLabelSoftMarginLoss
      • CosineEmbeddingLoss
      • MultiMarginLoss
      • TripletMarginLoss
    • Vision layers
      • PixelShuffle
      • Upsample
      • UpsamplingNearest2d
      • UpsamplingBilinear2d
    • DataParallel layers (multi-GPU, distributed)
      • DataParallel
      • DistributedDataParallel
    • Utilities
      • clip_grad_norm_
      • clip_grad_value_
      • weight_norm
      • remove_weight_norm
      • PackedSequence
      • pack_padded_sequence
      • pad_packed_sequence
      • pad_sequence
      • pack_sequence
  • torch.nn.functional
    • Convolution functions
      • conv1d
      • conv2d
      • conv3d
      • conv_transpose1d
      • conv_transpose2d
      • conv_transpose3d
    • Pooling functions
      • avg_pool1d
      • avg_pool2d
      • avg_pool3d
      • max_pool1d
      • max_pool2d
      • max_pool3d
      • max_unpool1d
      • max_unpool2d
      • max_unpool3d
      • lp_pool1d
      • lp_pool2d
      • adaptive_max_pool1d
      • adaptive_max_pool2d
      • adaptive_max_pool3d
      • adaptive_avg_pool1d
      • adaptive_avg_pool2d
      • adaptive_avg_pool3d
    • Non-linear activation functions
      • threshold
      • relu
      • hardtanh
      • relu6
      • elu
      • selu
      • leaky_relu
      • prelu
      • rrelu
      • glu
      • logsigmoid
      • hardshrink
      • tanhshrink
      • softsign
      • softplus
      • softmin
      • softmax
      • softshrink
      • log_softmax
      • tanh
      • sigmoid
    • Normalization functions
      • batch_norm
      • instance_norm
      • layer_norm
      • local_response_norm
      • normalize
    • Linear functions
      • linear
    • Dropout functions
      • dropout
      • alpha_dropout
      • dropout2d
      • dropout3d
    • Distance functions
      • pairwise_distance
      • cosine_similarity
    • Loss functions
      • binary_cross_entropy
      • poisson_nll_loss
      • cosine_embedding_loss
      • cross_entropy
      • hinge_embedding_loss
      • kl_div
      • l1_loss
      • mse_loss
      • margin_ranking_loss
      • multilabel_margin_loss
      • multilabel_soft_margin_loss
      • multi_margin_loss
      • nll_loss
      • binary_cross_entropy_with_logits
      • smooth_l1_loss
      • soft_margin_loss
      • triplet_margin_loss
    • Vision functions
      • pixel_shuffle
      • pad
      • upsample
      • upsample_nearest
      • upsample_bilinear
      • grid_sample
      • affine_grid
    • DataParallel functions (multi-GPU, distributed)
      • data_parallel
  • torch.nn.init
  • torch.optim
    • How to use an optimizer
      • Constructing it
      • Per-parameter options
      • Taking an optimization step
        • optimizer.step()
        • optimizer.step(closure)
    • Algorithms
    • How to adjust Learning Rate
  • torch.autograd
    • Locally disabling gradient computation
    • In-place operations on Tensors
      • In-place correctness checks
    • Variable (deprecated)
    • Tensor autograd functions
    • Function
    • Profiler
  • torch.distributions
    • Score function
    • Pathwise derivative
    • Distribution
    • ExponentialFamily
    • Bernoulli
    • Beta
    • Binomial
    • Categorical
    • Cauchy
    • Chi2
    • Dirichlet
    • Exponential
    • FisherSnedecor
    • Gamma
    • Geometric
    • Gumbel
    • Independent
    • Laplace
    • LogNormal
    • Multinomial
    • MultivariateNormal
    • Normal
    • OneHotCategorical
    • Pareto
    • Poisson
    • RelaxedBernoulli
    • RelaxedOneHotCategorical
    • StudentT
    • TransformedDistribution
    • Uniform
    • KL Divergence
    • Transforms
    • Constraints
    • Constraint Registry
  • torch.multiprocessing
    • Strategy management
    • Sharing CUDA tensors
    • Sharing strategies
      • File descriptor - file_descriptor
      • File system - file_system
  • torch.distributed
    • Basics
    • Initialization
      • TCP initialization
      • Shared file-system initialization
      • Environment variable initialization
    • Groups
    • Point-to-point communication
    • Collective functions
    • Multi-GPU collective functions
    • Launch utility
  • torch.utils.bottleneck
  • torch.utils.checkpoint
  • torch.utils.cpp_extension
  • torch.utils.data
  • torch.utils.ffi
  • torch.utils.model_zoo
  • torch.onnx
    • Example: End-to-end AlexNet from PyTorch to Caffe2
    • Limitations
    • Supported operators
    • Functions
  • torch.legacy

torchvision Reference

  • torchvision
    • torchvision.datasets
      • MNIST
      • Fashion-MNIST
      • EMNIST
      • COCO
        • Captions
        • Detection
      • LSUN
      • ImageFolder
      • DatasetFolder
      • Imagenet-12
      • CIFAR
      • STL10
      • SVHN
      • PhotoTour
    • torchvision.models
      • Alexnet
      • VGG
      • ResNet
      • SqueezeNet
      • DenseNet
      • Inception v3
    • torchvision.transforms
      • Transforms on PIL Image
      • Transforms on torch.*Tensor
      • Conversion Transforms
      • Generic Transforms
    • torchvision.utils
PyTorch
  • Docs »
  • PyTorch documentation
  • Edit on GitHub

PyTorch documentation¶

PyTorch is an optimized tensor library for deep learning using GPUs and CPUs.

Notes

  • Autograd mechanics
  • Broadcasting semantics
  • CUDA semantics
  • Extending PyTorch
  • Frequently Asked Questions
  • Multiprocessing best practices
  • Serialization semantics
  • Windows FAQ

Package Reference

  • torch
  • torch.Tensor
  • Tensor Attributes
  • torch.sparse
  • torch.cuda
  • torch.Storage
  • torch.nn
  • torch.nn.functional
  • torch.nn.init
  • torch.optim
  • torch.autograd
  • torch.distributions
  • torch.multiprocessing
  • torch.distributed
  • torch.utils.bottleneck
  • torch.utils.checkpoint
  • torch.utils.cpp_extension
  • torch.utils.data
  • torch.utils.ffi
  • torch.utils.model_zoo
  • torch.onnx
  • torch.legacy

torchvision Reference

  • torchvision
    • torchvision.datasets
    • torchvision.models
    • torchvision.transforms
    • torchvision.utils

Indices and tables¶

  • Index
  • Module Index
Next

© Copyright 2018, Torch Contributors.

Built with Sphinx using a theme provided by Read the Docs.