fan_out'  ` preserves the magnitudes in the backwards pass. 

  * **nonlinearity** – the non-linear function (  nn.functional  name), recommended to use only with ` 'relu'  ` or ` 'leaky_relu'  ` (default). 

Examples

    
    
    >>> w = torch.empty(3, 5)
    >>> nn.init.kaiming_normal_(w, mode='fan_out', nonlinearity='relu')
    

` torch.nn.init. ` ` orthogonal_ ` (  _tensor_ , _gain=1_ )  [ [source]
](_modules/torch/nn/init.html#orthogonal_) ¶

    

Fills the input  Tensor  with a (semi) orthogonal matrix, as described in
Exact solutions to the nonlinear dynamics of learning in deep linear neural
networks  \- Saxe, A. et al. (2013). The input tensor must have at least 2
dimensions, and for tensors with more than 2 dimensions the trailing
dimensions are flattened.

Parameters

    

  * **tensor** – an n-dimensional  torch.Tensor  , where  n  ≥  2  n \geq 2  n  ≥  2 

  * **gain** – optional scaling factor 

Examples

    
    
    >>> w = torch.empty(3, 5)
    >>> nn.init.orthogonal_(w)
    

` torch.nn.init. ` ` sparse_ ` (  _tensor_ , _sparsity_ , _std=0.01_ )  [
[source]  ](_modules/torch/nn/init.html#sparse_) ¶

    

Fills the 2D input  Tensor  as a sparse matrix, where the non-zero elements
will be drawn from the normal distribution  N  (  0  ,  0.01  )
\mathcal{N}(0, 0.01)  N  (  0  ,  0  .  0  1  )  , as described in  Deep
learning via Hessian-free optimization  \- Martens, J. (2010).

Parameters

    

  * **tensor** – an n-dimensional  torch.Tensor 

  * **sparsity** – The fraction of elements in each column to be set to zero 

  * **std** – the standard deviation of the normal distribution used to generate the non-zero values 

Examples

    
    
    >>> w = torch.empty(3, 5)
    >>> nn.init.sparse_(w, sparsity=0.1)
    

[ Next ![](_static/images/chevron-right-orange.svg) ](onnx.html "torch.onnx")
[ ![](_static/images/chevron-right-orange.svg) Previous ](jit_unsupported.html
"TorchScript Unsupported Pytorch Constructs")

* * *

© Copyright 2019, Torch Contributors.

Built with [ Sphinx ](http://sphinx-doc.org/) using a [ theme
](https://github.com/rtfd/sphinx_rtd_theme) provided by [ Read the Docs
](https://readthedocs.org) .

  * torch.nn.init 

![](https://www.googleadservices.com/pagead/conversion/795629140/?label=txkmCPmdtosBENSssfsC&guid=ON&script=0)

##  Docs

Access comprehensive developer documentation for PyTorch

[ View Docs ](https://pytorch.org/docs/stable/index.html)

##  Tutorials

Get in-depth tutorials for beginners and advanced developers

[ View Tutorials ](https://pytorch.org/tutorials)

##  Resources

Find development resources and get your questions answered

[ View Resources ](https://pytorch.org/resources)

[ ](https://pytorch.org/)

  * [ PyTorch ](https://pytorch.org/)
  * [ Get Started ](https://pytorch.org/get-started)
  * [ Features ](https://pytorch.org/features)
  * [ Ecosystem ](https://pytorch.org/ecosystem)
  * [ Blog ](https://pytorch.org/blog/)
  * [ Contributing ](https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md)

  * [ Resources ](https://pytorch.org/resources)
  * [ Tutorials ](https://pytorch.org/tutorials)
  * [ Docs ](https://pytorch.org/docs/stable/index.html)
  * [ Discuss ](https://discuss.pytorch.org)
  * [ Github Issues ](https://github.com/pytorch/pytorch/issues)
  * [ Brand Guidelines ](https://pytorch.org/assets/brand-guidelines/PyTorch-Brand-Guidelines.pdf)

  * Stay Connected 
  * Email Address 

[ ](https://www.facebook.com/pytorch) [ ](https://twitter.com/pytorch) [
](https://www.youtube.com/pytorch)

To analyze traffic and optimize your experience, we serve cookies on this
site. By clicking or navigating, you agree to allow our usage of cookies. As
the current maintainers of this site, Facebook’s Cookies Policy applies. Learn
more, including about available controls: [ Cookies Policy
](https://www.facebook.com/policies/cookies/) .

![](_static/images/pytorch-x.svg)

[ ](https://pytorch.org/)

  * [ Get Started ](https://pytorch.org/get-star