neulab/docprompting-conala huggingface dataset:
class torch.nn.Hardshrink(lambd=0.5) [source]
 
Applies the hard shrinkage function element-wise:  HardShrink(x)={x, if x>λx, if x<−λ0, otherwise \text{HardShrink}(x) = \begin{cases} x, & \text{ if } x > \lambda \\ x, & \text{ if } x < -\lambda \\ 0, & \text{ otherwise } \end{cases}  
 Parameters 
lambd – the λ\lambda  value for the Hardshrink formulation. Default: 0.5    Shape:

 Input: (N,∗)(N, *)  where * means, any number of additional dimensions Output: (N,∗)(N, *) , same shape as the input     Examples: >>> m = nn.Hardshrink()
>>> input = torch.randn(2)
>>> output = m(input)
class torch.nn.Hardsigmoid(inplace=False) [source]
 
Applies the element-wise function:  Hardsigmoid(x)={0if x≤−3,1if x≥+3,x/6+1/2otherwise\text{Hardsigmoid}(x) = \begin{cases} 0 & \text{if~} x \le -3, \\ 1 & \text{if~} x \ge +3, \\ x / 6 + 1 / 2 & \text{otherwise} \end{cases}  
 Parameters 
inplace – can optionally do the operation in-place. Default: False    Shape:

 Input: (N,∗)(N, *)  where * means, any number of additional dimensions Output: (N,∗)(N, *) , same shape as the input    Examples: >>> m = nn.Hardsigmoid()
>>> input = torch.randn(2)
>>> output = m(input)
class torch.nn.Hardswish(inplace=False) [source]
 
Applies the hardswish function, element-wise, as described in the paper: Searching for MobileNetV3.  Hardswish(x)={0if x≤−3,xif x≥+3,x⋅(x+3)/6otherwise\text{Hardswish}(x) = \begin{cases} 0 & \text{if~} x \le -3, \\ x & \text{if~} x \ge +3, \\ x \cdot (x + 3) /6 & \text{otherwise} \end{cases}  
 Parameters 
inplace – can optionally do the operation in-place. Default: False    Shape:

 Input: (N,∗)(N, *)  where * means, any number of additional dimensions Output: (N,∗)(N, *) , same shape as the input    Examples: >>> m = nn.Hardswish()
>>> input = torch.randn(2)
>>> output = m(input)
class torch.nn.Hardtanh(min_val=-1.0, max_val=1.0, inplace=False, min_value=None, max_value=None) [source]
 
Applies the HardTanh function element-wise HardTanh is defined as:  HardTanh(x)={1 if x>1−1 if x<−1x otherwise \text{HardTanh}(x) = \begin{cases} 1 & \text{ if } x > 1 \\ -1 & \text{ if } x < -1 \\ x & \text{ otherwise } \\ \end{cases}  
The range of the linear region [−1,1][-1, 1]  can be adjusted using min_val and max_val.  Parameters 
 
min_val – minimum value of the linear region range. Default: -1 
max_val – maximum value of the linear region range. Default: 1 
inplace – can optionally do the operation in-place. Default: False
    Keyword arguments min_value and max_value have been deprecated in favor of min_val and max_val.  Shape:

 Input: (N,∗)(N, *)  where * means, any number of additional dimensions Output: (N,∗)(N, *) , same shape as the input     Examples: >>> m = nn.Hardtanh(-2, 2)
>>> input = torch.randn(2)
>>> output = m(input)
class torch.nn.HingeEmbeddingLoss(margin=1.0, size_average=None, reduce=None, reduction='mean') [source]
 
Measures the loss given an input tensor xx  and a labels tensor yy  (containing 1 or -1). This is usually used for measuring whether two inputs are similar or dissimilar, e.g. using the L1 pairwise distance as xx , and is typically used for learning nonlinear embeddings or semi-supervised learning. The loss function for nn -th sample in the mini-batch is  ln={xn,ifyn=1,max⁡{0,Δ−xn},ifyn=−1,l_n = \begin{cases} x_n, & \text{if}\; y_n = 1,\\ \max \{0, \Delta - x_n\}, & \text{if}\; y_n = -1, \end{cases}  
and the total loss functions is  ℓ(x,y)={mean⁡(L),if reduction=‘mean’;sum⁡(L),if reduction=‘sum’.\ell(x, y) = \begin{cases} \operatorname{mean}(L), & \text{if reduction} = \text{`mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{`sum'.} \end{cases}  
where L={l1,…,lN}⊤L = \{l_1,\dots,l_N\}^\top .  Parameters 
 
margin (float, optional) – Has a default value of 1. 
size_average (bool, optional) – Deprecated (see reduction). By default, the losses are averaged over each loss element in the batch. Note that for some losses, there are multiple elements per sample. If the field size_average is set to False, the losses are instead summed for each minibatch. Ignored when reduce is False. Default: True
 
reduce (bool, optional) – Deprecated (see reduction). By default, the losses are averaged or summed over observations for each minibatch depending on size_average. When reduce is False, returns a loss per batch element instead and ignores size_average. Default: True
 
reduction (string, optional) – Specifies the reduction to apply to the output: 'none' | 'mean' | 'sum'. 'none': no reduction will be applied, 'mean': the sum of the output will be divided by the number of elements in the output, 'sum': the output will be summed. Note: size_average and reduce are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction. Default: 'mean'
     Shape:

 Input: (∗)(*)  where ∗*  means, any number of dimensions. The sum operation operates over all the elements. Target: (∗)(*) , same shape as the input Output: scalar. If reduction is 'none', then same shape as the input
class torch.nn.Identity(*args, **kwargs) [source]
 
A placeholder identity operator that is argument-insensitive.  Parameters 
 
args – any argument (unused) 
kwargs – any keyword argument (unused)    Examples: >>> m = nn.Identity(54, unused_argument1=0.1, unused_argument2=False)
>>> input = torch.randn(128, 20)
>>> output = m(input)
>>> print(output.size())
torch.Size([128, 20])
torch.nn.init.calculate_gain(nonlinearity, param=None) [source]
 
Return the recommended gain value for the given nonlinearity function. The values are as follows:   
nonlinearity gain   
Linear / Identity 11   
Conv{1,2,3}D 11   
Sigmoid 11   
Tanh 53\frac{5}{3}   
ReLU 2\sqrt{2}   
Leaky Relu 21+negative_slope2\sqrt{\frac{2}{1 + \text{negative\_slope}^2}}   
SELU 34\frac{3}{4}     Parameters 
 
nonlinearity – the non-linear function (nn.functional name) 
param – optional parameter for the non-linear function    Examples >>> gain = nn.init.calculate_gain('leaky_relu', 0.2)  # leaky_relu with negative_slope=0.2
torch.nn.init.constant_(tensor, val) [source]
 
Fills the input Tensor with the value val\text{val} .  Parameters 
 
tensor – an n-dimensional torch.Tensor
 
val – the value to fill the tensor with    Examples >>> w = torch.empty(3, 5)
>>> nn.init.constant_(w, 0.3)
torch.nn.init.dirac_(tensor, groups=1) [source]
 
Fills the {3, 4, 5}-dimensional input Tensor with the Dirac delta function. Preserves the identity of the inputs in Convolutional layers, where as many input channels are preserved as possible. In case of groups>1, each group of channels preserves identity  Parameters 
 
tensor – a {3, 4, 5}-dimensional torch.Tensor
 
groups (optional) – number of groups in the conv layer (default: 1)    Examples >>> w = torch.empty(3, 16, 5, 5)
>>> nn.init.dirac_(w)
>>> w = torch.empty(3, 24, 5, 5)
>>> nn.init.dirac_(w, 3)
torch.nn.init.eye_(tensor) [source]
 
Fills the 2-dimensional input Tensor with the identity matrix. Preserves the identity of the inputs in Linear layers, where as many inputs are preserved as possible.  Parameters 
tensor – a 2-dimensional torch.Tensor   Examples >>> w = torch.empty(3, 5)
>>> nn.init.eye_(w)
torch.nn.init.kaiming_normal_(tensor, a=0, mode='fan_in', nonlinearity='leaky_relu') [source]
 
Fills the input Tensor with values according to the method described in Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification - He, K. et al. (2015), using a normal distribution. The resulting tensor will have values sampled from N(0,std2)\mathcal{N}(0, \text{std}^2)  where  std=gainfan_mode\text{std} = \frac{\text{gain}}{\sqrt{\text{fan\_mode}}}  
Also known as He initialization.  Parameters 
 
tensor – an n-dimensional torch.Tensor
 
a – the negative slope of the rectifier used after this layer (only used with 'leaky_relu') 
mode – either 'fan_in' (default) or 'fan_out'. Choosing 'fan_in' preserves the magnitude of the variance of the weights in the forward pass. Choosing 'fan_out' preserves the magnitudes in the backwards pass. 
nonlinearity – the non-linear function (nn.functional name), recommended to use only with 'relu' or 'leaky_relu' (default).    Examples >>> w = torch.empty(3, 5)
>>> nn.init.kaiming_normal_(w, mode='fan_out', nonlinearity='relu')
torch.nn.init.kaiming_uniform_(tensor, a=0, mode='fan_in', nonlinearity='leaky_relu') [source]
 
Fills the input Tensor with values according to the method described in Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification - He, K. et al. (2015), using a uniform distribution. The resulting tensor will have values sampled from U(−bound,bound)\mathcal{U}(-\text{bound}, \text{bound})  where  bound=gain×3fan_mode\text{bound} = \text{gain} \times \sqrt{\frac{3}{\text{fan\_mode}}}  
Also known as He initialization.  Parameters 
 
tensor – an n-dimensional torch.Tensor
 
a – the negative slope of the rectifier used after this layer (only used with 'leaky_relu') 
mode – either 'fan_in' (default) or 'fan_out'. Choosing 'fan_in' preserves the magnitude of the variance of the weights in the forward pass. Choosing 'fan_out' preserves the magnitudes in the backwards pass. 
nonlinearity – the non-linear function (nn.functional name), recommended to use only with 'relu' or 'leaky_relu' (default).    Examples >>> w = torch.empty(3, 5)
>>> nn.init.kaiming_uniform_(w, mode='fan_in', nonlinearity='relu')
torch.nn.init.normal_(tensor, mean=0.0, std=1.0) [source]
 
Fills the input Tensor with values drawn from the normal distribution N(mean,std2)\mathcal{N}(\text{mean}, \text{std}^2) .  Parameters 
 
tensor – an n-dimensional torch.Tensor
 
mean – the mean of the normal distribution 
std – the standard deviation of the normal distribution    Examples >>> w = torch.empty(3, 5)
>>> nn.init.normal_(w)
torch.nn.init.ones_(tensor) [source]
 
Fills the input Tensor with the scalar value 1.  Parameters 
tensor – an n-dimensional torch.Tensor   Examples >>> w = torch.empty(3, 5)
>>> nn.init.ones_(w)
torch.nn.init.orthogonal_(tensor, gain=1) [source]
 
Fills the input Tensor with a (semi) orthogonal matrix, as described in Exact solutions to the nonlinear dynamics of learning in deep linear neural networks - Saxe, A. et al. (2013). The input tensor must have at least 2 dimensions, and for tensors with more than 2 dimensions the trailing dimensions are flattened.  Parameters 
 
tensor – an n-dimensional torch.Tensor, where n≥2n \geq 2 
 
gain – optional scaling factor    Examples >>> w = torch.empty(3, 5)
>>> nn.init.orthogonal_(w)
torch.nn.init.sparse_(tensor, sparsity, std=0.01) [source]
 
Fills the 2D input Tensor as a sparse matrix, where the non-zero elements will be drawn from the normal distribution N(0,0.01)\mathcal{N}(0, 0.01) , as described in Deep learning via Hessian-free optimization - Martens, J. (2010).  Parameters 
 
tensor – an n-dimensional torch.Tensor
 
sparsity – The fraction of elements in each column to be set to zero 
std – the standard deviation of the normal distribution used to generate the non-zero values    Examples >>> w = torch.empty(3, 5)
>>> nn.init.sparse_(w, sparsity=0.1)
torch.nn.init.uniform_(tensor, a=0.0, b=1.0) [source]
 
Fills the input Tensor with values drawn from the uniform distribution U(a,b)\mathcal{U}(a, b) .  Parameters 
 
tensor – an n-dimensional torch.Tensor
 
a – the lower bound of the uniform distribution 
b – the upper bound of the uniform distribution    Examples >>> w = torch.empty(3, 5)
>>> nn.init.uniform_(w)
torch.nn.init.xavier_normal_(tensor, gain=1.0) [source]
 
Fills the input Tensor with values according to the method described in Understanding the difficulty of training deep feedforward neural networks - Glorot, X. & Bengio, Y. (2010), using a normal distribution. The resulting tensor will have values sampled from N(0,std2)\mathcal{N}(0, \text{std}^2)  where  std=gain×2fan_in+fan_out\text{std} = \text{gain} \times \sqrt{\frac{2}{\text{fan\_in} + \text{fan\_out}}}  
Also known as Glorot initialization.  Parameters 
 
tensor – an n-dimensional torch.Tensor
 
gain – an optional scaling factor    Examples >>> w = torch.empty(3, 5)
>>> nn.init.xavier_normal_(w)
torch.nn.init.xavier_uniform_(tensor, gain=1.0) [source]
 
Fills the input Tensor with values according to the method described in Understanding the difficulty of training deep feedforward neural networks - Glorot, X. & Bengio, Y. (2010), using a uniform distribution. The resulting tensor will have values sampled from U(−a,a)\mathcal{U}(-a, a)  where  a=gain×6fan_in+fan_outa = \text{gain} \times \sqrt{\frac{6}{\text{fan\_in} + \text{fan\_out}}}  
Also known as Glorot initialization.  Parameters 
 
tensor – an n-dimensional torch.Tensor
 
gain – an optional scaling factor    Examples >>> w = torch.empty(3, 5)
>>> nn.init.xavier_uniform_(w, gain=nn.init.calculate_gain('relu'))
torch.nn.init.zeros_(tensor) [source]
 
Fills the input Tensor with the scalar value 0.  Parameters 
tensor – an n-dimensional torch.Tensor   Examples >>> w = torch.empty(3, 5)
>>> nn.init.zeros_(w)
class torch.nn.InstanceNorm1d(num_features, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) [source]
 
Applies Instance Normalization over a 3D input (a mini-batch of 1D inputs with optional additional channel dimension) as described in the paper Instance Normalization: The Missing Ingredient for Fast Stylization.  y=x−E[x]Var[x]+ϵ∗γ+βy = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta 
The mean and standard-deviation are calculated per-dimension separately for each object in a mini-batch. γ\gamma  and β\beta  are learnable parameter vectors of size C (where C is the input size) if affine is True. The standard-deviation is calculated via the biased estimator, equivalent to torch.var(input, unbiased=False). By default, this layer uses instance statistics computed from input data in both training and evaluation modes. If track_running_stats is set to True, during training this layer keeps running estimates of its computed mean and variance, which are then used for normalization during evaluation. The running estimates are kept with a default momentum of 0.1.  Note This momentum argument is different from one used in optimizer classes and the conventional notion of momentum. Mathematically, the update rule for running statistics here is x^new=(1−momentum)×x^+momentum×xt\hat{x}_\text{new} = (1 - \text{momentum}) \times \hat{x} + \text{momentum} \times x_t , where x^\hat{x}  is the estimated statistic and xtx_t  is the new observed value.   Note InstanceNorm1d and LayerNorm are very similar, but have some subtle differences. InstanceNorm1d is applied on each channel of channeled data like multidimensional time series, but LayerNorm is usually applied on entire sample and often in NLP tasks. Additionally, LayerNorm applies elementwise affine transform, while InstanceNorm1d usually don’t apply affine transform.   Parameters 
 
num_features – CC  from an expected input of size (N,C,L)(N, C, L)  or LL  from input of size (N,L)(N, L) 
 
eps – a value added to the denominator for numerical stability. Default: 1e-5 
momentum – the value used for the running_mean and running_var computation. Default: 0.1 
affine – a boolean value that when set to True, this module has learnable affine parameters, initialized the same way as done for batch normalization. Default: False. 
track_running_stats – a boolean value that when set to True, this module tracks the running mean and variance, and when set to False, this module does not track such statistics and always uses batch statistics in both training and eval modes. Default: False
     Shape:

 Input: (N,C,L)(N, C, L) 
 Output: (N,C,L)(N, C, L)  (same shape as input)    Examples: >>> # Without Learnable Parameters
>>> m = nn.InstanceNorm1d(100)
>>> # With Learnable Parameters
>>> m = nn.InstanceNorm1d(100, affine=True)
>>> input = torch.randn(20, 100, 40)
>>> output = m(input)
class torch.nn.InstanceNorm2d(num_features, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) [source]
 
Applies Instance Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension) as described in the paper Instance Normalization: The Missing Ingredient for Fast Stylization.  y=x−E[x]Var[x]+ϵ∗γ+βy = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta 
The mean and standard-deviation are calculated per-dimension separately for each object in a mini-batch. γ\gamma  and β\beta  are learnable parameter vectors of size C (where C is the input size) if affine is True. The standard-deviation is calculated via the biased estimator, equivalent to torch.var(input, unbiased=False). By default, this layer uses instance statistics computed from input data in both training and evaluation modes. If track_running_stats is set to True, during training this layer keeps running estimates of its computed mean and variance, which are then used for normalization during evaluation. The running estimates are kept with a default momentum of 0.1.  Note This momentum argument is different from one used in optimizer classes and the conventional notion of momentum. Mathematically, the update rule for running statistics here is x^new=(1−momentum)×x^+momentum×xt\hat{x}_\text{new} = (1 - \text{momentum}) \times \hat{x} + \text{momentum} \times x_t , where x^\hat{x}  is the estimated statistic and xtx_t  is the new observed value.   Note InstanceNorm2d and LayerNorm are very similar, but have some subtle differences. InstanceNorm2d is applied on each channel of channeled data like RGB images, but LayerNorm is usually applied on entire sample and often in NLP tasks. Additionally, LayerNorm applies elementwise affine transform, while InstanceNorm2d usually don’t apply affine transform.   Parameters 
 
num_features – CC  from an expected input of size (N,C,H,W)(N, C, H, W) 
 
eps – a value added to the denominator for numerical stability. Default: 1e-5 
momentum – the value used for the running_mean and running_var computation. Default: 0.1 
affine – a boolean value that when set to True, this module has learnable affine parameters, initialized the same way as done for batch normalization. Default: False. 
track_running_stats – a boolean value that when set to True, this module tracks the running mean and variance, and when set to False, this module does not track such statistics and always uses batch statistics in both training and eval modes. Default: False
     Shape:

 Input: (N,C,H,W)(N, C, H, W) 
 Output: (N,C,H,W)(N, C, H, W)  (same shape as input)    Examples: >>> # Without Learnable Parameters
>>> m = nn.InstanceNorm2d(100)
>>> # With Learnable Parameters
>>> m = nn.InstanceNorm2d(100, affine=True)
>>> input = torch.randn(20, 100, 35, 45)
>>> output = m(input)
class torch.nn.InstanceNorm3d(num_features, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False) [source]
 
Applies Instance Normalization over a 5D input (a mini-batch of 3D inputs with additional channel dimension) as described in the paper Instance Normalization: The Missing Ingredient for Fast Stylization.  y=x−E[x]Var[x]+ϵ∗γ+βy = \frac{x - \mathrm{E}[x]}{ \sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta 
The mean and standard-deviation are calculated per-dimension separately for each object in a mini-batch. γ\gamma  and β\beta  are learnable parameter vectors of size C (where C is the input size) if affine is True. The standard-deviation is calculated via the biased estimator, equivalent to torch.var(input, unbiased=False). By default, this layer uses instance statistics computed from input data in both training and evaluation modes. If track_running_stats is set to True, during training this layer keeps running estimates of its computed mean and variance, which are then used for normalization during evaluation. The running estimates are kept with a default momentum of 0.1.  Note This momentum argument is different from one used in optimizer classes and the conventional notion of momentum. Mathematically, the update rule for running statistics here is x^new=(1−momentum)×x^+momentum×xt\hat{x}_\text{new} = (1 - \text{momentum}) \times \hat{x} + \text{momentum} \times x_t , where x^\hat{x}  is the estimated statistic and xtx_t  is the new observed value.   Note InstanceNorm3d and LayerNorm are very similar, but have some subtle differences. InstanceNorm3d is applied on each channel of channeled data like 3D models with RGB color, but LayerNorm is usually applied on entire sample and often in NLP tasks. Additionally, LayerNorm applies elementwise affine transform, while InstanceNorm3d usually don’t apply affine transform.   Parameters 
 
num_features – CC  from an expected input of size (N,C,D,H,W)(N, C, D, H, W) 
 
eps – a value added to the denominator for numerical stability. Default: 1e-5 
momentum – the value used for the running_mean and running_var computation. Default: 0.1 
affine – a boolean value that when set to True, this module has learnable affine parameters, initialized the same way as done for batch normalization. Default: False. 
track_running_stats – a boolean value that when set to True, this module tracks the running mean and variance, and when set to False, this module does not track such statistics and always uses batch statistics in both training and eval modes. Default: False
     Shape:

 Input: (N,C,D,H,W)(N, C, D, H, W) 
 Output: (N,C,D,H,W)(N, C, D, H, W)  (same shape as input)    Examples: >>> # Without Learnable Parameters
>>> m = nn.InstanceNorm3d(100)
>>> # With Learnable Parameters
>>> m = nn.InstanceNorm3d(100, affine=True)
>>> input = torch.randn(20, 100, 35, 45, 10)
>>> output = m(input)
class torch.nn.intrinsic.ConvBn1d(conv, bn) [source]
 
This is a sequential container which calls the Conv 1d and Batch Norm 1d modules. During quantization this will be replaced with the corresponding fused module.
class torch.nn.intrinsic.ConvBn2d(conv, bn) [source]
 
This is a sequential container which calls the Conv 2d and Batch Norm 2d modules. During quantization this will be replaced with the corresponding fused module.
class torch.nn.intrinsic.ConvBnReLU1d(conv, bn, relu) [source]
 
This is a sequential container which calls the Conv 1d, Batch Norm 1d, and ReLU modules. During quantization this will be replaced with the corresponding fused module.
class torch.nn.intrinsic.ConvBnReLU2d(conv, bn, relu) [source]
 
This is a sequential container which calls the Conv 2d, Batch Norm 2d, and ReLU modules. During quantization this will be replaced with the corresponding fused module.
class torch.nn.intrinsic.ConvReLU1d(conv, relu) [source]
 
This is a sequential container which calls the Conv1d and ReLU modules. During quantization this will be replaced with the corresponding fused module.
class torch.nn.intrinsic.ConvReLU2d(conv, relu) [source]
 
This is a sequential container which calls the Conv2d and ReLU modules. During quantization this will be replaced with the corresponding fused module.
class torch.nn.intrinsic.qat.ConvBn2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=None, padding_mode='zeros', eps=1e-05, momentum=0.1, freeze_bn=False, qconfig=None) [source]
 
A ConvBn2d module is a module fused from Conv2d and BatchNorm2d, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of torch.nn.Conv2d and torch.nn.BatchNorm2d. Similar to torch.nn.Conv2d, with FakeQuantize modules initialized to default.  Variables 
 
~ConvBn2d.freeze_bn –  
~ConvBn2d.weight_fake_quant – fake quant module for weight
class torch.nn.intrinsic.qat.ConvBnReLU2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=None, padding_mode='zeros', eps=1e-05, momentum=0.1, freeze_bn=False, qconfig=None) [source]
 
A ConvBnReLU2d module is a module fused from Conv2d, BatchNorm2d and ReLU, attached with FakeQuantize modules for weight, used in quantization aware training. We combined the interface of torch.nn.Conv2d and torch.nn.BatchNorm2d and torch.nn.ReLU. Similar to torch.nn.Conv2d, with FakeQuantize modules initialized to default.  Variables 
~ConvBnReLU2d.weight_fake_quant – fake quant module for weight