Python | Tensorflow nn.softplus()

06 Jan, 2022
Tensorflow is an open-source machine learning library developed by Google. One of its applications is to develop deep neural networks.
The module tensorflow.nn provides support for many basic neural network operations.
An activation function is a function which is applied to the output of a neural network layer, which is then passed as the input to the next layer. Activation functions are an essential part of neural networks as they provide non-linearity, without which the neural network reduces to a mere logistic regression model. One of the many activation functions is the Softplus function which is defined as .
Traditional activation functions such as sigmoid and hyperbolic tangent have lower and upper bounds, whereas the softplus function outputs in the range (0, ∞). The derivative of the softplus function comes out to be , which is the sigmoid function. The softplus function is quite similar to the Rectified Linear Unit (ReLU) function, with the main difference being softplus function’ differentiability at the x = 0. The research paper “Improving deep neural networks using softplus units” by Zheng et al. (2015) suggests that softplus provides more stabilization and performance to deep neural networks than ReLU function. However, ReLU is generally preferred because of the ease in calculating it and its derivative. Calculation of activation function and its derivative is a frequent operation in neural networks, and ReLU provides faster forward and backward propagation when compared with softplus function.
The function nn.softplus() [alias math.softplus] provides support for softplus in Tensorflow.
Syntax: tf.nn.softplus(features, name=None) or tf.math.softplus(features, name=None)
Parameters:features: A tensor of any of the following types: float32, float64, int32, uint8, int16, int8, int64, bfloat16, uint16, half, uint32, uint64.name (optional): The name for the operation.
Return type: A tensor with the same type as that of features.
Code #1:
Python3
# Importing the Tensorflow libraryimport tensorflow as tf  # A constant vector of size 6a = tf.constant([1.0, -0.5, 3.4, -2.1, 0.0, -6.5], dtype = tf.float32)  # Applying the softplus function and# storing the result in 'b'b = tf.nn.softplus(a, name ='softplus')  # Initiating a Tensorflow sessionwith tf.Session() as sess:    print('Input type:', a)    print('Input:', sess.run(a))    print('Return type:', b)    print('Output:', sess.run(b))
Output:
Input type: Tensor("Const:0", shape=(6, ), dtype=float32)
Input: [ 1.        -0.5        3.4000001 -2.0999999  0.        -6.5      ]
Return type: Tensor("softplus:0", shape=(6, ), dtype=float32)
Output: [  1.31326163e+00   4.74076986e-01   3.43282866e+00   1.15519524e-01
   6.93147182e-01   1.50233845e-03]

 
Code #2: Visualization
Python3
# Importing the Tensorflow libraryimport tensorflow as tf  # Importing the NumPy libraryimport numpy as np  # Importing the matplotlib.pyplot functionimport matplotlib.pyplot as plt  # A vector of size 15 with values from -5 to 5a = np.linspace(-5, 5, 15)  # Applying the softplus function and# storing the result in 'b'b = tf.nn.softplus(a, name ='softplus')  # Initiating a Tensorflow sessionwith tf.Session() as sess:    print('Input:', a)    print('Output:', sess.run(b))    plt.plot(a, sess.run(b), color = 'red', marker = "o")     plt.title("tensorflow.nn.softplus")     plt.xlabel("X")     plt.ylabel("Y")       plt.show()
Output:
Input: [-5.         -4.28571429 -3.57142857 -2.85714286 -2.14285714 -1.42857143
 -0.71428571  0.          0.71428571  1.42857143  2.14285714  2.85714286
  3.57142857  4.28571429  5.        ]
Output: [ 0.00671535  0.01366993  0.02772767  0.05584391  0.11093221  0.21482992
  0.39846846  0.69314718  1.11275418  1.64340135  2.25378936  2.91298677
  3.59915624  4.29938421  5.00671535]

sanskar27jain
sagar0719kumar
Python-Library
Tensorflow
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Python Dictionary
Different ways to create Pandas Dataframe
Enumerate() in Python
Read a file line by line in Python
Python String | replace()
How to Install PIP on Windows ?
*args and **kwargs in Python
Python Classes and Objects
Iterate over a list in Python
Python OOPs Concepts