Repository: TheAlgorithms/Python
Pull request: Added Scaled Exponential Linear Unit Activation Function
Added Scaled Exponential Linear Unit Activation (SELU) Function under TheAlgorithms/Python/neural_network/activation_functions. Description of SELU taken from reference link provided in the top comment section in scaled_exponential_linear_unit.py
Fixes #9010 



* [X] Add an algorithm?

### Checklist:
* [X] I have read [CONTRIBUTING.md](https://github.com/TheAlgorithms/Python/blob/master/CONTRIBUTING.md).
* [X] This pull request is all my own work -- I have not plagiarized.
* [X] I know that pull requests will not be merged if they fail the automated tests.
* [X] This PR only changes one algorithm file.  To ease review, please open separate PRs for separate algorithms.
* [X] All new Python files are placed inside an existing directory.
* [X] All filenames are in all lowercase characters with no spaces or dashes.
* [X] All functions and variable names follow Python naming conventions.
* [X] All function parameters and return values are annotated with Python [type hints](https://docs.python.org/3/library/typing.html).
* [X] All functions have [doctests](https://docs.python.org/3/library/doctest.html) that pass the automated testing.
* [X] All new algorithms include at least one URL that points to Wikipedia or another similar explanation.
* [X] If this pull request resolves one or more open issues then the description above includes the issue number(s) with a [closing keyword](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue): "Fixes #ISSUE-NUMBER".


Problem statement:
Other Activation Functions
### Feature description

Hi, I'm relatively new to open source and found this repo friendly for early contributions. But is there a reason as to why there are only two activations (ELU and Leaky ReLU), under neural_network / activation_functions.? Also, can I contribute new activations into it?

Hints from issue discussion:
> But is there a reason as to why there are only two activations (ELU and Leaky ReLU), under neural_network / activation_functions.?

People simply haven't contributed any more activation functions. That being said, I think there are some activation functions currently placed under different directories (such as sigmoid in the maths directory)—they should be moved to the activation functions directory IMO, so feel free to open a PR to do so if you wish.

> Also, can I contribute new activations into it?

Yes, you can simply open a PR to do so. We don't assign issues in this repo, so there's no need to ask or open an issue about it.

Commit message: Update scaled_exponential_linear_unit.py

Patch to review:
diff --git a/neural_network/activation_functions/scaled_exponential_linear_unit.py b/neural_network/activation_functions/scaled_exponential_linear_unit.py
new file mode 100644
index 000000000000..a9f272d8f294
--- /dev/null
+++ b/neural_network/activation_functions/scaled_exponential_linear_unit.py
@@ -0,0 +1,44 @@
+"""
+Implements the Scaled Exponential Linear Unit or SELU function.
+The function takes a vector of K real numbers and two real numbers
+alpha(default = 1.6732) & lambda (default = 1.0507) as input and
+then applies the SELU function to each element of the vector.
+SELU is a self-normalizing activation function. It is a variant
+of the ELU. The main advantage of SELU is that we can be sure
+that the output will always be standardized due to its
+self-normalizing behavior. That means there is no need to
+include Batch-Normalization layers.
+References :
+https://iq.opengenus.org/scaled-exponential-linear-unit/
+"""
+
+import numpy as np
+
+
+def scaled_exponential_linear_unit(
+    vector: np.ndarray, alpha: float = 1.6732, _lambda: float = 1.0507
+) -> np.ndarray:
+    """
+    Applies the Scaled Exponential Linear Unit function to each element of the vector.
+    Parameters :
+        vector : np.ndarray
+        alpha : float (default = 1.6732)
+        _lambda : float (default = 1.0507)
+
+    Returns : np.ndarray
+    Formula : f(x) = _lambda * x if x > 0
+                     _lambda * alpha * (e**x - 1) if x <= 0
+    Examples :
+    >>> scaled_exponential_linear_unit(vector=np.array([1.3, 3.7, 2.4]))
+    array([1.36591, 3.88759, 2.52168])
+
+    >>> scaled_exponential_linear_unit(vector=np.array([1.3, 4.7, 8.2]))
+    array([1.36591, 4.93829, 8.61574])
+    """
+    return _lambda * np.where(vector > 0, vector, alpha * (np.exp(vector) - 1))
+
+
+if __name__ == "__main__":
+    import doctest
+
+    doctest.testmod()


Code review comments:
diff_hunk: @@ -0,0 +1,44 @@
+"""
+Implements the Scaled Exponential Linear Unit or SELU function.
+The function takes a vector of K real numbers and two real numbers
+alpha(default = 1.6732) & lambda (default = 1.0507) as input and
+then applies the SELU function to each element of the vector.
+SELU is a self-normalizing activation function. It is a variant
+of the ELU. The main advantage of SELU is that we can be sure
+that the output will always be standardized due to its
+self-normalizing behavior. That means there is no need to
+include Batch-Normalization layers.
+References :
+https://iq.opengenus.org/scaled-exponential-linear-unit/
+"""
+
+import numpy as np
+
+
+def scaled_exponential_linear_unit(
+    vector: np.ndarray, alpha: float = 1.6732, _lambda: float = 1.0507
path: neural_network/activation_functions/scaled_exponential_linear_unit.py
text: @user1:
Is there a reason why `_lambda` has an underscore? Is the user not meant to change this coefficient?

@author:
Yes, firstly lambda is a reserved keyword in python. Secondly both alpha and lambda are fixed constants.

@author:
The user can change these values, that may yield slightly different behaviour from the function, but the default values given to them are `alpha: float = 1.6732, _lambda: float = 1.0507`.

@user1:
> Yes, firstly lambda is a reserved keyword in python.

Oh yeah, duh 🤦

Could you rename the variable to something like `lambda_` instead? Having an underscore at the start of a variable name generally signifies that the user isn't supposed to use it.

@author:
Ah. Yes for sure, I'll get it done right away!

@author:
Done