Repository: OpenMined/PySyft

Issue:
Exponential implementation in SMPC
**Suggestion to implement exp in SMPC**

The interest of this is to allow computation of functions which derive of exponentials like sigmoid.

Let's say you share the tensor x in x0 and x1. You want to compute exp(x).
But exp(x) = exp(x0 + x1) = exp(x0) * exp(x1).
So you can compute locally exp(x0) and exp(x1), and **additively share** these values, perform an encrypted mul and then you obtain shares of exp(x0) * exp(x1) = exp(x).



Discussion:
Guess I will take this up :) 

@LaRiffle please notice that when doing operations in SMPC we are working with fixed_precision tensors. Exponential is not defined for fixed_precision tensors, we would need a polynomial approximation or other numerical methods that require several computations (including division which is really slow now)... 😞 
Why it won't work:
```python
x = torch.tensor([[3, 4]])
x_sh = x.share(alice, bob, crypto_provider=crypto_provider)

x0, x1 = x_sh.child.child['alice'], x_sh.child.child['bob']
x0 = x0.float()
x1 = x1.float()

exp_x0, exp_x1 = [torch.exp(x0), torch.exp(x1)]
print(alice._objects[exp_x0.id_at_location], bob._objects[exp_x1.id_at_location])
```
output
```python
(tensor([[inf, inf]]), tensor([[inf, inf]]))
```

Computers are so weak.
Ok so let's not give up here!

 ## Claim 1
> If I can compute privately exp of a bit then I'm good for any exp(integer)

Idea:
```python 
e = torch.exp(torch.tensor(1.))
# this...
e**5
# can be written
e**(1*2**2 + 0*2**1 + 1*2**0)
# or also
(e**1)**(2**2) * (e**0)**(2**1) * (e**1)**(2**0)
```

## Claim 2
> I can compute privately exp of a single bit 
```python
# Take a bit (0 or 1, here 0 for example)and share it in a *binary* field
x = torch.tensor([0])
x_sh = x.share(alice, bob, crypto_provider=crypto_provider, field=2)

# Access shares
x0, x1 = x_sh.child.child['alice'], x_sh.child.child['bob']
x0 = x0.float()
x1 = x1.float()
print(alice._objects[x0.id_at_location], bob._objects[x1.id_at_location])

# Compute privately the wrap field bit, which decrypts to 1 iff x0+x1 >= 2  
x0_sh = x0.fix_precision().share(alice, bob, crypto_provider=charlie).get()
x1_sh = x1.fix_precision().share(alice, bob, crypto_provider=charlie).get()
wrap_field = x0_sh * x1_sh 

# Compute exp of shares
exp_x0, exp_x1 = [torch.exp(x0), torch.exp(x1)]
alice._objects[exp_x0.id_at_location], bob._objects[exp_x1.id_at_location]

# Share the exp of shares
exp_x0_sh = exp_x0.fix_precision().share(alice, bob, crypto_provider=charlie).get()
exp_x1_sh = exp_x1.fix_precision().share(alice, bob, crypto_provider=charlie).get()

# Apply exp(x0 + x1) =  exp(x0) *  exp(x1) formula + a wrapping correction if needed
one = torch.tensor([1.]).fix_precision()
inv_exp_field_size = torch.exp(-torch.tensor([2.])).fix_precision()

exp_sh = exp_x0_sh * exp_x1_sh * (wrap_field * (inv_exp_field_size - one) + one)

# Open and get 1.0
exp_sh.get().float_prec()
```

Now, how practical is this is another question...
From my first implem, the price to pay should be around 1.5s for a float value, which can be amortized when using vectors
@LaRiffle I suggest you take a look in the [SCALE-MAMBA documentation](https://homes.esat.kuleuven.be/~nsmart/SCALE/Documentation.pdf). Note that some of those protocols might be slower versions that are used to support any `n`, and hence there may be better protocols for the two-party case.
@mortendahl we were discussing SCALE-MAMBA last week, @LaRiffle and I were thinking about trying to "hook" it for some MPC operations...
@andrelmfarias @LaRiffle not sure I understand what you mean by hooking here..?
We were thinking that maybe we could "hook" some operations from their github repository: https://github.com/KULeuven-COSIC/SCALE-MAMBA

To use their protocols without needing to implement them ourselves in PySyft.

I don't know if it's clear...

Resolving patch:
diff --git a/syft/frameworks/torch/tensors/interpreters/additive_shared.py b/syft/frameworks/torch/tensors/interpreters/additive_shared.py
--- a/syft/frameworks/torch/tensors/interpreters/additive_shared.py
+++ b/syft/frameworks/torch/tensors/interpreters/additive_shared.py
@@ -325,6 +325,9 @@ def add(self, shares: dict, other):
                 - a torch tensor
                 - a constant
         """
+        if isinstance(other, int):
+            other = torch.LongTensor([other])
+
         if isinstance(other, (torch.LongTensor, torch.IntTensor)):
             # if someone passes a torch tensor, we share it and keep the dict
             other = other.share(
@@ -356,10 +359,8 @@ def add(self, shares: dict, other):
 
         return new_shares
 
-    def __add__(self, other, **kwargs):
-        """Adds two tensors. Forwards command to add. See add() for more details."""
-
-        return self.add(other, **kwargs)
+    __add__ = add
+    __radd__ = add
 
     @overloaded.method
     def sub(self, shares: dict, other):
@@ -374,6 +375,9 @@ def sub(self, shares: dict, other):
                 - a constant
         """
 
+        if isinstance(other, int):
+            other = torch.LongTensor([other])
+
         if isinstance(other, (torch.LongTensor, torch.IntTensor)):
             # if someone passes a torch tensor, we share it and keep the dict
             other = other.share(
@@ -405,9 +409,10 @@ def sub(self, shares: dict, other):
 
         return new_shares
 
-    def __sub__(self, *args, **kwargs):
-        """Subtracts two tensors. Forwards command to sub. See .sub() for details."""
-        return self.sub(*args, **kwargs)
+    __sub__ = sub
+
+    def __rsub__(self, other):
+        return (self - other) * -1
 
     def _private_mul(self, other, equation: str):
         """Abstractly Multiplies two tensors
diff --git a/syft/frameworks/torch/tensors/interpreters/precision.py b/syft/frameworks/torch/tensors/interpreters/precision.py
--- a/syft/frameworks/torch/tensors/interpreters/precision.py
+++ b/syft/frameworks/torch/tensors/interpreters/precision.py
@@ -1,12 +1,12 @@
 import torch
 
 import syft
-from syft.workers.abstract import AbstractWorker
+from syft.frameworks.torch.tensors.interpreters.additive_shared import AdditiveSharingTensor
 from syft.generic.frameworks.hook import hook_args
+from syft.generic.frameworks.overload import overloaded
 from syft.generic.pointers.multi_pointer import MultiPointerTensor
 from syft.generic.tensor import AbstractTensor
-from syft.frameworks.torch.tensors.interpreters.additive_shared import AdditiveSharingTensor
-from syft.generic.frameworks.overload import overloaded
+from syft.workers.abstract import AbstractWorker
 
 
 class FixedPrecisionTensor(AbstractTensor):
@@ -142,8 +142,8 @@ def truncate(self, precision_fractional, check_sign=True):
     def add(self, _self, other):
         """Add two fixed precision tensors together.
         """
-        if isinstance(other, int):
-            scaled_int = other * self.base ** self.precision_fractional
+        if isinstance(other, (int, float)):
+            scaled_int = int(other * self.base ** self.precision_fractional)
             return getattr(_self, "add")(scaled_int)
 
         if isinstance(_self, AdditiveSharingTensor) and isinstance(other, torch.Tensor):
@@ -161,6 +161,7 @@ def add(self, _self, other):
         return response
 
     __add__ = add
+    __radd__ = add
 
     def add_(self, value_or_tensor, tensor=None):
         if tensor is None:
@@ -182,8 +183,8 @@ def __iadd__(self, other):
     def sub(self, _self, other):
         """Subtracts a fixed precision tensor from another one.
         """
-        if isinstance(other, int):
-            scaled_int = other * self.base ** self.precision_fractional
+        if isinstance(other, (int, float)):
+            scaled_int = int(other * self.base ** self.precision_fractional)
             return getattr(_self, "sub")(scaled_int)
 
         if isinstance(_self, AdditiveSharingTensor) and isinstance(other, torch.Tensor):
@@ -202,6 +203,9 @@ def sub(self, _self, other):
 
     __sub__ = sub
 
+    def __rsub__(self, other):
+        return (self - other) * -1
+
     def sub_(self, value_or_tensor, tensor=None):
         if tensor is None:
             result = self.sub(value_or_tensor)
@@ -225,7 +229,7 @@ def t(self, _self, *args, **kwargs):
 
     def mul_and_div(self, other, cmd):
         """
-        Hook manually mul and div to add the trucation/rescaling part
+        Hook manually mul and div to add the truncation/rescaling part
         which is inherent to these operations in the fixed precision setting
         """
         changed_sign = False
@@ -238,6 +242,10 @@ def mul_and_div(self, other, cmd):
         if isinstance(other, (int, torch.Tensor, AdditiveSharingTensor)):
             new_self = self.child
             new_other = other
+        elif isinstance(other, float):
+            raise NotImplementedError(
+                "Can't multiply or divide a FixedPrecisionTensor with a float value"
+            )
 
         elif isinstance(self.child, (AdditiveSharingTensor, MultiPointerTensor)) and isinstance(
             other.child, torch.Tensor
@@ -374,14 +382,17 @@ def pow(self, power):
         This uses the following trick:
          - Divide power by 2 and multiply base to itself (if the power is even)
          - Decrement power by 1 to make it even and then follow the first step
+
+        Args:
+            power (int): the exponent supposed to be an integer > 0
         """
         base = self
 
-        result = 1
+        result = None
         while power > 0:
             # If power is odd
             if power % 2 == 1:
-                result = result * base
+                result = result * base if result is not None else base
 
             # Divide the power by 2
             power = power // 2
@@ -443,6 +454,109 @@ def matmul(self, *args, **kwargs):
     __matmul__ = matmul
     mm = matmul
 
+    # Approximations:
+    def inverse(self, iterations=8):
+        """
+        Computes an approximation of the matrix inversion using Newton-Schulz
+        iterations
+        """
+        # TODO: should we add non-approximate version if self.child is a pure tensor?
+
+        assert len(self.shape) >= 2, "Can't compute inverse on non-matrix"
+        assert self.shape[-1] == self.shape[-2], "Must be batches of square matrices"
+
+        inverse = (0.1 * torch.eye(self.shape[-1])).fix_prec(**self.get_class_attributes()).child
+
+        for _ in range(iterations):
+            inverse = 2 * inverse - inverse @ self @ inverse
+
+        return inverse
+
+    def exp(self, iterations=8):
+        """
+        Approximates the exponential function using a limit approximation:
+        exp(x) = \lim_{n -> infty} (1 + x / n) ^ n
+
+        Here we compute exp by choosing n = 2 ** d for some large d equal to
+        iterations. We then compute (1 + x / n) once and square `d` times.
+
+        Args:
+            iterations (int): number of iterations for limit approximation
+
+        Ref: https://github.com/LaRiffle/approximate-models
+        """
+        return (1 + self / 2 ** iterations) ** (2 ** iterations)
+
+    def sigmoid(self, method="exp"):
+        """
+        Approximates the sigmoid function
+
+        Args:
+            self: the fixed precision tensor
+            method (str): (default = "exp")
+                "exp": Use the exponential approximation and the sigmoid definition
+                    sigmoid(x) = 1 / (1 + exp(-x))
+                "maclaurin": Use the Maclaurin / Taylor approximation, with polynomial
+                    interpolation of degree 5 over [-8,8]
+                    NOTE: This method is faster but not as precise as "exp"
+                    Ref: https://mortendahl.github.io/2017/04/17/private-deep-learning-with-mpc/#approximating-sigmoid
+        """
+
+        if method == "exp":
+            # Inverse can only be used on matrices
+            if len(self.shape) == 1:
+                one = self * 0 + 1
+                result = one / (1 + (self * -1).exp())
+            else:
+                result = (1 + (self * -1).exp()).inverse()
+
+        elif method == "maclaurin":
+            weights = (
+                torch.tensor([0.5, 1.91204779e-01, -4.58667307e-03, 4.20690803e-05])
+                .fix_precision(**self.get_class_attributes())
+                .child
+            )
+            degrees = [0, 1, 3, 5]
+
+            # initiate with term of degree 0 to avoid errors with tensor ** 0
+            one = self * 0 + 1
+            result = one * weights[0]
+            for i, d in enumerate(degrees[1:]):
+                result += (self ** d) * weights[i + 1]
+
+        return result
+
+    def log(self, iterations=2, exp_iterations=8):
+        """Approximates the natural logarithm using 8th order modified Householder iterations.
+        Recall that Householder method is an algorithm to solve a non linear equation f(x) = 0.
+        Here  f: x -> 1 - C * exp(-x)  with C = self
+
+        Iterations are computed by:
+            y_0 = some constant
+            h = 1 - self * exp(-y_n)
+            y_{n+1} = y_n - h * (1 + h / 2 + h^2 / 3 + h^3 / 6 + h^4 / 5 + h^5 / 7)
+
+        Args:
+            iterations (int): number of iterations for 6th order modified
+                Householder approximation.
+            exp_iterations (int): number of iterations for limit approximation of exp
+
+        Ref: https://github.com/LaRiffle/approximate-models
+        """
+
+        y = self / 31 + 1.59 - 20 * (-2 * self - 1.4).exp(iterations=exp_iterations)
+
+        # 6th order Householder iterations
+        for i in range(iterations):
+            h = [1 - self * (-y).refresh().exp(iterations=exp_iterations)]
+            for i in range(1, 5):
+                h.append(h[-1] * h[0])
+
+            y -= h[0] * (1 + h[0] / 2 + h[1] / 3 + h[2] / 4 + h[3] / 5 + h[4] / 6)
+
+        return y
+
+    # Binary ops
     @overloaded.method
     def __gt__(self, _self, other):
         result = _self.__gt__(other)
@@ -506,28 +620,26 @@ def addmm(bias, input_tensor, weight):
 
         module.addmm = addmm
 
-        def sigmoid(tensor):
-            """
-            Overloads torch.sigmoid to be able to use MPC
-            Approximation with polynomial interpolation of degree 5 over [-8,8]
-            Ref: https://mortendahl.github.io/2017/04/17/private-deep-learning-with-mpc/#approximating-sigmoid
-            """
+        def inverse(self):
+            return self.inverse()
 
-            weights = [0.5, 1.91204779e-01, -4.58667307e-03, 4.20690803e-05]
-            degrees = [0, 1, 3, 5]
+        module.inverse = inverse
 
-            max_degree = degrees[-1]
-            max_idx = degrees.index(max_degree)
+        def exp(tensor):
+            return tensor.exp()
 
-            # initiate with term of degree 0 to avoid errors with tensor ** 0
-            result = (tensor * 0 + 1) * torch.tensor(weights[0]).fix_precision().child
-            for w, d in zip(weights[1:max_idx], degrees[1:max_idx]):
-                result += (tensor ** d) * torch.tensor(w).fix_precision().child
+        module.exp = exp
 
-            return result
+        def sigmoid(tensor):
+            return tensor.sigmoid()
 
         module.sigmoid = sigmoid
 
+        def log(tensor):
+            return tensor.log()
+
+        module.log = log
+
         def tanh(tensor):
             """
             Overloads torch.tanh to be able to use MPC


Test patch:
diff --git a/test/efficiency_tests/assertions.py b/test/efficiency_tests/assertions.py
--- a/test/efficiency_tests/assertions.py
+++ b/test/efficiency_tests/assertions.py
@@ -16,7 +16,7 @@ def wrapper(*args, **kwargs):
             t0 = time.time()
             func(*args, **kwargs)
             dt = time.time() - t0
-            assert dt < max_time
+            assert dt < max_time, f"Test run in {round(dt, 2)} > {round(max_time, 2)} s"
 
         return wrapper
 
diff --git a/test/efficiency_tests/test_activations_time.py b/test/efficiency_tests/test_activations_time.py
--- a/test/efficiency_tests/test_activations_time.py
+++ b/test/efficiency_tests/test_activations_time.py
@@ -5,7 +5,7 @@
 
 
 @pytest.mark.parametrize("activation", ["tanh", "sigmoid"])
-@assert_time(max_time=1)
+@assert_time(max_time=10)
 def test_activation(activation, hook, workers):
 
     activation_func = torch.tanh if activation == "tanh" else torch.sigmoid
diff --git a/test/torch/tensors/test_additive_shared.py b/test/torch/tensors/test_additive_shared.py
--- a/test/torch/tensors/test_additive_shared.py
+++ b/test/torch/tensors/test_additive_shared.py
@@ -169,6 +169,28 @@ def test_add(workers):
 
     assert (z == (t + t)).all()
 
+    # with constant integer
+    t = torch.tensor([1.0, -2.0, 3.0])
+    x = t.fix_prec().share(alice, bob, crypto_provider=james)
+    c = 4
+
+    z = (x + c).get().float_prec()
+    assert (z == (t + c)).all()
+
+    z = (c + x).get().float_prec()
+    assert (z == (c + t)).all()
+
+    # with constant float
+    t = torch.tensor([1.0, -2.0, 3.0])
+    x = t.fix_prec().share(alice, bob, crypto_provider=james)
+    c = 4.2
+
+    z = (x + c).get().float_prec()
+    assert ((z - (t + c)) < 10e-3).all()
+
+    z = (c + x).get().float_prec()
+    assert ((z - (c + t)) < 10e-3).all()
+
 
 def test_sub(workers):
     bob, alice, james = (workers["bob"], workers["alice"], workers["james"])
@@ -211,6 +233,28 @@ def test_sub(workers):
 
     assert (z == (u - t)).all()
 
+    # with constant integer
+    t = torch.tensor([1.0, -2.0, 3.0])
+    x = t.fix_prec().share(alice, bob, crypto_provider=james)
+    c = 4
+
+    z = (x - c).get().float_prec()
+    assert (z == (t - c)).all()
+
+    z = (c - x).get().float_prec()
+    assert (z == (c - t)).all()
+
+    # with constant float
+    t = torch.tensor([1.0, -2.0, 3.0])
+    x = t.fix_prec().share(alice, bob, crypto_provider=james)
+    c = 4.2
+
+    z = (x - c).get().float_prec()
+    assert ((z - (t - c)) < 10e-3).all()
+
+    z = (c - x).get().float_prec()
+    assert ((z - (c - t)) < 10e-3).all()
+
 
 def test_mul(workers):
     torch.manual_seed(121)  # Truncation might not always work so we set the random seed
diff --git a/test/torch/tensors/test_precision.py b/test/torch/tensors/test_precision.py
--- a/test/torch/tensors/test_precision.py
+++ b/test/torch/tensors/test_precision.py
@@ -3,6 +3,7 @@
 import torch.nn as nn
 import torch.nn.functional as F
 
+from test.efficiency_tests.assertions import assert_time
 from syft.frameworks.torch.tensors.interpreters.precision import FixedPrecisionTensor
 
 
@@ -59,19 +60,6 @@ def test_fix_prec_inplace_registration(hook):
         assert hook.local_worker.get_obj(x.id) == torch.tensor([1.0]).fix_precision()
 
 
-def test_add_method():
-
-    t = torch.tensor([0.1, 0.2, 0.3])
-    x = t.fix_prec()
-
-    y = x + x
-
-    assert (y.child.child == torch.LongTensor([200, 400, 600])).all()
-    y = y.float_prec()
-
-    assert (y == t + t).all()
-
-
 @pytest.mark.parametrize("method", ["t", "matmul"])
 @pytest.mark.parametrize("parameter", [False, True])
 def test_methods_for_linear_module(method, parameter):
@@ -96,6 +84,17 @@ def test_methods_for_linear_module(method, parameter):
 def test_torch_add(workers):
     bob, alice, james = (workers["bob"], workers["alice"], workers["james"])
 
+    # Method syntax
+    x = torch.tensor([0.1, 0.2, 0.3]).fix_prec()
+
+    y = x + x
+
+    assert (y.child.child == torch.LongTensor([200, 400, 600])).all()
+    y = y.float_prec()
+
+    assert (y == torch.tensor([0.2, 0.4, 0.6])).all()
+
+    # Function syntax
     x = torch.tensor([0.1, 0.2, 0.3]).fix_prec()
 
     y = torch.add(x, x)
@@ -120,15 +119,39 @@ def test_torch_add(workers):
 
     assert (y == torch.tensor([40.0, -20.0, 20.0])).all()
 
-    # with AST
+    # with AdditiveSharingTensor
     t = torch.tensor([1.0, -2.0, 3.0])
     x = t.fix_prec()
     y = t.fix_prec().share(bob, alice, crypto_provider=james)
 
-    z = torch.add(y, x).get().float_prec()
+    z = torch.add(x, y).get().float_prec()
+    assert (z == torch.add(t, t)).all()
 
+    z = torch.add(y, x).get().float_prec()
     assert (z == torch.add(t, t)).all()
 
+    # with constant integer
+    t = torch.tensor([1.0, -2.0, 3.0])
+    x = t.fix_prec()
+    c = 4
+
+    z = (x + c).float_prec()
+    assert (z == (t + c)).all()
+
+    z = (c + x).float_prec()
+    assert (z == (c + t)).all()
+
+    # with constant float
+    t = torch.tensor([1.0, -2.0, 3.0])
+    x = t.fix_prec()
+    c = 4.2
+
+    z = (x + c).float_prec()
+    assert ((z - (t + c)) < 10e-3).all()
+
+    z = (c + x).float_prec()
+    assert ((z - (c + t)) < 10e-3).all()
+
 
 def test_torch_add_():
     x = torch.tensor([0.1, 0.2, 0.3]).fix_prec()
@@ -151,27 +174,6 @@ def test_torch_add_():
     assert (y == torch.tensor([0.15, 0.3, 0.45])).all()
 
 
-def test_torch_sub_():
-    x = torch.tensor([0.1, 0.2, 0.3]).fix_prec()
-
-    y = x.sub_(x)
-
-    assert (y.child.child == torch.LongTensor([0, 0, 0])).all()
-    y = y.float_prec()
-
-    assert (y == torch.tensor([0, 0, 0.0])).all()
-
-    x = torch.tensor([0.1, 0.2, 0.3]).fix_prec()
-    lr = torch.tensor(0.5).fix_prec()
-
-    y = x.sub_(lr, x)
-
-    assert (y.child.child == torch.LongTensor([50, 100, 150])).all()
-    y = y.float_prec()
-
-    assert (y == torch.tensor([0.05, 0.1, 0.15])).all()
-
-
 def test_torch_sub(workers):
     bob, alice, james = (workers["bob"], workers["alice"], workers["james"])
 
@@ -185,7 +187,7 @@ def test_torch_sub(workers):
 
     assert (z == torch.tensor([0.4, 0.6, 1.0])).all()
 
-    # with AST
+    # with AdditiveSharingTensor
     tx = torch.tensor([1.0, -2.0, 3.0])
     ty = torch.tensor([0.1, 0.2, 0.3])
     x = tx.fix_prec()
@@ -197,6 +199,49 @@ def test_torch_sub(workers):
     assert (z1 == torch.sub(ty, tx)).all()
     assert (z2 == torch.sub(tx, ty)).all()
 
+    # with constant integer
+    t = torch.tensor([1.0, -2.0, 3.0])
+    x = t.fix_prec()
+    c = 4
+
+    z = (x - c).float_prec()
+    assert (z == (t - c)).all()
+
+    z = (c - x).float_prec()
+    assert (z == (c - t)).all()
+
+    # with constant float
+    t = torch.tensor([1.0, -2.0, 3.0])
+    x = t.fix_prec()
+    c = 4.2
+
+    z = (x - c).float_prec()
+    assert ((z - (t - c)) < 10e-3).all()
+
+    z = (c - x).float_prec()
+    assert ((z - (c - t)) < 10e-3).all()
+
+
+def test_torch_sub_():
+    x = torch.tensor([0.1, 0.2, 0.3]).fix_prec()
+
+    y = x.sub_(x)
+
+    assert (y.child.child == torch.LongTensor([0, 0, 0])).all()
+    y = y.float_prec()
+
+    assert (y == torch.tensor([0, 0, 0.0])).all()
+
+    x = torch.tensor([0.1, 0.2, 0.3]).fix_prec()
+    lr = torch.tensor(0.5).fix_prec()
+
+    y = x.sub_(lr, x)
+
+    assert (y.child.child == torch.LongTensor([50, 100, 150])).all()
+    y = y.float_prec()
+
+    assert (y == torch.tensor([0.05, 0.1, 0.15])).all()
+
 
 def test_torch_mul(workers):
     bob, alice, james = (workers["bob"], workers["alice"], workers["james"])
@@ -343,6 +388,117 @@ def test_torch_dot(workers):
     assert torch.dot(x, y).float_prec() == 45
 
 
+@assert_time(max_time=6)
+def test_torch_inverse_approx(workers):
+    """
+    Test the approximate inverse with different tolerance depending on
+    the precision_fractional considered
+    """
+    alice, bob, james = workers["alice"], workers["bob"], workers["james"]
+
+    fix_prec_tolerance = {3: 1 / 100, 4: 1 / 100, 5: 1 / 100}
+
+    for prec_frac, tolerance in fix_prec_tolerance.items():
+        for t in [
+            torch.tensor([[0.4, -0.1], [-0.4, 2.0]]),
+            torch.tensor([[1, -0.6], [0.4, 4.0]]),
+            torch.tensor([[1, 0.2], [0.4, 4.0]]),
+        ]:
+            t_sh = t.fix_precision(precision_fractional=prec_frac).share(
+                alice, bob, crypto_provider=james
+            )
+            r_sh = t_sh.inverse()
+            r = r_sh.get().float_prec()
+            t = t.inverse()
+            diff = (r - t).abs().max()
+            norm = (r + t).abs().max() / 2
+
+            assert (diff / (tolerance * norm)) < 1
+
+
+@assert_time(max_time=10)
+def test_torch_exp_approx(workers):
+    """
+    Test the approximate exponential with different tolerance depending on
+    the precision_fractional considered
+    """
+    alice, bob, james = workers["alice"], workers["bob"], workers["james"]
+
+    fix_prec_tolerance = {3: 20 / 100, 4: 5 / 100, 5: 5 / 100}
+
+    for prec_frac, tolerance in fix_prec_tolerance.items():
+        cumsum = torch.zeros(5)
+        for i in range(10):
+            t = torch.tensor([0.0, 1, 2, 3, 4])
+            t_sh = t.fix_precision(precision_fractional=prec_frac).share(
+                alice, bob, crypto_provider=james
+            )
+            r_sh = t_sh.exp()
+            r = r_sh.get().float_prec()
+            t = t.exp()
+            diff = (r - t).abs()
+            norm = (r + t) / 2
+            cumsum += diff / (tolerance * norm)
+
+        cumsum /= 10
+        assert (cumsum < 1).all()
+
+
+@assert_time(max_time=40)
+def test_torch_sigmoid_approx(workers):
+    """
+    Test the approximate sigmoid with different tolerance depending on
+    the precision_fractional considered
+    """
+    alice, bob, james = workers["alice"], workers["bob"], workers["james"]
+
+    fix_prec_tolerance_by_method = {
+        "exp": {3: 5 / 100, 4: 1 / 100, 5: 1 / 100},
+        "maclaurin": {3: 7 / 100, 4: 15 / 100, 5: 15 / 100},
+    }
+
+    for method, fix_prec_tolerance in fix_prec_tolerance_by_method.items():
+        for prec_frac, tolerance in fix_prec_tolerance.items():
+            t = torch.tensor(range(-10, 10)) * 0.5
+            t_sh = t.fix_precision(precision_fractional=prec_frac).share(
+                alice, bob, crypto_provider=james
+            )
+            r_sh = t_sh.sigmoid(method=method)
+            r = r_sh.get().float_prec()
+            t = t.sigmoid()
+            diff = (r - t).abs().max()
+            norm = (r + t).abs().max() / 2
+
+            assert (diff / (tolerance * norm)) < 1
+
+
+@assert_time(max_time=45)
+def test_torch_log_approx(workers):
+    """
+    Test the approximate logarithm with different tolerance depending on
+    the precision_fractional considered
+    """
+    alice, bob, james = workers["alice"], workers["bob"], workers["james"]
+    fix_prec_tolerance = {3: 100 / 100, 4: 3 / 100, 5: 2 / 100}
+
+    for prec_frac, tolerance in fix_prec_tolerance.items():
+        cumsum = torch.zeros(9)
+        for i in range(10):
+            t = torch.tensor([0.1, 0.5, 2, 5, 10, 20, 50, 100, 250])
+            t_sh = t.fix_precision(precision_fractional=prec_frac).share(
+                alice, bob, crypto_provider=james
+            )
+            r_sh = t_sh.log()
+            r = r_sh.get().float_prec()
+            t = t.log()
+            diff = (r - t).abs()
+            norm = (r + t) / 2
+            cumsum += diff / (tolerance * norm)
+
+        cumsum /= 10
+        assert (cumsum.abs() < 1).all()
+
+
 def test_torch_conv2d(workers):
     bob, alice, james = (workers["bob"], workers["alice"], workers["james"])
     im = torch.Tensor(