Repository: OpenMined/PySyft
Pull request: Ability to seed the `IdProvider`, Verbosity Toggling from `TorchHook`
## Features
## Seed the IdProvider
During most of my debugging sessions (especially the GC ones), the need to trace the course of specific tensors was very important. And the way to do that was through their IDs as that is basically the identity of any syft object. 

These changes give an ability to seed the `IdProvider` in the following ways
 ```python
sy.ID_PROVIDER.seed()
sy.manual_seed()
hook = sy.TorchHook(th, seed=0)
```

## Set Verbosity from Hook
Also, the ability to toggle the verbosity of all `VirtualWorkers` through the hook is more comfortable rather toggling each worker individually. It can be done this way now.
 ```python
hook = sy.TorchHook(th, verbose=True)
```

## Toggle Verbosity Using Hook
Toggle the verbosity at will! This is super useful while debugging a specific line
eg. 
```python
a = torch.ones(1, 5)
b = torch.ones(1, 5)

hook.set_verbose(True)
a = a.encrypt(workers=[alice, bob], crypto_provider=crypto_provider)
hook.set_verbose(False)

b = b.encrypt(workers=[alice, bob], crypto_provider=crypto_provider)
print(f"Output: {(a*b).decrypt()}")
```
Output:
```
worker <VirtualWorker id:me #objects:0> sending (ObjectMessage tensor([[...]])) to <VirtualWorker id:alice #objects:6>
worker <VirtualWorker id:alice #objects:6> received ObjectMessage
worker <VirtualWorker id:me #objects:0> sending (ObjectMessage tensor([[...]])) to <VirtualWorker id:bob #objects:6>
worker <VirtualWorker id:bob #objects:6> received ObjectMessage

Output: tensor([[1., 1., 1., 1., 1.]])
```
Something like this can be included in the beginner tutorials about the VirutalWorkers to better understand what happens under the hood. :tada: 

## Type of change

Please mark the options that are relevant.

- [ ] Added/Modified tutorials
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

## Checklist:

* [ ] I have added tests for my changes
* [x] I did follow the [contribution guidelines](https://github.com/OpenMined/PySyft/blob/master/CONTRIBUTING.md)
* [ ] I have commented my code following [Google style](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html).


Problem statement:
matmul in AST error when verbose=True
**Describe the bug**
Matrix multiplication of AST throws an error when workers are verbose

**To Reproduce**

```
import torch
import syft as sy
hook = sy.TorchHook(torch)

alice = sy.VirtualWorker(hook=hook, id = "alice", verbose=True)
bob = sy.VirtualWorker(hook=hook, id = "bob", verbose=True)
james = sy.VirtualWorker(hook=hook, id = "james")


m = torch.tensor([[1, 2], [3, 4.0]])
x = m.fix_prec().share(bob, alice, crypto_provider=james)
y = (x @ x).get().float_prec()
```


**Traceback**

```
Traceback (most recent call last):
  File "/home/juanmixp/Documentos/PySyft/fii.py", line 12, in <module>
    y = (x @ x).get().float_prec()
  File "/home/juanmixp/Documentos/PySyft/syft/generic/frameworks/hook/hook.py", line 466, in overloaded_native_method
    response = method(*new_args, **new_kwargs)
  File "/home/juanmixp/Documentos/PySyft/syft/frameworks/torch/tensors/interpreters/precision.py", line 432, in matmul
    response = getattr(new_self, "matmul")(*new_args, **new_kwargs)
  File "/home/juanmixp/Documentos/PySyft/syft/frameworks/torch/tensors/interpreters/additive_shared.py", line 646, in matmul
    return self._private_mul(other, "matmul")
  File "/home/juanmixp/Documentos/PySyft/syft/frameworks/torch/tensors/interpreters/additive_shared.py", line 536, in _private_mul
    shares = spdz.spdz_mul(cmd, self, other, self.crypto_provider, self.field, self.dtype)
  File "/home/juanmixp/Documentos/PySyft/syft/frameworks/torch/mpc/spdz.py", line 40, in spdz_mul
    delta = delta.reconstruct()
  File "/home/juanmixp/Documentos/PySyft/syft/frameworks/torch/tensors/interpreters/additive_shared.py", line 321, in reconstruct
    pointer = ptr_to_sh.remote_get()
  File "/home/juanmixp/Documentos/PySyft/syft/generic/pointers/pointer_tensor.py", line 304, in remote_get
    self.owner.send_command(message=("mid_get", self, (), {}), recipient=self.location)
  File "/home/juanmixp/Documentos/PySyft/syft/workers/base.py", line 637, in send_command
    ret_val = self.send_msg(message, location=recipient)
  File "/home/juanmixp/Documentos/PySyft/syft/workers/base.py", line 290, in send_msg
    bin_response = self._send_msg(bin_message, location)
  File "/home/juanmixp/Documentos/PySyft/syft/workers/virtual.py", line 15, in _send_msg
    return location._recv_msg(message)
  File "/home/juanmixp/Documentos/PySyft/syft/workers/virtual.py", line 19, in _recv_msg
    return self.recv_msg(message)
  File "/home/juanmixp/Documentos/PySyft/syft/workers/base.py", line 326, in recv_msg
    response = self._message_router[type(msg)](msg)
  File "/home/juanmixp/Documentos/PySyft/syft/workers/base.py", line 468, in execute_tensor_command
    return self.execute_computation_action(cmd.action)
  File "/home/juanmixp/Documentos/PySyft/syft/workers/base.py", line 510, in execute_computation_action
    response = getattr(_self, op_name)(*args_, **kwargs_)
  File "/home/juanmixp/Documentos/PySyft/syft/frameworks/torch/tensors/interpreters/native.py", line 604, in mid_get
    self.child.mid_get()
  File "/home/juanmixp/Documentos/PySyft/syft/generic/object.py", line 112, in mid_get
    tensor = self.get()
  File "/home/juanmixp/Documentos/PySyft/syft/frameworks/torch/tensors/interpreters/additive_shared.py", line 240, in get
    shares.append(share.get())
  File "/home/juanmixp/Documentos/PySyft/syft/generic/pointers/pointer_tensor.py", line 334, in get
    tensor = ObjectPointer.get(self, user=user, reason=reason, deregister_ptr=deregister_ptr)
  File "/home/juanmixp/Documentos/PySyft/syft/generic/pointers/object_pointer.py", line 269, in get
    obj = self.owner.request_obj(self.id_at_location, self.location, user, reason)
  File "/home/juanmixp/Documentos/PySyft/syft/workers/base.py", line 750, in request_obj
    obj = self.send_msg(ObjectRequestMessage(obj_id, user, reason), location)
  File "/home/juanmixp/Documentos/PySyft/syft/workers/base.py", line 284, in send_msg
    print(f"worker {self} sending {message} to {location}")
  File "/home/juanmixp/Documentos/PySyft/syft/messaging/message.py", line 303, in __str__
    return f"({type(self).__name__} {(self.obj_id, self.user, self.reason)})"
AttributeError: 'ObjectRequestMessage' object has no attribute 'obj_id'
```

Hints from issue discussion:

Commit message: `_virtual_workers` as set
Verbose condition corrected
Fix obj_id missing but called

Patch to review:
diff --git a/syft/__init__.py b/syft/__init__.py
index 5572d8213f7..e91b8e1acdd 100644
--- a/syft/__init__.py
+++ b/syft/__init__.py
@@ -131,3 +131,7 @@ def pool():
     from syft.generic.id_provider import IdProvider
 
     ID_PROVIDER = IdProvider()
+
+
+def manual_seed(seed=0):
+    syft.ID_PROVIDER.seed(seed)
diff --git a/syft/frameworks/torch/hook/hook.py b/syft/frameworks/torch/hook/hook.py
index 71f1368e349..26ab0e26449 100644
--- a/syft/frameworks/torch/hook/hook.py
+++ b/syft/frameworks/torch/hook/hook.py
@@ -85,9 +85,15 @@ class TorchHook(FrameworkHook):
     """
 
     def __init__(
-        self, torch, local_worker: BaseWorker = None, is_client: bool = True, verbose: bool = True
+        self,
+        torch,
+        local_worker: BaseWorker = None,
+        is_client: bool = True,
+        verbose: bool = False,
+        seed=None,
     ):
-        """Initializes the hook.
+        """
+        Initializes the hook.
 
         Initialize the hook and define all the attributes pertaining to the
         torch hook in a special TorchAttibute class, that will be added in the
@@ -97,6 +103,9 @@ def __init__(
         # Save the provided torch module as an attribute of the hook
         self.torch = torch
         self.framework = self.torch
+        if seed is not None:
+            syft.manual_seed(seed)
+        self.verbose = verbose
 
         # Save the local worker as an attribute
         self.local_worker = local_worker
@@ -122,10 +131,14 @@ def __init__(
             # be agnostic to the means by which workers communicate (such as
             # peer-to-peer, sockets, through local ports, or all within the
             # same process)
-            self.local_worker = VirtualWorker(hook=self, is_client_worker=is_client, id="me")
+            self.local_worker = VirtualWorker(
+                hook=self, is_client_worker=is_client, id="me", verbose=verbose
+            )
         else:
             self.local_worker.hook = self
 
+        self._virtual_workers = {self.local_worker}
+
         self.to_auto_overload = {}
 
         self.args_hook_for_overloaded_attr = {}
@@ -828,3 +841,7 @@ def param_is_pointer_tensor(param):
             return total_norm
 
         self.torch.nn.utils.clip_grad_norm_ = clip_grad_norm_remote_
+
+    def set_verbose(self, flag):
+        for workers in self._virtual_workers:
+            workers.verbose = flag
diff --git a/syft/generic/id_provider.py b/syft/generic/id_provider.py
index 89c0fe07977..b3788beee6d 100644
--- a/syft/generic/id_provider.py
+++ b/syft/generic/id_provider.py
@@ -82,3 +82,7 @@ def get_recorded_ids(self, continue_recording=False):
             self.record_ids = False
             self.recorded_ids = list()
         return ret_val
+
+    @staticmethod
+    def seed(seed=0):
+        random.seed(seed)
diff --git a/syft/messaging/message.py b/syft/messaging/message.py
index 01d03492d41..83022454863 100644
--- a/syft/messaging/message.py
+++ b/syft/messaging/message.py
@@ -300,7 +300,7 @@ def __init__(self, obj_id, user, reason):
 
     def __str__(self):
         """Return a human readable version of this message"""
-        return f"({type(self).__name__} {(self.obj_id, self.user, self.reason)})"
+        return f"({type(self).__name__} {(self.obj_id if hasattr(self, 'obj_id') else '', self.user, self.reason)})"
 
     @staticmethod
     def simplify(worker: AbstractWorker, msg: "ObjectRequestMessage") -> tuple:
diff --git a/syft/workers/base.py b/syft/workers/base.py
index 1055bd6bb2f..8d870ec9d80 100644
--- a/syft/workers/base.py
+++ b/syft/workers/base.py
@@ -101,7 +101,7 @@ def __init__(
         data: Union[List, tuple] = None,
         is_client_worker: bool = False,
         log_msgs: bool = False,
-        verbose: bool = False,
+        verbose: bool = None,
         auto_add: bool = True,
         message_pending_time: Union[int, float] = 0,
     ):
@@ -112,7 +112,14 @@ def __init__(
         self.id = id
         self.is_client_worker = is_client_worker
         self.log_msgs = log_msgs
-        self.verbose = verbose
+        if verbose is None:
+            self.verbose = hook.verbose if hasattr(hook, "verbose") else False
+        else:
+            self.verbose = verbose
+
+        if isinstance(hook, sy.TorchHook) and hasattr(hook, "_virtual_workers"):
+            hook._virtual_workers.add(self)
+
         self.auto_add = auto_add
         self._message_pending_time = message_pending_time
         self.msg_history = list()


Code review comments:
diff_hunk: @@ -97,6 +103,9 @@ def __init__(
         # Save the provided torch module as an attribute of the hook
         self.torch = torch
         self.framework = self.torch
+        if seed is not None:
+            syft.manual_seed(seed)
path: syft/frameworks/torch/hook/hook.py
text: @user1:
```suggestion
            syft.ID_PROVIDER.seed(seed)
```

diff_hunk: @@ -131,3 +131,7 @@ def pool():
     from syft.generic.id_provider import IdProvider
 
     ID_PROVIDER = IdProvider()
+
+
+def manual_seed(seed=0):
+    syft.ID_PROVIDER.seed(seed)
path: syft/__init__.py
text: @user1:
```suggestion
```

diff_hunk: @@ -300,7 +300,7 @@ def __init__(self, obj_id, user, reason):
 
     def __str__(self):
         """Return a human readable version of this message"""
-        return f"({type(self).__name__} {(self.obj_id, self.user, self.reason)})"
+        return f"({type(self).__name__} {(self.obj_id if hasattr(self, 'obj_id') else '', self.user, self.reason)})"
path: syft/messaging/message.py
text: @user1:
This bypasses the error, but I think the underlying ssue is that the field on the message is called `object_id` (instead of `obj_id`.)

diff_hunk: @@ -112,7 +112,14 @@ def __init__(
         self.id = id
         self.is_client_worker = is_client_worker
         self.log_msgs = log_msgs
-        self.verbose = verbose
+        if verbose is None:
+            self.verbose = hook.verbose if hasattr(hook, "verbose") else False
+        else:
+            self.verbose = verbose
+
+        if isinstance(hook, sy.TorchHook) and hasattr(hook, "_virtual_workers"):
path: syft/workers/base.py
text: @user1:
It seems like `hasattr(hook, "_virtual_workers")` is always `True`, given that the hook sets `self._virtual_workers = {self.local_worker}`?

@author:
That would make it fail the first time while instantiating the "me" / local_worker from the torch hook.
Like [here](https://github.com/Syzygianinfern0/PySyft/blob/e9ecd4594129f0f307ff7fefaae136cbab8769ad/syft/frameworks/torch/hook/hook.py#L134)

diff_hunk: @@ -112,7 +112,14 @@ def __init__(
         self.id = id
         self.is_client_worker = is_client_worker
         self.log_msgs = log_msgs
-        self.verbose = verbose
+        if verbose is None:
+            self.verbose = hook.verbose if hasattr(hook, "verbose") else False
+        else:
+            self.verbose = verbose
+
+        if isinstance(hook, sy.TorchHook) and hasattr(hook, "_virtual_workers"):
+            hook._virtual_workers.add(self)
path: syft/workers/base.py
text: @user1:
This looks like it adds all workers to `hook._virtual_workers`. Either the name `_virtual_workers` or something about the logic doesn't quite make sense here. 🤔 

@author:
We keep track of the "virtual" workers (alice, bob, etc.) here. Hence the naming convention. 

@user1:
This is in BaseWorker though, which means all workers get added to this dictionary.