Repository: OpenMined/PySyft

Issue:
Some operations are still implemented in PassthroughTensor and are in the Syft Tensor AST, even though they're not supported
From the list of supported ops, you can see that some ops are not supported: argpartition, partition, cumprod, conj, etc

Some of these are still implemented in PassthroughTensor and still are included in the Syft Tensor API located in `syft/core/tensor/__init__.py` and thus show up when calling `dir(TensorPointer)`



![Image](https://user-images.githubusercontent.com/32711264/199150919-9fe02d50-bcc3-4d4f-acf5-deecf921450a.png)




Discussion:
Anything that doesn't reliably work should be `commented` out for the 0.7 release.
@IshanMi could this just be because the Pointer classes subclass from `PassthroughTensor`?

```python
class TensorWrappedPhiTensorPointer(Pointer, PassthroughTensor):

class TensorWrappedGammaTensorPointer(Pointer, PassthroughTensor):
```
There may need to be some cleanup here involving PassthroughTensor but it's as of yet unclear to me. The idea of PassthroughTensor was to be able to reduce the number of ops we had to implement by implementing some ops by using simpler ops. For example, sum() could be implemented as a series of __add__ operations at the pointer level. This should have alleviated the need for us to implement a sum() method for SMPC or DP as long as SMPC/DP had __add__ implemented. It'd probably be a bit slower, but would bring us to full API functionality faster.

If, however,. there are ops which PassthroughTensor defines but doesn't have functionality for, we should comment them out. However, I think PassthroughTensor is still a good idea, particularly for giving us lots of functionality quickly.
@iamtrask As I understand it PassthroughTensor is for the server side. Pointers are for the client side. Client side Pointers will never have a .child hence the issue.
Ah - so I suppose PassthroughTensor is doing two things when it should really only do one. 

1) if an op can be framed as merely a series of other tensor ops, implement it
2) if an op cannot be framed as a series of other ops, call self.child.op

The main value of passthrough tensor is (1). That's the bit I'd like for us to keep. We can delete the parts of the api for (2). (I don't know if those deletions will break things... they might)
> Ah - so I suppose PassthroughTensor is doing two things when it should really only do one.
> 
> 1. if an op can be framed as merely a series of other tensor ops, implement it
> 2. if an op cannot be framed as a series of other ops, call self.child.op
> 
> The main value of passthrough tensor is (1). That's the bit I'd like for us to keep. We can delete the parts of the api for (2). (I don't know if those deletions will break things... they might)

Does it make sense for a Pointer to implement its client side logic with calls to multiple actions as composition? All that does is send 3 sub actions and chain the results, rather than sending 1 action and having the server side do the composition. I think Pointers should have 0 knowledge of anything except "send this action", which is why they used to be derived directly.

As I understand the goal of subclassing is to provide default behavior, but what is the default behavior for an unimplemented method?

Looking at the code I think whats happening is because we have public data such as bounds and data subjects, its desirable that the generated local pointer from an operation mirrors the same behavior that would happen remotely.

If this was implemented as a DP Pointer parent class then it could do something like:
```python
def op(self, *args, *kwargs) -> Any:
  self.send_action(self.op.__name__, args, kwargs)
  min_vals = apply_op(op, self.min_vals, args, kwargs)
  max_vals = apply_op(op, self.max_vals, args, kwargs)
  data_subjects = apply_op(op, self.data_subjects, args, kwargs)
  return DPPointer.create(data_subjects.is_gamma, min_vals, max_vals, data_subjects)
```

Since the DP Tensor Pointer knows about its internal public bounds and datasubject attributes which require mutation and since DSA and bounds should be a `numpy`-like type ready to be called directly with the op.

If min_vals, max_vals, or data_subject happen to inherit from PassthroughTensor then they can get all the free composition locally as well, but the Pointer shouldnt know about this stuff.

Then the default behavior for any `op` on DPTensorPointer would be to send that op to the server and run it against its local public attributes, if we wanted to.

Looking at `MPCTensor` it looks like that uses a seperate `TensorPointer` which does not inherit from `PassthroughTensor`.

Without digging into it I guess theres probably some coupling and overlap particularly around the need for multiple return type signatures from methods which could result in either a DP or SMPC TensorPointer. 

I suggest we do two things here:
1) remove the bug by not subclassing from PassthroughTensor
2) separately look at our Pointer design and unify it around a set of constraints which allow for local public and remote private data to be executed of different types
It's not ideal for Pointer to implement ops as multiple ops because it means we're sending more messages than necessary. It's a fine way to get to a lot of functionality fast but the ideal state is ti have that all on server side.

Given that ideal state, it seems like a good decision for us to remove passthrough from pointer. Since SMPC functionality is so limited there's a part of me that wonders if leaving some of the methods that work by calling a series of other method on self would be worth it. But if we do a push on SMPC we can cross that bridge then. 

Resolving patch:
diff --git a/packages/syft/src/syft/core/tensor/autodp/gamma_tensor.py b/packages/syft/src/syft/core/tensor/autodp/gamma_tensor.py
--- a/packages/syft/src/syft/core/tensor/autodp/gamma_tensor.py
+++ b/packages/syft/src/syft/core/tensor/autodp/gamma_tensor.py
@@ -80,7 +80,7 @@
 
 
 @serializable(recursive_serde=True)
-class TensorWrappedGammaTensorPointer(Pointer, PassthroughTensor):
+class TensorWrappedGammaTensorPointer(Pointer):
     __name__ = "TensorWrappedGammaTensorPointer"
     __module__ = "syft.core.tensor.autodp.gamma_tensor"
     __attr_allowlist__ = [
diff --git a/packages/syft/src/syft/core/tensor/autodp/phi_tensor.py b/packages/syft/src/syft/core/tensor/autodp/phi_tensor.py
--- a/packages/syft/src/syft/core/tensor/autodp/phi_tensor.py
+++ b/packages/syft/src/syft/core/tensor/autodp/phi_tensor.py
@@ -67,7 +67,7 @@
 
 
 @serializable(recursive_serde=True)
-class TensorWrappedPhiTensorPointer(Pointer, PassthroughTensor):
+class TensorWrappedPhiTensorPointer(Pointer):
     __name__ = "TensorWrappedPhiTensorPointer"
     __module__ = "syft.core.tensor.autodp.phi_tensor"
     __attr_allowlist__ = [


Test patch: