Repository: scikit-hep/awkward

Issue:
Error when using ak.copy in v2
### Version of Awkward Array

1.9.0rc4

### Description and code to reproduce

```
import awkward._v2 as ak

array = ak.Array([
    [{"x": 1.1, "y": [1]}, {"x": 2.2, "y": [1, 2]}, {"x": 3.3, "y": [1, 2, 3]}],
    [],
    [{"x": 4.4, "y": [1, 2, 3, 4]}, {"x": 5.5, "y": [1, 2, 3, 4, 5]}]
])

ak.copy(array)
```

Expect a copy of the array, but I get this:

```
File ~/miniconda3/envs/research/lib/python3.9/site-packages/awkward/_v2/operations/ak_copy.py:57, in copy(array)
     10 """
     11 Returns a deep copy of the array (no memory shared with original).
     12 
   (...)
     51 a mutable third-party library, so this function allows you to make a true copy.
     52 """
     53 with ak._v2._util.OperationErrorContext(
     54     "ak._v2.fill_none",
     55     dict(array=array),
     56 ):
---> 57     return _impl(array)
...
...
...
TypeError: cannot pickle 'module' object
```


Discussion:
I took a quick look at this and I suspect that it's the `_module` object of the `nplike` associated with the various contents.

Having only just glanced over the code, I think we might want to replace the existing `deep_copy` method with a `__deepcopy__` implementation which ensures that nested layouts are also handled to avoid trying to pickle the module.

In the mean-time, if you really need a deep copy, you *could* use `ak.to_buffers` with `copy.deepcopy`, e.g.

```python
import copy

array_copy = ak.from_buffers(
    *copy.deepcopy(
        ak.to_buffers(array)
    )
)
```

Though in practice, there are not usually that many instances where one actually wants a deep copy given that Awkward Arrays are partially immutable.
That's sounding right. I had intended to look into this, but you beat me to it!

Now that I think of it, all of our arrays have a link to the `nplike` singleton, and the one corresponding to NumPy holds a reference to the actual `numpy` module. We don't want to copy that! So yes, we'll need a custom `__deepcopy__` (and probably also `__copy__`), and its implementation can be our `deep_copy`. It could possibly be renamed, as long as everything that's currently calling `deep_copy` goes and calls `__deepcopy__` instead.
> I took a quick look at this and I suspect that it's the `_module` object of the `nplike` associated with the various contents.
> 
> Having only just glanced over the code, I think we might want to replace the existing `deep_copy` method with a `__deepcopy__` implementation which ensures that nested layouts are also handled to avoid trying to pickle the module.
> 
> In the mean-time, if you really need a deep copy, you _could_ use `ak.to_buffers` with `copy.deepcopy`, e.g.
> 
> ```python
> import copy
> 
> array_copy = ak.from_buffers(
>     *copy.deepcopy(
>         ak.to_buffers(array)
>     )
> )
> ```
> 
> Though in practice, there are not usually that many instances where one actually wants a deep copy given that Awkward Arrays are partially immutable.

I was actually creating a copy to return a subset version of the array and keep a copy of the original, but just realized that it was not needed. Thanks for the advice.
(We should fix it anyway, thanks!)

What you need might be `ak.packed`. Ordinarily, when you slice something, it doesn't actually make a copy, so the original is still entirely in memory. Packing ensures that only the parts you can see in the array you have are linked, so that a subset actually uses less memory.

Resolving patch:
diff --git a/src/awkward/_v2/contents/content.py b/src/awkward/_v2/contents/content.py
--- a/src/awkward/_v2/contents/content.py
+++ b/src/awkward/_v2/contents/content.py
@@ -1480,15 +1480,13 @@ def with_parameter(self, key, value):
 
         return out
 
-    def deep_copy(self, memo=None):
+    def __deepcopy__(self, memo=None):
         cls = self.__class__
         new_instance = cls.__new__(cls)
-        memo = {"id(self)": new_instance}
+        memo = {id(self): new_instance}
         for k, v in self.__dict__.items():
             if k == "_nplike":
-                setattr(new_instance, k, v)
-            # elif isinstance(v, ak._v2.index.Index):
-            #     setattr(new_instance, k, copy.copy(v))
+                new_instance._nplike = v
             else:
                 setattr(new_instance, k, copy.deepcopy(v, memo))
         return new_instance
diff --git a/src/awkward/_v2/operations/ak_copy.py b/src/awkward/_v2/operations/ak_copy.py
--- a/src/awkward/_v2/operations/ak_copy.py
+++ b/src/awkward/_v2/operations/ak_copy.py
@@ -63,4 +63,4 @@ def _impl(array):
         allow_record=True,
         allow_other=False,
     )
-    return ak._v2._util.wrap(layout.deep_copy(), ak._v2._util.behavior_of(array))
+    return ak._v2._util.wrap(layout.__deepcopy__(), ak._v2._util.behavior_of(array))
diff --git a/src/awkward/_v2/record.py b/src/awkward/_v2/record.py
--- a/src/awkward/_v2/record.py
+++ b/src/awkward/_v2/record.py
@@ -1,6 +1,5 @@
 # BSD 3-Clause License; see https://github.com/scikit-hep/awkward-1.0/blob/main/LICENSE
 
-import copy
 from collections.abc import Iterable
 
 import awkward as ak
@@ -191,8 +190,8 @@ def to_list(self, behavior=None):
 
         return self._array[self._at : self._at + 1]._to_list(behavior, None)[0]
 
-    def deep_copy(self):
-        return Record(self._array.deep_copy(), copy.deepcopy(self._at))
+    def __deepcopy__(self, memo=None):
+        return Record(self._array.__deepcopy__(memo), self._at)
 
     def recursively_apply(
         self,


Test patch:
diff --git a/tests/v2/test_0511-copy-and-deepcopy.py b/tests/v2/test_0511-copy-and-deepcopy.py
--- a/tests/v2/test_0511-copy-and-deepcopy.py
+++ b/tests/v2/test_0511-copy-and-deepcopy.py
@@ -53,3 +53,39 @@ def test():
         {"x": 3, "y": 9},
     ]
     assert to_list(original) == [{"x": 1}, {"x": 2}, {"x": 3}]
+
+    array = ak._v2.Array(
+        [
+            [{"x": 1.1, "y": [1]}, {"x": 2.2, "y": [1, 2]}, {"x": 3.3, "y": [1, 2, 3]}],
+            [],
+            [{"x": 4.4, "y": [1, 2, 3, 4]}, {"x": 5.5, "y": [1, 2, 3, 4, 5]}],
+        ]
+    )
+
+    assert to_list(ak._v2.operations.copy(array)) == [
+        [{"x": 1.1, "y": [1]}, {"x": 2.2, "y": [1, 2]}, {"x": 3.3, "y": [1, 2, 3]}],
+        [],
+        [{"x": 4.4, "y": [1, 2, 3, 4]}, {"x": 5.5, "y": [1, 2, 3, 4, 5]}],
+    ]
+
+    a = ak._v2.Array(
+        [
+            {"x": 0, "y": 0.0},
+            {"x": 1, "y": 1.1},
+            {"x": 2, "y": 2.2},
+            {"x": 3, "y": 3.3},
+            {"x": 4, "y": 4.4},
+        ]
+    )
+    record = a[2].layout.__deepcopy__()
+    a["z"] = a.x**2
+
+    assert to_list(a) == [
+        {"x": 0, "y": 0.0, "z": 0},
+        {"x": 1, "y": 1.1, "z": 1},
+        {"x": 2, "y": 2.2, "z": 4},
+        {"x": 3, "y": 3.3, "z": 9},
+        {"x": 4, "y": 4.4, "z": 16},
+    ]
+    assert to_list(record) == {"x": 2, "y": 2.2}
+    assert to_list(a[2]) == {"x": 2, "y": 2.2, "z": 4}