Repository: ray-project/ray

Issue:
Numpy arrays are mutable in object store in ray >=0.8
My first ever github post and hopefully I am doing this right...

### System information
OS Platform: Linux RHEL7
Ray installed from binary
Ray version: 0.8.x onwards
Python version: 3.7.4
Exact command to reproduce: See example below.

### Describe the problem
Numpy arrays (and pandas dataframes) were immutable in 0.7.6, as described in the documentation. Any modification to a numpy array fetched with ray.get would result in error (`"ValueError: assignment destination is read-only"`). In 0.8 (tested 0.8.1 and 0.9.0 dev), remote numpy arrays are no longer immutable.

### Example

``` python
import ray
import numpy as np
ray.init("auto")  #  initialize as necessary in your local install

x = np.full((2, 2), 1.)
o = ray.put(x)
y = ray.get(o)
y[0, 0] = 9.
# no error

print(ray.get(o))

```
prints
[[9. 1.]
 [1. 1.]]

expected
[[1. 1.]
 [1. 1.]]

Numpy arrays are mutable in object store in ray >=0.8
My first ever github post and hopefully I am doing this right...

### System information
OS Platform: Linux RHEL7
Ray installed from binary
Ray version: 0.8.x onwards
Python version: 3.7.4
Exact command to reproduce: See example below.

### Describe the problem
Numpy arrays (and pandas dataframes) were immutable in 0.7.6, as described in the documentation. Any modification to a numpy array fetched with ray.get would result in error (`"ValueError: assignment destination is read-only"`). In 0.8 (tested 0.8.1 and 0.9.0 dev), remote numpy arrays are no longer immutable.

### Example

``` python
import ray
import numpy as np
ray.init("auto")  #  initialize as necessary in your local install

x = np.full((2, 2), 1.)
o = ray.put(x)
y = ray.get(o)
y[0, 0] = 9.
# no error

print(ray.get(o))

```
prints
[[9. 1.]
 [1. 1.]]

expected
[[1. 1.]
 [1. 1.]]



Discussion:
It is a good catch. Thanks so much.

This issue could be related to the new serializer we are using. I am going to fix it.
It is a good catch. Thanks so much.

This issue could be related to the new serializer we are using. I am going to fix it.

Resolving patch:
diff --git a/python/ray/cloudpickle/cloudpickle_fast.py b/python/ray/cloudpickle/cloudpickle_fast.py
--- a/python/ray/cloudpickle/cloudpickle_fast.py
+++ b/python/ray/cloudpickle/cloudpickle_fast.py
@@ -411,6 +411,16 @@ def _property_reduce(obj):
     return property, (obj.fget, obj.fset, obj.fdel, obj.__doc__)
 
 
+def _numpy_frombuffer(buffer, dtype, shape, order):
+    # Get the _frombuffer() function for reconstruction
+    from numpy.core.numeric import _frombuffer
+    array = _frombuffer(buffer, dtype, shape, order)
+    # Unfortunately, numpy does not follow the standard, so we still
+    # have to set the readonly flag for it here.
+    array.setflags(write=not buffer.readonly)
+    return array
+
+
 def _numpy_ndarray_reduce(array):
     # This function is implemented according to 'array_reduce_ex_picklebuffer'
     # in numpy C backend. This is a workaround for python3.5 pickling support.
@@ -443,10 +453,7 @@ def _numpy_ndarray_reduce(array):
         # (gh-12745).
         return array.__reduce__()
 
-    # Get the _frombuffer() function for reconstruction
-    import numpy.core.numeric as numeric_mod
-    from_buffer_func = numeric_mod._frombuffer
-    return from_buffer_func, (buffer, array.dtype, array.shape, order)
+    return _numpy_frombuffer, (buffer, array.dtype, array.shape, order)
 
 
 class CloudPickler(Pickler):


Test patch:
diff --git a/python/ray/tests/test_basic.py b/python/ray/tests/test_basic.py
--- a/python/ray/tests/test_basic.py
+++ b/python/ray/tests/test_basic.py
@@ -478,6 +478,15 @@ def f():
     assert wr() is None
 
 
+def test_deserialized_from_buffer_immutable(ray_start_regular):
+    x = np.full((2, 2), 1.)
+    o = ray.put(x)
+    y = ray.get(o)
+    with pytest.raises(
+            ValueError, match="assignment destination is read-only"):
+        y[0, 0] = 9.
+
+
 def test_passing_arguments_by_value_out_of_the_box(ray_start_regular):
     @ray.remote
     def f(x):