Repository: pytorch/pytorch

Issue:
torch.load() may store unexpected values into tensors on big-endian
### 🐛 Describe the bug

The current `torch.load()` interprets data as native-endian if a given pickle file has no `byteorder` record.

This behavior causes practical issues on big-endian machines when we load parameters for many DNN models saved on little-endian machines (e.g. x86_64). As a result, tensors would have incorrect values. Practically, it would be better to handle the pickle file w/o `byteorder` record as little-endian. 

In addition, we could add an option to handle the pickle file w/o `byteorder` record as native-endian to keep the current behavior.

```python
import torch

x = torch.load(`path to pickle file w/o byteorder record`)
```

### Versions

```
Collecting environment information...
PyTorch version: 2.1.0a0+gite7681b5
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.2 LTS (s390x)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: Could not collect
CMake version: version 3.22.1
Libc version: glibc-2.35

Python version: 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.15.0-71-generic-s390x-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: False

CPU:
Architecture:                    s390x
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Big Endian
CPU(s):                          8
On-line CPU(s) list:             0-7
Vendor ID:                       IBM/S390
Machine type:                    3931
Thread(s) per core:              1
Core(s) per socket:              1
Socket(s) per book:              1
Book(s) per drawer:              1
Drawer(s):                       8
CPU dynamic MHz:                 5200
CPU static MHz:                  5200
BogoMIPS:                        26315.00
Dispatching mode:                horizontal
Flags:                           esan3 zarch stfle msa ldisp eimm dfp edat etf3eh highgprs te vx vxd vxe gs vxe2 vxp sort dflt vxp2 nnpa sie
Hypervisor:                      z/VM 7.2.0
Hypervisor vendor:               IBM
Virtualization type:             full
L1d cache:                       1 MiB (8 instances)
L1i cache:                       1 MiB (8 instances)
L2 cache:                        256 MiB (8 instances)
L3 cache:                        256 MiB
NUMA node(s):                    1
NUMA node0 CPU(s):               0-7
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Mmio stale data:   Not affected
Vulnerability Retbleed:          Not affected
Vulnerability Spec store bypass: Not affected
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; etokens
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected

Versions of relevant libraries:
[pip3] numpy==1.24.3
[pip3] torch==2.1.0a0+gite7681b5
[conda] Could not collect
```

cc @mruberry @mikaylagawarecki


Discussion:
Hi, @kiszk thanks for the report! To clarify, how are you saving these objects, are you using `torch.save`? Did you disable [this argument](https://github.com/pytorch/pytorch/blob/main/torch/serialization.py#L457), I believe if `_disable_byteorder_record=False` (the default) it should be saving the endian-ness
Thank you for your comment. Your comment is correct.   
Unfortunately, I do not know how the given file is saved since I did not create the original (problematic) pickle file myself.
Hm, it is interesting that the pickle file creator explicitly sets `_disable_byteorder_record=True`

Could a workaround here possibly be to add a byteorder record to the zipfile? e.g. something like 

```
from torch.serialization import _open_zipfile_writer

with _open_zipfile_writer(f) as opened_zipfile:
     zip_file.write_record('byteorder', 'little', len('little'))
torch.load(f)
```
Unfortunately, there are many zip files w/o a byteorder record practically. For example, many DNN models saved on x86 using previous versions of PyTorch do not have.

When we download those files from repositories using https, there is no chance to add a byteorder record.
Hey my apologies, I had not noticed that writing the byteorder record was a very recent addition and most checkpoints would not have it. 

Following up on this, from offline discussion, when there is no byteorder mark we could either go with the path of  

1) assume little endian
2) adding a kwarg (like `default_load_endian` as you have done) 
         (i) with default `little`
         (ii) with default `native`
3) adding a global variable that indicates the default loading endianness
         (i) with default `little`
         (ii) with default `native`
         
(3)(ii) here is the solution that we feel makes the most sense here. Firstly setting native-endian as the default is BC-compatible and secondly using a global variable ensures that users who don't explicitly call`load` itself (e.g. they could be perhaps calling an API that might wrap load like `from_pretrained`) would still be able to toggle this.
         
 


Resolving patch:
diff --git a/torch/serialization.py b/torch/serialization.py
--- a/torch/serialization.py
+++ b/torch/serialization.py
@@ -1326,12 +1326,12 @@ def _load(zip_file, map_location, pickle_module, pickle_file='data.pkl', overall
         byteorderdata = zip_file.get_record(byteordername)
         if byteorderdata not in [b'little', b'big']:
             raise ValueError('Unknown endianness type: ' + byteorderdata.decode())
-    elif get_default_load_endianness() == LoadEndianness.LITTLE:
+    elif get_default_load_endianness() == LoadEndianness.LITTLE or \
+            get_default_load_endianness() is None:
         byteorderdata = b'little'
     elif get_default_load_endianness() == LoadEndianness.BIG:
         byteorderdata = b'big'
-    elif get_default_load_endianness() == LoadEndianness.NATIVE or \
-            get_default_load_endianness() is None:
+    elif get_default_load_endianness() == LoadEndianness.NATIVE:
         pass
     else:
         raise ValueError('Invalid load endianness type')
@@ -1339,14 +1339,14 @@ def _load(zip_file, map_location, pickle_module, pickle_file='data.pkl', overall
     if not zip_file.has_record(byteordername) and \
             get_default_load_endianness() is None and \
             sys.byteorder == 'big':
-        # Default behaviour should be changed in future
+        # Default behaviour was changed
         # See https://github.com/pytorch/pytorch/issues/101688
         warnings.warn("The default load endianness for checkpoints without a byteorder mark "
-                      "on big endian machines will be changed from 'native' to 'little' endian "
-                      "in a future release, to avoid this behavior please use "
+                      "on big endian machines was changed from 'native' to 'little' endian, "
+                      "to avoid this behavior please use "
                       "torch.serialization.set_default_load_endianness to set "
                       "the desired default load endianness",
-                      DeprecationWarning)
+                      UserWarning)
 
     def load_tensor(dtype, numel, key, location):
         name = f'data/{key}'


Test patch:
diff --git a/test/test_serialization.py b/test/test_serialization.py
--- a/test/test_serialization.py
+++ b/test/test_serialization.py
@@ -14,6 +14,7 @@
 import pickle
 import shutil
 import pathlib
+import platform
 from copy import deepcopy
 from itertools import product
 
@@ -3859,6 +3860,47 @@ def test_serialization_load_bom_data_cfloat(self):
             self.assertTrue(torch.equal(tensor_be_no_bom, tensor_le_bom))
             self.assertTrue(torch.equal(tensor_be_no_bom, tensor_be_bom))
 
+    @unittest.skipIf(platform.machine() != 's390x', "s390x-specific test")
+    def test_serialization_warning_s390x(self):
+        data_be_no_bom = (b'PK\x03\x04\x00\x00\x08\x08\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+                          b'\x00\x00\x00\x00\x00\x19\x00\t\x00tensor.double.BE/data.pklFB\x05\x00ZZZZZ\x80\x02'
+                          b'ctorch._utils\n_rebuild_tensor_v2\nq\x00((X\x07\x00\x00\x00storageq\x01ctorch\n'
+                          b'DoubleStorage\nq\x02X\x01\x00\x00\x000q\x03X\x03\x00\x00\x00cpuq\x04K\x04tq\x05'
+                          b'QK\x00K\x02K\x02\x86q\x06K\x02K\x01\x86q\x07\x89ccollections\nOrderedDict\nq\x08'
+                          b')Rq\ttq\nRq\x0b.PK\x07\x08S\xd3\xba&\x9b\x00\x00\x00\x9b\x00\x00\x00PK\x03\x04\x00'
+                          b'\x00\x08\x08\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+                          b'\x00\x17\x00 \x00tensor.double.BE/data/0FB\x1c\x00ZZZZZZZZZZZZZZZZZZZZZZZZZZZZ'
+                          b'?\xc9^|\xff\xa4v\x97\xbf\xe9\xb0\x8dP\x8c\xbc\xce\xbf\xd3\xdb\xb7[\xef\x0e\xdc?\xde'
+                          b'\x00\xf9Q\x08\xb14PK\x07\x083@\x82/ \x00\x00\x00 \x00\x00\x00PK\x03\x04\x00\x00'
+                          b'\x08\x08\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+                          b'\x18\x00\x1a\x00tensor.double.BE/versionFB\x16\x00ZZZZZZZZZZZZZZZZZZZZZZ3\nPK\x07'
+                          b'\x08\xd1\x9egU\x02\x00\x00\x00\x02\x00\x00\x00PK\x01\x02\x00\x00\x00\x00\x08\x08'
+                          b'\x00\x00\x00\x00\x00\x00S\xd3\xba&\x9b\x00\x00\x00\x9b\x00\x00\x00\x19\x00\x00'
+                          b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00tensor.double.BE/da'
+                          b'ta.pklPK\x01\x02\x00\x00\x00\x00\x08\x08\x00\x00\x00\x00\x00\x003@\x82/ '
+                          b'\x00\x00\x00 \x00\x00\x00\x17\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+                          b'\xeb\x00\x00\x00tensor.double.BE/data/0PK\x01\x02\x00\x00\x00\x00\x08\x08\x00\x00'
+                          b'\x00\x00\x00\x00\xd1\x9egU\x02\x00\x00\x00\x02\x00\x00\x00\x18\x00\x00\x00\x00'
+                          b'\x00\x00\x00\x00\x00\x00\x00\x00\x00p\x01\x00\x00tensor.double.BE/versionPK\x06\x06'
+                          b',\x00\x00\x00\x00\x00\x00\x00\x1e\x03-\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03'
+                          b'\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\xd2\x00\x00\x00\x00'
+                          b'\x00\x00\x00\xd2\x01\x00\x00\x00\x00\x00\x00PK\x06\x07\x00\x00\x00\x00\xa4\x02\x00'
+                          b'\x00\x00\x00\x00\x00\x01\x00\x00\x00PK\x05\x06\x00\x00\x00\x00\x03\x00\x03\x00'
+                          b'\xd2\x00\x00\x00\xd2\x01\x00\x00\x00\x00')
+
+        current_load_endian = get_default_load_endianness()
+
+        buf_be_no_bom = io.BytesIO(data_be_no_bom)
+
+        try:
+            set_default_load_endianness(None)
+            with self.assertWarnsRegex(UserWarning, "The default load endianness for checkpoints "
+                                       "without a byteorder mark on big endian machines "
+                                       "was changed from 'native' to 'little' endian"):
+                tensor_be_no_bom = torch.load(buf_be_no_bom)
+        finally:
+            set_default_load_endianness(current_load_endian)
+
     @parametrize('weights_only', (True, False))
     @unittest.skipIf(IS_WINDOWS, "NamedTemporaryFile on windows")
     def test_serialization_mmap_loading(self, weights_only):