Repository: dask/dask

Issue:
test_with_spec_integer_method fails on s390x
**Describe the issue**:

I'm not sure if this is a big-endian problem, or some other thing; I'm not really sure what the test is about.

```pytb
________________________ test_with_spec_integer_method _________________________
[gw2] linux -- Python 3.12.0 /usr/bin/python3
    def test_with_spec_integer_method():
        from dask.dataframe.io.demo import ColumnSpec, DatasetSpec, with_spec
    
        spec = DatasetSpec(
            npartitions=1,
            nrecords=5,
            column_specs=[
                ColumnSpec(prefix="pois", dtype=int, method="poisson"),
                ColumnSpec(prefix="norm", dtype=int, method="normal"),
                ColumnSpec(prefix="unif", dtype=int, method="uniform"),
                ColumnSpec(prefix="binom", dtype=int, method="binomial", args=(100, 0.4)),
                ColumnSpec(prefix="choice", dtype=int, method="choice", args=(10,)),
                ColumnSpec(prefix="rand", dtype=int, random=True, low=0, high=10),
                ColumnSpec(prefix="rand", dtype=int, random=True),
            ],
        )
        ddf = with_spec(spec, seed=42)
        res = ddf.compute()
>       assert res["pois1"].tolist() == [1002, 985, 947, 1003, 1017]
E       assert [1021, 1006, 1024, 1030, 930] == [1002, 985, 947, 1003, 1017]
E         At index 0 diff: 1021 != 1002
E         Full diff:
E         - [1002, 985, 947, 1003, 1017]
E         + [1021, 1006, 1024, 1030, 930]
../../../BUILDROOT/python-dask-2023.7.1-1.fc39~bootstrap.s390x/usr/lib/python3.12/site-packages/dask/dataframe/io/tests/test_demo.py:338: AssertionError
```

**Minimal Complete Verifiable Example**:

```
pytest
```

**Anything else we need to know?**:

**Environment**:

- Dask version: 2023.7.1
- Python version: 3.12b4
- Operating System: Fedora Rawhide
- Install method (conda, pip, source): source


Discussion:
Still failing with 2023.8.1, though I don't think much changed on this code anyway.

If I've understood the code, it appears to be a wrapper around `np.random.RandomState.poisson`, but I'm not sure of the exact parameters. Perhaps if I had that, I could check if this were a NumPy bug.
This was reported to Debian as https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1050526

I'm pretty sure I figured out a solution to this.  Eventually the test calls dask.util.random_state_data(1, 42) on x86 the output of that call start with this list on two different machines:

In [9]: random_state_data(1, 0)
Out[9]: 
[array([2357136044, 2546248239, 3071714933, 3626093760, 2588848963,
        3684848379, 2340255427, 3638918503, 1819583497, 2678185683,
        2774094101, 1650906866, 1879422756, 1277901399, 3830135878,
         243580376, 4138900056, 1171049868, 1646868794, 2051556033,
        3400433126, 3488238119, 2271586391, 2061486254, 2439732824,
        1686997841, 3975407269, 3590930969,  305097549, 1449105480,
         374217481, 2783877012,   86837363, 1581585360, 3576074995,


But on s390x it starts with 

In [16]: random_state_data(1, 42)
Out[16]: 
[array([1725751647, 3007179467, 1543725811,  244708654, 1794401211,
        1205115335, 3165536665,  338480024, 1725362215, 2031624818,
        3527536423, 3606353689, 1251073550, 3394605429, 1471725021,
        1961717077, 1672274585, 1743491620, 2537440437, 2191564966,
        2500216069,  889024526,   16993528, 1474942136, 3958708949,
        2650621168,  635132726, 2164863744, 3205794862, 3146973694,
         345633582, 2688816030, 3419529805,  961385884,  359683718,


Digging further there's a place where the random number gets initialized from a buffer using random bytes converted into uint32 using numpy.frombuffer.

I added some code to force little endian byte order on the buffer, and
after that the random_state_data call returns the same list of values on s390x.
So it will hopefully then generate the same random values used in the tests  as on x86_64.


```python
--- dask-2023.8.0+dfsg.orig/dask/utils.py
+++ dask-2023.8.0+dfsg/dask/utils.py
@@ -426,7 +426,9 @@ def random_state_data(n: int, random_sta
         random_state = np.random.RandomState(random_state)
 
     random_data = random_state.bytes(624 * n * 4)  # `n * 624` 32-bit
integers
-    l = list(np.frombuffer(random_data, dtype=np.uint32).reshape((n, - 1)))
+    dt = np.dtype(np.uint32)
+    dt = dt.newbyteorder("<")
+    l = list(np.frombuffer(random_data, dtype=dt).reshape((n, -1)))
     assert len(l) == n
     return l
```

I released a Debian package with this fix I'll be able to check to see if it worked in a few hours to tomorrow morning.
That looks reasonable, though note you can simplify the dtype to be `np.frombuffer(random_data, dtype="<u4")`.
Thanks for the patch; I've successfully build on s390x on Fedora with it: https://koji.fedoraproject.org/koji/taskinfo?taskID=105380481

Resolving patch:
diff --git a/dask/utils.py b/dask/utils.py
--- a/dask/utils.py
+++ b/dask/utils.py
@@ -557,7 +557,7 @@ def random_state_data(n: int, random_state=None) -> list:
         random_state = np.random.RandomState(random_state)
 
     random_data = random_state.bytes(624 * n * 4)  # `n * 624` 32-bit integers
-    l = list(np.frombuffer(random_data, dtype=np.uint32).reshape((n, -1)))
+    l = list(np.frombuffer(random_data, dtype="<u4").reshape((n, -1)))
     assert len(l) == n
     return l
 


Test patch: