Repository: dask/fastparquet

Issue:
Writer timestamp conversion is backwards for datetime64[ns] to TIMESTAMP_MICROS
Appending to a Parquet file with a TIMESTAMP_MICROS column using a datetime64[ns] DataFrame column should silently truncate the ns timestamp to µs by dividing by 1000. Currently it multiplies by 1000 instead which causes the file data to be incorrect.

The cause is in `fastparquet.writer` `convert(data, se)`:
```python
        try:
            out = data.values.view("int64") * factor
        except KeyError:
            breakpoint()
```
The multiplication should be integer division, the same as in `timeshift(indata, outdata, factor)`. Fastparquet and PyArrow both read this back correctly when the fix is applied. Tangentially, I think the breakpoint() call should be removed so it can't drop into pdb in production code, if it's reachable at all. 🙂

I can do a PR for this if you'd like.

**Minimal Complete Verifiable Example**:

```python
import sys
import pandas as pd
import fastparquet as fp

FILE_PATH = 'out.parquet'

def main():
    # Write datetime64[us] to TIMESTAMP_MICROS
    df0 = pd.DataFrame({'Date': ['2025-01-01', pd.NaT]}, index=[0, 1], dtype='datetime64[us]')
    print(df0)
    print(f'Writing {df0['Date'].dtype}\n')
    fp.write(FILE_PATH, df0, append=False)

    # Append datetime64[ns] to TIMESTAMP_MICROS
    df1 = pd.DataFrame({'Date': ['2025-07-31', pd.NaT]}, index=[2, 3], dtype='datetime64[ns]')
    print(df1)
    print(f'Appending {df1['Date'].dtype}\n')
    fp.write(FILE_PATH, df1, append=True)

    # Read back
    pf = fp.ParquetFile(FILE_PATH)
    print(f'Read schema:\n{pf.schema}\n')
    df_actual = pf.to_pandas()
    print(f'Read dataframe:\n{df_actual}\n')

    df_expected = pd.concat([df0, df1.astype('datetime64[us]')])
    if not df_expected.equals(df_actual):
        print('Fail')
        sys.exit(1)
    print('Pass')

if __name__ == '__main__':
    main()
```

**Environment**:
python 3.13.5
fastparquet 2024.11.0
pandas 2.3.1
numpy 2.3.1

Operating System: Linux
Install method: conda


Discussion:
martindurant: CI fixed in https://github.com/dask/fastparquet/pull/962

Fix (patch):
diff --git a/fastparquet/writer.py b/fastparquet/writer.py
index a30d40d..57eecfe 100644
--- a/fastparquet/writer.py
+++ b/fastparquet/writer.py
@@ -321,10 +321,7 @@ def convert(data, se):
         else:
             unit = [k for k, v in se.logicalType.TIMESTAMP.unit._asdict().items() if v is not None][0]
             factor = time_factors[(unit, part)]
-        try:
-            out = data.values.view("int64") * factor
-        except KeyError:
-            breakpoint()
+        out = data.values.view("int64") // factor
     else:
         raise ValueError("Don't know how to convert data type: %s" % dtype)
     return out


Test changes:
diff --git a/fastparquet/test/test_timestamp_conversion_960.py b/fastparquet/test/test_timestamp_conversion_960.py
new file mode 100644
index 0000000..f5a2d80
--- /dev/null
+++ b/fastparquet/test/test_timestamp_conversion_960.py
@@ -0,0 +1,23 @@
+import pandas as pd
+
+import fastparquet as fp
+
+def test_github_960(request, tmp_path):
+    """
+    https://github.com/dask/fastparquet/issues/960
+    """
+    file_path = tmp_path / f'{request.node.name}.parquet'
+    # Write datetime64[us] to TIMESTAMP_MICROS
+    df0 = pd.DataFrame({'Date': ['2025-01-01', pd.NaT]}, index=[0, 1], dtype='datetime64[us]')
+    fp.write(file_path, df0, append=False)
+
+    # Append datetime64[ns] to TIMESTAMP_MICROS
+    df1 = pd.DataFrame({'Date': ['2025-07-31', pd.NaT]}, index=[2, 3], dtype='datetime64[ns]')
+    fp.write(file_path, df1, append=True)
+
+    # Read back
+    pf = fp.ParquetFile(file_path)
+    df_actual = pf.to_pandas()
+
+    df_expected = pd.concat([df0, df1.astype('datetime64[us]')])
+    assert df_expected.equals(df_actual)