Repository: mne-tools/mne-python

Issue:
How to fix: “UnicodeDecodeError: ‘ascii’ codec can’t decode byte”
how to solve this problem?
linux operating system 18.01

```python
sample_data_folder = mne.datasets.testing.data_path()
sample_data_raw_file = os.path.join(sample_data_folder, 'NihonKohden', 'DA0935F5.EEG')

raw = mne.io.read_raw_nihon(sample_data_raw_file)
```
```
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-36-62ba074d16ef> in <module>
      2 sample_data_raw_file = os.path.join(sample_data_folder, 'NihonKohden', 'DA0935F5.EEG')
      3 
----> 4 raw = mne.io.read_raw_nihon(sample_data_raw_file)

~/anacondalele/envs/mne/lib/python3.8/site-packages/mne/io/nihon/nihon.py in read_raw_nihon(fname, preload, verbose)
     44     mne.io.Raw : Documentation of attribute and methods.
     45     """
---> 46     return RawNihon(fname, preload, verbose)
     47 
     48 

<decorator-gen-224> in __init__(self, fname, preload, verbose)

~/anacondalele/envs/mne/lib/python3.8/site-packages/mne/io/nihon/nihon.py in __init__(self, fname, preload, verbose)
    352 
    353         # Get annotations from LOG file
--> 354         annots = _read_nihon_annotations(fname, orig_time=info['meas_date'])
    355         self.set_annotations(annots)
    356 

~/anacondalele/envs/mne/lib/python3.8/site-packages/mne/io/nihon/nihon.py in _read_nihon_annotations(fname, orig_time)
    247             n_logs = np.fromfile(fid, np.uint8, 1)[0]
    248             fid.seek(t_blk_address + 0x14)
--> 249             t_logs = np.fromfile(fid, '|S45', n_logs).astype('U45')
    250 
    251             for t_log in t_logs:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe7 in position 7: ordinal not in range(128)
```



Discussion:
Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get back to you soon. 🚴🏽‍♂️

I'm not really familiar with the Nihon Kohden format, but the error occurs because the reader tries to decode a non-ASCII character:

```
np.fromfile(fid, '|S45', n_logs)
```

I'm not sure why it is first decoded as ASCII and then converted to Unicode, because changing the whole line 249 to

```
t_logs = np.fromfile(fid, '|U45', n_logs)
```

would  probably solve the problem. But again, I don't know if non-ASCII characters are allowed by the format.

@fraimondo @larsoner any idea?
Also, @leticiabragas2 the file `DA0935F5.EEG` is not part of the MNE testing data sets - is this a file you recorded yourself?
Can you share all files that are part of the `DA0935F5` data set? Specifically, I need the EEG data file (extension `.EEG`).
Hi, I'm having the same issue. Indeed, it is related to non-ASCII characters in the annotations. I attached  [Examplefiles.zip](https://github.com/mne-tools/mne-python/files/6442306/test.zip). [EDFBrowser](https://www.teuniz.net/edfbrowser/) converts without issues, so it doesn't seem to be an issue of the .EEG format itself.

The suggestion:
`t_logs = np.fromfile(fid, '|U45', n_logs)`
proposed by @cbrnr does not work, it raises another [ValueError](https://github.com/mne-tools/mne-python/files/6442368/error.txt) related to encoding, the decoding via ASCII and cast to utf-8 seems to be necessary. 

As there is no way to suppress loading the annotations, the only way for me to load these files for now is to remove the annotations from the folder.


Resolving patch:
diff --git a/mne/io/nihon/nihon.py b/mne/io/nihon/nihon.py
--- a/mne/io/nihon/nihon.py
+++ b/mne/io/nihon/nihon.py
@@ -240,15 +240,25 @@ def _read_nihon_annotations(fname, orig_time):
         n_logblocks = np.fromfile(fid, np.uint8, 1)[0]
         all_onsets = []
         all_descriptions = []
+        encodings = ('utf-8', 'latin1')
         for t_block in range(n_logblocks):
             fid.seek(0x92 + t_block * 20)
             t_blk_address = np.fromfile(fid, np.uint32, 1)[0]
             fid.seek(t_blk_address + 0x12)
             n_logs = np.fromfile(fid, np.uint8, 1)[0]
             fid.seek(t_blk_address + 0x14)
-            t_logs = np.fromfile(fid, '|S45', n_logs).astype('U45')
-
+            t_logs = np.fromfile(fid, '|S45', n_logs)
             for t_log in t_logs:
+                for enc in encodings:
+                    try:
+                        t_log = t_log.decode(enc)
+                    except UnicodeDecodeError:
+                        pass
+                    else:
+                        break
+                else:
+                    warn(f'Could not decode log as one of {encodings}')
+                    continue
                 t_desc = t_log[:20].strip('\x00')
                 t_onset = datetime.strptime(t_log[20:26], '%H%M%S')
                 t_onset = (t_onset.hour * 3600 + t_onset.minute * 60 +


Test patch: