Repository: sktime/sktime

Issue:
[BUG] load_from_tsfile UnicodeDecodeError: 'utf-8'
**Describe the bug**
`load_from_tsfile` may raise a UnicodeDecodeError.

**To Reproduce**

```python
import sktime
from sktime.datasets import load_from_tsfile

DATA_PATH = './Monash_UEA_UCR_Regression_Archive/'

'''
You can edit load_from_tsfile_to_dataframe function in sktime/datasets/_readers_writers/ts.py 
Replace with open(full_file_path_and_name, encoding="utf-8") encoding with "cp1252".
'''
train_x, train_y = load_from_tsfile(os.path.join(DATA_PATH, "BIDMC32HR/BIDMC32HR_TRAIN.ts"))
```

**Expected behavior**
No decode error

**Proposed solution**
Add an optional parameter `encoding="utf-8"` to `load_from_tsfile` and `load_from_tsfile_to_dataframe`, to allow user to manually select encoding.
If you agree, I will take care of the PR 😄

**Additional context**
* Windows 10
* Python 3.12.2
* sktime 0.26.0

```
(venv/Lib/site-packages/sktime/datasets/_readers_writers/ts.py#line=76), in load_from_tsfile_to_dataframe(full_file_path_and_name, return_separate_X_and_y, replace_missing_vals_with)
     75 # Parse the file
     76 with open(full_file_path_and_name, encoding="utf-8") as file:
---> 77     for line in file:
     78         # Strip white space from start/end of line and change to
     79         # lowercase for use below
     80         line = line.strip().lower()
     81         # Empty lines are valid at any point in a file

File <frozen codecs>:322, in decode(self, input, final)

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 1158: invalid start byte
```

**Versions**
<details>
System:
    python: 3.12.2 (tags/v3.12.2:6abddd9, Feb  6 2024, 21:26:36) [MSC v.1937 64 bit (AMD64)]
executable: C:\Users\cyril\Documents\Development\FunWithTSER\venv\Scripts\python.exe
   machine: Windows-10-10.0.19045-SP0

Python dependencies:
          pip: 24.0
       sktime: 0.26.0
      sklearn: 1.4.1.post1
       skbase: 0.7.2
        numpy: 1.26.4
        scipy: 1.12.0
       pandas: 2.1.4
   matplotlib: 3.8.3
       joblib: 1.3.2
        numba: None
  statsmodels: None
     pmdarima: None
statsforecast: None
      tsfresh: None
      tslearn: None
        torch: None
   tensorflow: None
tensorflow_probability: None

</details>



Discussion:
Yes, propoesd fix sounds good, go ahead!

Small note, we should be careful to not change existing default behaviour in a non-failing case, so adding a parameter and making current the default seems "safest".

Further, it seems tricky to test - do you have any good suggestions how to test this without downloading anything?
In my opinion, we just give some user the possibility to modify the encoding, in all other cases, this does not apply any changes, so we should not risk “breaking” anything.

If you want to set up tests, one possibility could be to convert current files to a different encoding, but I don't know the project well enough yet to know what's in place.
> In my opinion, we just give some user the possibility to modify the encoding,

Makes sense.

>  one possibility could be to convert current files to a different encoding, but I don't know the project well enough yet to know what's in place.

There are no converters, so this may have to be done manually. Some datasets used for testing are in `sktime/datasets/data`.

Resolving patch:
diff --git a/sktime/datasets/_readers_writers/ts.py b/sktime/datasets/_readers_writers/ts.py
--- a/sktime/datasets/_readers_writers/ts.py
+++ b/sktime/datasets/_readers_writers/ts.py
@@ -29,6 +29,7 @@ def load_from_tsfile_to_dataframe(
     full_file_path_and_name,
     return_separate_X_and_y=True,
     replace_missing_vals_with="NaN",
+    encoding="utf-8",
 ):
     """Load data from a .ts file into a Pandas DataFrame.
 
@@ -43,6 +44,8 @@ def load_from_tsfile_to_dataframe(
     replace_missing_vals_with: str
        The value that missing values in the text file should be replaced
        with prior to parsing.
+    encoding: str
+        encoding is the name of the encoding used to read file using the open function.
 
     Returns
     -------
@@ -73,7 +76,7 @@ class values.
     class_val_list = []
     line_num = 0
     # Parse the file
-    with open(full_file_path_and_name, encoding="utf-8") as file:
+    with open(full_file_path_and_name, encoding=encoding) as file:
         for line in file:
             # Strip white space from start/end of line and change to
             # lowercase for use below
@@ -610,6 +613,7 @@ def load_from_tsfile(
     replace_missing_vals_with="NaN",
     return_y=True,
     return_data_type="nested_univ",
+    encoding="utf-8",
 ):
     """Load time series .ts file into X and (optionally) y.
 
@@ -637,6 +641,8 @@ def load_from_tsfile(
             "numpy2d"/"np2d"/"numpyflat": 2D np.ndarray (instance, time index)
             "pd-multiindex": pd.DataFrame with 2-level (instance, time) MultiIndex
         Exception is raised if the data cannot be stored in the requested type.
+    encoding: str
+        encoding is the name of the encoding used to read file using the open function.
 
     Returns
     -------
@@ -674,6 +680,7 @@ def load_from_tsfile(
         full_file_path_and_name=full_file_path_and_name,
         return_separate_X_and_y=True,
         replace_missing_vals_with=replace_missing_vals_with,
+        encoding=encoding,
     )
 
     X = convert(X, from_type="nested_univ", to_type=return_data_type)


Test patch: