Repository: freqtrade/freqtrade

Issue:
FreqAI : Issue with custom objective XGBoost classifier
Operating system: Windows 10
Python Version: 3.12.0
Freqtrade Version: freqtrade docker-2023.11-dev-1657fd1f

I run everything with docker using freqtradeorg/freqtrade:develop_freqairl (Image hash : 04465eb86ae602128ab43469ff2d192db3708b9b0fb690dcea7f3b602e4e02ec)

I created a custom XGBoost classifier in order to introduce my custom loss function. I defined my loss function as a top level fucntion above my class (class MyXGBoostClassifier(BaseClassifierModel):)

```
def my_loss(y_true, y_pred):
    ...
    return grad, hess

class MyXGBoostClassifier(BaseClassifierModel):
    def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
        ...

        model_training_parameters = deepcopy(self.model_training_parameters)
        model_training_parameters['objective'] = my_loss
        model = xgb.XGBClassifier(**model_training_parameters)
        model.fit(X=X, y=y, eval_set=eval_set, sample_weight=train_weights, xgb_model=init_model)

        return model

```

I have the following issue when running backtesting :

```
  File "/freqtrade/user_data/strategies/mystrategy.py", line 289, in populate_indicators
    df = self.freqai.start(df, metadata, self)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/freqtrade/freqtrade/freqai/freqai_interface.py", line 161, in start
    dk = self.start_backtesting(dataframe, metadata, self.dk, strategy)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/freqtrade/freqtrade/freqai/freqai_interface.py", line 365, in start_backtesting
    self.dd.save_data(self.model, pair, dk)
  File "/freqtrade/freqtrade/freqai/data_drawer.py", line 474, in save_data
    dump(model, save_path / f"{dk.model_filename}_model.joblib")
  File "/home/ftuser/.local/lib/python3.11/site-packages/joblib/numpy_pickle.py", line 553, in dump
    NumpyPickler(f, protocol=protocol).dump(value)
  File "/usr/local/lib/python3.11/pickle.py", line 487, in dump
    self.save(obj)
  File "/home/ftuser/.local/lib/python3.11/site-packages/joblib/numpy_pickle.py", line 355, in save
    return Pickler.save(self, obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/pickle.py", line 603, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/local/lib/python3.11/pickle.py", line 717, in save_reduce
    save(state)
  File "/home/ftuser/.local/lib/python3.11/site-packages/joblib/numpy_pickle.py", line 355, in save
    return Pickler.save(self, obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
    ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/pickle.py", line 972, in save_dict
    self._batch_setitems(obj.items())
  File "/usr/local/lib/python3.11/pickle.py", line 998, in _batch_setitems
    save(v)
  File "/home/ftuser/.local/lib/python3.11/site-packages/joblib/numpy_pickle.py", line 355, in save
    return Pickler.save(self, obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
    ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/pickle.py", line 1071, in save_global
    raise PicklingError(
_pickle.PicklingError: Can't pickle <function my_loss at 0x7f8728254040>: it's not found as MyXGBoostClassifier.my_loss
```

Apparently freqtrade failed to impoed my custom loss function or somthing liek that but I can't find any info about this issue.
Is there an example of how to implement a custom freqai XGBoost objective function ?
FreqAI : Issue with custom objective XGBoost classifier
Operating system: Windows 10
Python Version: 3.12.0
Freqtrade Version: freqtrade docker-2023.11-dev-1657fd1f

I run everything with docker using freqtradeorg/freqtrade:develop_freqairl (Image hash : 04465eb86ae602128ab43469ff2d192db3708b9b0fb690dcea7f3b602e4e02ec)

I created a custom XGBoost classifier in order to introduce my custom loss function. I defined my loss function as a top level fucntion above my class (class MyXGBoostClassifier(BaseClassifierModel):)

```
def my_loss(y_true, y_pred):
    ...
    return grad, hess

class MyXGBoostClassifier(BaseClassifierModel):
    def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
        ...

        model_training_parameters = deepcopy(self.model_training_parameters)
        model_training_parameters['objective'] = my_loss
        model = xgb.XGBClassifier(**model_training_parameters)
        model.fit(X=X, y=y, eval_set=eval_set, sample_weight=train_weights, xgb_model=init_model)

        return model

```

I have the following issue when running backtesting :

```
  File "/freqtrade/user_data/strategies/mystrategy.py", line 289, in populate_indicators
    df = self.freqai.start(df, metadata, self)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/freqtrade/freqtrade/freqai/freqai_interface.py", line 161, in start
    dk = self.start_backtesting(dataframe, metadata, self.dk, strategy)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/freqtrade/freqtrade/freqai/freqai_interface.py", line 365, in start_backtesting
    self.dd.save_data(self.model, pair, dk)
  File "/freqtrade/freqtrade/freqai/data_drawer.py", line 474, in save_data
    dump(model, save_path / f"{dk.model_filename}_model.joblib")
  File "/home/ftuser/.local/lib/python3.11/site-packages/joblib/numpy_pickle.py", line 553, in dump
    NumpyPickler(f, protocol=protocol).dump(value)
  File "/usr/local/lib/python3.11/pickle.py", line 487, in dump
    self.save(obj)
  File "/home/ftuser/.local/lib/python3.11/site-packages/joblib/numpy_pickle.py", line 355, in save
    return Pickler.save(self, obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/pickle.py", line 603, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/local/lib/python3.11/pickle.py", line 717, in save_reduce
    save(state)
  File "/home/ftuser/.local/lib/python3.11/site-packages/joblib/numpy_pickle.py", line 355, in save
    return Pickler.save(self, obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
    ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/pickle.py", line 972, in save_dict
    self._batch_setitems(obj.items())
  File "/usr/local/lib/python3.11/pickle.py", line 998, in _batch_setitems
    save(v)
  File "/home/ftuser/.local/lib/python3.11/site-packages/joblib/numpy_pickle.py", line 355, in save
    return Pickler.save(self, obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
    ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/pickle.py", line 1071, in save_global
    raise PicklingError(
_pickle.PicklingError: Can't pickle <function my_loss at 0x7f8728254040>: it's not found as MyXGBoostClassifier.my_loss
```

Apparently freqtrade failed to impoed my custom loss function or somthing liek that but I can't find any info about this issue.
Is there an example of how to implement a custom freqai XGBoost objective function ?


Discussion:
> I run everything with docker using freqtradeorg/freqtrade:develop_freqairl

I'd still ask you to get used to fill out the issue template. Those who frequently omit the template may have their issues closed without comment.
We're asking some very basic questions - by deleting the issue template, you're explicitly choosing not to answer our questions.

Filling out the template helps reduce our workload, and we greatly appreciate that.

On top of that -  `:develop[*]` is a moving target, so the version you're using is not the latest one (it could be from day, yesterday, or from 2 months ago, depending on the last time you actually pulled).

---

Well what the error says is that pickle (and with that, python) wasn't able to find the function as it was detected (`MyXGBoostClassifier.my_loss`).

Ad hock, i'd say it (falsely) assumes the function is part of the class (i guess it could be in the class?), or it could be an accidental naming collision, maybe with the filename ? 

> I run everything with docker using freqtradeorg/freqtrade:develop_freqairl

I'd still ask you to get used to fill out the issue template. Those who frequently omit the template may have their issues closed without comment.
We're asking some very basic questions - by deleting the issue template, you're explicitly choosing not to answer our questions.

Filling out the template helps reduce our workload, and we greatly appreciate that.

On top of that -  `:develop[*]` is a moving target, so the version you're using is not the latest one (it could be from day, yesterday, or from 2 months ago, depending on the last time you actually pulled).

---

Well what the error says is that pickle (and with that, python) wasn't able to find the function as it was detected (`MyXGBoostClassifier.my_loss`).

Ad hock, i'd say it (falsely) assumes the function is part of the class (i guess it could be in the class?), or it could be an accidental naming collision, maybe with the filename ? 


Resolving patch:
diff --git a/freqtrade/freqai/data_drawer.py b/freqtrade/freqai/data_drawer.py
--- a/freqtrade/freqai/data_drawer.py
+++ b/freqtrade/freqai/data_drawer.py
@@ -12,7 +12,6 @@
 import pandas as pd
 import psutil
 import rapidjson
-from joblib import dump, load
 from joblib.externals import cloudpickle
 from numpy.typing import NDArray
 from pandas import DataFrame
@@ -471,7 +470,8 @@ def save_data(self, model: Any, coin: str, dk: FreqaiDataKitchen) -> None:
 
         # Save the trained model
         if self.model_type == 'joblib':
-            dump(model, save_path / f"{dk.model_filename}_model.joblib")
+            with (save_path / f"{dk.model_filename}_model.joblib").open("wb") as fp:
+                cloudpickle.dump(model, fp)
         elif self.model_type == 'keras':
             model.save(save_path / f"{dk.model_filename}_model.h5")
         elif self.model_type in ["stable_baselines3", "sb3_contrib", "pytorch"]:
@@ -558,7 +558,8 @@ def load_data(self, coin: str, dk: FreqaiDataKitchen) -> Any:  # noqa: C901
         if dk.live and coin in self.model_dictionary:
             model = self.model_dictionary[coin]
         elif self.model_type == 'joblib':
-            model = load(dk.data_path / f"{dk.model_filename}_model.joblib")
+            with (dk.data_path / f"{dk.model_filename}_model.joblib").open("rb") as fp:
+                model = cloudpickle.load(fp)
         elif 'stable_baselines' in self.model_type or 'sb3_contrib' == self.model_type:
             mod = importlib.import_module(
                 self.model_type, self.freqai_info['rl_config']['model_type'])


Test patch: