Repository: intel/ipex-llm
Pull request: Chronos : add scheduler for Autoformer
## Description

add scheduler for Autoformer to fix https://github.com/intel-analytics/BigDL/issues/5096

## Related Link or Issue

https://github.com/intel-analytics/BigDL/issues/5096

## How was this PR tested?

- [x] jenkins passed (please provide link):
http://10.112.231.51:18888/job/BigDL-Chronos-PR-Validation/1131/
http://10.112.231.51:18888/job/BigDL-PRNV-chronos-Python-Spark-2.4-py37-Horovod/1270/

Problem statement:
Chronos : can't reproduce paper's result as lack of scheduler
I can't reproduce paper's result on the setting of Autoformer 97-720 forecast.
I find the reason is current implementation of Autoformer lack of a lr scheduler comparing with the source code.

Hints from issue discussion:

Commit message: add scheduler

Patch to review:
diff --git a/python/chronos/src/bigdl/chronos/model/autoformer/Autoformer.py b/python/chronos/src/bigdl/chronos/model/autoformer/Autoformer.py
index 1e3837788f3..c26c9365d93 100644
--- a/python/chronos/src/bigdl/chronos/model/autoformer/Autoformer.py
+++ b/python/chronos/src/bigdl/chronos/model/autoformer/Autoformer.py
@@ -179,7 +179,9 @@ def predict_step(self, batch, batch_idx):
         return outputs
 
     def configure_optimizers(self):
-        return getattr(optim, self.optim)(self.parameters(), lr=self.lr)
+        optimizer = getattr(optim, self.optim)(self.parameters(), lr=self.lr)
+        scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones=[3,4,5,6,7,8,9,10], gamma=0.5, verbose=True)
+        return [optimizer], [scheduler]
 
 
 def model_creator(config):


Code review comments:
diff_hunk: @@ -179,7 +179,9 @@ def predict_step(self, batch, batch_idx):
         return outputs
 
     def configure_optimizers(self):
-        return getattr(optim, self.optim)(self.parameters(), lr=self.lr)
+        optimizer = getattr(optim, self.optim)(self.parameters(), lr=self.lr)
+        scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones=[3,4,5,6,7,8,9,10], gamma=0.5, verbose=True)
path: python/chronos/src/bigdl/chronos/model/autoformer/Autoformer.py
text: @user1:
I think we should allow users to set scheduler / milestones (I think this might be a better choice). Furthermore ,they may be allowed to disable scheduler.

@author:
done