CRediT authorship contribution statement

**Mohamed Habib Jabeur:** Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Resources, Software, Visualization, Writing – original draft, Writing – review & editing, Validation. **Sonia Mahjoub:** Conceptualization, Formal analysis, Funding acquisition, Investigation, Project administration, Supervision, Validation, Writing – original draft, Writing – review & editing, Resources, Visualization. **Cyril Toublanc:** Formal analysis, Funding acquisition, Investigation, Project administration, Resources, Supervision, Validation, Visualization, Writing – review & editing. **Veronique Cariou:** Conceptualization, Formal analysis, Funding acquisition, Investigation, Project administration, Resources, Supervision, Validation, Writing – review & editing, Visualization.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

Data will be made available on request.

References

Alves, F. F., Nogueira, T. H., De Souza, M. C., & Ravetti, M. G. (2021). Approaches for the joint resolution of lot-sizing and scheduling with infeasibilities occurrences. *Computers & Industrial Engineering, **155*, Article 107176. https://doi.org/10.1016/j.cie.2021.107176

Babaei, M., Mohammadi, M., & Ghomi, S. M. T. F. (2014). A genetic algorithm for the simultaneous lot sizing and scheduling problem in capacitated flow shop with complex setups and backlogging. *The International Journal of Advanced Manufacturing Technology, **70***, 125–134. https://doi.org/10.1007/s00170-013-5252-y

Basán, N. P., Cóccola, M. E., Dondo, R. G., Guarnaschelli, A., Schweickardt, G. A., & Méndez, C. A. (2020). A reactive-iterative optimization algorithm for scheduling of air separation units under uncertainty in electricity prices. *Computers & Chemical Engineering, **142***, Article 107050. https://doi.org/10.1016/j.compchemeng.2020.107050

Carvalho, D. M., & Nascimento, M. C. V. (2022). Hybrid matheuristics to solve the integrated lot sizing and scheduling problem on parallel machines with sequence-dependent and non-triangular setup. *European Journal of Operational Research, **296***, 158–173. https://doi.org/10.1016/j.ejor.2021.03.050

Cheng, L., Tang, Q., Zhang, L., & Yu, C. (2022). Scheduling flexible manufacturing cell with no-idle flow-lines and job-shop via Q-learning-based genetic algorithm. *Computers & Industrial Engineering, **169***, Article 108293. https://doi.org/10.1016/j.cie.2022.108293

Duarte, J. L. R., Fan, N., & Jin, T. (2020). Multi-process production scheduling with variable renewable integration and demand response. *European Journal of Operational Research, **281***, 186–200.

Eurostat. Energy. Transport and environment statistics - 2019 edition. 2019th ed. Luxembourg: Publications Office of the European Union; 2019. https://doi.org/10.2785/660147.

Golari, M., Fan, N., & Jin, T. (2017). Multistage stochastic optimization for production-inventory planning with intermittent renewable energy. *Production and Operations Management, **26***, 409–425. https://doi.org/10.1111/poms.12657

Gomes, E. R., & Kowalczyk, R. (2009). Dynamic analysis of multiagent Q-learning with -greedy exploration 8. In *Proceedings of the 26th annual international conference on machine learning*.

Kelley, M. T., Baldick, R., & Baldea, M. (2019). Demand response operation of electricity intensive chemical processes for reduced greenhouse gas emissions: Application to an air Separation unit. *Sustainable Chemistry & Engineering, **7**(2), 1909–1922. https://doi.org/10.1021/acssuschemeng.8b03927

Kelley, M. T., Baldick, R., & Baldea, M. (2020). Demand response scheduling under uncertainty: Chance- constrained framework and application to an air separation unit. *AIChE Journal*. https://doi.org/10.1002/aic.16273

Kelley, M. T., Pattison, R. C., Baldick, R., & Baldea, M. (2018). An MILP framework for optimizing demand response operation of air separation units. *Applied Energy, **222***, 951–966. https://doi.org/10.1016/j.apenergy.2017.12.127

Lei, K., Guo, P., Zhao, W., Wang, Y., Qian, L., Meng, X., & Tang, L. (2022). A multi-action deep reinforcement learning framework for flexible job-shop scheduling problem. *Expert Systems with Applications, **205***, Article 117796. https://doi.org/10.1016/j.eswa.2022.117796

Li, J., Sang, H., Han, Y., Wang, C., & Gao, K. (2018). Efficient multi-objective optimization algorithm for hybrid flow shop scheduling problems with setup energy consumptions. *Journal of Cleaner Production, **181***, 584–598.

Lin, J., Li, Y.-Y., & Song, H.-B. (2022). Semiconductor final testing scheduling using Q-learning based hyper-heuristic. *Expert Systems with Applications, **187***, Article 115978. https://doi.org/10.1016/j.eswa.2022.115978

Mahdieh, M., Bijari, M., & Clark, A. (2011). Simultaneous lot sizing and scheduling in a flexible flow line. *Journal of Industrial and Systems Engineering, **5***, 107–119.

Masmoudi, O., Yalaoui, A., Ouazene, Y., & Chehade, H. (2017). Lot-sizing in a multi-stage flow line production system with energy consideration. *International Journal of Production Research, **55***, 1640–1663.

Mazyavkina, N., Sviridov, S., Ivanov, S., & Burnaev, E. (2021). Reinforcement learning for combinatorial optimization: A survey. *Computers & Operations Research, **134***, Article 105400. https://doi.org/10.1016/j.cor.2021.105400

Van Moffaert, K. V., & Nowe, A. (2014). Multi-objective reinforcement Learning using sets of Pareto dominating policies. *The Journal of Machine Learning Research, **15***, 3483–3512.

Özdamar, L., & Barbarasoglu, G. (1999). Hybrid heuristics for the multi-stage capacitated lot sizing and loading problem. *Journal of the Operational Research Society, **50***, 810–825.

Peinado-Guerrero, M. A., & Villalobos, J. R. (2022). Using inventory as energy storage for demand-side management of manufacturing operations. *Journal of Cleaner Production, **375***, Article 134213. https://doi.org/10.1016/j.jclepro.2022.134213

R Core Team. (2021). *R: A language and environment for statistical computing*. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org/.

Rodoplu, M., Arbaoui, T., & Yalaoui, A. (2020). A fix-and-relax heuristic for the single-item lot-sizing problem with a flow-shop system and energy constraints. *International Journal of Production Research, **58***, 6532–6552. https://doi.org/10.1080/00207543.2019.1683249

Shrouf, F., Ordieres-Meré, J., García-Sánchez, A., & Ortega-Mier, M. (2014). Optimizing the production scheduling of a single machine to minimize total energy consumption costs. *Journal of Cleaner Production, **67***, 197–207. https://doi.org/10.1016/j.jclepro.2013.12.024

Sun, Z., Li, L., Fernández, M., & Wang, J. (2014). Inventory control for peak electricity demand reduction of manufacturing systems considering the tradeoff between production loss and energy savings. *Journal of Cleaner Production, **82***, 84–93. https://doi.org/10.1016/j.jclepro.2014.06.071

Trevino-Martinez, S., Sawhney, R., & Sims, C. (2022). Energy-carbon neutrality optimization in production scheduling via solar net metering. *Journal of Cleaner Production, **380***, Article 134627. https://doi.org/10.1016/j.jclepro.2022.134627

Valledor, P., Gomez, A., Priore, P., & Puente, J. (2018). Solving multi-objective rescheduling problems in dynamic permutation flow shop environments with disruptions. *International Journal of Production Research, **56***, 6363–6377. https://doi.org/10.1080/00207543.2018.1468095

Wang, L., Hu, X., Wang, Y., Xu, S., Ma, S., Yang, K., Liu, Z., & Wang, W. (2021). Dynamic job-shop scheduling in smart manufacturing using deep reinforcement learning. *Computer Networks, **190***, Article 107969. https://doi.org/10.1016/j.comnet.2021.107969

Wang, S., Mason, S. J., & Gangammanavar, H. (2020). Stochastic optimization for flow-shop scheduling with on-site renewable energy generation using a case in the United States. *Computers & Industrial Engineering, **149***, Article 106812.

Yan, Q., Wang, H., & Wu, F. (2022). Digital twin-enabled dynamic scheduling with preventive maintenance using a double-layer Q-learning algorithm. *Computers & Operations Research, **144***, Article 105823. https://doi.org/10.1016/j.cor.2022.105823

Xiao, J., Yang, H., Zhang, C., Zheng, L., & Gupta, J. N. D. (2015). A hybrid lagrangian-simulated annealing-based heuristic for the parallel-machine capacitated lot-sizing and scheduling problem with sequence-dependent setup times. *Computers & Operations Research, **63***, 72–82. https://doi.org/10.1016/j.cor.2015.04.010

Zou, F., Yen, G. G., Tang, L., & Wang, C. (2021). A reinforcement learning approach for dynamic multi-objective optimization. *Information Sciences, **546***, 815–834. https://doi.org/10.1016/j.ins.2020.08.101
CRediT authorship contribution statement

**Mohamed Habib Jabeur:** Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Resources, Software, Visualization, Writing – original draft, Writing – review & editing, Validation. **Sonia Mahjoub:** Conceptualization, Formal analysis, Funding acquisition, Investigation, Project administration, Supervision, Validation, Writing – original draft, Writing – review & editing, Resources, Visualization. **Cyril Toublanc:** Formal analysis, Funding acquisition, Investigation, Project administration, Resources, Supervision, Validation, Visualization, Writing – review & editing. **Veronique Cariou:** Conceptualization, Formal analysis, Funding acquisition, Investigation, Project administration, Resources, Supervision, Validation, Writing – review & editing, Visualization.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

Data will be made available on request.

References

Alves, F. F., Nogueira, T. H., De Souza, M. C., & Ravetti, M. G. (2021). Approaches for the joint resolution of lot-sizing and scheduling with infeasibilities occurrences. *Computers & Industrial Engineering, 155*, Article 107176. https://doi.org/10.1016/j.cie.2021.107176

Babaei, M., Mohammadi, M., & Ghomi, S. M. T. F. (2014). A genetic algorithm for the simultaneous lot sizing and scheduling problem in capacitated flow shop with complex setups and backlogging. *The International Journal of Advanced Manufacturing Technology, 70*, 125–134. https://doi.org/10.1007/s00170-013-5252-y

Basán, N. P., Cóccola, M. E., Dondo, R. G., Guarnaschelli, A., Schweickardt, G. A., & Méndez, C. A. (2020). A reactive-iterative optimization algorithm for scheduling of air separation units under uncertainty in electricity prices. *Computers & Chemical Engineering, 142*, Article 107050. https://doi.org/10.1016/j.compchemeng.2020.107050

Carvalho, D. M., & Nascimento, M. C. V. (2022). Hybrid matheuristics to solve the integrated lot sizing and scheduling problem on parallel machines with sequence-dependent and non-triangular setup. *European Journal of Operational Research, 296*, 158–173. https://doi.org/10.1016/j.ejor.2021.03.050

Cheng, L., Tang, Q., Zhang, L., & Yu, C. (2022). Scheduling flexible manufacturing cell with no-idle flow-lines and job-shop via Q-learning-based genetic algorithm. *Computers & Industrial Engineering, 169*, Article 108293. https://doi.org/10.1016/j.cie.2022.108293

Duarte, J. L. R., Fan, N., & Jin, T. (2020). Multi-process production scheduling with variable renewable integration and demand response. *European Journal of Operational Research, 281*, 186–200.

Eurostat. Energy. Transport and environment statistics - 2019 edition. 2019th ed. Luxembourg: Publications Office of the European Union; 2019. https://doi.org/10.2785/660147.

Golari, M., Fan, N., & Jin, T. (2017). Multistage stochastic optimization for production-inventory planning with intermittent renewable energy. *Production and Operations Management, 26*, 409–425. https://doi.org/10.1111/poms.12657

Gomes, E. R., & Kowalczyk, R. (2009). Dynamic analysis of multiagent Q-learning with -greedy exploration 8. *In Proceedings of the 26th annual international conference on machine learning*.

Kelley, M. T., Baldick, R., & Baldea, M. (2019). Demand response operation of electricity intensive chemical processes for reduced greenhouse gas emissions: Application to an air Separation unit. *Sustainable Chemistry & Engineering, 7*(2), 1909–1922. https://doi.org/10.1021/acssuschemeng.8b03927

Kelley, M. T., Baldick, R., & Baldea, M. (2020). Demand response scheduling under uncertainty: Chance- constrained framework and application to an air separation unit. *AIChE Journal*. https://doi.org/10.1002/aic.16273

Kelley, M. T., Pattison, R. C., Baldick, R., & Baldea, M. (2018). An MILP framework for optimizing demand response operation of air separation units. *Applied Energy, 222*, 951–966. https://doi.org/10.1016/j.apenergy.2017.12.127

Lei, K., Guo, P., Zhao, W., Wang, Y., Qian, L., Meng, X., & Tang, L. (2022). A multi-action deep reinforcement learning framework for flexible job-shop scheduling problem. *Expert Systems with Applications, 205*, Article 117796. https://doi.org/10.1016/j.eswa.2022.117796

Li, J., Sang, H., Han, Y., Wang, C., & Gao, K. (2018). Efficient multi-objective optimization algorithm for hybrid flow shop scheduling problems with setup energy consumptions. *Journal of Cleaner Production, 181*, 584–598.

Lin, J., Li, Y.-Y., & Song, H.-B. (2022). Semiconductor final testing scheduling using Q-learning based hyper-heuristic. *Expert Systems with Applications, 187*, Article 115978. https://doi.org/10.1016/j.eswa.2022.115978

Mahdieh, M., Bijari, M., & Clark, A. (2011). Simultaneous lot sizing and scheduling in a flexible flow line. *Journal of Industrial and Systems Engineering, 5*, 107–119.

Masmoudi, O., Yalaoui, A., Ouazene, Y., & Chehade, H. (2017). Lot-sizing in a multi-stage flow line production system with energy consideration. *International Journal of Production Research, 55*, 1640–1663.

Mazyavkina, N., Sviridov, S., Ivanov, S., & Burnaev, E. (2021). Reinforcement learning for combinatorial optimization: A survey. *Computers & Operations Research, 134*, Article 105400. https://doi.org/10.1016/j.cor.2021.105400

Van Moffaert, K. V., & Nowe, A. (2014). Multi-objective reinforcement Learning using sets of Pareto dominating policies. *The Journal of Machine Learning Research, 15*, 3483–3512.

Özdamar, L., & Barbarasoglu, G. (1999). Hybrid heuristics for the multi-stage capacitated lot sizing and loading problem. *Journal of the Operational Research Society, 50*, 810–825.

Peinado-Guerrero, M. A., & Villalobos, J. R. (2022). Using inventory as energy storage for demand-side management of manufacturing operations. *Journal of Cleaner Production, 375*, Article 134213. https://doi.org/10.1016/j.jclepro.2022.134213

R Core Team. (2021). *R: A language and environment for statistical computing*. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org/.

Rodoplu, M., Arbaoui, T., & Yalaoui, A. (2020). A fix-and-relax heuristic for the single-item lot-sizing problem with a flow-shop system and energy constraints. *International Journal of Production Research, 58*, 6532–6552. https://doi.org/10.1080/00207543.2019.1683249

Shrouf, F., Ordieres-Meré, J., García-Sánchez, A., & Ortega-Mier, M. (2014). Optimizing the production scheduling of a single machine to minimize total energy consumption costs. *Journal of Cleaner Production, 67*, 197–207. https://doi.org/10.1016/j.jclepro.2013.12.024

Sun, Z., Li, L., Fernández, M., & Wang, J. (2014). Inventory control for peak electricity demand reduction of manufacturing systems considering the tradeoff between production loss and energy savings. *Journal of Cleaner Production, 82*, 84–93. https://doi.org/10.1016/j.jclepro.2014.06.071

Trevino-Martinez, S., Sawhney, R., & Sims, C. (2022). Energy-carbon neutrality optimization in production scheduling via solar net metering. *Journal of Cleaner Production, 380*, Article 134627. https://doi.org/10.1016/j.jclepro.2022.134627

Valledor, P., Gomez, A., Priore, P., & Puente, J. (2018). Solving multi-objective rescheduling problems in dynamic permutation flow shop environments with disruptions. *International Journal of Production Research, 56*, 6363–6377. https://doi.org/10.1080/00207543.2018.1468095

Wang, L., Hu, X., Wang, Y., Xu, S., Ma, S., Yang, K., Liu, Z., & Wang, W. (2021). Dynamic job-shop scheduling in smart manufacturing using deep reinforcement learning. *Computer Networks, 190*, Article 107969. https://doi.org/10.1016/j.comnet.2021.107969

Wang, S., Mason, S. J., & Gangammanavar, H. (2020). Stochastic optimization for flow-shop scheduling with on-site renewable energy generation using a case in the United States. *Computers & Industrial Engineering, 149*, Article 106812.

Yan, Q., Wang, H., & Wu, F. (2022). Digital twin-enabled dynamic scheduling with preventive maintenance using a double-layer Q-learning algorithm. *Computers & Operations Research, 144*, Article 105823. https://doi.org/10.1016/j.cor.2022.105823

Xiao, J., Yang, H., Zhang, C., Zheng, L., & Gupta, J. N. D. (2015). A hybrid lagrangian-simulated annealing-based heuristic for the parallel-machine capacitated lot-sizing and scheduling problem with sequence-dependent setup times. *Computers & Operations Research, 63*, 72–82. https://doi.org/10.1016/j.cor.2015.04.010

Zou, F., Yen, G. G., Tang, L., & Wang, C. (2021). A reinforcement learning approach for dynamic multi-objective optimization. *Information Sciences, 546*, 815–834. https://doi.org/10.1016/j.ins.2021.08.101