# Smart Grid for Industry Using Multi-Agent Reinforcement Learning
## 5 Case Study
### Set-Up  
The production system was derived from a contract manufacturing business for surface treatment. The portion of the factory site having the main energy demand consists of five machines in total, where jobs with a fixed lot size are produced. In every resource, between two and four different job types  
Figure 8: Reward assignment in every time step. can be machined, each requiring a specific set-up state. Additional parameters like stochastic events and delays as well as storage costs are considered, like those discussed previously. The modelling data of the production system are briefly summarized in Table 2.  
On the energy supply side, data from a solar power plant near the production site were used as a VPS. In addition, the company had a local CHP plant with a maximum generation capacity of 32 kW and a lithium battery system with a total capacity of 35 kWh. To settle load deviation from the public grid, a tolerance of \(\pm 5\%\) and costs of 200 \(\upmu\)/MWh were implied. Considering the electricity market, the energy prices at the German intraday market from the period October 2018 to September 2019 were used. In addition, it was assumed that a price forecast would be available at a standard deviation of 5%. The energy demand of a production shift was determined by the number of jobs to be machined, which is thus not constant. In order to still be able to generate comparable scenarios, the following dimensioning of the elements of energy supply was assumed: there exists a solar power plant which provides roughly 25% of the total energy demand; 55% has already been purchased in advance, thus forming a constant base load, and the rest can be provided by the CPS. Comparable scenarios were calculated in this way, the used data is summarized in Table 3.  
In order to generate sufficient training data, a simple training data generator was introduced. At the beginning of every training episode, arbitrary jobs were generated for every production resource. Thereby, the overall net utilization, without considering set-up, was random at between 60 and 80%. On the energy side, a representative day of the solar power and market data was extracted for every episode and changed by a normal distribution. A sufficient amount of realistic training data was able to be provided in this way. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Category** & **Parameter** & **Value** \\ \hline \multirow{4}{*}{Production resource and jobs} & Total number of resources & 5 \\  & Number of job types, which can be machined on the same resource & 2–4 \\  & Machining duration of a job depending on the job type & 21–42 min \\  & Average power consumption of a job depending on the job type & 20–48 kW \\  & Set-up duration of a job depending on the job type & 6–12 min \\ \hline \multirow{4}{*}{Stochastic events} & Probability for the arrival of rush jobs during a shift & \multirow{2}{*}{5\% (\(\alpha=2.5\%\))} \\  & (normally distributed) & \\ \cline{1-1}  & Probability for a resource breakdown in every time step & 1\% \\ \cline{1-1}  & Average duration of a breakdown (normally distributed) & 12 min (\(\sigma=3\) min) \\ \hline \multirow{2}{*}{Costs} & Delay costs of a job & 48–88 \(\upmu\)/h \\  & Storage costs of a job & 5.2–11 \(\upmu\)/h \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Modelling data of the production system.