Package: building-energy-storage-simulation
Building Energy Storage SimulationThe Building Energy Storage Simulation serves as OpenAI gym (nowgymnasium) environment
for Reinforcement Learning. The environment represents a building with an energy storage (in form of a battery) and a
solar energy system. The building is connected to a power grid with time varying electricity prices. The task is to
control the energy storage so that the total cost of electricity are minimized.The inspiration of this project and the data profiles come from theCityLearnenvironment. Anyhow, this project focuses on the ease of usage and the simplicity of its implementation. Therefore, this project serves as playground for those who want to get started with reinforcement learning for energy management system control.InstallationBy using pip just:pip install building-energy-storage-simulationor if you want to continue developing the package:git clone https://github.com/tobirohrer/building-energy-storage-simulation.git && cd building-energy-storage-simulation
pip install -e .[docs,tests]Usagefrombuilding_energy_storage_simulationimportEnvironment,BuildingSimulationsimulation=BuildingSimulation()env=Environment(building_simulation=simulation)env.reset()env.step(1)...Important note:This environment is implemented by usinggymnasium(the proceeder of OpenAI gym). Meaning, if you are using a reinforcement learning library likeStable Baselines3make sure it supportsgymnasiumenvironments.Task DescriptionThe simulation contains a building with an energy load profile attached to it. The load is always automatically covered byprimarily using electricity generated by the solar energy system,and secondary by using the remaining required electricity "from the grid"When energy is taken from the grid, costs are incurred that can vary depending on the time (if a price profile is passed
aselectricity_pricetoBuildingSimulation). The simulated building contains a battery which be controlled bycharginganddischargingenergy. The goal is to find control strategies optimize the use of the energy storage
by e.g. charging whenever electricity prices are high or whenever there is a surplus of solar generation. It is important
to note that no energy can be fed into the grid. This means any surplus of solar energy which is not used to charge the
battery is considered lost.Reward$$ r_t = -1 * electricity_consumed_t * electricity_price_t$$Note, that the termelectricity_consumedcannot be negative. This means, excess energy from the solar
energy system which is not consumed by the electricity load or by charging the battery is considered lost
(electricity_consumedis 0 in this case).Action SpaceActionMinMaxCharge-11The actions lie in the interval of [-1;1]. The action represents a fraction of the maximum energy which can be retrieved from the battery (or used to charge the battery) per time step.1 means maximum charging the battery. The maximum charge per time step is defined by the parametermax_battery_charge_per_timestep.-1 means maximum discharging the battery, meaning "gaining" electricity out of the battery0 means don't charge or dischargeObservation SpaceIndexObservationMinMax0State of Charge (in kWh)0battery_capacity[1; n]Forecast Electric Load (in kWh)Min of Load ProfileMax of Load Profile[n+1; 2*n]Forecast Solar Generation (in kWh)Min of Generation ProfileMax of Generation Profile[2n+1; 3*n]Electricity Price (in € cent per kWh)Min of Price ProfileMax of Price ProfileThe length of the observation depends on the length of the forecast ($n$) used. By default, the simulation uses a forecast length of 4.
This means 4 time steps of an electric load forecast, 4 time steps of a solar generation forecast and 4 time steps of the
electric price profile are included in the observation.
In addition to that, the information about the current state of charge of the battery is contained in the observation.The length of the forecast can be defined by setting the parameternum_forecasting_stepsof theEnvironment().Episode EndsThe episode ends if themax_timestepsof theEnvironment()are reached.Example SolutionsThe folderexample_solutionscontains three different example solutions to solve the problem
described.By applying deep reinforcement learning using the frameworkstable-baselines3.By formulating the problem as optimal control problem (OCP) usingpyomo. In this case, it
is assumed that the forecast for the price, load and generation data for the whole period is available.By model predictive control, which solves the optimal control problem formulation from 2. in each time step in a closed loop manner.
In contrast to 2. only a forecast of a fixed length is given in each iteration.Note that the execution of the example solutions requires additional dependencies which are not specified insidesetup.py.
Therefore, make sure to install the required python packages defined inrequirements.txt. Additionally, an installation
of theipoptsolver is required in order to solve the optimal control problem
(by using conda, simply runconda install -c conda-forge ipopt).Code DocumentationThe documentation is available athttps://building-energy-storage-simulation.readthedocs.io/Contribute & ContactAs I just started with this project, I am very happy for any kind of
contribution! In case you want to contribute, or if you have any
questions, contact me viadiscord.