# Simulation-Supported Testing of Smart Energy Product Prototypes
## 4 Discussion and Conclusions  
The operation of the three HEMP prototypes was successfully tested using a simulation environment, proving the usefulness of this tool for quickly and accurately validating the operation of SEPS using simulated PV production and load profiles. The tests presented a simple, quick visualisation of how these prototypes would operate in households without the need to involve the end users themselves. This approach can be useful during the early product development phase for rapidly testing several modes of operation and determining which one is best suited for achieving the intended purpose of a given SEPS. Although the proposed testing sequences were relatively simple, they provided a clear demonstration of how the HEMPs would operate in practice, and there is potential for improving the accuracy of these tests by designing more complex testing sequences. Future simulation testing of the presented HEMP concepts should also explore the potential for incorporating other methods such as co-simulation in order to obtain more accurate results. These tests would require the development of an agent-based model of product influence on user behaviour and the resulting impact on the residual load profile in order to upscale the physical device to a large number of simulated devices. The scenario simulations and end user testing served as a more extensive test on HEMP performance, which helped identify some of the advantages and limitations of the current designs. For instance, tests on the Bodhi concept helped identify a recurring daily R\({}_{\text{B}}\) profile but were limited by inaccurate predictions for the household's energy budget, highlighting the importance of correctly estimating this type of parameter during its operation. LightInsight, on the other hand, was able to present simple, intuitive information about how energy flows in a household, but was found to be useful for only a small fraction of the day since by definition net consumption takes place whenever the sun is down. The rapid fluctuations observed in the energy profiles also meant that the proposed transition states were extremely rare, meaning they should be restructured to better respond to the observed user behaviour. Overall, the proposed feedback algorithms required only basic energy data in order to determine feedback to users. However, developing more complex algorithms that use other variables as inputs and respond to changes in use patterns could help improve the effectiveness of these concepts in the future or even add new functions, such as the scheduling algorithm for smart appliances presented in [27]. The scenario-based simulations were also intended to replicate the conditions observed during the end user testing. User test performance for the LightInsight concept was more closely approximated by Scenario 1 with a root mean square error (RMSE) between both datasets of 1.58 followed by Scenarios 3, 4 and 2 (RMSE = 1.61, 1.96 and 2.43, respectively). For Bodhi, the summer scenario (RMSE = 0.31) matched the user tests more closely than the winter scenario (RMSE = 0.35). Overall, while the simulations approximated some of the general trends observed in the user tests, they are still far from  
Figure 12: LightInsight performance during testing phase. Background colour corresponds to the light colour shown by the prototype LEDs; the yellow line indicates the balance point between energy consumption and production. being a significant predictor of SEPS operation. Comparing HEMP performance during user tests to their reference measurements, on the other hand, revealed that the concepts did not always seem to achieve their intended purpose. Testing on LightInsight resulted in an increase in both the average load (25%) and the peak load (3%) compared to the reference phase, and a more deficient match between energy supply and demand. User tests with Bodhi, on the other hand, had a positive impact since the overall energy consumption showed a significant decrease averaging 27% less than in the reference phase.