BCStream - a data streaming based system for processing energy consumption data and integrating with social media

Lowering energy consumption is one of the biggest challenges in today’s society. Each and everyday we use a large amount of electricity to support our lifestyle. According to the U.S. Energy Information Administration, the energy usage will increase by 56% by 2040. Thus new innovative solutions to reduce power consumption are in great need. Consumers typically do not have the information they need to take proactive decisions about their energy usage. Therefore, the potential for saving energy might rely on the awareness and involvement in energy conservation and waste reduction. One of smart grid’s visions is to enable utilities and consumers to more effectively monitor, control energy usage and cost. Advanced Metering Infrastructure (AMI) is an important part of smart grid, it is a two-way communication system between smart meters and utilities, where smart meters transmit consumption data to the utilities. Smart meter is a digital metering device, which not only shows real-time feedback of the energy consumption but also supports remote communication with utilities. Consumers can receive their power consumption data in real-time, which allows them to make informed choices about their energy usage. In this sense, smart grids, AMIs and smart meters can be used to make people aware of their energy consumption. Additionally, energy usage trends can be illustrated to consumers by applying different queries to extract energy consumption data from the AMIs. At the same time, social media is one of the biggest communication tools used to reach millions of people. By integrating energy usage data produced by AMIs on social media, consumers can inspire and influence others to match the energy they use with their needs and lifestyles. As more and more smart meters are installed across power grids, huge volumes of data are generated. In order to capture and analyze large data sets generated by smart meters within AMIs, challenges such as how to cope with huge volumes of data appears. Data Stream Processing models, utilized to process the huge amount of continuous data and extract interesting data, emerge as a possible approach to transform these real-time operational data into applied insights. This master thesis presents BCStream, a data streaming based application, which consumes energy consumption data on the fly and produces feedback to users on social media in order to raise their energy usage awareness. BCStream utilizes a Stream Processing Engine to process a large amount of continuous data in real-time. In order to show that it is possible to address the aforementioned challenges, we implement and evaluate BCStream using several use-cases. These use-cases provide consumers different insights of their energy usage. For instance, in the first use-case, the proposed system determines at which hour of a day and which day of a week the power is consumed the most. In the second use-case, BCStream helps users find the top five houses using the most energy in the consumers’ neighborhood. Finally, the third use-case locates top five most energy consuming devices per household. In order to evaluate the performance of BCStream, real energy readings are modified and used as input data to the use-cases. In order to conduct the evaluation, we use a single machine that relies on a multi-core architecture to demonstrate good performance (between 160,000 and 170,000 energy consumption readings) and scalability.