Package: smartmeter-datacollector
Smart Meter Data CollectorThesmartmeter-datacollectortool is a Python3 software which allows you to continuously retrieve data from supported smart meters (listed below).
The acquired values can be forwarded to one or more data sinks like a MQTT broker or logger.This tool is an open source software funded byEKZ (Elektrizitätswerke des Kantons Zürich)and developed bySupercomputing Systems AG. The goal of this voluntary initiative is to create a tool that can read, decode and decrypt data (if necessary) from the local interface of various smart meters and convert it into human-readable and simple formats. This should eliminate the complexity of decoding DLMS and similar protocols for the user. The user has the data in a format, which he can easily use in various use-cases ranging from simple visualization (seeMethod 1: Raspberry Pi image with demo) to integration into energy management algorithms or into custom home automation software.The following smart meters are supported (seeWiki/Homefor detailed information).Landis+Gyr E450:Data pushed by smart meter over CII interface (wired M-Bus, HDLC, DLMS/COSEM).Landis+Gyr E360:Data pushed by smart meter over P1 interface (HDLC, DLMS/COSEM only, no DSMR).Iskraemeco AM550:Data pushed by smart meter over P1 interface (HDLC, DLMS/COSEM only, no DSMR).Kamstrup OMNIPOWER with HAN-NVE:Data pushed by smart meter over insertedHAN-NVE module(wired M-Bus, HDLC, DLMS/COSEM).Note: All smart meters integrated so far push binary data encoded with HDLC (IEC 62056-46) and DLMS/COSEM. Both unencrypted and encrypted DLMS messages are accepted by the software.The following data sinks are implemented:MQTT (v3.1.1):EncryptionUnencrypted connectionEncrypted connectionTLSoptional custom CA certificateAuthenticationUnauthenticatedAuthenticated with username / passwordAuthenticated with client certificateLogger tostdoutsmartmeter-datacollectoris fully configurable through a.iniconfiguration file. Thesmartmeter-datacollector-configuratorwebinterface can help to create and modify the configuration.Smart Meter Data CollectorHow to installMethod 1: Raspberry Pi image with demoMethod 2: Python packagePython RequirementsInstallationMethod 3: Debian packageDebian RequirementsInstallationHow to useConfigurationSpecificationsmartmeter-datacollector-configuratorRun manuallyConfigurationRun as a systemd serviceConfigurationHow to developDevelopment RequirementsInstallationDebian / UbuntuCheckout the codeSetup Development EnvironmentCustom Commands / WorkflowsAcknowledgementsHow to installThere are different methods how to usesmartmeter-datacollector.Raspberry Pi image with demoPython packageDebian packageMethod 1: Raspberry Pi image with demoFor a very easy first time usage ofsmartmeter-datacollectorwe provide aRaspberry Pi image(based on Raspberry Pi OS) which contains the following parts:smartmeter-datacollectoras a systemd servicesmartmeter-datacollector-configuratorwebinterfacedemomosquiottoas a local MQTT broker to publish the measurements fromsmartmeter-datacollectorTelegrafto collect measurements published bysmartmeter-datacollectorover MQTT and store them in InfluxDBInfluxDBto store the measurementsGrafanato visualize the measurementsSeeWiki/Demofor detailed setup instructions.Method 2: Python packagesmartmeter-datacollectorcan be installed as a Python3 package either fromPyPior manually using areleased wheel.The Python3 package does not contain any infrastructure to runsmartmeter-datacollectorin the background or to automatically start it during a boot sequence. If such infrastructure is required either seeMethod 3: Debian packageor provide it yourself.Python RequirementsPython >= 3.7 (tested with 3.8)InstallationInstall the package either as global Python package or in a virtualenv withpython3-mpipinstallsmartmeter-datacollectorSimilarly thesmartmeter-datacollector-configuratorwebinterface can be installed withpython3-mpipinstallsmartmeter-datacollector-configuratorMethod 3: Debian packagesmartmeter-datacollectoris also available as a Debian (.deb) package from thereleaseswhich installs it system wide. The Debian package includes a systemd service file which enablessmartmeter-datacollectorto automatically start after booting the system.Debian RequirementsDistributionDebianReleaseBullseyeCPU architectureamd64armhfInstallationDownload the Debian package fromreleasesand install it withsudoaptinstall./python3-smartmeter-datacollector_*.debSimilarly thesmartmeter-datacollector-configuratorwebinterface can be installed with a Debian package from itsreleaseswithsudoaptinstall./python3-smartmeter-datacollector-configurator_*.debHow to useThe usage ofsmartmeter-datacollectordepends on the installation method. Independent of the installation method a.iniconfiguration file is required to properly runsmartmeter-datacollector.ConfigurationManually create configurationSeeWiki/Configurationfor more details on the available configuration options.smartmeter-datacollector-configuratorTo simplify the process of generating a valid.iniconfiguration forsmartmeter-datacollectorthe companionsmartmeter-datacollector-configuratorwebinterface can be used. It supportsa graphical approach to manage the configurationinput validation to avoid invalid configurationsloading / saving / discarding a configurationrestartingsmartmeter-datacollector(only if installed as a Debian package)SeeWiki/smartmeter-datacollector-configuratorfor more details on the webinterface and the configuration possibilities.Run manuallyRunsmartmeter-datacollectorwith the following command:smartmeter-datacollector --config datacollector.iniThe following command line arguments are supported:-h, --help: Shows the help output ofsmartmeter-datacollector.-c, --config CONFIG: Path to the.iniconfiguration file.-s,--saveconfig: Create a default.iniconfiguration file.-d, --dev: Enable development mode.ConfigurationThe configuration file can be located anywhere and use any filename. If no.iniconfiguration file is specified a default configuration with the following options is used:Landys+Gyr E450 smart meter in unencrypted mode connected to/dev/ttyUSB0LOGGERsinkMQTTsink connected to a local broker without encryption or authenticationRun as a systemd serviceWhensmartmeter-datacollectorhas been installed as a Debian package it provides a systemd .service file namedpython3-smartmeter-datacollector.service. Therefore the service can be managed using thesystemctlcommand. SeeRun as a systemd servicefor possible commands.How to developHelp from the community for thesmartmeter-datacollectorproject is always welcome. Please follow the next few chapters to setup a working development environment.Development RequirementsPython >= 3.8, <= 3.10pipenvOptional software packages (Debian bullseye/ Ubuntu 22.04)python3-alldebhelperdh-pythonInstallationDebian / UbuntuTo install the listed minimal requirements run the following command:sudo apt install git python3 pipenvTo install the optional requirements also run the following command:sudo apt install python3-all debhelper dh-pythonCheckout the codeUsegitto clone / checkoutsmartmeter-datacollectorfrom GitHub usinggit clone https://github.com/scs/smartmeter-datacollector.gitSetup Development Environmentsmartmeter-datacollectorusespipenvto manage its dependencies and setup a virtual environment. Run the following command to setup the initial development environment:pipenv install --devThis will install all runtime and development dependencies forsmartmeter-datacollectorin a new virtual environment. Now you are ready to start working onsmartmeter-datacollector.Custom Commands / Workflowssmartmeter-datacollectoroffers a few custompipenv runcommands to simplify certain development workflows:build_checkusestwineto check if the built Python package will be accepted byPiPI.build_debbuilds a Debian package for the current development platform.build_srcdebbuilds a Debian source package which can be used to build a Debian (binary) package for any platform (e.g. usingpbuilder)buildbuilds a Python package which can be uploaded toPyPIusingtwine.debianizecreates adebian/directory used to build Debian source / binary packages.format_checkchecks if the code follows theautopep8code formatting rules.formatautomatically adjusts the code to follow theautopep8code formatting rules.isort_checkchecks if the order of the import statements is correct usingisort.isortautomatically re-orders the import statements usingisort.licensemakes sure every Python (*.py) file contains the proper license header.lint_checkchecks if the code follows thepylintrules defined inpyproject.toml.lintautomatically adjust the code to follow thepylintrules defined inpyproject.toml.setup_checkchecks whether the dependencies defined inPipfile/Pipfile.lockare in sync withsetup.py.setupsynchronizes the dependencies defined inPipfile/Pipfile.lockwithsetup.py.Make sure to runformat_check/format,isort_check/isort,lint_check/lint,license,setup_check/setupbefore committing changes to the repository to avoid unnecessary development cycles.smartmeter-datacollectorusesGitHub Actionsto check if these rules apply.Acknowledgementssmartmeter-datacollectorand its companion projectsmartmeter-datacollector-configuratorhave been developed bySupercomputing Systems AGon behalf of and funded byEKZ.