Package: htcollector
htcollectorGather incoming temperature and humidity updates from a small swarm of Shellyht devices.IntroInstallation and useSecurityInstallation as a libraryAPI DocumentationIntroShelly HT devices can be configured to send their measurements to a configurable server.
The server is called using a simple HTTP GET request likehttp://mymachine:8083/sensorlog?hum=24&temp=42.38&id=shellyht-1234FAwhere the parthttp://mymachine:8083/sensorlogis something you configure in the sensor itself.htcollectoris a solution that processes these requests and logs the data in a database.
It also provides means to associate meaningful labels with the station-ids and generate html or json data with the latest measurements.Installation and usehtcollectoris comes astwo Docker imagesthan can be run on your docker installation.To spin up a server you need to perform the following steps:Download a Docker compose file and an environment fileYou can of course adapt these files to your needsDownload and start your containersA simpledocker compose upwill take care of bothConfigure your Shelly HT devices to point to your serverThis is documented in the Shelly HT manualAssociate names with the Shelly HT station idsSo that you can more easily identify your Shelly HT devicesAfter this, you can access the most recent measurments onhttp://myserver:8083/all(wheremyserveris the name of the machine you have started the docker containers on)The steps are document in more detail in the following section.Installation: Step by stephtcollectoris provided as an easy to useDocker solution:You can simply download just a file with environment variables and a docker compose file and start that up withdocker compose.
Inbash(on a a system withcurlinstalled) that would look like (watch the long curl lines!):mkdirhtcollector-dockercdhtcollector-docker
mkdirdockercddocker
curlhttps://raw.githubusercontent.com/varkenvarken/shellyhtcollector2/master/docker/.env>.env
curlhttps://raw.githubusercontent.com/varkenvarken/shellyhtcollector2/master/docker/docker-compose.yml>docker-compose.yml
dockercompose-fdocker-compose.ymlup-dThis will download two images and start them as two containers: one running a MariaDB server and one running the htcollector.By default the htcollector container is configured to listen on port 8083 for incoming measurements from the devices,
but this can be changed of course (or you can change the configuration of your Shelly HT devices).When measurements start getting logged, you can go to the same webserver to get an html page with the last measurements:http://mymachine:8083/allThe image will show the last measurement of any device that has contacted the server, along with the name of the device and the time of the last measurement.
It also shows a small graph that depicts the temperature in the last 24 hours.To associate a label with a station, simply go to:http://mymachine:8083/namesIf you click on theChangelink of the device you want to change the label of, you can enter a name.!!! noteTested with Docker version 20.10.12, build 20.10.12-0ubuntu2~20.04.1 / Docker Compose version v2.6.0A note on securityhtcollectorhasnosafety features whatsoever: Authentication is not required and connections are not encrypted. Anybody can connect to the server and log measurements and anybody can change device names.You could consider running it behind a reverse proxy with https enabled and that would prevent people from eavesdropping on the measurements sent from the devices to the server, and from the name changes, but then you would need to implement some form of authentication as well and that would still not prevent injection of false measurements, even if you would implement some form of whitelisting, because the device ids of Shelly HT devices can be changed, so I did not bother to implement any kind of securit feature.What I did do on my home network is to have all my Shelly HT devices connect to a password protected WIFI network that is on a separate VLAN where also the server with htcollector is connected. This does not prevent unwanted access or snooping completely, but would require an attacker to be in range of the WIFI network and crack the encrypted WIFI connection. Not impossible but unlikely to be worth the trouble to intercept a few temperature measurements but if your Shelly devices are interconnected to control other devices you will need to design your security setup with a bit more care perhaps. You have been warned.If you would like to implement authentication for example, have a look at the section below so you can adapt the Python package yourself.Installation as a libraryYou can of course build your own solution on top of the htcollector module but that takes a few steps. Installing the htcollector package from PyPi is simple enough but it also depends on the MariaDB connector.Install the mariadb connector for PythonThemariadbpython module is not pure Python and depends onlibmariadb, so setup is less straight forward than you would hope:wgethttps://downloads.mariadb.com/MariaDB/mariadb_repo_setupecho"d4e4635eeb79b0e96483bd70703209c63da55a236eadd7397f769ee434d92ca8  mariadb_repo_setup"|sha256sum-c-
chmod+xmariadb_repo_setup
sudo./mariadb_repo_setup--mariadb-server-version="mariadb-10.6"sudoaptinstalllibmariadb3libmariadb-dev
sudopython3-mpipinstallmariadbHere we have chosen to install it for everyone, but if you used a virtual environment your could do away with thesudosInstall the packageInstall it directly from PyPipipinstallhtcollectoror alternatively, download it from GitHubgitclonehttps://github.com/varkenvarken/shellyhtcollector2.gitcdshellyhtcollector2
pythonsetup.pyinstall!!! noteIt might be a good idea to do this inside a virtual environment!Running the serverAssuming you have MariaDB running on the same machine with a database (schema)shellyht, and that the user defined in the environment variable DBUSER has enough privileges to create a tables, the following command will create the necessary tables if not yet present and start listening on port 1883 for incoming connections:nohuppython3-mshellyhtcollector&API documentationAvailable on the GitHub pages of this repoArchitecture overviewThe archictecture is not very complicated, just a http server and a database, each typically running inside a Docker container:flowchart LR
    s1[shellyht] --> docker
    s2[shellyht] --> docker
    s3[shellyht] --> docker
    s4[shellyht] --> docker
    subgraph docker
        w[webserver] <--> d[database]
    endImplementation detailsA ShellyHT can be configured to log temperature and humidity changes to a certain host:port using HTTP.The path of a typical GET request will look for example like:/sensorlog?hum=54&temp=23.4&id=shellyht-6A566FYou will need to configure your Shelly HT devices to connect to the server using this path, so you would typically enterhttp://myserver:8083/sensorlog!!! noteThe Webserver will *only* accept GET requests that have that exact format!