Package: learninghouse
learningHouse ServiceIntroductionTheLearningHouse Serviceprovides machine learning algorithms based on the scikit-learn python library as a RESTful API. Its purpose is to offer smart home enthusiasts an easy way to teach their homes.Contact and FeedbackIf you have any questions, please contact us on Discord.Please share your ideas on what you want to teach your home, suggestions or problems by opening anissue. We are really looking forward to your feedback.Installation and ConfigurationInstall and update using pip.pip install -U learninghouseInstall and update using dockerdocker pull ghcr.io/learninghouseservice/learninghouse:latestPrepare configuration directorymkdir -p brainsThebrainsdirectory holds the model configuration as a json file. The models are the brains of your learning house.There will be one subdirectory per brain, where all files relevant for a brain will be stored.
The brain subdirectory needs aconfig.jsonfile holding the basic configuration. The service will store atraining_data.csvfile holding
all data from your sensors and an object dump of the trained model to a file calledtrained.pkl.Service configurationThe service is configured by environment variables. The following options can be set:Environment Variabledefault (production/development)descriptionLEARNINGHOUSE_ENVIRONMENTproductionChoose the default environment settings: production or development.LEARNINGHOUSE_HOST127.0.0.1Set the address that the service should bind to. (use 0.0.0.0 for all available)LEARNINGHOUSE_PORT5000Set the port on which the service should listen.LEARNINGHOUSE_BASE_URLNot setSet the base URL for external access, for example, the hostname of your Docker host.LEARNINGHOUSE_CONFIG_DIRECTORY./brainsDefine the directory where all configuration data goes.LEARNINGHOUSE_OPENAPI_FILE/learninghouse_api.jsonProvide the file URL path to the OpenAPI JSON file.LEARNINGHOUSE_DOCS_URL/docsDefine the URL path for the interactiveAPI documentation. If you leave it empty, the documentation will be disabled.LEARNINGHOUSE_JWT_SECRETGenerated on startupFor administration authentication, a JWT is generated after login. This JWT is signed with a secret. By default, it is generated on startup, which will invalidate existing JWTs on each restart.LEARNINGHOUSE_JWT_EXPIRE_MINUTES10The refresh token of JWTs will expire after a given amount of minutes.LEARNINGHOUSE_LOGGING_LEVELINFOSet logging level to DEBUG, INFO, WARNING, ERROR, CRITICALLEARNINGHOUSE_DEBUG(False/True)The debugger will be automatically activated in the development environment. For security reasons, it is recommended not to activate it in production.LEARNINGHOUSE_RELOAD(False/True)The source will be automatically reloaded in the development environment. For security reasons, it is recommended not to activate it in production.Example configurationYou can download.env.exampleand rename it to.env. Inside, you can modify the default configuration values to meet your needs in this file.Run the serviceIn the consoleCopy the.env.examplefile to .env and modify it according to your needs.Then, simply runlearninghouseto start the service. By default, the service will listen onhttp://localhost:5000/.With docker:docker run --name learninghouse --rm -v brains:/learninghouse/brains -p 5000:5000 -e "TZ=Europe/Berlin" ghcr.io/learninghouseservice/learninghouse:latestUIFor configuration purposes, there is a small user interface that can be found athttp://localhost:5000/ui.SecurityThe service is protected by different authentication and authorization mechanisms. For administration, you can log in via the UI.Fallback passwordOn the first run, the service is set to use the fallback passwordlearninghousefor the administrator account. Until this is changed, all other endpoints will be deactivated.You can change the password on the initial login screen of the UI.Security notice: Unless you use a proxy setup for SSL security of your connection, only use a separate password for your learninghouse.API KeyYou can use your administration access for training and prediction endpoints, but we also recommend using an API key mechanism for application access. There are two roles for API key authorization:userfor the prediction endpoint andtrainerfor the training and prediction endpoints.You can add more API keys via the UI.Your API key will only be displayed once and cannot be requested again. So save it for your usage.
If you forget it, you will have to delete this API key and recreate it.You have to provide this API key for all requests, either as a query parameter?api_key=YOURSECRETKEYor as a header fieldX-LEARNINGHOUSE-API-KEY: YOURSECRETKEY.You can also test the API key by logging in to the UI.Brains and Sensors ConfigurationSensors ConfigurationSend data from all sensors to thelearningHouse Service, especially when training your brains. The service will save all data fields, even if they are not currently used as afeature. The service will choose the best feature set each time you train a brain.In general, sensor data can be divided into two different types.Numerical datacan be processed directly by your models, whileCategorical dataneeds to be preprocessed by the service in order to be used as afeature.Categorical datacan be identified using a simple rule:Non-numerical values, orNumerical values that can be described using terms.Here are some examples of categorical data:pressure_trend: Values of 'falling', 'rising', 'consistent'month_of_year: 1('January'), 2('February'), ...weather_condition: 'sunny', 'cloudy'switch: 'ON', 'OFF'To enable the service to use the data from your sensors asfeaturesfor your brain, you need to provide the service with information about the data type. You can add each sensor you want to use via the UI.For example, add the following sensors:Nametypeazimuthnumericalelevationnumericalrain_gaugenumericalpressurenumericalpressure_trend_1hcategoricaltemperature_outsidenumericaltemperature_trend_1hcategoricallight_statecategoricalExample brainThe brain determines whether it is dark enough to switch on the light. It utilizes a machine learning algorithm called RandomForestClassifier.To add a new brain via the UI, use your administration account and provide the following parameters.FieldValueNamedarknessTypedClassifierDependent encodeTrueTest size0.2Estimators100Max depth5Configuration ParametersEstimatorThe LearningHouse Service can predict values using an estimator. An estimator can be of typeclassifier, which is best suited for categorical outputs, such as true and false. If you want to predict a numerical value, such as the setpoint of a heating equipment, use the typeregressorinstead.For both types, theLearningHouse Serviceuses a machine learning algorithm called random forest estimation. This algorithm builds a "forest" of decision trees with yourfeaturesand takes the mean of the predictions of all of them to give you the best result. For more details, see the API description of scikit-learn.Estimator typeAPI ReferenceRandomForestRegressorhttps://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html#sklearn.ensemble.RandomForestRegressorRandomForestClassifierhttps://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifierYou can adjust the number of decision trees by using theestimators(default: 100) option. You can also adjust the maximum depth of each tree by using themax depth(default: 5) option. Both options are optional. Try resizing these values to optimize the accuracy of your model.Dependent variableThedependentvariable is the one that must be included in the training data and is predicted by the trained brain. It is the same as thenamevariable.Thedependentvariable must be a number. If it is not a number, but a string or boolean (true/false) as shown in the example, setdependent encodeto yes.Test sizeThe LearningHouse service only uses a portion of your training data to train the brain. The remaining portion, specified bytest size, is used to score the accuracy of your brain.You can specify thetest sizeas a percentage using floating point numbers between 0.01 and 0.99, or as an absolute number of data points using integer numbers.For example, atest sizeof 20% (0.2) should be sufficient to start with.An accuracy score between 80% and 90% is considered good. Scores below 80% indicate that the brain is underfitted, while scores above 90% indicate that the brain is overfitted. Both cases can result in poor predictions for new data points. You can try adjusting theestimatorconfigurationto improve the score.Training of the brain will start when there are at least 10 data points.Changing configuration via RESTful APIYou can also change the configuration of sensors and brains using the API. Please refer to the interactiveAPI documentationwhen the service is running.API DocumentationWhen the service is running, you can access an interactive API documentation by calling the URLhttp://localhost:5000/docs.Train the brainTo train, send a PUT request to the service:You need administration JWT or API key roletrainerfor this request (seeSecurity)# URL is http://<host>:5000/api/brain/:name/training
curl --location --request PUT 'http://localhost:5000/api/brain/darkness/training' \
    --header 'Content-Type: application/json' \
    --header 'X-LEARNINGHOUSE-API-KEY: YOURSECRETKEY' \
    --data-raw '{
        "azimuth": 321.4441223144531,
        "elevation": -19.691608428955078,
        "rain_gauge": 0.0,
        "pressure": 971.0,
        "pressure_trend_1h": "falling",
        "temperature_outside": 23.0,
        "temperature_trend_1h": "rising",
        "light_state": false,
        "darkness": true
}'You can send either a fieldtimestampwith your dataset containing a UNIX-Timestamp or the service will add this information with its current time. The service generates some further time-relevant fields inside the training dataset that you can also use asfeatures. These aremonth_of_year,day_of_month,day_of_week,hour_of_day, andminute_of_hour.If one of your sensors is not working at the moment and therefore not sending a value, the service will add a value using the following rules. Forcategorical data, all categorical columns will be set to zero. Fornumerical data, the mean of all known training set values (seeTest size) for thisfeaturewill be assumed.To train the brain with existing data, for example after a service update, use a POST request without data:You need an administrator JWT or API key with the roletrainerfor this request (seeSecurity).# URL is http://host:5000/api/brain/:name/training
curl --location \
    --header 'X-LEARNINGHOUSE-API-KEY: YOURSECRETKEY' \
    --request POST 'http://localhost:5000/api/brain/darkness/training'To obtain information about a trained brain, use a GET request:You will need an administrator JWT or API key with the role oftraineroruserfor this request (seeSecurity).# URL is http://host:5000/api/brain/:name/info
curl --location \
    --header 'X-LEARNINGHOUSE-API-KEY: YOURSECRETKEY' \ 
    --request GET 'http://localhost:5000/brain/darkness/info'PredictionTo predict a new data set with your brain, send a POST request:You need an administrator JWT or API key with the roletraineroruserfor this request (seeSecurity).# URL is http://host:5000/api/brain/:name/prediction
curl --location --request POST 'http://localhost:5000/api/brain/darkness/prediction' \
    --header 'Content-Type: application/json' \
    --header 'X-LEARNINGHOUSE-API-KEY: YOURSECRETKEY' \
    --data-raw '{    
        "azimuth": 321.4441223144531,
        "elevation": -19.691608428955078,
        "rain_gauge": 0.0,
        "pressure_trend_1h": "falling"
    }'If one of your sensors used as afeaturein the brain is not working at the moment and is not sending a value, the service will handle this by using the following rules. Forcategorical data, all categorical columns will be set to zero. Fornumerical data, the mean of all known training set values (seeTest size) for thisfeaturewill be assumed.