Package: polaris-ml
PolarisPython3 tool to analyze a satellite set of telemetry to understand links/dependencies among different subsystems. The telemetry is currently retrieved from theSatNOGS Network.If you want toknow more:join ourMatrix roomread theproject wikiread the blog postAnalyzing Lightsail-2 Telemetry with Polarislook at thedemo siteIf you are a developer, and want to contribute to Polaris check outCONTRIBUTE.md.Project structurecontrib/-codethatisnotdirectlydependentonPolaris,butisusedintheproject
docs/-Somedocumentationontheproject(thoughmoreisinthewiki)polaris/-Projectsourcecodecommon/-ModulescommontoallofPolarisfetch/-Moduletofetchandpreparedatafortheanalysisviz/-Moduletovisualizetheanalysisresultslearn/-Moduletoperformthedataanalysisbatch/-Moduletoperformbatchoperationsconvert/-Moduletoconvertgraphoutputfromlearntootherfileformatspolaris.py-Polarisentrypoint

tests/-Projectunittests
playground/-ExploratorytestsInstallationpip3installpolaris-mlWe recommend to install it inside a Python virtual environment:# Create the virtual env$python3-mvenv.venv# Activate it$source.venv/bin/activate# Upgrade Pip before installing Polaris$(.venv)pipinstall--upgradepip# Install Polaris from Pypi$(.venv)pipinstallpolaris-mlNote:If you run into problems installing Polaris via pip,make
sure you've upgraded pip itselfand are using a clean, new, separate
virtual environment -- this solves most problems.Running the code$(.venv)polaris--help
Usage:polaris[OPTIONS]COMMAND[ARGS]...Toolforanalyzingsatellitetelemetry

Options:--versionShowtheversionandexit.--helpShowthismessageandexit.

Commands:batchRunpolariscommandsinbatchmodeconvertConvertpolarisgraphfile(supportedformats:gexf)fetchDownloaddataset(s)learnAnalyzedatavizDisplayingresults# To fetch and decode data from the SatNOGS network and space weather sources, run:$(.venv)polarisfetch-s2019-08-10-e2019-10-5--cache_dir/tmp/LightSail_2LightSail-2/tmp/normalized_frames.json# Note: this may take some time.# If the normalizer for your satellite does not exist, you can run polaris fetch# with the --skip_normalizer flag. The result with and without the normalizer# (without/with --skip-normalizer) are bound to be slightly different.# The normalizer is mainly present to give you, the satellite operator, an# intuitive (SI) value for fields (instead of arbitrarily scaled/shifted# values). It is easy to create and you can get started with the process using# the snippet at https://gitlab.com/librespacefoundation/polaris/polaris/-/snippets/2006696$(.venv)polarisfetch-s2019-08-10-e2019-10-5--cache_dir/tmp/LightSail_2--skip_normalizerLightSail-2/tmp/normalized_frames.json# Data will be saved at /tmp/normalized_frames.json$(.venv)head/tmp/normalized_frames.json[{"time":"2019-09-12 08:14:42","measurement":"","tags":{"satellite":"","decoder":"Lightsail2","station":"","observer":"","source":"",[...]# To learn from that data, run:$(.venv)polarislearn-g/tmp/new_graph.json/tmp/normalized_frames.json# Note: depending on your hardware, this may take some time.# Note: `polaris learn` uses your dedicated (CUDA enabled) GPU by default#       to suppress this behaviour, you can utilise the --force-cpu flag.$(.venv)polarislearn-g/tmp/new_graph.json/tmp/normalized_frames.json--force_cpu# To see a visualization of these results, run:$(.venv)polarisviz/tmp/new_graph.json# Then visit http://localhost:8080 in your browserConfiguring PolarisIt is possible to override the default parameters used in the ai processes of Polaris by your own using configuration files.Learn cross correlation process (generating graph) :{"use_gridsearch":false,"random_state":43,"test_size":0.2,"gridsearch_scoring":"neg_mean_squared_error","gridsearch_n_splits":6,"dataset_cleaning_params":{"col_max_na_percentage":100,"row_max_na_percentage":100},"model_cpu_params":{"objective":"reg:squarederror","n_estimators":81,"learning_rate":0.1,"n_jobs":1,"predictor":"cpu_predictor","tree_method":"auto","max_depth":8},"model_params":{"objective":"reg:squarederror","n_estimators":80,"learning_rate":0.1,"n_jobs":1,"max_depth":8}}To use it, add the-lorlearn_config_filecommand line parameter when calling learn:$polarislearn-g/tmp/graph.json/tmp/normalized_frames-l../xcorr_cfg.json## Batch operationsBatchoperationsallowautomationofrepeatedsteps.Forexample:

-running`polarisfetch`sothatitfetchesthelatestdataforaparticularsatellite,thenrunning`polarislearn`toupdatethemodel

-running`polarisfetch`,`polarislearn`and`polarisviz`aspartofanintegrationtestThe`polarisbatch`commandiscontrolledbyaJSONconfigurationfile;anexamplecanbefoundat`polaris/common/polaris_config.json.EXAMPLE`.```bash
$(.venv)polarisbatch--config_filepolaris/common/polaris_config.json.EXAMPLEInfluxDBWith the addition of space weather recently, influxdb support has been added to Polaris. To create the requireddocker-compose.ymlfile and start and stop the docker container, run:$python
>>>fromvinvelivaanilai.storageimportcommon# To create the path>>>common.create_docker_compose("/path/to/docker-compose.yml","/path/to/storage")# To start influxdb>>>common.start_docker_compose("/path/to/docker-compose.yml")# To stop influxdb>>>common.stop_docker_compose("/path/to/docker-compose.yml")To store in and fetch from influxdb use the flags--store_in_influxdband--fetch_from_influxdbrespectively.polarisfetch-s2019-08-10-e2019-10-5--cache_dir/tmp/LightSail_2LightSail-2/tmp/normalized_frames.json--store_in_influxdb
$polarisfetch-s2019-08-10-e2019-10-5--cache_dir/tmp/LightSail_2LightSail-2/tmp/normalized_frames.json--fetch_from_influxdbMLflowInstalling Polaris will install MLflow as a dependency. At this time Polaris is using MLflow during the cross check dependencies process, and the database is stored in the current working directory under the mlruns folder.To view the logs in MLflow, run this command in the directory that holds themlrunsfolder (by default, this is the project root directory):mlflowuiThis command will start the tracking ui server athttp://localhost:5000.Working on documentationDocumentation is hosted on readthedocs.io.  We use theMyst parser, and write documentation in Markdown.To work on documentation, install the docs dependencies like so:# Yes, with the square brackets:pipinstall-e.[docs]Documentation files are in thedocs/directory.  To build the HTML files, run:cddocs/
makehtmlGenerated files will be in thedocs/_builddirectory, and can be viewed with your favourite browser.