Package: pandiocli
PandioCLI - Pandio.com Machine Learning CLI ToolThis repository contains the PandioCLI tool to develop and deploy machine learning for streaming data.Quick LinksPandio.com/PandioML-Pandio.com-Getting Started-Quick Start-PyPi PandioML-PyPi PandioCLIInstallationpip install pandiocliRequirementsPython 3.5 - 3.8
PIP > 20.0.0Commandspandiocli function generate --project_name exampleGenerates a project template in the current working directory at./example./example/function.pyThis is the file where all of your logic should be placed../example/requirements.txtThis file should contain all the necessary Python packages to powerfunction.py. The contents of this will automatically be installed for you when deploying to Pandio's platform. When running locally, make sure to install as you normally wouldpip install -r requirements.txt./example/config.pyThis contains non-sensitive configuration parameters for the project. Sensitive configuration parameters are set via the PandioCLI.Acceptable values are:'FUNCTION_NAME': 'exampleFunction123',
'CONNECTION_STRING': 'pulsar://localhost:6651',
'ADMIN_API': 'http://localhost:8080',
'TENANT': 'public',
'NAMESPACE': 'default',
'INPUT_TOPICS': ['non-persistent://public/default/in'],
'OUTPUT_TOPICS': ['non-persistent://public/default/out'],
'LOG_TOPIC': 'non-persistent://public/default/log',
'ARTIFACT_STORAGE': "./artifacts"pandiocli function upload --project_folder path_to_folderPackage up your function project and upload it to Pandio's platform.pandiocli dataset generate --project_name exampleGenerates a project template in the current working directory at./example./example/dataset.pyThis is the file where all of your logic should be placed.Three things need to be defined to complete the dataset:__init__Establish a connection or load your data. Returns an iterable.nextReturns a single record from the dataset.schemaDefines the schema used for the dataset.For more information on schemas, see the Schema Registry../example/wrapper.pyThis is a wrapper class for the dataset to allow it to work on the Pandio platform.Note: You should not need to ever modify this file../example/requirements.txtThis file should contain all the necessary Python packages to powerdataset.py. The contents of this will automatically be installed for you when deploying to Pandio's platform. When running locally, make sure to install as you normally wouldpip install -r requirements.txt./example/config.pyThis contains non-sensitive configuration parameters for the project. Sensitive configuration parameters are set via the PandioCLI.Acceptable values are:'FUNCTION_NAME': 'exampleFunction123',
'CONNECTION_STRING': 'pulsar://localhost:6651',
'ADMIN_API': 'http://localhost:8080',
'TENANT': 'public',
'NAMESPACE': 'default',
'INPUT_TOPICS': ['non-persistent://public/default/in'],
'OUTPUT_TOPICS': ['non-persistent://public/default/out'],
'LOG_TOPIC': 'non-persistent://public/default/log'Additional parameter of--typecan be specified to generate a dataset with a template.Currently supported templates are:mysqltrinocsvpandiocli dataset upload --project_folder path_to_folderPackage up your dataset project and upload it to Pandio's platform.pandiocli config showThis will output the current configuration for the PandioCLIpandiocli config fileThis will output the current configuration file location for the PandioCLIpandiocli config resetThis will delete all settings for the PandioCLIpandiocli config set --key PANDIO_TOKEN --value ABC123This command allows you to manually set the configuration parameters for PandioCLIThese values are first set when you use the register command.PANDIO_CLUSTERPANDIO_TENANTPANDIO_NAMESPACEPANDIO_CLUSTER_TOKENPANDIO_EMAILPANDIO_DATA_TOKENNote: These values can be found from inside of your Pandio.com Dashboardpandiocli test --project_folder folder_name --dataset_name FormSubmissionGenerator --loops 1000This is a helper method to running thefolder_name/runner.pyfile manually with Python. It includes performance metrics which is helpful to debug excessive resource usage such as memory leaks.project_folderis the relative path to the project folder from where the command is being executed.dataset_nameis the name of thepandioml.datadatasets and generators available inside of PandioML or the relative path to the folder of the dataset generated by thepandiocli dataset generatecommand.loopsis the number of events to process. Most streams of data are infinite, so this allows iterative testing with limited data.pipeline_nameis the number of events to process. Most streams of data are infinite, so this allows iterative testing with limited data.pandiocli register your@email.comThis command registers a Pandio.com account for you. An email with a link to verify your registration will be sent.Once the link is clicked, the local PandioCLI will be configured successfully with your new Pandio account.If you already have a Pandio.com account, you'll need to use thepandiocli configcommand to manually set the configuration with values inside of the Pandio.com Dashboard.ContributingAll contributions are welcome.The best ways to get involved are as follows:IssuesThis is a great place to report any problems found with PandioCLI. Bugs, inconsistencies, missing documentation, or anything that acted as an obstacle to using PandioCLI.DiscussionsThis is a great place for anything related to PandioCLI. Propose features, ask questions, highlight use cases, or anything else you can imagine.If you would like to submit a pull request to this library, please read thecontributor guidelines.LicensePandioCLI is licensed under theSSPL license.