Package: octo-bots-python
Octo BotsThe bots infrastructure gives easy to define easy to work with interface for actions on events and background jobsEach event that is triggered via github webhook, will be sent to the bots manager to execute different pre-defined botsEach bot is defined within a yaml file and can define its operations and filters on which the bot can run onAlong with that, background jobs may also be defined to run every X time and perform operations, similar to the botsStarting a new bots serverThe first thing that needs to be done in order to create a bots server, is define its configurationThe yaml configuration of the server may look as follows:settings:parallel-background-jobs:10bots-endpoint:/eventsclient-validity-time-minutes:10parallel-bots:truecredentials:github-app-credentials:api-url:https://api.github.comapp-name:octo-bot-appcertificate-path:../data/cacert.pemapp-creds-path:../data/github/octo_bot_github_app_creds.ymlcheckmarx-credentials:api-url:https://checkmarx.api.comcreds-path:../data/checkmarx/octo_bot_checkmarx_creds.ymlbots:-./bots/github_checkmarx_bot.ymljobs:-./jobs/github_stale_job.ymlWhere:settings - the global bots server settingsparallel-background-jobs: Maximum amount of running parallel background jobsbots-endpoint: Listening endpoint which events will be sent toclient-validity-time-minutes: For how long a client can be valid for connection (github/checkmarx)parallel-bots: Should the bots defined run in parallel or sequentialcredentials - credentials for each client, currently supportsgithub-app-credentialscheckmarx-credentialsbots - List of files that define different botsjobs - List of files that define different background jobsOnce the configuration is done, the bots server can be started, either by the bots_executor.py or by a wrapping script for example:flask_app=Flask(__name__)bots_manager=BotsManager.create_bots_manager(args.config_path,flask_app)bots_manager.start_bots_manager()flask_app.run(ssl_context="adhoc",host="0.0.0.0",port=8443)Defining a new botEach bot is defined with a list of operations and a list of filtersThe list of filters will assert whether the bots needs to run its operations or not on any eventA set of bots are defined in a yaml configuration file, for example:bots:-name:Checkmarx scanparallel:trueoperations:-pull-request-checkmarx:risk-scheme:min-high-vuls:1min-medium-vuls:20min-low-vuls:100min-info-vuls:10000scan-timeout:30 minutesfilters:-events-filter:events:-name:pull_requestactions:-opened-reopened-ready_for_review-synchronizeThe above bot will define a name bot calledCheckmarx scanThe bot will run its operations in parallel (currently only one operation)The operation that will be ran is calledpull-request-checkmarxAnd it has a set of parameters, such as therisk-schemeAlong with that, a list of filters are also set, such as the events-filter, which u can filter out which webhooks will trigger this bot, such as the pull_request webhookThe operation itself is defined in a python script, which overrides a base Operationoperations/github/pull_request_checkmarx_operation.pyOnce the bot is ready, it can be set within the main yaml configurationDefining a new background jobThe same idea as the bot is applied here
for example:jobs:-name:Close stale issues and pull requestsparallel:trueoperations:-close-stale:close-issues:trueclose-prs:truestale-expiration:60 daysstale-comment:>This issue has been automaticlly marked as stale because it has not hadrecent activity. It will be closed and can be later re-opened.Thanks for your contributions.exempt-labels:-hotfix-releaseevery:"30minutes"For example here the job is ran every 30 minutes, and will clean up stale pull requests and issues, close them and write a comment (github will also send a mail to the subscribers of the repo accordingly)Defining github app credentialsIn order to connect to github, we also need to define which application we are working withTo do so, we also define a seperate creds yaml file, which can be locally (or in the future from conjur):app-id:1client-id:some_client_idclient-secret:some_client_secretprivate-key-path:/some/path/to/app/keywebhook-secret:some_webhook_secretThe above are the credentials which will be used in the main settings, to generate access tokens to github as the github app (bot) to act upon him and make actionsAdding a new operationIn order to add a new operation, we would need to create a new class which inherits the Operation classThe operation class is defined as follows:operations/operation.pyclassOperation:@abc.abstractstaticmethoddefcreate_operation(config:dict)->'Operation':pass@abc.abstractstaticmethoddefoperation_type()->str:pass@abc.abstractmethoddefexecute_operation(self,clients:Dict[str,BotsBaseClient],headers:dict,event:dict):passWhere:create_operation: Receives the config dict from the yaml where we defined the operation and its parameters, and returns the created operationoperation_type: The operation type that we defined its name in the yamlexecute_operation: The actual operation logic, with inputs of a set of clients which we can use (github, checkmarx) and the validated event itself, after all the filters we definedOnce the operation was implemented, we can use it on the yaml as seen aboveAdding a new filterSimilarly, in order to add a new filter, we would implement the following interfacefilters/filter.pyclassFilter:@abc.abstractstaticmethoddefcreate_filter(config:dict)->"Filter":pass@abc.abstractstaticmethoddeffilter_type()->str:pass@abc.abstractmethoddeffilter_event(self,clients:Dict[str,BotsBaseClient],headers:dict,event:dict)->bool:passWhere:create_filter: Receives the config dict from the yaml where we defined the filter and its parameters, and returns the created filterfilter_type: The filter type that we defined its name in the yamlexecute_filter: The actual filter logic, with inputs of a set of clients which we can use (github, checkmarx) and the event itself, which we will filter (if it needs to be filtered, we will return True)Adding a new clientTo add a new client, such as github or checkmarx, we would need to implement two classes, one for the credentials, and one for the actual client itselfThe credentials class is defined as followsclassBotsBaseCredentials:@abstractmethoddefcreate_authenticated_client(self,validity_time)->BotsBaseClient:pass@abstractstaticmethoddefcreds_type()->str:pass@abstractstaticmethoddefclient_type()->str:passWhere:create_authenticated_client: Will create the client itself per the credentials, alrdy authenticated to the resourcecreds_type: Name of the creds typeclient_type: Name of the clientAnd the client is defined as followsw:classBotsBaseClient:@abstractmethoddefis_valid_client(self)->bool:pass@abstractmethoddefvalidate_request(self,request:Any)->bool:pass@abstractstaticmethoddefclient_type()->str:passWhere:is_valid_client: Checks if the client itself is valid to be used, if not it will be recreated via the credsvalidate_request: Tries and validates a given request if it can be even used in the first placeclient_type: Name of the clientOnce the above two are defined, the client will arrive to the operations as one of the clients in the dictionary, and can be casted and used accordingly