catops 1.0.0
Highly trained cats for managing servers.
License: MIT
Requires Python: >=2.7.0


<h1 align="center" >CatOps</h1>

Highly trained cats for managing servers.

![Dedicated server support agent.](https://github.com/BBOXX/CatOps/blob/master/docs/catops.jpg)

## What is CatOps

CatOps is a very simple NoOps framework for deploying your own ChatOps bot.

Commands you wish to add to your CatOps implementation are added to a `plugins`
folder in the same directory, and will then be automatically imported and callable
using the function name.

## Why CatOps

- NoOps.
  - Deploy, rewrite, and redeploy FaaS easily with no worrying about setting up and managing servers.
  - Only charged when CatOps is called.

- Codify common maintenance procedures.
  - Perform high level actions without intimate low level knowledge.
  - Prevent errors doing complicated but routine tasks. 

- Unify documentation.
  - CatOps can act as a unified go-to location for help, merging/pooling all documentation into one place.

- Transparency.
  - Team members can see all actions taken by others in solving a problem. Organic learning.
  - No 'go-to' person for certain maintenance tasks.
  - Everyone aware of server changes. No-one surprised that the server is down if they see `/meow restart server` in the chat.
  - Spread knowledge; everyone becomes equally capable of solving problems.
  - Out of date help messages or documentation is more obvious to everyone.

- Context-aware suggestions, suggest actions and display help depending on context.
  - Docs/procedures/etc are useful, but can be too much to read through, hard to find, not up to date. 
  - Reduce clutter when trying to figure out next actions. 

- Reduce context switching.
  - No need for bash, Linux, ssh or VPN to fix most server issues.
  - No checking server logs.
  - Easily accesible and readble output.

- Control access.
  - Only gives necessary access, no unnecessary ssh-ing into production!

## Features

- Completely NoOps. 
- Easily extensible.
- Pay per invocation.
- Provider agnostic.

## Quick Start

1. Install catops `pip install catops`
2. Run `meow install [--template] [--target-dir]`
3. Adjust the template according to your needs e.g. add Slack OAuth tokens, adjust service names etc.
4. Install serverless dependencies `npm install` in the template directory.
5. Run `serverless deploy`
6. Configure your Slack app (i.e. set Slash command/Bot endpoint URLs to appropriate URLs)

## Example

### Handler

Every Lambda function needs a handler, which takes arguments `(event, context)`. In this case, it is necessary to respond instantly to the request with a `200` so the handler below calls the actual functionality asynchronously and then returns a `200` response.

#### Example Handler

```python handler.py
import json
from six.moves.urllib.parse import parse_qs
import requests
import boto3
from catops import convert_dispatch


def respond(event, context):
    """Call handler.main asynchronously and then return instant response."""
    lambda_client = boto3.client('lambda')
    response = {'statusCode':'200'}
    # Call actual function asynchronously
    lambda_client.invoke(
        FunctionName='CatOps-dev-dispatcher',
        InvocationType='Event',
        Payload=json.dumps(event))
    return response


def main(event, context):
    """Main lamda function logic, to be called asynchronously."""
    params = parse_qs(event.get('body'))
    payload = convert_dispatch(params)
    username =  params.get('user_name', ['catops'])[0] 

    # Post to Slack channel
    response_url = params.get('response_url')
    if type(response_url) is list:
        response_url = response_url[0]
    r = requests.post(response_url, data=json.dumps(payload))
    if not r.ok:
        print(r)
        print(r.reason)
        print(r.text)
    return
```

### Plugins

CatOps works around plugins.

- Plugins are python files stored in the 'plugins/' directory.
- CatOps scans this directory for valid functions to import.
- All files and/or functions starting with `_` are ignored. (`_` means they are private and will not be added to the CatOps dispatcher)
- Other functions are added to the CatOps dispatcher and can be called via `/catops <functionname> [argv]`
- All functions need to have the arguments `(argv, params)`
    - argv will contain the arguments passed to the function e.g. for `/catops role --user t.user`, argv will contain `['role', '--user', 't.user']
    - params will contain the parse Lambda event body, which contains all the information from Slack e.g. `{"text": ... , "username": ..., "response_url": ...}`.

#### Example plugin

```python plugins/example.py
"""example.py - example plugin for ChatOps."""
from catops import create_slack_payload

def ping(argv, params):
    """Check is working."""
    text = '@{} Meow!'.format(params.get('user_name', ['CatOps'])[0]),
    return create_slack_payload(text=text)
```

### Serverless configuration

```yaml serverless.yml
service: CatOps

package:
  include:
    - handler.py
    - plugins/**

custom:
  pythonRequirements:
    slim: true

provider:
  name: aws
  stage: ${opt:stage, 'dev'}
  runtime: python3.6
  profile: serverless
  # Permissions for the lambda function
  # If using boto3, ensure correct permissions
  # have been granted to the lambda function
  iamRoleStatements:
    - Effect: Allow
      Action:
        - lambda:InvokeFunction
        - lambda:InvokeAsync
      Resource: "*"

functions:
  dispatcher:
    handler: handler.main
  respond:
    handler: handler.respond
    events:
      - http:
          path: ping
          method: post

plugins:
  - serverless-python-requirements
```

### Deploy and Test

```bash
serverless deploy
serverless invoke --function dispatcher --path /path/to/json/data --log
```

See [examples](https://github.com/BBOXX/CatOps/tree/master/examples) and [templates](https://github.com/BBOXX/CatOps/tree/master/catops/templates) for more.

## Useful functions

```python
# auth.py
def verify_request(event, slack_secret, timeout=True):
    """Verify that Lambda event came from Slack"""

def get_user_slack(event, oauth, team_id=None, channel_id=None):
    """Check whether user exists and is in specified team and channel.

    Arguments:
        event       - AWS Lambda event
        oauth       - Slack OAUTH token
        team_id     - Slack team_id (workspace, i.e. BBOXX)
        channel_id  - Channel user must be in
    Returns:
        False, err_msg
        True, user_dict with id, name, team_id, channel_id, email
    """

# dispatcher.py
def dispatch(command, params=None, plugin_dir='plugins/'):
    """Create Dispatcher object and run parse_command on (command, params)"""

# helpers.py
def get_slack_colour(level):
    """Return Slack colour value based on log level."""

def get_text(params):
    """Return event_text from parse_qs event.get('body')."""

def create_slack_attachment(fallback=None,
                            color=None,
                            pretext=None,
                            author_name=None,
                            author_link=None,
                            author_icon=None,
                            title=None,
                            title_link=None,
                            text=None,
                            fields=None,
                            image_url=None,
                            thumb_url=None,
                            footer=None,
                            footer_icon=None,
                            ts=None
    ):
    """Create slack attachment payload
    See https://api.slack.com/docs/message-attachments for more info.

    Arguments:
        fallback - Required plain-text summary of the attachment
        [color] - Colour of the attachment
        [pretext] - Optional text that appears above the attachment block
        [author_name]
        [author_link]
        [author_icon] - URL to author icon
        [title] - Title of the attachment
        [title_link]
        [text] - Optional text that appears inside the attachment
        [fields] - Array of dicts containing more values
        [image_url] - URL to image attached
        [thumb_url] - URL to image thumbnail
        [footer] - Footer message
        [footer_icon] - URL to footer icon
        [ts] - timestamp
    """

def create_slack_payload(response_type='ephemeral', text="", attachments=None):
    """Create a Slack payload formatted correctly."""

def convert_dispatch(params, convert_function=None, plugin_dir='plugins/'):
    """Call dispatch and convert the output accordingly into a payload."""

# slack_handler.py

class SlackHandler(logging.Handler):
    """Logger slack_handler which posts json log body to lambda_url."""

# install.py

def install(argv=None):
    """Install catops serverless Slack template."""

# parser.py 
class ArgumentParserError(Exception):
    """Error raised by ArgumentParser"""
    pass


class CatParser(argparse.ArgumentParser):
    """Overrides error method to throw an error instead of exiting"""
    def error(self, message):
        raise ArgumentParserError(message)

```

## Installation

```bash
sudo apt-get install npm
sudo npm install -g serverless
npm install serverless-python-requirements
pip install catops
```

Install `serverless-python-requirements` in the same dir as `serverless.yml`.

## Limitations

- Passive rather than active; needs to be triggered (e.g. by Slack slash commands (could run it every command))
- Limitations of FaaS
  - Max size (256MB for AWS Lambda)
  - Execution time limit (5 minute for AWS Lambda)
  - No state (recommend using a cloud-based database for state e.g. DynamoDB for AWS)
- No autocomplete inside of Slack.