Package: service-framework
Service FrameworkNoteThis library is currently in alpha. While I use it for a few private repos this library can be changes at any moment.InstallingAs a Librarypip install service_frameworkFor DebuggingTo install locally for testing run the following command from the base directory.pip install -e .RunningFrom Command LineRun the following from the directory the Service Framework was locally built.
As this directory houses the packaged Service Framework as a python "Egg" file.python -m service_framework

# OR

service_frameworkUseful Command Line ArgumentsBelow are a few useful arguments...# Used for Service Setup
-a  Addresses Path (Relative)
-c  Config Path
-m  Main Mode Flag
-s  Service Path (Relative)
-wt Min Wait Time between each service loop

# Used for Logging
-cl Console Log Level
-f  File Log Level
-bc Backup Count (Number of hourly log files to keep)
-l  Log Path (Relative)As An ObjectTo allow multiple services to be run in sync and programmatically created or destroyed it has been neatly packaged into a class for use.
TheService Classshould mirror the same requirements as running the framework from the command line.
Quite frankly, because running it from the command line just uses this class underneath.
Below are the commands on how to use this class:# Useage Example

from importlib import import_module
from service_framework import Service

service_relative_path = './path_to_service_file/service_specification.py'
service_module = import_module('package.path.to.module')
config = {'some_config_key': 'some_config_value'}
addresses = {'look below this can get complicated'}

my_service = Service(service_path=service_relative_path, config=config, addresses=addresses)
also_my_service = Service(service_module=service_module, config=config, addresses=addresses)


# To run the service as a "main service" or in "main mode" in the background.
my_service.run_service_as_main()

# To run the service regularly and in the background/non-blocking.
my_service.run_service()

# To stop the service's execution.
my_service.stop_service()

# To run the service in it's respective mode and block (stop execution past this line):
my_service.run_service_as_main_blocking()
my_service.run_service_blocking()Within One ThreadTo allow multiple services to be run synchronous within a single thread, for performance reasons, multiple services can be runInline.An example can be find within theInline Service Unit Tests.# Usage Example
from service_framework import InlineServices
from service_module import requester_service, responder_service

requester_path = './requester_service.py'
responder_path = './responder_service.py'

# Inline Services from Path
inline = InlineServices()
inline.set_main_service('requester', requester_path)
inline.add_service('replyer', replyer_path)
inline.add_relation('requester', 'request', 'replyer', 'reply')

# This will run the "main_service" main function and continue on it's return
inline.start()

# Inline Services from Modules
inline2 = InlineServices()
inline2.set_main_service_by_module('requester', requester_path)
inline2.add_service_by_module('replyer', replyer_path)
inline2.add_relation('requester', 'request', 'replyer', 'reply')

# Configs can be provided when a main, or regular, service is added
requester_config = {'nim_req_to_send': 10}
replyer_config = {'response_text': 'TO_SEND'}
inline3 = InlineServices()
inline3.set_main_service_by_module('requester', requester_path, requester_config)
inline3.add_service_by_module('replyer', replyer_path, replyer_config)
inline3.add_relation('requester', 'request', 'replyer', 'reply')Useful Optional ArgumentsHere are some other, optional parameters, that can be passed into the Service object:console_loglevel='INFO'
log_path=None
file_loglevel='INFO'
backup_count=24
service_loop_min_wait_time=0These Parameters are the same as their command line counterparts.LoggingTo log inside of your service file import the logger that's pre-created.from service_framework import get_logger

LOG = get_logger()Addresses PathIf the addresses path is provided, the addresses JSON file will be loaded into the service framework.
This allows the framework to setup each connection of the services connections.
Example:# Command
-a "./addresses.json"

# Addresses.json
{
  "in": {
    "connection_name": {
      "socket_name": "127.0.0.1:8001",
      "socket_name_2: "127.0.0.1:8002"
    },
    "connection_name_2": {
      "socket_name": "256.128.65.1:9000"
    },
  },
  "out": {}
}Config PathThis is the relative path to the config json file that will be passed into the service. This file is not required but is useful if one wants to pass configuration information into a service.
Example:# Command:
-c "./config.json"

# Example Config.json:
{
  "required": {
    "config_property_1": 12345,
    "config_property_2": ["list_item_1", "list_item_2", "list_item_3"],
    "config_property_3": {
      "key_1": "value_1",
      "key_2": "value_2"
    }
  },
  "optional": {
    "config_property_4": "TestTestTest"
  }
}Main Mode FlagThis flag is used to start the service in "main_mode".
This modeDOES NOTallow the use of inbound connections at all.
It simply runs the provided main function in the service and all the connections corresponding with the provided models.
The main function must have a predefined signature seen below.# Example main function
def main(to_send, config):Service PathThis is the relative path to the service python file that will be used for starting and running the service.# Command
-s "./service.json"Custom Environmental ArgumentsAny additional arguments that are passed will automatically be added to the config. See below for an example...# Calling the below
python -m service_framework -s ./service.py --random_variable HELLO

# Will cause a config similar to below
config = {
    'random_variable': 'HELLO',
}Help CommandUse the below command to get a helpful output for the service framework.python -m service_framework -hBuilding BlocksAn abstract overview of the building blocks.ConnectionsConnections are the glue that holds services together.
Any communication between services should be done via connections.Leading Edge ServicesThese are services that use the "main mode".
If all services are plotted as a graph where they are connected by connections these services have no inbound connections from other services.ServicesA service is a block of code that does one thing and one thing well.
The goal of this framework is to make spinning up and dealing with a large number of services easily.ImplementationLeading Edge ServicesRequired MethodsMain MethodOnly Method called in this serviceex.main(to_send, config)Optional MethodsSetup Configex.setup_config(config)This method is called before the main method.This method is used to update the config before it's used.This method merges the responses with the provided config. (Overwrites keys)ex.return {
    'required': {
        'argument_1': 'first_argument',
    },
    'optional': {
        'argument_2': 'the_second_argument',
    },
}Setup Connectionsex.setup_connections(config)This method is called at the start of the serviceThis method must return all arguments for the given connection name.ex.return {
  'connection_name_1': {
    'argument_1': 'value',
    'argument_2': 123, 
  },
  'connection_name_2': {
    'argument_1': 'test_test',
  },
}Sigint Handlerex.sigint_handler(sigint, frame, to_send, config)This method is called whenever a sigint is provided.Sigterm Handlerex.sigterm_handler(signum, frame, to_send, config)This method is called whenever a sigterm is provided.Optional ModelsEach Model is only required if it's used in the leading edge service.config_modelconnection_modelsServicesRequired MethodsMethod for each "in" Connection ModelThis method is called whenever a message from this connection is receivedThe mapping between function and model is located in the modelex.function_name_in_conn_model(args, to_send, config)Optional MethodsSetup Connections Methodex.setup_connections(config)This method is only required if a connection has required argumentsThis method is called at the start of the serviceThis method must return all arguments for the given connection name.ex.return {
  'connection_name_1': {
    'argument_1': 'value',
    'argument_2': 123, 
  },
  'connection_name_2': {
    'argument_1': 'test_test',
  },
}Sigint Handlerex.sigint_handler(sigint, frame, to_send, config)This method is called whenever a sigint is provided.Sigterm Handlerex.sigterm_handler(sigint, frame, to_send, config)This method is called whenever a sigterm is provided.Init Functionex.init_function(to_send, config)This method is called before the main method or service framework.Optional ModelsModels are only needed if used in the service.config_modelconnection_modelsParameters Passed"args"::dictWhen this function is called these are the "arguments" passed. These arguments are pulled from the in connector that is associated with this function."to_send"::functiondef to_send(connection_name, args):
    """
    connection_name::str The name of the connection defined in the corresponding model.
    args::{'str': value} This is a dict of the args to be passed to the connection.
    """This function takes the provided args and sends them to the desired connection."config"::dictThis is a dictionary that is a culmination of config file data and environment variables.Model LayoutConfig ModelOverviewIt's a key value store of config information that is passed into the model.Exampleconfig_model = {
    'required': {       # Optional
        'symbol': str,
    },
    'optional': {},     # Optional
}Connection ModelsOverviewTriggers work/functions between different services.Exampleconnection_models = {
    'in': {
        'cancel_order': {
            'connection_type': 'replyer',                  # Required
            'required_creation_arguments': {               # Optional
                'connection_function': on_new_cancel_order, # Optional
            },
            'optional_creation_arguments': {},             # Optional
            'required_connection_arguments': {},           # Depends on the Connection
            'optional_connection_arguments': {},           # Depends on the Connection
            'required_arguments': {                        # Optional
                'order_id': str,
            },
            'optional_arguments': {},                      # Optional
            'required_return_arguments': {                 # Optional
                'order_id': str,
            },
            'optional_return_arguments': {},               # Optional
        }
    },
    'out': {
        'cancel_order_on_coinbase': {
            'connection_type': 'external_target',     # Required
            'required_creation_arguments': {          # Optional
                'target': 'CbproClient.cancel_order', 
            },
            'optional_creation_arguments': {}         # Optional
            'required_connection_arguments': {},      # Depends on the Connection
            'optional_connection_arguments': {},      # Depends on the Connection
            'required_arguments': {                   # Optional
                'order_id': str,
            },
            'optional_arguments': {},                 # Optional
            'required_return_arguments': None,        # Optional
            'optional_return_arguments': None,        # Optional
        },
    },
}Connection TypesInReplyerUsed to wrap a ZMQ "Replyer" socket.
Triggers a provided method on new requester message.Link to Replyer FileSubscriberUsed to wrap a ZMQ "Subscriber" socket.
Triggers a provided method on a new published message.
Also has XSUB support.Link to Subscriber FileOutExternal TargetUsed to wrap an external call and make sure all of the arguments are properly formatted and returned.
Can useto_send('external_target_name', {'args': 'here'}to send a payload to the external service.PublisherUsed to wrap a ZMQ "Publisher" socket.
Can useto_send('publisher_conn_name', {'args': 'here'}to send a payload to the connected subscribers.
Also has XPUB support.Link to Publisher FileRequesterUsed to wrap a ZMQ "Requester" socket.
Can useto_send('requester_conn_name', {'args': 'here'}to send a payload to the connected replyer.Link to Requester FileField and Argument ExamplesDictionary# Abstract Model Representation
'dictionary_argument': {Type: Type},

# Actually Passed
'dictionary_argument': {Decimal: Decimal},Service Framework Running WorkflowLoad Service File to get modelsConfig:Get config file locationGet additional environmental variablesRun setup_config functionCheck config parametersAddresses:Load addresses configRun setup_addresses if availableConnections:Load Connections ConfigRun setup_connections functionGet connections (in and out)Sig Handlers:Sets up Sigint and sigterm handling if neededRuns sigint_handler and sigterm_handlerInit Function:Run init_function before running main or service loopRuns either main or service loopHonestly, check theentrance_pointmethod