Package: ryuso
RYU Sequential OrchestratorThis is a specification framework that can automatically generate template code and a JSON-over-socket server that allows clients to execute programs defined by a spec.This framework was initially developed to formalise the backend operations for theGRC Wallet Botwhen I realised that what I made can be extended from transaction processing to pretty much any process in an atomic and optionally reversible fashion.The concept is that you have a problem to solve that is made up of a set of steps that can't be looped. This problem might make changes to data or other operations that need to be reversed, and for each step, you may specify how to reverse it if it has run successfully. Each step may depend on other steps and whenever an error occurs, the framework automatically reverses the operations starting one-before the erroneous step and working backwards.Python packageYou can install this package using pippip install ryusoThe nameThe name is based on the Japanese name for the "Yakuza" game franchise (Ryu Ga Gotoku) which I am completely obsessed with. I chose the name because many of the Yakuza games are highly sequential singleplayer games and I couldn't think of anything else that wasn't already taken.OverviewThe framework operates over a simple send and then receive socket.The client sends a JSON payload to the RYU server telling it what program to run and what inputs to initialise it with.The server queues these tasks in the socket accept queue and accepts them on a single threadOne-by one, the server will read and verify the tasks before running the appropriate hooks according to the specIf a hook fails, then it will iterate back through the hooks (in reverse) and execute their rectification functions (if any are specified) starting from the previous hook.The resulting data pool is placed on an output queue and the program run is flagged as successful or unsuccessful.A programA program is a specific function you can call from the framework. It consists of hooks which will run in a particular sequence. This is not an actually defined function in code, rather it abstractly defines the sequence of functions that accomplish a task.A hookA hook is either a defined function (in code) that can be run and referenced by any program spec, or it is a reference to a program (yet to be implemented). Hooks operate on a data pool (key-value store made accessible to all children of a program) and accomplish a single, defined task. Hooks can optionally contain reversal functions which intend on reversing the actions of the hook.It is important that hooks achieve a single operation, eg, a data-write because then it is far easier to maintain atomicity and data integrity through rectification.A dependencyA type of hook that can be attached to an existing hook which is run before that hook is executed. This introduces recursiveness into the system, making it possible to make very large hooks comprising of many dependencies.Request bodyThe body of a RYU request consists of the following:{
  "program" : "my_program",
  "data_pool" : {
    "a" : "variable A",
    "b" : 2.345
  },
  "metadata" : {}
}Where:programis the name of the program you wish to rundata_poolis a dictionary structure containing the initial data poolmetadataData that can be optionally logged or acts as flags in the RYUSO. Any data stored here will be automatically converted to a magic variable and be stored in the data_pool.Multi-threading and parallelismThis framework is a single-threaded framework. However, there is nothing stopping a user of this framework to launch more than one RYUSO server for a particular spec. So long as you have some method of load balancing and logical mutual exclusion for all data operations, you can safely parallelise all RYUSO operations by redirecting tasks to the appropriate (or idle) server.SpecificationThe following is a spec definition which must be followed in order to create a valid RYUSO spec.Spec keyRequiredData TypeDescriptionExampleversionFalseStringCurrent version or build of the spec1.2.3nameTrueStringThe name of the spec containing only alphanumeric characters, spaces, underscores and hyphens. The module name will be a lower-case and underscore-spaced version of this.My transaction processingprogramsTrueDictionaryThe dictionary ofprogramwhere the key is the name of the program. The contents are defined below inprogram.*NAprogram.hooksTrueList of stringsThe list of references to hooks or programs in the spec. These hooks will run in the defined sequence.["hooks.check_balance", "programs.make_deposit"]program.argsTrueDictionaryThe dictionary ofargwhere the key is the name of the argument. This defines the initial data pool.NAarg.typeTrueStringOne of {string, int, float, list, dict, bool}"bool"arg.descriptionFalseStringA description of the variable and what it is for."Tells the TX clearer to run"arg.defaultFalseAnyThe default value for this argument. If this represents a program input, it will be automatically added to the data pool if no real parameter is provided. If this is a hook input, it will be used to generate code which uses the default value parameter for the python dictionary .get() function. If this is an output, it will be automatically added to the output if the argument doesn't exist in the data pool.NAarg.requiredTrueBooleanTells the spec whether this argument must be in the initial data pool. For outputs, it determines whether or not it has to be in the output data pool.trueprogram.outputsTrueDictionaryThe dictionary ofargwhere the key is the name of the argument. This defines the output data pool and returns only these args if they exist in the pool.NAprogram.descriptionFalseStringA description of what the program achieves and how it works"Will commit a transaction and optionally generates currency when the generate flag is set"hooksTrueDictionaryThe dictionary ofhookwhere the key is the name of the hook.NAhook.referenceTrueStringThe reference to the function from the root of the built package."my_orchestrator.util.math.is_prime"hook.argsTrueDictionaryThe dictionary ofargwhere the key is the name of the argument. This defines the requirements for the contents inside of the initial data pool for the hook. This is checked before the referenced function is run and after the dependencies are executed.NAhook.dependenciesFalseList of stringsThe list of references to hooks or programs in the spec. These hooks will run in the defined sequence. They will run before the referenced function is executed.["hooks.has_balance", "hooks.prep_working_dir"]hook.rectifiersFalseList of stringsThe list of references to hooks or programs in the spec. These hooks will be run in-order when the framework asks this hook to reverse its actions.["hooks.undo_commit_balance", "hooks.unwrap_tx_data"]hook.descriptionFalseStringThe string describing what the hook is suppoosed to accomplish"Checks whether the user's balance is > 5 and fails otherwise"