code-augmentor-support 2.3.0
Enables the use of Python 3 to write code generator scripts for Code Augmentor
Requires Python: >=3.5,<4.0
Keywords: code-generation,data-driven,macro-processor

# code-augmentor-support

This package enables the use of Python 3 as a scripting platform to generate code to serve the goals of Code Augmentor.

Code Augmentor is a set of libraries, plugins and tools for bringing code generation techniques to every programmer. For a more detailed explanation please visit the main Code Augmentor Github repository [here](https://github.com/aaronicsubstances/code-augmentor).

As far as this package and Python 3 developers are concerned, it is enough to think of Code Augmentor as (1) a command-line application, (2) which is configured to run an [Apache Ant](https://ant.apache.org) XML build file, (3) which in turn runs a Python 3 package or project written by a programmer, (4) with the aim of generating code for Python 3 or  another target programming language, (5) using this package as a dependency.


## Install

`pip install code-augmentor-support`

If using poetry, can rather use `poetry install code-augmentor-support`

## Example

Below is a main script demonstrating how to set up the library for use with functions defined in two client modules snippets.py and worker.py.

It requires input and ouput file command-line arguments, and optional third argument to enable verbose logging.

```
python main.py test-augCodes.json actual.json
```

Or if using poetry,
```
poetry run python main.py test-augCodes.json actual.json
```

### main.py

```python
import os.path
import re
import sys

from code_augmentor_support.tasks import ProcessCodeTask

import snippets
import worker

FUNCTION_NAME_REGEX = re.compile(r'^((snippets|worker)\.)[a-zA-Z]\w*$')
def callUserFunction(functionName, augCode, context):
    # validate name.
    if not FUNCTION_NAME_REGEX.search(functionName):
        raise Exception("Invalid/Unsupported function name: " + functionName)

    # name is valid. make function call "dynamically".
    result = eval(functionName + '(augCode, context)')
    return result

instance = ProcessCodeTask()
instance.inputFile = sys.argv[1]
instance.outputFile = sys.argv[2]
if len(sys.argv) > 3:
    instance.verbose = bool(sys.argv[3])
instance.execute(callUserFunction)
if instance.allErrors:
    print(str(len(instance.allErrors)) + " error(s) found.", file=sys.stderr)
    for error in instance.allErrors:
        print(instance.generateStackTrace(error), file=sys.stderr)
    sys.exit(1)
```

### snippets.py

```python
def generateSerialVersionUID(augCode, context):
    return "private static final int serialVersionUID = 23L;"
```

### worker.py

```python
def stringify(augCode, context):
    g = context.newGenCode()
    for i in range(len(augCode.args)):
        s = '"' + augCode.args[i]
        if i < len(augCode.args) - 1:
            s += augCode.lineSeparator + '" +'
        else:
            s += '"'
        g.contentParts.append(context.newContent(s, True))
    return g
```

### test-augCodes.json (sample input file)

```json
{ "genCodeStartMarker": "//:GS:", "genCodeEndMarker": "//:GE:", "embeddedStringMarker": "//:STR:", "embeddedJsonMarker": "//:JSON:", "skipCodeStartMarker": "//:SS:", "skipCodeEndMarker": "//:SE:", "augCodeMarker": "//:AUG:", "inlineGenCodeMarker": "//:GG:", "nestedLevelStartMarker": "//:AUG_S:", "nestedLevelEndMarker": "//:AUG_E:" }
{"fileId":1,"dir":"src","relativePath":"A1.py","augmentingCodes":[{"id":1,"augCodeTokenType":6,"indent":"","lineNumber":1,"lineSeparator":"\n","blocks":[{"stringify":false,"jsonify":false,"content":" Snippets.generateSerialVersionUID "}]}]}
{"fileId":2,"dir":"src","relativePath":"B2.py","augmentingCodes":[{"id":1,"augCodeTokenType":6,"indent":"","lineNumber":1,"lineSeparator":"\n","blocks":[{"stringify":false,"jsonify":false,"content":" Worker.stringify "},{"stringify":true,"jsonify":false,"content":" SELECT * FROM contacts "},{"stringify":true,"jsonify":false,"content":" WHERE contacts.id = ? "}]},{"id":2,"augCodeTokenType":6,"indent":"","lineNumber":19,"lineSeparator":"\n","blocks":[{"stringify":false,"jsonify":false,"content":" Snippets.generateSerialVersionUID "},{"stringify":false,"jsonify":true,"content":"{ \"name\": \"expired\", \"type\": \"boolean\" } "}]}]}

```

### actual.json (sample output file)

```json
{}
{"fileId":1,"generatedCodes":[{"id":1,"contentParts":[{"content":"private static final int serialVersionUID = 23L;","exactMatch":false}]}]}
{"fileId":2,"generatedCodes":[{"id":1,"contentParts":[{"content":"\" SELECT * FROM contacts \n\" +","exactMatch":true},{"content":"\" WHERE contacts.id = ? \"","exactMatch":true}]},{"id":2,"contentParts":[{"content":"private static final int serialVersionUID = 23L;","exactMatch":false}]}]}

```

## Usage

The library's functionality is contained in the method `execute` of the class `ProcessCodeTask` in the `tasks` module of this package. The `execute` method takes a function object used for evaluating code generation requests and producing generated code snippets.

Instances of `ProcessCodeTask` have the following properties and methods:

   * `inputFile` - path to the code generation request. Must be the aug code file result of running the *code_aug_prepare* Ant task.
   * `outputFile` - path for writing out code generation response. Will be used as the gen code file input to the *code_aug_complete* Ant task.
   * `verbose` - boolean property which can be used with default verbose logging mechansim to enable printing of verbose mesages to standard output.
   * `beforeAllFilesHook` - optional function that will be called once, before any file of aug codes is read for processing. Called with a single context argument.
   * `afterAllFilesHook` - optional function that will be called once, after all files of aug codes are read and processed. Called with a single context argument.
   * `beforeFileHook` - optional function that will be called every time just after a file of aug codes (even if empty) has been read, before the aug codes are processed. Called with a single context argument.
   * `afterFileHook` - optional function that will be called every time just after all aug codes (even if empty) of a file have been processed. Called with a single context argument.
   * `allErrors` - array which contains any errors encountered during execution.
   * `generateStackTrace(e)` - takes an error object and returns a string stack trace of it and any nested cause error objects.
   
These methods can be overriden in a subclass:
   * `logVerbose`, `logInfo`, `logWarn` - methods which are called with a format string, *args, and **kwargs, when a verbose message, normal message, or warning message is issued. By default all normal and warning messages are printed to standard output, and verbose messages are ignored.

The `evalFunction` function argument of the `execute` method is called with 3 arguments. The first is name of a function to invoke in the current Python scope, and the remaining two are an augmenting code object and a helper instance of the `ProcessCodeContext` class from the `models` module of this package. These remaining two arguments are the arguments passed to the function to be invoked.

The `evalFunction` is called with every augmenting code object encountered in the input file. It is expected to in turn call client-defined functions dynamically and receive from them a correponding generated code object to be written to the output file. As a convenience, it can return strings, content parts, and arrays of generated code objects.


### Properties and Methods of `ProcessCodeContext` instances

   * *header* - JSON object resulting from parsing first line of input file.
   * *globalScope* - a dictionary provided for use by clients which remains throughout parsing of entire input file.
   * *fileScope* - a dictionary provided for use by clients which is recreated at the start of processing every line of input file.
   * *fileAugCodes* - object with propery of an array of augmenting code objects for the file currently being processed.
   * *fileGenCodes* - object with property of an array of generated code objects for the file currently being processed.
   * *augCodeIndex* - index of `augCode` parameter in `fileAugCodes->augmentingCodes` array. Can be equal to -1 if called in a situation (e.g. inside hooks) where there is no augmenting code object in focus.
   * *newGenCode()* - convenience method available to clients for creating a generated code object with empty `contentParts` array property.
   * *newContent(content, exactMatch=false)* - convenience method available to clients for creating a new content part object with properties set with arguments supplied to the function.
   * *getScopeVar(name)* - gets a variable from `fileScope` dictionary with given name, or from `globalScope` dictionary if not found in `fileScope`.
   * *setScopeVar(name, value)* - sets/replaces a variable in `fileScope` dictionary with given name.
   * *setGlobalScopeVar(name, value)* - sets/replaces a variable in `globalScope` dictionary with given name.
   * *addError(message, e)* - used for adding to pending list of errors during aug code processing. First parameter is an error message which will be augmented with the current state of the context object. The second parameter is an optional error object which will become the \_\_cause\_\_ property of the error to be created and added to the list of pending errors.

### Reserved 'code' Prefix

CodeAugmentor supplies utility functions and variables by reserving the **code** prefix. As such scripts should avoid naming aug code processing functions and variables in fileScope/globalScope with that prefix.

The following variables are provided by default in `context.globalScope`:

   * *code_indent* - set with value of four spaces.


### Note on JSON serialization

This library deserializes JSON objects into instances of the class [types.SimpleNamespace](https://docs.python.org/3/library/types.html#types.SimpleNamespace). It similarly requires objects to be serialized to be either dictionaries or instances of `types.SimpleNamespace`. 

By so doing clients are provided with the convenience that any arbitrary field can be set on a JSON object, and it will get serialized. 

The default Python behaviour will apply though, when trying to access non-existent fields on a deserialized JSON object by raising an error. In that case, the function `hasattr` will have to be called first.

## Further Information

For more information on the structure of augmenting code object, generated code object and other considerations, refer to [wiki](https://github.com/aaronicsubstances/code-augmentor/wiki/Documentation-for-Code-Generator-Scripts) in the main Code Augmentor repository.


## Building and Testing Locally

[Poetry](https://python-poetry.org/) is the build system for this project. So it is required to set up it first.

   * Clone repository locally
   * Install project dependencies with `poetry install`
   * With all dependencies present locally, test project with `poetry run pytest tests -s`

*NB:*

   1. Had to perform the following step on Windows 10 (2022-09-27) to get Poetry 1.1.4 working without freezing:
       * run `poetry config experimental.new-installer false`