Package: zswag
Zswagzswag is a set of libraries for using/hosting zserio services through OpenAPI.Table of Contents:ComponentsSetupFor Python UsersFor C++ UsersOpenAPI Generator CLIGenerator Usage ExampleDocumentation extractionServer Component (Python)Using the Python ClientC++ ClientClient Environment SettingsHTTP Proxies and AuthenticationSwagger User InterfaceOpenAPI Options InteroperabilityHTTP methodRequest BodyURL Blob ParameterURL Scalar ParameterURL Array ParameterURL Compound ParameterServer URL Base PathAuthentication SchemesComponentsThe zswag repository contains two main libraries which provide
OpenAPI layers for zserio Python and C++ clients. For Python, there
is even a generic zserio OpenAPI server layer.The following UML diagram provides a more in-depth overview:Here are some brief descriptions of the main components:zswagclis a C++ Library which exposes the zserio OpenAPI service clientOAClientas well as the more genericOpenApiClientandOpenApiConfigclasses.
The latter two are reused for the Python client library.zswagis a Python Library which provides both a zserio Python service client
(OAClient) as well as a zserio-OpenAPI server layer based on Flask/Connexion
(OAServer). It also contains the command-line toolzswag.gen, which can be
used to generate an OpenAPI specification from a zserio Python service class.pyzswagclis a binding library which exposes the C++-based OpenApi
parsing/request functionality to Python.Please consider it "internal".httpclis a wrapper aroundcpp-httplib,
HTTP request configuration and OS secret storage abilities based on
thekeychainlibrary.SetupFor Python UsersSimply runpip install zswag.Note: This only works with ...64-bit Python 3.8.x,pip --version>= 19.364-bit Python 3.9.x,pip --version>= 19.3Note:On Windows, make sure that you have theMicrosoft Visual C++ Redistributable Binariesinstalled. You can find the x64 installer here:https://aka.ms/vs/16/release/vc_redist.x64.exeFor C++ UsersUsing CMake, you can ...🌟run tests.🌟build the zswag wheels for a custom Python version.🌟integrate the C++ client into a C++ project.Dependencies are managed via CMake'sFetchContentmechanism and Conan 2.0.
Make sure you have a recent version of CMake (>= 3.24) and Conan (>= 2.0.5) installed.The basic setup follows the usual CMake configure/build steps:mkdirbuild&&cdbuild
cmake..
cmake--build.Note:The Python environment used for configuration will be used
to build the resulting wheels. After building, you will find the Python
wheels underbuild/bin/wheel.To run tests, just execute CTest at the top of the build directory:cdbuild&&ctest--verboseOpenAPI Generator CLIAfter installingzswagvia pip asdescribed above,
you can runpython -m zswag.gen, a CLI to generate OpenAPI YAML files.
The CLI offers the following optionsusage: Zserio OpenApi YAML Generator [-h] -s service-identifier -i
                                     zserio-or-python-path
                                     [-r zserio-src-root-dir]
                                     [-p top-level-package] [-c tags [tags ...]]
                                     [-o output] [-b BASE_CONFIG_YAML]

optional arguments:
  -h, --help
        show this help message and exit
  -s service-identifier, --service service-identifier

        Fully qualified zserio service identifier.

        Example:
            -s my.package.ServiceClass

  -i zserio-or-python-path, --input zserio-or-python-path

        Can be either ...
        (A) Path to a zserio .zs file. Must be either a top-
            level entrypoint (e.g. all.zs), or a subpackage
            (e.g. services/myservice.zs) in conjunction with
            a "--zserio-source-root|-r <dir>" argument.
        (B) Path to parent dir of a zserio Python package.

        Examples:
            -i path/to/schema/main.zs         (A)
            -i path/to/python/package/parent  (B)

  -r zserio-src-root-dir, --zserio-source-root zserio-src-root-dir

        When -i specifies a zs file (Option A), indicate the
        directory for the zserio -src directory argument. If
        not specified, the parent directory of the zs file
        will be used.

  -p top-level-package, --package top-level-package

        When -i specifies a zs file (Option A), indicate
        that a specific top-level zserio package name
        should be used.

        Examples:
            -p zserio_pkg_name

  -c tags [tags ...], --config tags [tags ...]

        Configuration tags for a specific or all methods.
        The argument syntax follows this pattern:

           [(service-method-name):](comma-separated-tags)

        Note: The -c argument may be applied multiple times.
        The `comma-separated-tags` must be a list of tags
        which indicate OpenApi method generator preferences.
        The following tags are supported:

        get|put|post|delete : HTTP method tags
                query|path| : Parameter location tags
                header|body
                  flat|blob : Flatten request object,
                              or pass it as whole blob.
          (param-specifier) : Specify parameter name, format
                              and location for a specific
                              request-part. See below.
            security=(name) : Set a particular security
                              scheme to be used. The scheme
                              details must be provided through
                              the --base-config-yaml.
         path=(method-path) : Set a particular method path.
                              May contain placeholders for
                              path params.

        A (param-specifier) tag has the following schema:

            (field?name=...
                  &in=[path|body|query|header]
                  &format=[binary|base64|hex]
                  [&style=...]
                  [&explode=...])

        Examples:

          Expose all methods as POST, but `getLayerByTileId`
          as GET with flat path-parameters:

            `-c post getLayerByTileId:get,flat,path`

          For myMethod, put the whole request blob into the a
          query "data" parameter as base64:

            `-c myMethod:*?name=data&in=query&format=base64`

          For myMethod, set the "AwesomeAuth" auth scheme:

            `-c myMethod:security=AwesomeAuth`

          For myMethod, provide the path and place myField
          explicitely in a path placeholder:

            `-c 'myMethod:path=/my-method/{param},...
                 myField?name=param&in=path&format=string'`

        Note:
            * The HTTP-method defaults to `post`.
            * The parameter 'in' defaults to `query` for
              `get`, `body` otherwise.
            * If a method uses a parameter specifier, the
              `flat`, `body`, `query`, `path`, `header` and
              `body`-tags are ignored.
            * The `flat` tag is only meaningful in conjunction
              with `query` or `path`.
            * An unspecific tag list (no service-method-name)
              affects the defaults only for following, not
              preceding specialized tag assignments.

  -o output, --output output

        Output file path. If not specified, the output will be
        written to stdout.

  -b BASE_CONFIG_YAML, --base-config-yaml BASE_CONFIG_YAML

        Base configuration file. Can be used to fully or partially
        substitute --config arguments, and to provide additional
        OpenAPI information. The YAML file must look like this:

          method: # Optional method tags dictionary
            <method-name|*>: <list of config tags>
          securitySchemes: ... # Optional OpenAPI securitySchemes
          info: ...            # Optional OpenAPI info section
          servers: ...         # Optional OpenAPI servers section
          security: ...        # Optional OpenAPI global securityGenerator Usage exampleLet's consider the following zserio service saved undermyapp/services.zs:package services;

struct Request {
  int32 value;
};

struct Response {
  int32 value;
};

service MyService {
  Response myApi(Request);
};An OpenAPI fileapi.yamlforMyServicecan now be
created with the followingzswag.geninvocation:cdmyapp
python-mzswag.gen-sservices.MyService-iservices.zs-oapi.yamlYou can further customize the generation using-cconfiguration
arguments. For example,-c get,flat,pathwill recursively "flatten"
the zserio request object into it's compound scalar fields usingx-zserio-request-partfor all methods.
If you want to change OpenAPI parameters only for one particular
method, you can prefix the tag config argument with the method
name (-c methodName:tags...).Documentation ExtractionWhen invokingzswag.genwith-i zserio-filean attempt
will be made to populate the service/method/request/response
descriptions with doc-strings that are extracted from the zserio
sources.For structs and services, the documentation is expected to be
enclosed by/*! .... !*/markers preceding the declaration:/*!### My Markdown Struct DocI choose to __highlight__ this word.!*/structMyStruct{...};For service methods, a single-line doc-string is parsed which
immediately precedes the declaration:/** This method is documented. */ReturnTypemyMethod(ArgumentType);Server ComponentTheOAServercomponent gives you the power to marry a zserio-generated app
server class with a user-written app controller and a fitting OpenAPI specification.
It is based onFlaskandConnexion.Implementation choice regarding HTTP response codes:The server as implemented
here will return HTTP code400(Bad Request) when the user request could not
be parsed, and500(Internal Server Error) when a different exception occurred while
generating the response/running the user's controller implementation.Integration ExampleWe consider the samemyappdirectory with aservices.zszserio file
as already used in theOpenAPI Generator Example.Note:myappmust be available as a module (it must be
possible toimport myapp).We recommend to run the zserio Python generator invocation
inside themyappmodule's__init__.py, like this:importzseriofromos.pathimportdirname,abspathworking_dir=dirname(abspath(__file__))zserio.generate(zs_dir=working_dir,main_zs_file="services.zs",gen_dir=working_dir)A server script likemyapp/server.pymight then look as follows:importzswagimportmyapp.controllerascontrollerfrommyappimportworking_dir# This import only works after zserio generation.importservices.apiasservicesapp=zswag.OAServer(controller_module=controller,service_type=services.MyService.Service,yaml_path=working_dir+"/api.yaml",zs_pkg_path=working_dir)if__name__=="__main__":app.run()The server script above references two important components:AnOpenAPI file(myapp/api.yaml): Upon startup,OAServerwill output an error message if this file does not exist. The
error message already contains the correct command to
invoke theOpenAPI Generator CLIto generatemyapp/api.yaml.Acontroller module(myapp/controller.py): This file provides
the actual implementations for your service endpoints.For the current example,controller.pymight look as follows:importservices.apiasservices# Written by youdefmy_api(request:services.Request):returnservices.Response(request.value*42)Using the Python ClientThe generic Python client talks to any zserio service that is running
via HTTP/REST, and provides an OpenAPI specification of it's interface.Integration ExampleAs an example, consider a Python module calledmyappwhich has the
samemyapp/__init__.pyandmyapp/services.zszserio definition aspreviously mentioned. We consider
that the server is providing its OpenAPI spec underlocalhost:5000/openapi.json.In this setting, a clientmyapp/client.pymight look as follows:fromzswagimportOAClientimportservices.apiasservicesopenapi_url="http://localhost:5000/openapi.json"# The client reads per-method HTTP details from the OpenAPI URL.# You can also pass a local file by setting the `is_local_file` argument# of the OAClient constructor.client=services.MyService.Client(OAClient(openapi_url))# This will trigger an HTTP request under the hood.client.my_api(services.Request(1))As you can see, an instance ofOAClientis passed into the constructor
for zserio to use as the service client's transport implementation.Note:While connecting, the client will also use ...Persistent HTTP configuration.Additional HTTP query/header/cookie/proxy/basic-auth configs passed
into theOAClientconstructor using an instance ofzswag.HTTPConfig.
For example:fromzswagimportOAClient,HTTPConfigimportservices.apiasservicesconfig=HTTPConfig()\.header(key="X-My-Header",val="value")\# Can be specified.cookie(key="MyCookie",val="value")\# multiple times..query(key="MyCookie",val="value")\#.proxy(host="localhost",port=5050,user="john",pw="doe")\.basic_auth(user="john",pw="doe")\.bearer("bearer-token")\.api_key("token")client=services.MyService.Client(OAClient("http://localhost:8080/openapi.",config=config))# Alternative when specifying api-key or bearerclient=services.MyService.Client(OAClient("http://localhost:8080/openapi.",api_key="token",bearer="token"))Note:The additionalconfigwill only enrich, not overwrite the
default persistent configuration. If you would like to prevent persistent
config from being considered at all, setHTTP_SETTINGS_FILEto empty,
e.g. viaos.environ['HTTP_SETTINGS_FILE']=''C++ ClientThe generic C++ client talks to any zserio service that is running
via HTTP/REST, and provides an OpenAPI specification of its interface.
When using the C++OAClientwith your zserio schema, make sure
that the flags-withTypeInfoCodeand-withReflectionCodeare passed to the zserio C++ emitter.Integration ExampleAs an example, we consider themyappdirectory which contains aservices.zszserio definition aspreviously mentioned.We assume that zswag is added tomyappas aGit submoduleundermyapp/zswag.Next tomyapp/services.zs, we place amyapp/CMakeLists.txtwhich describes our project:project(myapp)# If you are not interested in building zswag Python# wheels, you can set the following option:# set(ZSWAG_BUILD_WHEELS OFF)# If your compilation environment does not provide# libsecret, the following switch will disable keychain integration:# set(ZSWAG_KEYCHAIN_SUPPORT OFF)# This is how C++ will know about the zswag lib# and its dependencies, such as zserio.if(NOTTARGETzswag)FetchContent_Declare(zswagGIT_REPOSITORY"https://github.com/ndsev/zswag.git"GIT_TAG"v1.5.0"GIT_SHALLOWON)FetchContent_MakeAvailable(zswag)endif()find_package(OpenSSLCONFIGREQUIRED)target_link_libraries(httplibINTERFACEOpenSSL::SSL)# This command is provided by zswag to easily create# a CMake C++ reflection library from zserio code.add_zserio_library(${PROJECT_NAME}-zserio-cppWITH_REFLECTIONROOT"${CMAKE_CURRENT_SOURCE_DIR}"ENTRYservices.zsTOP_LEVEL_PKGmyapp_services)# We create a myapp client executable which links to# the generated zserio C++ library and the zswag client# library.add_executable(${PROJECT_NAME}client.cpp)# Make sure to link to the `zswagcl` targettarget_link_libraries(${PROJECT_NAME}${PROJECT_NAME}-zserio-cppzswagcl)Theadd_executablecommand above references the filemyapp/client.cpp,
which contains the code to actually use the zswag C++ client.#include"zswagcl/oaclient.hpp"#include<iostream>#include"myapp_services/services/MyService.h"usingnamespacezswagcl;usingnamespacehttpcl;namespaceMyService=myapp_services::services::MyService;intmain(intargc,char*argv[]){// Assume that the server provides its OpenAPI definition hereautoopenApiUrl="http://localhost:5000/openapi.json";// Create an HTTP client to be used by our OpenAPI clientautohttpClient=std::make_unique<HttpLibHttpClient>();// Fetch the OpenAPI configuration using the HTTP clientautoopenApiConfig=fetchOpenAPIConfig(openApiUrl,*httpClient);// Create a Zserio reflection-based OpenAPI client that// uses the OpenAPI configuration we just retrieved.autoopenApiClient=OAClient(openApiConfig,std::move(httpClient));// Create a MyService client based on the OpenApi-Client// implementation of the zserio::IServiceClient interface.automyServiceClient=MyService::Client(openApiClient);// Create the request objectautorequest=myapp_services::services::Request(2);// Invoke the REST endpoint. Mind that your method-// name from the schema is appended with a "...Method" suffix.autoresponse=myServiceClient.myApiMethod(request);// Print the responsestd::cout<<"Got "<<response.getValue()<<std::endl;}Note:While connecting,HttpLibHttpClientwill also use ...Persistent HTTP configuration.Additional HTTP query/header/cookie/proxy/basic-auth configs passed
into theOAClientconstructor using an instance ofhttpcl::Config.
You can include this class via#include "httpcl/http-settings.hpp".
The additionalConfigwill only enrich, not overwrite the
default persistent configuration. If you would like to prevent persistent
config from being considered at all, setHTTP_SETTINGS_FILEto empty,
e.g. viasetenv.Client Environment SettingsBoth the Python and C++ Clients can be configured using the following
environment variables:Variable NameDetailsHTTP_SETTINGS_FILEPath to settings file for HTTP proxies and authentication, seenext sectionHTTP_LOG_LEVELVerbosity level for console/log output. Set todebugfor detailed output.HTTP_LOG_FILELogfile-path (including filename) to redirect console output. The log will rotate with three files (HTTP_LOG_FILE,HTTP_LOG_FILE-1,HTTP_LOG_FILE-2).HTTP_LOG_FILE_MAXSIZEMaximum size of the logfile, in bytes. Defaults to 1GB.HTTP_TIMEOUTTimeout for HTTP requests (connection+transfer) in seconds. Defaults to 60s.HTTP_SSL_STRICTSet to any nonempty value for strict SSL certificate validation.Persistent HTTP Headers, Proxy, Cookie and AuthenticationBoth the PythonOAClientand C++HttpLibHttpClientread a YAML file
stored under a path which is given by theHTTP_SETTINGS_FILEenvironment
variable. The YAML file contains a list of HTTP-related configs that are
applied to HTTP requests based on a regular expression which is matched
against the requested URL.For example, the following entry would match all requests due to the.*url-match-pattern:-url:.*basic-auth:user:johndoekeychain:keychain-service-stringproxy:host:localhostport:8888user:testkeychain:...cookies:key:valueheaders:key:valuequery:key:valueapi-key:valueNote:Forproxyconfigs, the credentials are optional.Theapi-keysetting will be applied under the correct
cookie/header/query parameter, if the service
you are connecting to uses anOpenAPIapiKeyauth scheme.Passwords can be stored in clear text by setting apasswordfield instead
of thekeychainfield. Keychain entries can be made with different tools
on each platform:Linuxsecret-toolmacOSadd-generic-passwordWindowscmdkeySwagger User InterfaceIf you have installedpip install "connexion[swagger-ui]", you can view
API docs of your service under[/prefix]/ui.OpenAPI Options InteroperabilityThe Server, Clients and Generator offer various degrees of freedom
regarding the OpenAPI YAML file. The following sections detail which
components support which aspects of OpenAPI. The difference in compliance
is mostly due to limited development scopes. If you are missing a particular
OpenAPI feature for a particular component, feel free to create an issue!Note:For all options that are not supported byzswag.gen, you
will need to manually edit the OpenAPI YAML file to achieve the desired
configuration. You will also need to edit the file manually to fill in
meta-info (provider name, service version, etc.).HTTP methodTo change theHTTP method, the desired method name is placed
as the key under the method path, such as in the following example:paths:/methodName:{get|post|put|delete}:...Component SupportFeatureC++ ClientPython ClientOAServerzswag.gengetpostputdelete✔️✔️✔️✔️patch❌️❌️❌️❌️Note:Patch is unsupported, because the required semantics of
a partial object update cannot be realized in the zserio transport
layer interface.Request BodyA server can instruct clients to transmit their zserio request object in the
request body when using HTTPpost,putordelete.
This is done by setting the OpenAPIrequestBody/contenttoapplication/x-zserio-object:requestBody:content:application/x-zserio-object:schema:type:stringComponent SupportFeatureC++ ClientPython ClientOAServerzswag.genapplication/x-zserio-object✔️✔️✔️✔️URL Blob ParameterZswag tools support an additional OpenAPI method parameter
field calledx-zserio-request-part. Through this field,
a service provider can express that a certain request parameter
only contains a part of, or the whole zserio request object.
When parameter contains the whole request object,x-zserio-request-partshould be set to an asterisk (*):parameters:-description:''in:query|path|headername:parameterNamerequired:truex-zserio-request-part:"*"schema:format:string|byte|base64|base64url|hex|binaryAbout theformatspecifier value:Bothstringandbinaryresult in a raw URL-encoded string buffer.Bothbyteandbase64result in a standard Base64-encoded value.
Thebase64urloption indicates URL-safe Base64 format.Thehexencoding produces a hexadecimal encoding of the request blob.Note:When a parameter is passed within=path, its valuemust not be empty. This holds true for strings and bytes,
but also for arrays (see below).Component SupportFeatureC++ ClientPython ClientOAServerzswag.genx-zserio-request-part: *✔️✔️✔️✔️format: string✔️✔️✔️✔️format: byte✔️✔️✔️✔️format: hex✔️✔️✔️✔️URL Scalar ParameterUsingx-zserio-request-part, it is also possible to transfer
only a single scalar (nested) member of the request object:parameters:-description:''in:query|path|headername:parameterNamerequired:truex-zserio-request-part:"[parent.]*member"schema:format:string|byte|base64|base64url|hex|binaryIn this case,x-zserio-request-partshould point to a scalar type,
such asuint8,float32,stringetc.Theformatvalue effect remains as explained above. A small
difference exists for integer types: Their hexadecimal representation
will be the natural numeric one, not binary.Component SupportFeatureC++ ClientPython ClientOAServerzswag.genx-zserio-request-part: <[parent.]*member>✔️✔️✔️✔️URL Array ParameterThex-zserio-request-partmay also point to an array member of
the zserio request struct, like so:parameters:-description:''in:query|path|headerstyle:form|simple|label|matrixexplode:true|falsename:parameterNamerequired:truex-zserio-request-part:"[parent.]*array_member"schema:format:string|byte|base64|base64url|hex|binaryIn this case,x-zserio-request-partshould point to an array of
scalar types. The array will be encoded according
to theformat, style and explodespecifiers.FeatureC++ ClientPython ClientOAServerzswag.genx-zserio-request-part: <[parent.]*array_member>✔️✔️✔️✔️style: simple✔️✔️✔️✔️style: form✔️✔️✔️✔️style: label✔️✔️❌✔️style: matrix✔️✔️❌✔️explode: true✔️✔️✔️✔️explode: false✔️✔️✔️✔️URL Compound ParameterIn this case,x-zserio-request-partpoints to a zserio compound struct
instead of a field with a scalar value.This is currently not supported.Component SupportFeatureC++ ClientPython ClientOAServerzswag.genx-zserio-request-part: <[parent.]*compound_member>❌️❌️❌️❌️Server URL Base PathOpenAPI allows for aserversfield in the spec that lists URL path prefixes
under which the specified API may be reached. The OpenAPI clients
looks into this list to determine a URL base path from
the first entry in this list. A sample entry might look as follows:servers:
- http://unused-host-information/path/to/my/apiThe OpenAPI client will then call methods with your specified host
and port, but prefix the/path/to/my/apistring.Component SupportFeatureC++ ClientPython ClientOAServerzswag.genservers✔️✔️✔️✔️Authentication SchemesTo facilitate the communication of authentication needs for the whole or parts
of a service, OpenAPI allows forsecuritySchemesandsecurityfields in the spec.
Please refer to the relevant parts of theOpenAPI 3 specificationfor some
examples on how to integrate these fields into your spec.Zswag currently understands the following authentication schemes:HTTP Basic Authorization:If a called endpoint requires HTTP basic auth,
zswag will verify that the HTTP config contains basic-auth credentials.
If there are none, zswag will throw a descriptive runtime error.HTTP Bearer Authorization:If a called endpoint requires HTTP bearer auth,
zswag will verify that the HTTP config contains a header with the
key nameAuthorizationand the valueBearer <token>,case-sensitive.API-Key Cookie:If a called endpoint requires a Cookie API-Key,
zswag will either applytheapi-keysetting, or verify that the
HTTP config contains a cookie with the required name,case-sensitive.API-Key Query Parameter:If a called endpoint requires a Query API-Key,
zswag will either apply theapi-keysetting, or verify that the
HTTP config contains a query key-value pair with the required name,case-sensitive.API-Key Header:If a called endpoint requires an API-Key Header,
zswag will either apply theapi-keysetting, or verify that the
HTTP config contains a header key-value pair with the required name,case-sensitive.Note: If you don't want to pass your Basic-Auth/Bearer/Query/Cookie/Header
credential through yourpersistent config,
you can pass ahttpcl::Config/HTTPConfigobject to theOAClient/OAClient.
constructor in C++/Python with the relevant detail.Component SupportFeatureC++ ClientPython ClientOAServerzswag.genHTTP Basic-AuthHTTP Bearer-AuthCookie API-KeyHeader API-KeyQuery API-Key✔️✔️✔️(**)✔️OpenID ConnectOAuth2❌️❌️✔️(**)❌️(**): The server support for all authentication schemes depends on your
configuration of the WSGI server (Apache/Nginx/...) which wraps the zswag Flask app.