Package: gnar-gear
Gnar Gear: Gnarly Python AppsPart of Project Gnar:base•gear•piste•off-piste•edge•powder•genesis•patrolGet started with Project Gnar onJoin Project Gnar onSupport Project Gnar onGnar Gear sets up a powerful Flask-based Python service with two lines of code:fromgnar_gearimportGnarApp...GnarApp('my_gnarly_app',production=True,port=80).run()Installationpip3installgnar-gearFeature ListFlask app with auto blueprint registrationFlask WSGI Development ServerBjoern WSGI Production ServerWhy Bjoern? Check outthese benchmarks!And alsothese benchmarksPostgres database connection viaPostgres.pySES client connection viaBoto 3JWT configuration viaFlask-JWT-ExtendedPeer requests - HTTP requests to other microservices in the appExternal requests - Convenience wrapper aroundrequestsSQS Message Polling and Sendingargon2_cffi.PasswordHasherinstanceArgon2was the winner of the2015 Password Hashing CompetitionLogger configurationError handler with tracebackOverridable and extendable class-based designDevelopment ModeFlask WSGI server in fault-tolerant debug (watch & reload) modeThe module loader logs the stack trace of any modules that fail to loadIf a module fails to load, the app continues to watch for file changesCORS "disabled" (Access-Controlresponse headers set), so you don't need to use other means to circumvent CORSProduction ModeBjoern WSGI serverCORS "enabled" (Access-Controlresponse headers are not set)RequirementsBjoernBjoernrequireslibev(high performance event loop)Installlibevwithbrew install libevon Mac, or find your platform-specific installation commandhereApplication StructureGnarAppexpects to be instantiated inmain.pyat<top-level-module>/app, i.e. the minimum app folder structure is+ <top-level-module>
    + app
        main.py
    __init__.pyIt is recommended (not required) to place your apis in segregated folders underappand the tests in atestfolder under the<top-level-module>, e.g.+ <top-level-module>
    + app
        + admin
            apis.py
            constants.py
            services.py
        + user
            apis.py
            constants.py
            services.py
        __init__.py
        main.py
    + test
        constants.py
        test_app.py
    __init__.pyBlueprintsEachFlaskBlueprintmust be assigned to a global-levelblueprintvariable in its module, e.g.fromflaskimportBlueprint,jsonifyapi_name='user'url_prefix='/{}'.format(api_name)blueprint=Blueprint(api_name,__name__,url_prefix=url_prefix)^^^^^^^^^@blueprint.route('/get',methods=['GET'])defuser_get():returnjsonify({'status':'ok'})By default, theGnarApppicks up everyblueprintin an auto-scan of the application code.OverviewTheGnarAppclass provides a highly configurable, feature-rich, production-ready Flask-based app.ParametersArgs (required)name: The name of the application's top-level moduleproduction: Boolean flag indicating whether or not the build is in production modeport: The port to bind to the WSGI serverKwargs (optional)env_prefix: Environment variable prefix (defaults toGNAR)log_level: Log level overrideseeconfigure_loggerfor log level overviewblueprint_modules: List of modules to find Flask blueprints (default is auto-scan)seeconfigure_blueprintsfor blueprints overviewsqs: List (or single dict) of SQS queues to pollseeconfigure_sqs_pollsfor SQS polling overviewno_db: Boolean flag - specifyTrueif the app does not need a Postgres connectionno_jwt: Boolean flag - specifyTrueif the app does not use JWT headers (i.e. non-api services)Overridable BehaviorGnarApp.runsimply calls a set of steps in the class. Here is an example of how to override any of the steps:defpostconfig():log.info('My Postconfig Step!')ga=GnarApp('my_gnarly_app',production=True,port=80)ga.postconfig=postconfigga.run()Run StepsThe run steps rely on a set ofenvironment variableswhich use a configurable prefix
(i.e. theenv_prefixparameter). The defaultenv_prefixisGNAR. An example of a Gnar environment variable
using a custom prefix isGnarApp( ..., env_prefix='MY_APP')and then instead of readingGNAR_LOG_LEVEL, theconfigure_loggerstep will readMY_APP_LOG_LEVEL.preconfigNo default behavior - provided as an optional initial step in the app configuration.configure_flaskAttaches aFlaskinstance to the Gnar app.configure_loggerAttaches the root logger tosys.stdout.Sets the logging level to the first defined:log_levelparameterGNAR_<app name>_LOG_LEVELenvironment variable, e.g.GNAR_MY_GNARLY_APP_LOG_LEVELGNAR_LOG_LEVELenvironment variableINFOReminder:Valid settings(in increasing order of severity) areDEBUG,INFO,WARNING,ERROR,CRITICALSets the log format to the first defined:GNAR_LOG_FORMAT'%(asctime)s %(levelname)-8s %(name)s:%(lineno)d   %(message)s', e.g.:2018-07-09 15:41:46.420 INFO     gear.gnar_app:75   Logging at INFOSets the log formatdefault_msec_formatto the first defined:GNAR_LOG_FORMAT_MSEC'%s.%03d'(e.g..001)configure_argon2Attaches anargon2_cffi.PasswordHasherinstance to the Gnar app.Reads the following environment variables[TYPE: DEFAULT]topass into the Argon2 instance:GNAR_ARGON2_TIME_COST:[INT: 2]Number of iterations to performGNAR_ARGON2_MEMORY_COST:[INT: 512]Amount of memory (in KB) to useGNAR_ARGON2_PARALLELISM:[INT: 2]Number of parallel threads (changes the resulting hash value)GNAR_ARGON2_HASH_LEN:[INT: 16]Length of the hash in bytesGNAR_ARGON2_SALT_LEN:[INT: 16]Length of random salt to be generated for each password in bytesGNAR_ARGON2_ENCODING:[STR: 'utf-8']Encoding to use when a string is passed intohashorverifyNotefrom the docs:Only tweak these if you’ve determined using CLI that these defaults are too slow or too fast for your use case.To hash a password usingArgon2:from<top-level-module>.mainimportapphash=app.generate_password_hash(<plaintextpassword>)# ORhash=app.argon2.hash(<plaintextpassword>)Note that this creates arandomly salted, memory-hard hash using the Argon2i algorithm.To validate a password withArgon2:from<top-level-module>.mainimportappis_valid=app.check_password_hash(<passwordhashfromdatabase>,<plaintextpassword>)# ORis_valid=app.argon2.verify(<passwordhashfromdatabase>,<plaintextpassword>)Note thatapp.argon2.verifyraises an exceptionif the password is invalid whereasapp.check_password_hashdoes not.configure_databaseCreates a Postgres database connection and attaches it to the Gnar appReads the following environment variables to set thehost,dbname,user,passwordconnection string parameters, respectively:GNAR_PG_ENDPOINTGNAR_PG_DATABASEGNAR_PG_USERNAMEGNAR_PG_PASSWORDNote: ThePostgres APIprimarily consists ofrun,one, andallattach_instanceAttaches theGnarAppinstance to theapp.mainmodule. This enables easy access to the Gnar app from anywhere in the application usingfrom<top-level-module>.mainimportappTheGnarApp's runtime assets aredb,argon2,flask,check_password_hash,generate_password_hash, andget_ses_clientFor example, to fetch one result (orNone) from the database:app.db.one("SELECT * FROM foo WHERE bar='buz'")configure_blueprintsBy default,GnarAppauto-scans every Python moduleunder theappfolder for blueprints.EachFlaskBlueprintmust be assigned to a global-levelblueprintvariable in its module.If you prefer to skip the auto-scan, you can provide a list (or single string) of blueprint modules.Each item in the list of module names may use one of two formats:Without a.in the module name:GnarAppwill look for the module in<top-level-module>.app.<module name>.apisWith a.in the module:GnarAppwill look for the module in<top-level-module>.app.<module name>configure_errorhandlerDefines a generic (Exception-level) Flask error handler which:Logs the error message and itstraceback(format_exec)Returns a 200-level json response containing{"error": <error message>, "traceback": <traceback>}configure_jwtSets the FlaskJWT_SECRET_KEYvariable to the value of theGNAR_JWT_SECRET_KEYenvironment variable.Sets the FlaskJWT_ACCESS_TOKEN_EXPIRESvariable to the value of theGNAR_JWT_ACCESS_TOKEN_EXPIRES_MINUTESenvironment variable (default 15 mins).Attaches aJWTManagerinstance to theGnarApp.Defines functions forexpired_token_loader,invalid_token_loader, andunauthorized_loaderwhich return meaningful error messages as 200-level json responses containing{"error": <error message>}.configure_sqs_pollsConfigures a set of polls to run at specified intervals to receive messages from SQS queues.Specified via thesqskwarg as a list of dicts (or a single dict) with a set of required (*) and optional properties. See theAWS ReceiveMessage docsfor more details on the optional properties:*queue_name: The name of the SQS queue - the queue URL is retrieved from AWS with this name*callback: The function to call with each received message. The message will be deleted from the queue unless the callback function returnsFalse.interval_seconds: The interval (in seconds) between receive message requests - default 60attribute_names: A list of attributes to return with each messagemax_number_of_messages: The maximum number of messages to return with each request - 1 to 10, default 1message_attribute_names: Metadata to include with the message -more detailsreceive_request_attempt_id: The token used for deduplication of ReceiveMessage calls (FIFO queues only)visibility_timeout: The duration (in seconds) that the received messages are hidden from subsequent retrieve requestswait_time_seconds: The duration (in seconds) for which the call waits for a message to arrive in the queue before returningEach message received by the callback includes the following properties. See theAWS Message docsfor more details on these properties:Body: Message contentsAttributes: Map of requested attributesMD5OfBody: MD5 Digest of the message bodyMD5OfMessageAttributes: MD5 digest of the message attribute stringMessageAttributes: Map of custom message metadataNotes:Messages are processed sequentially in the order they are receivedThecallbackblocks message processing until it's completeAfter all received messages are processed, the queue is immediately checked for new messagesOnce thereceive_messagecall returns no messages, aTimerwith the specifiedinterval_secondsstarts the next loopReads the following environment variables to set theregion_name,aws_access_key_id, andaws_secret_access_keyparameters of theboto3.clientcall, respectively:GNAR_SQS_REGION_NAMEGNAR_SQS_ACCESS_KEY_IDGNAR_SQS_SECRET_ACCESS_KEYExample:defreceive_sqs_message(message):redis_connection.set(message['MessageId'],message['Body'],3600)sqs={'queue_name':'gnar-queue','callback':receive_sqs_message}GnarApp('piste',production,port,sqs=sqs).run()configure_after_requestAdds a JWT Authorization header (Bearer token) to responses which received a valid JWT token in the request.In development mode, adds CORS headers to the response (so that you don't need to bother circumventing CORS).postconfigNo default behavior - provided as an optional initial step in the app configuration.Runtime FunctionalitypeerHandles requests to other microservices in a Kubernetes app.Usage:from<top-level-module>.mainimportapppeer=app.peer(<<servicename>>)Requires environment variables in the form:production:<< SERVICE >>_SERVICE_PORT, e.g.PISTE_SERVICE_PORTdevelopment:GNAR_<< SERVICE >>_SERVICE_PORT, e.g.GNAR_PISTE_SERVICE_PORTIn production, the environment variable is automatically set by KubernetesIn development, the environment variable should point to another Gnar Gear app on localhost on a separate portThe environment variable value must be in the form<< ip address >>:<< port >>, e.g.exportGNAR_PISTE_SERVICE_PORT='127.0.0.1:9401'peer MethodsEach peer method accepts all thekwargsof the underlyingrequestsmethod in addition to anauto_authproperty.Ifauto_authisTrueand the current route is protected, an Authorization header is added to the request.Both microservices must use the sameGNAR_JWT_SECRET_KEYfor the authorization to work.Usesrequestsunder the hood:request(method, path, **kwargs): Constructs and sends a requesthead(path, **kwargs): Sends a HEAD requestget(path, params=None, **kwargs): Sends a GET requestpost(path, data=None, json=None, **kwargs): Sends a POST requestput(path, data=None, **kwargs): Sends a PUT requestpatch(path, data=None, **kwargs): Sends a PATCH requestdelete(path, **kwargs): Sends a DELETE requestExample:from<top-level-module>.mainimportappresponse=app.peer('piste').post('login',{'email':'ic3b3rg@gmail.com'})externalConvenience wrapper aroundrequestsExample:from<top-level-module>.mainimportappresponse=app.external.get('https://api.pwnedpasswords.com/range/5ce7d')generate_password_hash / check_password_hashConvenience wrappers forapp.argon2.hashandapp.argon2.verifyUsage:from<top-level-module>.mainimportapphash=app.generate_password_hash(<plaintextpassword>)is_valid=app.check_password_hash(<passwordhashfromdatabase>,<plaintextpassword>)get_ses_clientExposed as runtime functionality (as opposed to creating the client at initialization) because AWS will close the client after a short period of timeReturns an SES connection (using boto3)Reads the following environment variables to set theregion_name,aws_access_key_id, andaws_secret_access_keyparameters of theboto3.clientcall, respectively:GNAR_SES_REGION_NAMEGNAR_SES_ACCESS_KEY_IDGNAR_SES_SECRET_ACCESS_KEYUsage:from<top-level-module>.mainimportappapp.get_ses_client().send_email(...)See theBoto 3 Docsfor thesend_emailrequest syntax.send_sqs_messageSends a message to an SQS queue.Args (required):queue_name: The name of the SQS queue - the queue URL is retrieved from AWS with this namemessage_body: Message contentsKwargs (optional) - See theAWS SendMessage docsfor more details on these properties:delay_seconds: The length of time, in seconds, for which to delay a specific messagemessage_attributes: Metadata to include with the messagemessage_deduplication_id: The token used for deduplication of sent messages (FIFO queues only)message_group_id: The tag that specifies that a message belongs to a specific message group (FIFO queues only)Returns a dict with the following attributes:MessageId: Unique message identifierMD5OfMessageAttributes: MD5 digest of the message attribute stringMD5OfMessageBody: MD5 digest of the message bodySequenceNumber: Unique, non-consecutive, 128 bit string (FIFO queues only)Reads the following environment variables to set theregion_name,aws_access_key_id, andaws_secret_access_keyparameters of theboto3.clientcall, respectively:GNAR_SQS_REGION_NAMEGNAR_SQS_ACCESS_KEY_IDGNAR_SQS_SECRET_ACCESS_KEYExample:from<top-level-module>.mainimportappmessage=app.send_sqs_message('gnar-queue','The gnarliest gear in the world 🤙')Environment VariablesThe environment variables (with configurable prefix) used byGnarAppare:GNAR_ARGON2_ENCODINGGNAR_ARGON2_HASH_LENGNAR_ARGON2_MEMORY_COSTGNAR_ARGON2_PARALLELISMGNAR_ARGON2_SALT_LENGNAR_ARGON2_TIME_COSTGNAR_JWT_SECRET_KEYGNAR_LOG_LEVELGNAR_PG_DATABASEGNAR_PG_ENDPOINTGNAR_PG_PASSWORDGNAR_PG_USERNAMEGNAR_SES_ACCESS_KEY_IDGNAR_SES_REGION_NAMEGNAR_SES_SECRET_ACCESS_KEYGNAR_SQS_REGION_NAMEGNAR_SQS_ACCESS_KEY_IDGNAR_SQS_SECRET_ACCESS_KEYSee the relevant sections above for detailsMade withbyBrien Givens