Package: open-alchemy.package-database
DatabaseThe database has a facade exposes a series of functions that enable services to
personalize responses.InterfaceThe interface for the database is defined as theTDatabaseclass here:open_alchemy/package_database/types.pyIt can be retrieved using:fromopen_alchemyimportpackage_databasedatabase_instance=package_database.get()Note that theSTAGEenvironment variable needs to be set. The possible
values are:TEST: DynamoDB is assumed to be running athttp://localhost:8000PROD: a connection is established to the AWS hosted DynamoDBTablesSpecsStores information about the specs for a user. The following access patterns
are expected:count the number of models for a user,create or update a spec record for a user,get the latest version of a spec for a user,list all specs for a user,retrieve a particular spec for a user,delete a particular spec for a user,list all versions of a spec for a user anddelete all specs for a user.Count Models for a UserCounts the number of models a user has defined.Input:sub: unique identifier for the user.Output:The sum of the latestmodel_countfor each spec for the user.Algorithm:filter by thesubandupdated_at_idto start withlatest#andsum over themodel_countof each record.Create or Update a SpecInput:sub,name: the name of the spec,version: the version of the spec,model_count: the number of models in the spec,title(optional): the title of the spec anddescription(optional): the description of the spec.Output:Algorithm:calculate theidof the spec usinghttps://packaging.pypa.io/en/latest/utils.html#packaging.utils.canonicalize_namebased on thename,calculateupdated_atbased on he current EPOCH time usinghttps://docs.python.org/3/library/time.html#time.timeand convert to an integer represented as a string,calculate the value forupdated_at_idby joining a zero paddedupdated_atto 20 characters andidwith a#and forid_updated_atby joiningidandupdated_atwith a#,save the item to the database,create another item but uselatestforupdated_atwhen generatingupdated_at_idandid_updated_atGet Latest Spec VersionRetrieve the latest version of a spec.Input:subandname.Output:The latestversionof the spec.Algorithm:calculate theidof the spec usinghttps://packaging.pypa.io/en/latest/utils.html#packaging.utils.canonicalize_namebased on thename,Retrieve the item using thesubpartition key andupdated_at_idsort key
equal tolatest#<id>andreturn the version of the item.List SpecsReturns information about all the available specs for a user.Input:sub.Output:A list of dictionaries with theid,name,updated_at,version,model_countandtitleanddescriptionif they are defined.Algorithm:filter items using thesubpartition key andupdated_at_idstarting withlatest#andconvert the items to dictionaries.Get SpecRetrieve a particular spec for a user.Input:subandname.Output:dictionary with theid,name,updated_at,version,model_countandtitleanddescriptionif they are definedAlgorithm:calculate theidof the spec usinghttps://packaging.pypa.io/en/latest/utils.html#packaging.utils.canonicalize_namebased on thename,get the item based onsubandupdated_at_idwith the valuelatest#<id>,handle the case where it is not found by returningNoneandconvert the item to a dictionary.Delete SpecDelete a particular spec for a user.Input:subandname.Output:Algorithm:calculate theidof the spec usinghttps://packaging.pypa.io/en/latest/utils.html#packaging.utils.canonicalize_namebased on thename,query theid_updated_at_indexlocal secondary index by filtering forsubandid_updated_atstarting with<id>#anddelete all returned items.List Spec VersionsReturns information about all the available versions of a spec for a user.Input:subandname.Output:A list of dictionaries with theid,name,updated_at,version,model_countandtitleanddescriptionif they are defined.Algorithm:calculate theidof the spec usinghttps://packaging.pypa.io/en/latest/utils.html#packaging.utils.canonicalize_namebased on thename,query theid_updated_at_indexlocal secondary index by filtering forsubandid_updated_atstarting with<id>#,filter out any items whereupdated_at_idstarts withlatest#andconvert the items to dictionaries.Delete All Specs for a UserInput:sub.Output:Algorithm:Delete all entries forsub.Spec Propertiessub: A string that is the partition key of the table.id: A string.name: A string.updated_at: A string.version: A string.title: An optional string.description: An optional string.model_countA number.updated_at_id: A string that is the sort key of the table.id_updated_at: A string that is the sort key of theidUpdatedAtlocal secondary index of the table.CredentialsStores credentials for a user. The following access patterns are expected:list available credentials for a user,create or update credentials for a user,retrieve particular credentials for a user,check that a public and secret key combination exists and retrieve thesubfor it,delete particular credentials for a user anddelete all credentials for a user.List CredentialsList all available credentials for a user.Input:sub.Output:list of dictionaries with theid,public_keyandsaltkeys.Algorithm:use thesubpartition key to retrieve all credentials for the user andmap the items to a dictionary.Create or Update CredentialsCreate or update credentials for a user.Input:sub: unique identifier for the user,id: unique identifier for the credentials,public_key: public identifier for the credentials,secret_key_hash: a hash of the secret key for the credentials that is safe
to store,salt: a random value used to generate the credentials.Output:Algorithm:create and store an item based on the input.Retrieve CredentialsIf the credential with the id exists, return it. Otherwise, returnNone.Input:sub: unique identifier for the user andid: unique identifier for the credential.Output:id,public_key,salt.Algorithm:Use thesubpartition key andidsort key to check whether
an entry exists,if an entry exists, return thepublic_keyandsaltandreturnNone.Retrieve UserCheck that the public key exists and retrieve the user and salt for it.Input:public_key.Output:sub,saltandsecret_key_hash.Algorithm:check whether an entry exists using thepublic_keypartition key for thepublicKeyglobal secondary indexif it does not exist, returnNoneandretrieve and return thesub,saltandsecret_key_hash.Delete a Credential for a UserInput:subandid.Output:Algorithm:Delete all entries forsubandid.Delete All Credentials for a UserInput:sub.Output:Algorithm:Delete all entries forsub.Credentials Propertiessub: A string that is the partition key of the table.id: A string that is the sort key of the table.public_key: A string that is the partition key of thepublicKeyglobal secondary index.secret_key_hash: Bytes.salt: Bytes.testExecutes the tests defined attests.buildBuilds the database package defined at..release-requiredHas 2 outputs:result: whether a release to PyPI is required based on the latest released
version and the version configured in the project andproject-version: the version configured in the code base.deployDeploys the CloudFormation stack for the database defined at../infrastructure/lib/database-stack.ts.releaseIf theresultoutput fromrelease-requiredis true, the package is deployed
to both test and production PyPI.Irrespective of whether the release was executed, the version of the package
defined in the code base is installed from both test and production PyPI and
the tests defined at../test/database/testsare
executed against the deployed infrastructure on AWS.Periodic Production TestsThe workflow is defined here:../.github/workflows/production-test-database.yaml.Executes the tests defined at../test/database/testsagainst a configured version of the package and against the currently deployed
infrastructure on AWS.Pytest PluginA pytest plugin is made available to make testing easier. It is defined atopen_alchemy/package_database/pytest_plugin.py.It requires thedynaliteNPM package to be installed in the project usingnpm install --save-dev dynaliteto run a local dynamoDB instance athttp://localhost:8000. More information on the package is here:https://www.npmjs.com/package/dynalite.FixturesAll fixtures that have an effect but yieldNoneare prefixed with_so that
tools like pylint do not complain about unused arguments for test functions._databaseSpins up the database at the start of the tests and tars down the database at
the end. This fixture is unlikely to be useful as no tables are created. The
table specific fixtures depend on the_databasefixture so it is not
necessary to include this fixture in any tests._specs_tableCreates thepackage.specstable before all the tests and deletes it after all
tests complete._clean_specs_tableDeletes all items from thepackage.specstable before and after each test._credentials_tableCreates thepackage.credentialstable before all the tests and deletes it after
all tests complete._clean_credentials_tableDeletes all items from thepackage.credentialstable before and after each
test.InfrastructureThe CloudFormation stack is defined here:../infrastructure/lib/database-stack.ts.CI-CDThe workflow is defined here:../.github/workflows/ci-cd-database.yaml.There are a few groups of jobs in the CI-CD:test: runs the tests for the package in supported python versions,build: builds the database package,deploy: deploys database infrastructure to AWS,release-required: determines whether a release to PyPI is required andrelease: a combination of deploying to test and production PyPI and
executing tests on the published packagesProduction TestsThe tests against the deployed database facade are defined here:../test/database/.The workflow that periodically executes the tests is defined here:../.github/workflows/production-test-database.yaml.