Package: resc-backend
Repository Scanner Backend (RESC-Backend)Table of contentsAbout the componentGetting startedPrerequisitesRun RESC Web service locally from sourceRun RESC Web service locally through MakeRun locally using dockerTestingRun unit tests, linting and import checks locallyRun Newman tests locallyRun OWASP ZAP API Security tests locallyCreate a migration for database changesUse Alembic to create a new migration scriptUse the --autogenerate parameterRunning migration and rollbackDocumentationAbout the componentThe RESC-backend component includes database models, RESC Web service, Alembic scripts for database migration, RabbitMQ users, and queue creation.Getting startedThese instructions will help you to get a copy of the project up and running on your local machine for development and testing purposes.PrerequisitesGitDocker DesktopPython (v3.9.0 or higher)Install odbc 17 sql server driver for your OSDownload for WindowsDownload for MacDownload for Linuxsudo apt install unixodbc-devRun RESC Web service locallyRun RESC Web service locally from sourcePreviewEnsure resc database is up and running locally.You can connect RESC web service to database, if you have already deployed RESC through helm in Kubernetes.Open the Git Bash terminal from /components/resc-backend folder and run below commands.Create virtual environment:pipinstallvirtualenv
virtualenvvenvsourcevenv/Scripts/activateInstall resc_backend package:pipinstallpyodbc==4.0.32
pipinstall-e.Set environment variables:sourcedb.envexportMSSQL_SCHEMA=masterexportMSSQL_DB_PORT=30880exportMSSQL_PASSWORD="<enter password for local database>"Run Web service:uvicornresc_backend.resc_web_service.api:app--workers1Openhttp://127.0.0.1:8000in a browser to access the API.Run RESC Web service locally through makeNote:This procedure has been only tested in Linux and Mac. It may not work in machines running the Apple M1 chip due to lack of support from MSSQL docker image.Prerequisites:InstallMakeon your system.Update MSSQL_PASSWORD (password you want to set for local database) in db.env file.PreviewCreate Python virtual environment and install resc_backend package:makeenvRun database locally:makedbThis target will run a local MSSQL instance in a container calledresc-db. It creates and populates the resc database schema using alembic and the sql script located intest_data/database_dummy_data.sqlNote:: This target will also try to remove the DB container if it already exists.If you want to remove this container, run:make cleandbRun Web service:makerwsOpenhttp://127.0.0.1:1234in a browser to access the API.Clean up:makecleanRun locally using dockerPreviewRun the RESC-Backend docker image locally with the following commands:Pull the docker image from registry:dockerpullrescabnamro/resc-backend:latestAlternatively, build the docker image locally by running following command:
Ensure resc database is up and running locally.You can connect RESC web service to database, if you have already deployed RESC through helm in Kubernetes.Open the Git Bash terminal from /components/resc-backend folder and run below commands.Update MSSQL_PASSWORD value in the docker run command.dockerbuild-trescabnamro/resc-backend:latest.Use the following command to run the RESC backend:sourcedb.env
dockerrun-p8000:8000-eDB_CONNECTION_STRING-eMSSQL_ODBC_DRIVER-eMSSQL_USERNAME-eRESC_REDIS_CACHE_ENABLE-eAUTHENTICATION_REQUIRED-eMSSQL_DB_HOST="host.docker.internal"-eMSSQL_PASSWORD="<enter password for local database>"-eMSSQL_SCHEMA="master"-eMSSQL_DB_PORT=30880--nameresc-backendrescabnamro/resc-backend:latestuvicornresc_backend.resc_web_service.api:app--workers1--host0.0.0.0--port8000Openhttp://127.0.0.1:8000in a browser to access the API.Testing(Back to top)Run unit tests, linting and import checks locally:See below commands for running various (unit/linting) tests locally. To run these tests you need to installtox. This can be done on Linux and Windows with Git Bash.Run below commands to make sure that the unit tests are running and that the code matches quality standards:pipinstalltox# install tox locallytox-v-esort# Run this command to validate the import sortingtox-v-elint# Run this command to lint the code according to this repository's standardtox-v-epytest# Run this command to run the unit teststox-v# Run this command to run all of the above testsRun Newman tests locally:If you don't provide any argument to the script, then the default image value will be usedcdtests/newman_tests
./run_newman_tests.shIf you can override the images by providing below arguments to the script.cdtests/newman_tests
./run_newman_tests.sh-b<resc-backendimage:tag>-d<resc-databaseimage:tag>-n<newmanimage:tag>Example:./run_newman_tests.sh-b'rescabnamro/resc-backend:latest'-d'mcr.microsoft.com/azure-sql-edge:1.0.7'-n'postman/newman:5.3.1-alpine'Run OWASP ZAP API Security tests locally:If you don't provide any argument to the script, then the default image value will be usedcdtests/zap_tests
./run_run_zap_api_tests.shIf you can override the images by providing below arguments to the script.cdtests/zap_tests
./run_run_zap_api_tests.sh-b<resc-backendimage:tag>-d<resc-databaseimage:tag>-z<zapimage:tag>

Example:./run_newman_tests.sh-b'rescabnamro/resc-backend:latest'-d'mcr.microsoft.com/azure-sql-edge:1.0.7'-n'owasp/zap2docker-weekly'Create a migration for database changes(Back to top)Use Alembic to create a new migration scriptPreviewThis command will create a new revision script in the ./alembic/versions directoryalembicrevision-m"<revision summary>"The filename is prefixed with the revision identifier used by Alembic to keep track of the revision history.
Make sure that the down_revision variable contains the identifier of the previous revision.
For instance:#d330d086edfe_first_revision.pyrevision='d330d086edfe'down_revision=None
...#e653f899efgh_second_revision.pyrevision='e653f899efgh'down_revision='d330d086edfe'The generated script contains two functions:The upgrade function that contains the revision changes.The downgrade function that revert these changes.Use the --autogenerate parameterPreviewAlembic provide an --autogenerate parameter to help revision scripts creation. It can output the necessary changes to apply,  by comparing the current database schema
and the model stated in Python. To create that revision make sure you have a connection to a running database with an up-to-date schema version.alembicrevision--autogenerate-m"<revision summary>"Note:Autogenerate cannot detect all the required changes.The created revision script must be carefully checked and tested.Running migration and rollbackPreviewTo upgrade/downgrade the database schema use the following:# Upgrade to specified revision identifieralembicupgrade<revision_identifier># Upgarde to latestalembicupgradehead# Upgrade to the next revisionalembicupgrade+1# Run next revision from a specific revisionalembicupgrade<revision_identifier>+1# Downgrade to base (no revision applied)alembicdowngradebase# Downgrade to the previous revisionalembicdowngrade-1Note:A list of needed changes and a table containing alembic revision history are created during the first revision.You can also check current revision information:alembiccurrentAnd the revision history:alembichistory--verboseDocumentation(Back to top)Alembic documentationConnect to database using Azure Data StudioHow to create a connection stringConnect and view the database tables from Intellij IDEA Ultimate edition