Package: s3app
YOTRONis a consultancy company which is focused on DevOps, Cloudmanagement and
Data Management. Visit us onwww.yotron.de.S3AppS3 (Simple Storage Solution) is a object storage services to provide a web bases storages which works with simple http requests. It is known for its
scalability, data availability, security and performance. Introduced by Amazon in AWS, other
provider of S3 compatible object storages are available just like possibilities for self-hosted solutions.S3App simplifiesthe access to a S3Buckets with a provider independent web based frontend which allows
the visualizing and the management of the content of S3 buckets with an S3 provider independent web application, ....For further information and the manual, please seeYOTRON/s3appS3App URLsThe project contains code, build packages, container ... . Below an overview:TypeProviderURLS3App Manualyotron.dehttps://www.yotron.de/s3app/Containerhub.docker.comhttps://hub.docker.com/r/yotronpublic/s3appPython Package (PyPi)pypi.orghttps://pypi.org/project/s3app/HELM packageartifacthub.io / helm.yotron.dehttps://artifacthub.io/packages/helm/yotron-helm-charts/s3appCode/Contributiongithub.comhttps://github.com/yotron/s3app/Problems/Feedbackgithub.comhttps://github.com/yotron/s3app/issuesInstallationWe provide two installation methods. A native installation on any OS which supports Python3 and a installation in Kubernetes via HELM.
Per default, S3App runs on a sqlite database. Sqlite is recommended only for testing and on a standalone installation.In production, you should use a PostgreSQL database for a better data persistency and to allow a HA setup with more S3App nodes.The S3App has no TLS-termination ("https"). We recommend to use a Reverse Proxy like NGINX or Apache Web Server in front of S3App Web App for TLS termination.The HELM project of S3App for Kubernetes contains all needed and recommended components. You can optionally install a NGINX Reverse Proxy and a PostgreSQL database with the project,
but you can also use separated applications.NativePrerequisitesPython3Python-pipfor dependency installationOpenLDAP for LDAP authenticationSSL for secured LDAP authenticationrecommended: python3-venv to create a virtual environment for Python3recommended:PostgreSQLdatabase for the S3App metadatarecommended: A reverse proxy with TLS termination (e.g. NGINX).Prerequisites Python and pipS3App supports all current Python3 versions. It is tested with Python 3.9.x and 3.10.xTo check which Python version is running:vagrant@bullseye:~$ python3 -V
Python 3.9.2To check which Pip version is running:vagrant@bullseye:~$ pip3 -V
pip 20.3.4 from /home/vagrant/s3app/lib/python3.9/site-packages/pip (python 3.9)It is possible, that in your distribution Python must be called withpythonandpip.If you need a Python installation from the scratch. Python3, Pip3 and python3-venv are available in the main OS distribution.For Debian/Ubuntu:sudo apt-get install python3 python3-pip python3-venvFor RedHat/CentOS:sudo yum install python3 python3-pip python3-venvPrerequisites OpenLDAP and SSLFor authentication packages for authentication like OpenLDAP and SSL are neededFor Debian/Ubuntu:sudo apt-get install libsasl2-dev python-dev libldap2-dev libssl-devFor RedHat/CentOS:sudo yum install python-devel openldap-develInstallationThere are a lot of variants how to start a Python web project. We use a Python virtual environment for setting up S3App on a virtual environment.
Depend on your environment and knowledge in Python the process can be different.Create a virtual environment of PythonCreate a virtual environment for S3Appvagrant@bullseye:~$ python3 -m venv venvActivate your virtual environmentThis steps is to repeat every time you want to restart S3App.vagrant@bullseye:~$ source venv/bin/activate
(venv) vagrant@bullseye:~$Install all Python dependencies needed for the project:(s3app) vagrant@bullseye:~$ pip install s3app --upgradeStart S3App server(s3app) vagrant@bullseye:~$ s3app-run --host=0.0.0.0 --port=8080
2022-10-06 15:21:07,382:INFO:flask_appbuilder.base:Registering class S3View on menu 
2022-10-06 15:21:07,383:INFO:flask_appbuilder.baseviews:Registering route /s3/<string:prefixUrl> ('GET',)
2022-10-06 15:21:07,383:INFO:flask_appbuilder.baseviews:Registering route /s3/ ('GET',)
...
2022-10-10 11:18:22,384:INFO:flask_appbuilder.base:Registering class S3View on menu 
2022-10-10 11:18:22,385:WARNING:flask_appbuilder.base:View already exists S3View ignoring
2022-10-10 11:18:22,385:INFO:flask_appbuilder.base:Registering class S3IndexView on menu
2022-10-10 11:18:22,385:INFO:flask_appbuilder.baseviews:Registering route / ('GET',)
2022-10-10 11:18:22,424:INFO:waitress:Serving on http://0.0.0.0:8080Open Browser and start S3App with one of the IPs mentioned above:Your will automatically redirected tohttp://x.x.x.x:8080/loginLogin as a admin userThe default credentials for the admin user are:Username: adminPassword: adminPlease see themanualto allow access to buckets.HELMYou find the installation instructions for KuberneteshereManualThe manual for S3App you findhereConfigurationThe dotenv file andS3APP_CONF_FILEvariableYou can customize S3App with a setting defined in a dotenv file. The dotenv file can be chosen freely and you can put the file in a folder of your choice but
the file must be reachable by the S3App App.Additionally, you need a environment variableS3APP_CONF_FILEwith the path to your config file.For example, for Linux you can set the environment variable with:export S3APP_CONF_FILE=/etc/s3app/.s3appwith.s3appas the dotenv file name.The file contains setting to the following topics:customization of your S3AppAuthenticationThe parameter of the config fileS3APPnameexampledescriptionpossible valuesdefaultS3APP_APP_NAMEMy Fancy S3 AppThe name of your S3App. It is display in the head of your application.S3APP_APP_ICONhttps://myserver.com/my_own_logo.svgThe icon of your S3App as a html link. It is display in the head of your application.S3APP_LOG_LEVELinfoLogLevel for the Logging.fatal, error, warning, info, debuginfoS3APP_LOG_FILE/var/log/s3app.logWhen set logs are written to the mentioned file. If not set it is written to console (stdout).not setS3APP_DB_TYPEpostgresqlType of database to use. Can be a SQLite or a PostgreSQL Databasesqlite, postgressqliteS3APP_PG_DB_HOSTk8s-1.mydatabse.comOnly if db-type postgres: Host name of the PostresSQL server.S3APP_PG_DB_PORT5432Only if db-type postgres: Port number of the PostresSQL server.5432S3APP_PG_DB_NAMEs3appOnly if db-type postgres: Name of the PostgreSQL Database.S3APP_PG_DB_USER_NAMEs3appOnly if db-type postgres: Username to authenticate against the PostgreSQL DatabaseS3APP_PG_DB_USER_PWs3appOnly if db-type postgres: Password to authenticate against the PostgreSQL DatabaseS3APP_SECRET_KEYthisIsMyHiddenSecretKeyA key which used to sign session cookies for protection against cookie data tampering. In production please change it.Authentication generalnameexampledescriptionpossible valuesdefaultS3APP_AUTH_TYPEdatabaseParameter to define the authentication method. It can be an authentication via LDAP or via the default database with name and password.ldap, databasedatabaseAUTH_USER_REGISTRATIONParameter to define if a user can self registrate to S3App. With LDAP it must be set to True.True, FalseFalseAUTH_USER_REGISTRATION_ROLEDefault role a user, when registered or authenticated via LDAP firstly.S3User, AdminS3UserAUTH_ROLES_MAPPINGMapping a LDAP group onto a S3App roleS3Manager,S3UserorAdmin.S3APP_SESSION_LIFETIME1800Seconds of inactivity after which a user must re-login.600Example of a role mappingAUTH_ROLES_MAPPING = {
    "cn=s3user,ou=groups,dc=example,dc=com": ["S3Users"],
    "cn=s3appadmins,ou=groups,dc=example,dc=com": ["Admin"],
}Authentication LDAPTo allow authentication with LDAP, a LDAP group must be mapped onto one the S3App RoleS3Manager,S3UserorAdmin. After the first authentication a new user, the user must be added to theS3AccessorS3Group. Please see themanual of S3App.nameexampledescriptionpossible valuesdefaultAUTH_LDAP_SERVERldap://ldap.example.comThe URL of the LDAP server.AUTH_LDAP_USE_TLSFalseIf the LDAP server allows TLS secured communication set to True.True, FalseAUTH_LDAP_FIRSTNAME_FIELDgivenNameName of the field of the person LDAP entity with the given name.AUTH_LDAP_LASTNAME_FIELDsnName of the field of the person LDAP entity with the last name.AUTH_LDAP_EMAIL_FIELDemailName of the field of the person LDAP entity with the email address.AUTH_LDAP_USERNAME_FORMATuid=%s,ou=users,dc=example,dc=comDistinguished name of the user to authenticate.%swill be replaced by the username of the S3App login.AUTH_LDAP_APPEND_DOMAINexample.comWhen a username always has a domain appendix.AUTH_LDAP_SEARCHou=users,dc=example,dc=comLDAP search string if a user is part of an organizational unit.AUTH_LDAP_UID_FIELDuidWhen using a LDAP search the filed name with the username of the organizational unit.AUTH_LDAP_BIND_USERldapadminThe bind user used for authentication against LDAP.AUTH_LDAP_BIND_PASSWORDmyHiddenPasswordThe password of the bin user sed for authentication against LDAP.AUTH_LDAP_SEARCH_FILTER(memberOf=cn=myTeam,ou=teams,dc=example,dc=com)Filter the user which are allowed to access S3App generally.AUTH_LDAP_GROUP_FIELDmemberOfWhen using AUTH_ROLES_MAPPING the name of the field with the role DN.You find further information about how to configure LDAP against Microsoft AD or OpenLDAPhere.The start parameterTo start of the S3App server simply calls3app-runas shown above. You have has the following parameter:nameshortexampledescriptiondefaultHOST-s, --host127.0.0.1Listener Host IP. 0.0.0.0 for all IPs.0.0.0.0PORT-P, --port8090Listener Host Port8080THREADS-t --threads1Threads for parallelization4Withs3app-run -hyou get the possible configuration:(s3app) vagrant@bullseye:~$ s3app-run -h
...
usage: s3app-run [-h] [-s HOST] [-p PORT] [-t THREADS]
Start parameter for the S3App web server.
optional arguments:
-h, --help            show this help message and exit
-s HOST, --host HOST  Listener Host IP. Default: 0.0.0.0
-p PORT, --port PORT  Listener Host Port. Default: 8080.
-t THREADS, --threads THREADS  Threads for parallelization. Default: 4.