Unstable Slope Management Program (USMP)

System Architecture Document

(Version 1.2)

Mike P. Wittie

Montana State University

mwittie@cs.montana.edu
Table of Contents

1. Introduction

2. Architectural Overview

2.1. Amazon Web Services (AWS) deployment

2.2. Resources Used

3. Database

3.1. Slope Information Form Data

3.2. Maintenance Form Data

3.3. New Slope Event Data

3.4. User Account Data

4. Web Server

4.1. Data Privacy

4.2. Data Security

5. Webpage

5.1. Data Quality Assurance

5.2. Data search and export

5.3. Links to other databases

6. Mobile App

7. Support Ticket System (Helprace)
1. Introduction

This document details the architecture and implementation of the USMP system. We describe

the individual system components, including the database, server backend, website, and mobile

app.

2. Architectural Overview

The architecture of the USMP systems contains a LAMP stack (a Linux server, Apache web

server, MySQL database, and PHP programming language for server-side scripts), a mobile

app built on React Native, and a webpage built on JavaScript/jQuery with a dependency on

Google Maps. The website also connects with Helprace to support customer feedback and

support tickets. The connectivity between the different components is shown below.

2.1. Amazon Web Services (AWS) deployment

Currently, the system is deployed on AWS. The web server functionality is divided between a

production instance and a development instance. The production server servers pages on

usmp.info, while the development server instances on testing.usmp.info. The DNS entries for

the system use godaddy.com DNS, where the usmp.info domain is registered. The database

runs on a separate, actively backed up, AWS RDS instance. There are two databases: ‘usmp’

supports usmp.info, while ‘usmp_mwittie’ supports the testing instance.

2.2. Resources Used

As of June 11th, 2019 the system uses the following resources:

● AWS EC2 compute nodes

○ two t2.micro instances

● AWS RDS database
○ one db.t2.micro instance with 7MB of 20GB used

● AWS S3 document storage

○ images 5822 Objects - 5.9 GB

○ documents 4 Objects - 1.8 MB

The AWS hosting hosts have not yet exceeded $40 in a given month, but could do so if the

traffic to the site increases as the system gains popularity.

Additional required system resources are:

● iOS development license - $99 per year

● Android development license - $15 per year

● The system uses the free tiers/features from Google Maps and Helprace

3. Database

USMP relies on a MySQL relational database for storage of structured data recorded through

the website and mobile app forms.

3.1. Slope Information Form Data

Slope information data is stored in SITE_INFORMATION table and several associated tables,

as shown below.
There are several important constraints not represented in the data structure:

● A site (rated slope) has a unique SITE_INFORMATION.SITE_ID. Each edit to the site

shares the same SITE_INFORMATION.SITE_ID, but has a unique

SITE_INFORMATION.ID.

● The associated tables have foreign keys to individual edits onto

SITE_INFORMATION.ID

● Each site has one or more HAZARD_TYPE values. The HAZARD_LINK table reflects

this many to many relationship through its foreign keys.
● Each slope is either a Rockfall, or a Landslide. There is not field representing this in

SITE_INFORMATION. Instead a slope will have either and associated row in

ROCKFALL_PRELIMINARY_RATING, or LANDSLIDE_PRELIMINARY_RATING.

● Uploaded images and documents are stored on disk, with file system paths to the

images stored in the PHOTOS and DOCUMENTS tables.

3.2. Maintenance Form Data

Maintenance Form data is stored in the MAINTENANCE_FORM table shown below.

3.3. New Slope Event Data

Maintenance Form data is stored in the SLOPE_EVENT table shown below. Photos for Slope

Events are also stored in the PHOTOS table with SI_ID as the foreign key.
3.4. User Account Data

User account data is stored in the USERS table shown below.

A user’s EMAIL serves as their login. We store the hash of the password in

USERS.PASSWORD, but not the password itself. USERS.PERMISSIONS stores the

administrative domains, within which a user may update slopes. For example, a user whose
PERMISSIONS field contains ‘NPS’, may create and update forms for slopes located in areas

administered by the National Park Service agency. The permissions may be more fine-grained.

For example, a user may be restricted to the Rocky Mountain Region of the Forest Service with

the PERMISSIONS set to ‘FSRockyMountainRegion’, or a single park, with the PERMISSIONS

set to ‘NPSIMRCHIRICAHUA’. Additional options include restrictions to States, Territories, and

Counties.

The PERMISSIONS field may contain a combination of areas for users permitted to operate

across them. This is particularly useful for users working in geographically adjacent

administrative domains. The more general permissions, for example at the Agency level, take

precedence over more restrictive areas, such as regions. And so, a user with permissions to

operate in NPS, will not be restricted further if their PERMISSIONS field also includes the ‘IMR’

region.

Finally, the PERMISSIONS field may contain the keyword ‘root’ to give a user administrative

privileges.

Users with administrative permissions may change the permissions of other users through the

tab of the website.

To add a new user, the administrator enters their username and password and selects the

permissions for the user in the field. The ‘Permissions’ field is a multi-select

field, where multiple permissions may be select with a control-click. To finish adding a user, the

administrator selects the button.
To change the permissions of an existing user, the administrator selects the

link on the tab. After scrolling down to find the right user, the administrator

presses the button, to load the user’s permissions into the ‘Permissions’ field. The

administrator can select and unselect permissions using the control-click. To record the

selection, the administrator presses the button (which replaces the ‘Add

User’ button) below the ‘Permissions’ field.
Finally, the administrator may delete a user by pressing the ‘Show users’ link, then scrolling

down to the right users, and ultimately pressing the link next to their name.

4. Web Server

The code on the web server acts as a REST API for the webpage and the mobile app to make

changes to the shared database state. The code is organized as follows:
These PHP scripts support API calls from the website and the mobile or provide supporting

functions for these calls.

● Scripts in account_management_php support user management functions on

http://usmp.info/client/account_management.php
● Scripts in authentication support user login and logout functions on

http://usmp.info/client/login.php

● Scripts in new_site_php enable the creation of new slopes on

http://usmp.info/client/new_site.php. Scripts in edit_site_php, on the other hand, support

the loading of information into the form at http://usmp.info/client/edit_site.php. These

scripts also support fetching of data from tables associated with the

SITE_INFORMATION table.

● Scripts in new_slope_event_php enable the creation of new slope events on

http://usmp.info/client/new_observer_form.php. Scripts in edit_slope_event_php, on the

other hand, support the loading of information into the form at

http://usmp.info/client/edit_slope_event.php.

● Scripts in maintenance enable the creation of new maintenance forms on

http://usmp.info/client/new_maintenance_form.php. Scripts in

edit_maintenance_form_php, on the other hand, support the loading of information into

the form at http://usmp.info/client/edit_maintenance_form.php.

● Scripts in map_php fetch sites to put pins on http://usmp.info/client/map.php and provide

information to select the correct icon color.

● Scripts in search_php support map search functionality, but running queries based on

criteria specified on the website, and with export functionality, by creating an export

excel file.

● Scripts in shared support functions for deleting sites and photos as well as obtaining

dropdown options for search functions. These scripts are experimental at this point.

● Scripts in site_photos_php support fetching photos from the database and the file

system.

● constants.php holds website configuration, such as database paths and passwords.

● Image_upload_test.php is a demo program showing how to downscale images before

uploading them to the server.

● The directory contains all the photos for rates slopes linked to

from the PHOTOS and DOCUMENTS tables.

4.1. Data Privacy

Personally identifying information (PII) in this system includes email addresses. These are

associated with user login and are accessible to any system administrator with root level

permissions. Additionally, the new slope event form collects observer name and email

information. These data are available to any authorized user of the system.

4.2. Data Security

Login and session information is protected by the transport security layer (TLS) and asymmetric

key encryption based on certificates configured and installed by https://letsencrypt.org/. Access

to the database is controlled by AWS security groups firewall, which restricts raw access to the

database only to developer machines and the web servers placed in specific IP subnets.
5. Webpage

The client code controls the functionality of the website.

The directories support the following functionalities:

● The CSS directory contains cascading style sheet files the control aspects of the website

such as fonts, colors, and cell spacings

● The files directory contains files that are linked to from the website, including manuals

and videos.

● Images contains all the image artworks for the website, including organization logos,

icons, etc.

● The bootstrap and jquery-ui directories are necessary imports for JavaScript calls in the

browser.

The js directory contains most of the scripts that control the front end functionality. The directory

is organized as follows:

The directories mirror the functionality implemented on the server. Scripts in each directory

issue POST calls to the server .php files and then process the responses. There are a few

important points to note:

● The js/localforage directory was used for offline storage on the website in its early

version. This code is now disabled.

● The QRA folder implements the QRA module functionality completely on the client side.

● The shared directory has a number of scripts that supports functionality across the

different forms. For example:

○ agency_info.js and get_dropdown_options.js fetch a tree of agency affiliations for

drop-down menus

○ equations.js implements calculations in form fields

○ field_regex.js (along with field_regex.file in individual folders) supports checking

field formats and providing messages on how to correct them

○ ImageTools.js supports scaling down images before uploading to the server

○ include* scripts enable inclusion of HTML forms, for example, ,

in PHP scripts that show the different forms

○ sha1.js is used to rename files before upload

○ validate* scripts enable input error checking on server side before upload

Finally, the website functionality is offered through a number of pages in the client/ directory:

The *.php files correspond to pages viewable by users. The *.html files are the markup text for

display and they are included by relevant *.js files. For example, form_table.html is included by

both client/new_site.php and client/edit_site.php to display the slope rating form. The *.html files
also call JavaScript/jQuery functions to trigger calculations and error checking whenever the

input into the form changes.

5.1. Data Quality Assurance

The system implements data quality control (QC) functions during data entry. Each form field

accepts data in a particular format that reflects the semantic of a form. The individual field

formats are described as regular expressions in field_regex.js files. These files also contain

error messages displayed to the user if their input does not match the prescribed format.

The website first checks field formats dynamically during data entry whenever a user switches to

a different field and displays a browser alert with the appropriate format error message. Field

formats are also checked whenever data is submitted. This second mechanism will show errors

for required fields that have been left blank. The list of required fields is included in

validate_(form_name).js files.

5.2. Data search and export

The website supports exporting of form data as comma separated value (CSV) files. To access

the search functionality users click on the form-specific dropdown on the map page.

They will be given a series of dropdowns to form a conjunctive query based on form fields. In

the example above, a user may search forms that match a particular form ID, which is one of the

fields in the slope rating form. Users may also select other fields and ranges for values, for

example, the Preliminary Rating between 100 and 200.

To export the data a user clicks the ‘Search on Map’ button, which will submit the search to the

server/search_map.php script, which will transform the criteria into a SQL query. The results

of the query will contain the map view to the sites that match the criteria. A user may then click

the ‘Export as CSV’ button, which will send a request to the server/export_search.php script
which will generate and download the file containing data for the sites visible in the map

window.

5.3. Links to other databases

The slope rating form also supports links to with other database systems.

The above fields allow users to enter an alternate database name, form id, and description.

These data may be used as keys to entries in other systems that describe the same slope and

are visible in CSV files exported through the map search and reporting functionality.

6. Mobile App

The USMP mobile application is implemented using the React Native platform, which compiles

JavaScript code to create native Android and iOS applications. The code is organized as

follows:
● The android and ios directories contain the compiled code for deployment through the

Google Play and iOS App stores.

● The src directory contains the React Native implementation of the app

○ The components directory contains visual input elements used in forms for

accepting input.

○ The logos directory contains agency logos used in the splash page of the app

○ The styles folder defines visual aspects of the application

○ The USMP directory contains the bulk of the app’s functionality. The individual .js

files correspond to the different app views (pages) For example, NewSite.js,

contains both the form layout as well as calculations of the different fields.

EditSite.js enables editing of a site that has already been created, whose data is

pulled from the database by site id. Offsite.js allows loading of a site from for

editing from local storage.

Local storage is implemented using a Realm database, specified in realm.js. New

forms stored in this database are submitted to the server using uploadCode.js.

Finally, userLogin.js/userLogoug.js will implement login permissions for users.

This functionality has been disabled for the current field trials.

● The index.android.js and index.ios.js are configuration files needed to generate iOS and

Android app code for deployment.
7. Support Ticket System (Helprace)

To facilitate the interaction between USMP users and the development team, the website

integrates with Helprace.com. Every page on the website has the link in

the bottom right corner. Clicking on the link brings up the following menu:

To submit a question, suggestion, or an issue, the user can fill out the Subject and Message

fields. Submissions may be made anonymously, or if the user expects a response, with the

Email field filled out.

Helprace will notify the development via email. Our goal is to reply to all inquiries within 48

hours. If the inquiry requires changes to the system implementation, we will notify the user that

their issue has been included in our internal tracking system. We will then notify them when we

have a time estimate on a resolution once the issue makes it into our next development sprint.

Finally, we will notify the user when the solution to their issue has been implemented and what

steps they need to take, if any, to use the newest version of our software.