Package: tools-barebone
tools-barebonetools-bareboneis a framework to develop and deploy small web applications,
implemented in Python using Flask Jinja2 templates.It can be used as a starting point to develop new tools for theMaterials Cloud Tools section.It provides:A common layout used for every tool in Materials cloudThe Materials Cloud themeCommon API endpointsCommon widgets, e.g., file upload functionality for crystal structuresWeb server settingsScripts to deploy the tool in a docker containerPrerequisitesdocker>= v18.09How to use a tools-barebone frameworkThetools-bareboneframework provides basic templates, that can be extended to develop
a new tool for Materials Cloud.Here we briefly explain how thetools-barebonetemplate (shown
on the left side of the figure below) can be extended to develop a new tool calledcustom-tool(shown on the right side).Tools barebone templateNew tool template1. Get the most recenttools-barebonedocker image from DockerHubBrowse to theTags page on DockerHub for materialscloud/tools-bareboneand find the most recent tagged version (in the formX.Y.Z, e.g. 1.0.0). You can also use thelatesttag, but we strongly suggest that you use a pinned version, rather than latest: this will ensure that your tool will continue to work also if future, incompatible versions are released.Then, run the following command to get the image:docker pull materialscloud/tools-bareboneYou can try to see if everything works by running a container from this image, e.g. by runningdocker run -p 8090:80 materialscloud/tools-barebone:latestand then connecting tohttp://localhost:8090with your browser.You should see a website similar to the image on the left above.If you want, you can also install the tools-barebone package (use the same version as the one you picked above) usingpip install tools-barebone==X.Y.ZThis is not technically required, but it is useful e.g. if your editor has inspection capabilities and needs to see the package, or if you are using a linter.2. New tool:custom-toolTo write the new tool calledcustom-tool, create a new directorycustom-tool. Let us first create the minimum set of files required
in this new tool:mkdir custom-tool
cd custom-tool

# create configuration file
touch config.yaml

# create Dockerfile file
touch Dockerfile

# create file for python requirements
touch requirements.txt

# create the folder in which you will put the
# python code for the backend
mkdir compute
touch compute/__init__.py

# create user templates folder
mkdir user_templates
cd user_templates
touch ack.html                              # add acknowledgement text here
touch about.html                            # add information about this tool here
touch how_to_cite.html                      # add tool citation here
touch additional_content.html               # additional functionality if any, otherwise empty file3. Configuration settingsTheconfig.yamlfile contains the configuration details used in this new tool like window title,
page title, list of html templates, etc. Update theconfig.yamlfile and add HTML templates
that will be shown in the section about the tool, for tool citation text and for the acknowledgements.As an example of the most common variables to be set in theconfig.yamlfile, we provide here an example
here below:window_title: "Materials Cloud Tools: an example app"
page_title: "A simple tool example"

about_section_title: "About this new tool"

# If True, a structure selection block will be shown and it will provide a common set of parsers.
# In this case, you will have to provide an endpoint
# `/compute/process_structure/` to process the results
# as shown later.
use_upload_structure_block: True

# If you have an upload block and want to have some parsers first, you can specify their internal
# name as a list. Those from the list will be shown first (NOTE! if the name is unknown, it is ignored).
# All the remaining ones, if any, are shown afterwards in a default order.
upload_structure_block_order: ['cif-pymatgen', 'xsf-ase']

templates:
  how_to_cite: "how_to_cite.html"
  about: "about.html"
  select_content: "visualizer_select_example.html" # what to show in the selection page (below the upload structure block, if present)
  upload_structure_additional_content: "upload_structure_additional_content.html" # if the upload structure block is present, you can add additional content right above the 'submit' button, if you want (e.g. a disclaimer, terms of use, ...)

# Add here more sections to the accordion shown on the top of the selection page
additional_accordion_entries:
#  - header: "What is this?"
#    template_page: what_is.html
  - header: "Acknowledgements"
    template_page: ack.html4. Create the DockerfileOnce the files are ready, we can write aDockerfilethat extends thetools-bareboneimage (with the tag you have chosen earlier),
to build and run the docker container forcustom-tool.The snippet below shows a minimalDockerfilefile that achieves this goal. You can create such a file insidecustom-tool/Dockerfile.
The commands that you need and that are specific tocustom-toolcan be added at the bottom of the file.
Remember to replace theLABELstring.FROM materialscloud/tools-barebone:X.Y.Z

LABEL maintainer="Developer Name <developer.email@xxx.yy>"

# Python requirements
COPY ./requirements.txt /home/app/code/requirements.txt
# Run this as sudo to replace the version of pip

RUN pip3 install -U 'pip>=10' setuptools wheel
# install packages as normal user (app, provided by passenger)

USER app
WORKDIR /home/app/code
# Install pinned versions of packages
RUN pip3 install --user -r requirements.txt
# Go back to root.
# Also, it should remain as user root for startup
USER root

# Copy various files: configuration, user templates, the actual python code, ...
COPY ./config.yaml /home/app/code/webservice/static/config.yaml
COPY ./user_templates/ /home/app/code/webservice/templates/user_templates/
COPY ./compute/ /home/app/code/webservice/compute/

# If you put any static file (CSS, JS, images),
#create this folder and put them here
# COPY ./user_static/ /home/app/code/webservice/user_static/

###
# Copy any additional files needed into /home/app/code/webservice/
###

# Set proper permissions on files just copied
RUN chown -R app:app /home/app/code/webservice/5. Test it!You can now build the Docker image, and then launch the container as follows.First, go in the top folder of your tool, where theDockerfilesits. Then, run this command:docker build -t custom-tools . && docker run -p 8091:80 --rm --name=custom-tools-instance custom-toolsYou can now connect tohttp://localhost:8091and check if your results starts to look like the right panel of the images above, and it contains the text that you were expecting.6. Fine tune the textBefore looking into the backend python logic, you can now fine-tune the templates that you have written before. Change theconfig.yamland the various templates. Then, run again the docker build+run commands of the previous sections, and refresh your browser until you are happy with the results.7. Backend implementationNow it is time to work on the python backend implementation.tools-bareboneuses theFlaskframework, so you might want to look into its documentation to discover all advanced features. Here we describe only how to make a minimal working tool.You will put the code in thecomputefolder you created before. You can add any number of python files in it, and load them usingfrom compute.XXX import YYY(thecomputefolder will be in the python path).You will need however to have some minimal content in thecompute/__init__.pyfile.In particular, you will need at least to define ablueprintas follows:importflaskblueprint=flask.Blueprint("compute",__name__,url_prefix="/compute")You can then add your views.
If you are using the structure upload block (see comments in the description of theconfig.yamlsection), you will need to define at least a/compute/process_structure/endpoint. Here is a minimal working example, that you can use as a starting point. Note that here we are going to use the parsing functionality provided directly by thetools-barebonepackage.fromtools_barebone.structure_importersimportget_structure_tuple,UnknownFormatErrorimportio@blueprint.route("/process_structure/",methods=["POST"])defprocess_structure():"""Example view to process a crystal structure."""# check if the post request has the file part, otherwise redirect to first pageif"structurefile"notinflask.request.files:# This will redirect the user to the selection page, that is called `input_data` in tools-barebonereturnflask.redirect(flask.url_for("input_data"))# Get structure, file format, file content, and form data (needed for additional information, e.g. cell in the case of a XYZ file)structurefile=flask.request.files["structurefile"]fileformat=flask.request.form.get("fileformat","unknown")filecontent=structurefile.read().decode("utf-8")fileobject=io.StringIO(str(filecontent))form_data=dict(flask.request.form)# Usetry:structure_tuple=get_structure_tuple(fileobject,fileformat,extra_data=form_data)exceptUnknownFormatError:# You can use the flask.flash functionality to send a message# back to the structure selection page; this# will be shown in a red box on the topflask.flash("Unknown format '{}'".format(fileformat))returnflask.redirect(flask.url_for("input_data"))exceptException:# Let's deal properly with any exception, to avoid to get a 500 error.# Feel free to do better error management here,# or to pass additional information via flask.flashflask.flash("I tried my best, but I wasn't able to load your ""file in format '{}'...".format(fileformat))returnflask.redirect(flask.url_for("input_data"))# If we are here, the file was retrieved.# It will contain a tuple of length three, with:# - the 3x3 unit cell (in angstrom)# - a Nx3 list of atomic coordinates (in fractional coordinates)# - a list of integer atomic numbers of length N# As an example, we just create a string representation of the JSON# and send it back to the user, to be rendered in a formimportjsondata_for_template={"structure_json":json.dumps({"cell":structure_tuple[0],"atoms":structure_tuple[1],"numbers":structure_tuple[2],},indent=2,sort_keys=True,)}returnflask.render_template("user_templates/custom-tool.html",**data_for_template)In order to make it work, the last step is to create auser_templates/custom-tool.htmlfile, e.g. with the following minimal content:<!DOCTYPE html><html><head><metacharset="utf-8"><metahttp-equiv="X-UA-Compatible"content="IE=edge"><metaname="viewport"content="width=device-width, initial-scale=1"><!-- Add CSS, JS, ... here, e.g, these from tools-barebone;  --><linkhref="../../static/css/bootstrap.min.css"rel="stylesheet"><linkrel="stylesheet"type="text/css"href="../../static/css/jquery-ui.1.12.1.min.css"/><linkrel="stylesheet"type="text/css"href="../../static/css/visualizer_base.min.css"/><linkrel="stylesheet"type="text/css"href="../../static/css/visualizer_input.min.css"/><scriptsrc="../../static/js/jquery-3.1.0.min.js"></script><scriptsrc="../../static/js/jquery-ui.1.12.1.min.js"></script><!-- If you add things in a user_static folder, you will be able to access it via ../../user_static/xxx --><title>Custom-tools example return page</title><!-- Keep this, it's needed to make the tool embeddable in an iframe; it's provided by tools-barebone --><scriptsrc="../../static/js/iframeResizer.contentWindow.min.js"></script></head><body><divid='container'><divid='maintitle'><h1style="text-align: center;">Tools-example return page</h1></div><h2>Successfully parsed structure tuple</h2><p><codeid='structureJson'>{{structure_json}}</code></p></div><!-- Important: leave this tag as the *very last* in your page, just before the end of the body --><!-- It is needed to properly detect the size of the iframe --><divstyle="position: relative"data-iframe-height></div></body></html>You can now build and run again the container, and you should see the parsed results, in JSON form, in the page once you upload a structure.8. Additional viewsYou can now continue adding views to your application, inside the blueprint. Check the Flask documentation for more information. Here, we just show an example to create a view for some Terms of use.First create auser_viewsfolder in the top folder of your application, and create a filetermsofuse.htmlinside it. Complete it with the full HTML code that you want to send to the user. Rembember also to add the correctCOPYline to theDockerfile.Then, create the Flask view for it, in thecompute/__init__.pyfile:importosVIEW_FOLDER=os.path.join(os.path.dirname(os.path.realpath(__file__)),os.pardir,"user_views")@blueprint.route("/termsofuse/")deftermsofuse():"""View for the terms of use"""returnflask.send_from_directory(VIEW_FOLDER,"termsofuse.html")The page will be accessible under the url/compute/termsofuse/.Finally, if e.g. you want to show a link to it in the Structure Upload block, right before the Submit button, you can add the following line in thetemplatesdictionary:templates:# ...upload_structure_additional_content:"upload_structure_additional_content.html"and create a fileupload_structure_additional_content.htmlin theuser_templatesfolder, e.g. with the following content:<divclass='row'style="text-align:center"><pclass='small'>By continuing, you agree with the<ahref="../compute/termsofuse/"target="_blank">terms of use</a>of this service.</p></div>Some examplesAn example based ontools-barebone, with additional Python backend functionality, is provided in thetools-exampletool.For a more advanced tool, you can also check out thetools-seekpathtool or thetools-phonon-dispersion, for instance.Here you can see also an example of how the python code in the backend is implemented (check the implementation of the API endpoints inside thecomputesubfolder).
You can also get inspiration for the setup of tests with pytest, of continuous integration with GitHub actions, on how to setup pre-commit hooks, etc.