Deploying Static Sites with statit
Tags: http, guide
Published: 2023-07-13 12:16:36

---
title: Deploying Static Sites with statit
published: true
date: 2023-07-12 23:40:00 UTC
tags: HTTP,Guide
canonical_url: https://kevincox.ca/2023/07/12/statit-static-site-deploy/
---

<main style="--width:min(36em,100%);display:grid;grid-template-columns:1fr var(--width) 1fr;overflow:auto;padding:0;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">In my earlier post <a href="https://kevincox.ca/2021/08/24/atomic-deploys/" style="box-sizing:border-box;margin:0;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray">You Don’t Want Atomic Deploys</a> I explained why atomic deployments are a lie because the web is a fundamentally distributed system. <strong style="box-sizing:border-box;margin:0">You should read that first</strong> as this article uses the terminology explained there.</p>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">In that post I only gave a high-level overview of how to solve the problem. I have since created my own static site deployment tool that is now powering most of my sites (including this one). I’m going to give a quick overview of the tool and how it solves the problems.</p>
<aside style="background:#333;box-shadow:4px 4px 8px #000;box-sizing:border-box;color:#fff;grid-column:2;margin:.2em 1em .6em"><p style="box-sizing:border-box;margin:.4em"><strong style="box-sizing:border-box;margin:0">Note</strong>: This tool was made by me and is mostly focused on deploying my sites following my conventions. However I did attempt to put most of my conventions behind flags to make it customizable. That being said the user-base is still very small so it may be missing customization options that would be needed to make it work for your site. For example <a href="https://aws.amazon.com/s3/" style="box-sizing:border-box;margin:0;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray">S3</a> is the only target currently supported. Patches are definitely welcome for most customization options, please open a feature request to discuss.</p></aside><p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">I was previously deploying my sites to <a href="https://ipfs.tech/" style="box-sizing:border-box;margin:0;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray">IPFS</a>. I liked this because people could easily archive them and access them over IPFS even if my host was down. In order to make the site available over HTTP at my domain I was using Cloudflare’s CNAME gateway. This was really cool, you could just CNAME <code style="background:#000;border:1px solid #555;border-radius:.3em;box-sizing:border-box;color:#fff;margin:0;padding:0 .2em;white-space:pre-wrap">kevincox.ca</code> to their domain and they would automatically serve <code style="background:#000;border:1px solid #555;border-radius:.3em;box-sizing:border-box;color:#fff;margin:0;padding:0 .2em;white-space:pre-wrap">/ipns/kevincox.ca</code>. This worked great until one day <a href="https://discuss.ipfs.tech/t/cname-cross-user-banned-at-cloudflare-ipfs-com/5884" style="box-sizing:border-box;margin:0;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray">they broke it with no explanation</a>, so I needed to find another solution.</p>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">The solution I ended up switching to was uploading to <a href="https://aws.amazon.com/s3/" style="box-sizing:border-box;margin:0;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray">S3</a> and using <a href="https://aws.amazon.com/cloudfront/" style="box-sizing:border-box;margin:0;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray">CloudFront</a> as a CDN. This provides good performance and cheap pay-as-you-go pricing. I tend to spend just a couple of cents a month on S3 requests. It is quite possible that if my site gets more popular the costs may actually go down as I will be more likely to get more cache hits in the CDN layer.</p>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">There are many tools for uploading files to S3 but surprisingly few that support setting headers in a flexible way and nearly none that support cleanup of old files in a way that suits websites. So I made my own, <a href="https://gitlab.com/kevincox/statit#easy-static-site-deployment" style="box-sizing:border-box;margin:0;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray">statit</a>. It is now in use by a big handful of my own sites, including this one!</p>
<h2 id="usage" style="box-sizing:border-box;grid-column:2;margin:1.5em 1rem .8em"><a href="https://kevincox.ca/2023/07/12/statit-static-site-deploy/#usage" style="box-sizing:border-box;color:inherit!important;margin:0;text-decoration:inherit;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray;text-shadow:inherit!important">Usage</a></h2>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">Basic usage is quite simple:</p>
<div style="background:#222;border-bottom:1px solid #444;border-top:1px solid #444;box-sizing:border-box;display:grid;grid-column:2;grid-column:1/4;grid-template-columns:1fr var(--width) 1fr;margin:0 0 .8em;overflow-x:auto;padding:0"><pre style="box-sizing:border-box;grid-column:2;margin:.4em 1rem"><code style="background:none!important;box-sizing:border-box;color:#fff;margin:0;white-space:pre-wrap;white-space:pre">statit <span style="box-sizing:border-box;color:#aaf;margin:0;padding:0">--bucket</span><span style="box-sizing:border-box;color:#f5a3a3;font-weight:700;margin:0;padding:0">=</span>my-bucket deploy www/ <span style="box-sizing:border-box;color:#aaf;margin:0;padding:0">--dry-run</span>
</code></pre></div>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">Most of the defaults should be suitable for most sites. The most important thing is to configure your <a href="https://kevincox.ca/2021/08/24/atomic-deploys/#how-to-deploy-static-sites" style="box-sizing:border-box;margin:0;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray">assets and entries</a>. This is done with the <code style="background:#000;border:1px solid #555;border-radius:.3em;box-sizing:border-box;color:#fff;margin:0;padding:0 .2em;white-space:pre-wrap"><span style="box-sizing:border-box;color:#aaf;margin:0;padding:0">--assets</span></code> flag. Simply declare the paths that should be considered assets and everything else will be considered an entry. The default is <code style="background:#000;border:1px solid #555;border-radius:.3em;box-sizing:border-box;color:#fff;margin:0;padding:0 .2em;white-space:pre-wrap">a/<span style="box-sizing:border-box;color:#8585e0;font-weight:700;margin:0;padding:0">**</span></code> (everything inside the <code style="background:#000;border:1px solid #555;border-radius:.3em;box-sizing:border-box;color:#fff;margin:0;padding:0 .2em;white-space:pre-wrap">a/</code> directory). But if you keep your assets in different locations you can pass flags such as <code style="background:#000;border:1px solid #555;border-radius:.3em;box-sizing:border-box;color:#fff;margin:0;padding:0 .2em;white-space:pre-wrap"><span style="box-sizing:border-box;color:#aaf;margin:0;padding:0">--assets</span> <span style="box-sizing:border-box;color:#b3d733;margin:0;padding:0">'scripts/**'</span> <span style="box-sizing:border-box;color:#aaf;margin:0;padding:0">--assets</span> <span style="box-sizing:border-box;color:#b3d733;margin:0;padding:0">'styles/**'</span></code>.</p>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">statit will automatically guess an appropriate content-type based on the file extension, however if this is not suitable you can override it for specific files with the <code style="background:#000;border:1px solid #555;border-radius:.3em;box-sizing:border-box;color:#fff;margin:0;padding:0 .2em;white-space:pre-wrap"><span style="box-sizing:border-box;color:#aaf;margin:0;padding:0">--content-type</span></code> option. For example <code style="background:#000;border:1px solid #555;border-radius:.3em;box-sizing:border-box;color:#fff;margin:0;padding:0 .2em;white-space:pre-wrap"><span style="box-sizing:border-box;color:#aaf;margin:0;padding:0">--content-type</span> <span style="box-sizing:border-box;color:#b3d733;margin:0;padding:0">'pgp.key'</span> <span style="box-sizing:border-box;color:#b3d733;margin:0;padding:0">'application/pgp-keys'</span></code> can be used to see the proper headers for PGP keys. The first argument is a glob pattern just like the <code style="background:#000;border:1px solid #555;border-radius:.3em;box-sizing:border-box;color:#fff;margin:0;padding:0 .2em;white-space:pre-wrap"><span style="box-sizing:border-box;color:#aaf;margin:0;padding:0">--assets</span></code> flag.</p>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">In <code style="background:#000;border:1px solid #555;border-radius:.3em;box-sizing:border-box;color:#fff;margin:0;padding:0 .2em;white-space:pre-wrap"><span style="box-sizing:border-box;color:#aaf;margin:0;padding:0">--dry-run</span></code> mode <code style="background:#000;border:1px solid #555;border-radius:.3em;box-sizing:border-box;color:#fff;margin:0;padding:0 .2em;white-space:pre-wrap">statit</code> will print a list of the modifications it would make. If it looks right remove the <code style="background:#000;border:1px solid #555;border-radius:.3em;box-sizing:border-box;color:#fff;margin:0;padding:0 .2em;white-space:pre-wrap"><span style="box-sizing:border-box;color:#aaf;margin:0;padding:0">--dry-run</span></code> to actually deploy the site.</p>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">For a complete list of options see statit’s <a href="https://gitlab.com/kevincox/statit/-/blob/master/README.md" style="box-sizing:border-box;margin:0;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray">README</a> and <a href="https://gitlab.com/kevincox/statit/-/blob/master/src/args.rs" style="box-sizing:border-box;margin:0;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray"><code style="background:#000;border:1px solid #555;border-radius:.3em;box-sizing:border-box;color:#fff;margin:0;padding:0 .2em;white-space:pre-wrap">statit <span style="box-sizing:border-box;color:#aaf;margin:0;padding:0">--help</span></code></a></p>
<h2 id="method-of-operation" style="box-sizing:border-box;grid-column:2;margin:1.5em 1rem .8em"><a href="https://kevincox.ca/2023/07/12/statit-static-site-deploy/#method-of-operation" style="box-sizing:border-box;color:inherit!important;margin:0;text-decoration:inherit;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray;text-shadow:inherit!important">Method of Operation</a></h2>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">What statit does is fairly simple, consisting of a few main steps.</p>
<h3 id="read-existing-state" style="box-sizing:border-box;grid-column:2;margin:.4em 1rem .8em"><a href="https://kevincox.ca/2023/07/12/statit-static-site-deploy/#read-existing-state" style="box-sizing:border-box;color:inherit!important;margin:0;text-decoration:inherit;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray;text-shadow:inherit!important">Read Existing State</a></h3>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">The first step is reading the existing state. This is important for skipping unnecessary operations (like already up-to-date files) and cleanup.</p>
<h3 id="process-new-version" style="box-sizing:border-box;grid-column:2;margin:.4em 1rem .8em"><a href="https://kevincox.ca/2023/07/12/statit-static-site-deploy/#process-new-version" style="box-sizing:border-box;color:inherit!important;margin:0;text-decoration:inherit;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray;text-shadow:inherit!important">Process New Version</a></h3>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">Next statit scans the input directory and collects information about the new version.</p>
<ol style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">
<li style="box-sizing:border-box;margin:0">List files.</li>
<li style="box-sizing:border-box;margin:0">Record file contents.</li>
<li style="box-sizing:border-box;margin:0">Resolve file metadata such as headers and asset/entry status.</li>
</ol>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">Step 3 is customizable through various command-line flags. For example the <code style="background:#000;border:1px solid #555;border-radius:.3em;box-sizing:border-box;color:#fff;margin:0;padding:0 .2em;white-space:pre-wrap"><span style="box-sizing:border-box;color:#aaf;margin:0;padding:0">--cache</span></code> flag can be used to adjust the caching settings for particular files.</p>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">At the end of this step we have a complete image of what the live site should look like when the deploy is complete.</p>
<h3 id="upload-assets" style="box-sizing:border-box;grid-column:2;margin:.4em 1rem .8em"><a href="https://kevincox.ca/2023/07/12/statit-static-site-deploy/#upload-assets" style="box-sizing:border-box;color:inherit!important;margin:0;text-decoration:inherit;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray;text-shadow:inherit!important">Upload Assets</a></h3>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">This is where the deployment really starts. The assets are uploaded to the bucket. New files will be publicly accessible but should be unreferenced at this point, so visitors will not yet experience any change.</p>
<h3 id="upload-entries" style="box-sizing:border-box;grid-column:2;margin:.4em 1rem .8em"><a href="https://kevincox.ca/2023/07/12/statit-static-site-deploy/#upload-entries" style="box-sizing:border-box;color:inherit!important;margin:0;text-decoration:inherit;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray;text-shadow:inherit!important">Upload Entries</a></h3>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">This is where the new site is made live! The entries are uploaded. At this point (caching aside) new visitors will see the new content. However, removed pages will still resolve to the old version.</p>
<h3 id="write-manifest" style="box-sizing:border-box;grid-column:2;margin:.4em 1rem .8em"><a href="https://kevincox.ca/2023/07/12/statit-static-site-deploy/#write-manifest" style="box-sizing:border-box;color:inherit!important;margin:0;text-decoration:inherit;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray;text-shadow:inherit!important">Write Manifest</a></h3>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">statit writes manifests to store deployment-related metadata. This allows for proper cleanup of old versions. Currently, the manifest essentially consists of a list of known files.</p>
<h3 id="cleanup" style="box-sizing:border-box;grid-column:2;margin:.4em 1rem .8em"><a href="https://kevincox.ca/2023/07/12/statit-static-site-deploy/#cleanup" style="box-sizing:border-box;color:inherit!important;margin:0;text-decoration:inherit;text-decoration:underline;-webkit-text-decoration-color:gray;text-decoration-color:gray;text-shadow:inherit!important">Cleanup</a></h3>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">In one step statit performs two types of cleanup.</p>
<ol style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">
<li style="box-sizing:border-box;margin:0">Remove deleted entries.</li>
<li style="box-sizing:border-box;margin:0">Remove obsolete assets.</li>
</ol>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">Part 1 is important to ensure that removed pages are made immediately unavailable. This should result in your 404 page appearing where content was deleted.</p>
<p style="box-sizing:border-box;grid-column:2;margin:0 1rem .8em">Part 2 is a cleanup process that isn’t critical to site function but reduces storage usage. statit will remove any assets that have been unused for at least 7 days. This period should be the longest timeframe for which an out-of-date asset can be requested. For many sites this is your longest entry cache plus the max load time. So if you cache pages for 1h than 2h should be very safe. However for sites that lazily load content (such as single-page applications with lazy code loading) users can potentially be using an older version of your site for much longer. 7 days is chosen for now as it should be conservative enough for most use cases, it will be made configurable in the future.</p></main>