opened
greysteil
username_0
There are 82 commits on master waiting to be deployed.

If there's anything blocking this going out then I'd love to help :octocat:.
🚢 Deploy!
issue
created
RedSparr0w
username_1
It has been almost 3 months since the last deploy (Mar 25) so hopefully there will be one soon.
🤞
comment
created
greysteil
username_0
Ah, 👍, and thanks both for all your work on this project!
comment
created
espadrine
username_2
The commit currently deployed is 57a1bf2f082be3c0bcef4250f6a5e783a02fc9cf, which is from May 30.

(I typically deploy every two weeks ; although last week-end was spent with the step-family, to prepare my sister's wedding. We were in a rural area of French Bourgogne, which is known for its wine, but not for its Internet access.)

@username_0 What fundamental change are you not seeing? Is there a commit that should have been deployed, but hasn't? Maybe there was a regression?
comment
created
greysteil
username_0
The commit that I (selfishly) want deployed is ca58d84447db73c52d3a67f82979e539d29ed2e0, but when I saw the badge saying there were 81 commits awaiting deploy and looked into it I figured a deploy was needed more generally, hence the PR.

Is there any way that we can tell which commit is the latest deployed and fix the badge on the readme? I wouldn't have created this PR and caused this trouble if I'd known it was only a few weeks since the last deploy.

<a href="https://github.com/badges/shields/compare/gh-pages...master">
        <img src="https://img.shields.io/github/commits-since/badges/shields/gh-pages.svg?label=commits%20to%20be%20deployed"
            alt="commits to be deployed"></a>
comment
created
paulmelnikow
username_3
Thanks for deploying the servers! For some reason gh-pages is not updating:

<img width="1126" alt="screen shot 2018-06-20 at 11 10 55 am" src="https://user-images.githubusercontent.com/1487036/41667357-be767996-747a-11e8-983a-153b323368a7.png">

Hence the status badge being wrong, as well…
comment
created
paulmelnikow
username_3
I know there's a bunch of tooling in the front-end deploy. If it's not working, let me know and I can fix whatever is broken and deploy 57a1bf2.
comment
created
espadrine
username_2
@username_0 No trouble! You're right to ask.

@username_3 I didn't notice that… I'll look into what is wrong tonight.

By the way, with Sentry working, do you feel OK with making deployments yourself? It is not ideal, but you have the ability to deploy individual servers or all of them at once, so this deployment workflow may work for you (it is pretty close to what I do):
1. Look at new commits compared to the latest deployed hash.
2. Check that it works on localhost.
3. Deploy a single server (say s0).
4. Look it up by IP by loading https://192.99.59.72/index.html in your browser.
5. Check Sentry for new errors for a bit.
6. If it looks OK, deploy all servers.
7. Keep an eye on Sentry.
comment
created
espadrine
username_2
using build directory: /home/tyl/file/github/gh-badges/.next
  copying "static" directory
  exporting path: /

Switched to and reset branch 'gh-pages'
~/file/github/gh-badges/next/c8b571e7-c012-44f1-9a20-e4eba9321dd0/page ~/file/github/gh-badges
mv: cannot stat '_error': No such file or directory
…
```

I don't quite know what it is; you are welcome to take a stab at it!
comment
created
RedSparr0w
username_1
Thanks for the update @username_2 :smile:,
I probably should have tried some of the badges instead of relying solely on the gh-pages branch.

As for the above problem would changing this line to something like this fix it:
```diff
-pushd next/*/page && mv {_,}error && popd && \
// If the _error directory is needed:
+pushd next/*/page && (mv {_,}error || mkdir error\_error) && popd && \
// If not:
+pushd next/*/page && (mv {_,}error || true) && popd && \
```
comment
created
platan
username_4
Can we somehow update `gh-pages`? Or should we wait for the next deployment? <a href="https://github.com/badges/shields/compare/gh-pages...master">
        <img src="https://img.shields.io/github/commits-since/badges/shields/gh-pages.svg?label=commits%20to%20be%20deployed"
            alt="commits to be deployed"></a> badge and [Deployment status in PR](https://github.com/badges/shields/issues/1708) feature are based on a content of this branch.
comment
created
paulmelnikow
username_3
@username_2,

As a stopgap, this seems okay. It's possible that things could break in a way that crashes the servers without getting a message to Sentry. Without logs, we won't know what's causing the problems. We also can't monitor server resources, see our traffic, or control the load balancer.

Could we make a plan to get this project onto dedicated hosting, so that no part of it is dependent on any one person?
comment
created
espadrine
username_2
We can switch the VPS instances one by one to a shared account. That said, SSH access will still need to be heavily restricted. Also, having a proper set up, where each deployment requires the validation of another member, and all SSH sessions are logged in a tamper-proof way, and can be inspected by other members, would require quite a bit of work.
comment
created
chris48s
username_5
I'm not going to express a strong preference for a particular solution because I don't have access to the logs/monitoring etc so its difficult for me to make an informed judgement on the best way forward given traffic, finances, etc.

I would like to see us move towards better performance, more frequent deploys and the ability to share ops burden with more maintainers (without compromising security, etc). Happy to pitch in with that where appropriate.

One thing I will add to @username_3 's suggestion is that because everything is served up from behind cloudfront, it should be possible to test a new setup (e.g: PaaS) without doing a 'big bang' migration. We could load-balance some % of traffic to a new solution and some to the existing 3xVPS setup. That would allow a low-risk trial to be conducted on real-world traffic with option to back out or conduct phased migration.
comment
created
paulmelnikow
username_3
I don't have access to logs or monitoring either, though I could speak to the finances. For a while I was arguing against new hosting for cost reasons. However things have changed. We've gotten some big donations and have some good-sized regular donations too. I took the liberty of reaching out to Zeit about donating hosting, though if that doesn't come through, we could pay our own way.

Clearing up any uncertainty about the cost would be another good reason for the load-balancer approach you're suggesting. We can get better cost numbers that way, and still have an option to back things out.
comment
created
paulmelnikow
username_3
A week ago I wrote @username_2, following up on the above. I’m concerned. If we don't fix our reliability problems now, I'm afraid our users will go elsewhere and this project will die.
comment