Repository: certbot/certbot

Issue:
Running with 192MB RAM is not enough
`python-cryptography` apparently tries to compile a pretty big C binding or something like that, and gcc proceeds to ICE due to lack of free memory.

The `letsencrypt-auto` tool should know its memory requirements and ask whether you really want to proceed if you haven’t enough.



Discussion:
Got the same problem on a 512MB Digitalocean droplet. So no chance of testing it.

this could be problematic.. anything can be done about it ? I think 512MB minimum VPS size would be a good target to aim for. 

Won't compile on a DO 512MB droplet even with `--param ggc-min-expand=0 --param ggc-min-heapsize=8192`. I think the only option for now is to wait for an operating system package.

OS packages are already available for Debian experimental, Arch, and FreeBSD.  More should be on the way.

But if you really need to work around this in the meantime, there are workarounds:
1. You could also probably just rsync the venv directory `~/.local/share/letsencrypt` from a machine with more RAM.
2. Try [making a swapfile](http://www.cyberciti.biz/faq/linux-add-a-swap-file-howto/) to give yourself more memory.  If your instance is spinning-disk-backed, this may be horribly slow, but SSD should be okay.

> OS packages are already available for Debian experimental, Arch, and FreeBSD. More should be on the way.

The native package that’s giving trouble (python-cryptography) is already available on many distributions (notable exception being current LTS of Ubuntu), but of incompatible version (?).

> You could also probably just rsync the venv directory ~/.local/share/letsencrypt from a machine with more RAM.

That’s way too naive. letsencrypt depends on native libraries and these must be properly cross-compiled to “just-work” after being copy-pasted.

---

That being said, it is pretty hard to make letsencrypt-auto to use already installed python packages. I had to do whole installation (into virtual environ) manually after installing all the dependencies from the OS.

Probably the first good step would be an option to use OS’s python packages that are available, only building and installing missing ones.

Gave the Debian package a go, and it seems `python-letsencrypt` has got a problem with it. `python-letsencrypt : Dependencies: python-acme (= 0.0.0.dev20151114) but 0.0.0.dev20151114-1 is to be installed` - it's asking explicitly for `dev20151114` whereas the only package available is `dev20151114-1`. 

Though this is only tangentially related to this ticket.

[Another full trace of the issue](http://ix.io/msn), provided by someone on IRC.

still an issue ? i only tested on 1GB VPS.  minimum memory requirements ?

128M VPS on BuyVM crashing with the same problem. I only found out about this problem after searching for the error I was encountering. 128M is pitifully small, but it works great for tiny single web app that doesn't have a lot of traffic. I could upgrade to something larger, but that would be quite a waste. What is the target size LetsEncrypt is shooting to support?

@huanga, one thing you could try is to build `simp_le` binary out of https://github.com/kuba/simp_le/tree/master/pyi and copy it over to your server - memory requirements should be minimal.

another person hit https://community.letsencrypt.org/t/letsencrypt-auto-error-errno-12-cannot-allocate-memory/6359/

We're working on a fix for this in cffi+cryptography. Will update as
there's progress.

On Tue, Dec 8, 2015 at 8:23 PM, George Liu (eva2000) <
notifications@github.com> wrote:

> another person hit
> https://community.letsencrypt.org/t/letsencrypt-auto-error-errno-12-cannot-allocate-memory/6359/
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/letsencrypt/letsencrypt/issues/1081#issuecomment-163076690
> .

## 

"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084

@alex cheers thanks for the heads up :+1: 

If you want something super lightweight and easy to install with few dependencies...

https://github.com/Daplie/node-letsencrypt-cli

``` bash
# Install node.js
curl -L bit.ly/iojs-min | bash

# Install LetsEncrypt
npm install -g letsencrypt-cli

# get certs
letsencrypt certonly \
  --agree-tos --email john.doe@example.com \
  --standalone \
  --domains example.com,www.example.com \
  --server https://acme-staging.api.letsencrypt.org/directory \

# take a look
ls ~/letsencrypt/etc/live
```

We don't have all of the apache config and whatnot, but if you just want your certs quick and easy - especially on low-mem and low-power platforms (DO, RPi), it'll do.

One way I use to avoid this issue is by restarting my VM/droplet. worked with DigitalOcean 512MB.

any movement on this ? @bmw @jsha @pde 

@the7th restarting your droplet gave you enough memory?

@macdonjo Yeah. Somehow it does. Try restarting your droplet. It "might" work.

I'm using a Digital Ocean 512Mb droplet to run three django apps through gunicorn/nginx. At first I did not get letsencrypt installed and got the error above. As soon as I shut the 3 apps down, installation went fine. Problem seems indeed to be memory related.

Same issue - I am trying to use on a 128MB VM.  I can't increase swop past 64MB.  I think lets encrypt should be usable even on low end boxes (Cent OS 6 - so maybe I need to wait for native packages?)

I was able to install certs on OpenVz/Ramnode 128MB RAM / 64 MB VRAM container with Ubuntu 14.04 **32-bit**.

The 32bit is the key. I was **not** able to install on Ubuntu 14.04 **64-bit**.

Hitting this on Ramnode OpenVZ 128MB with 64-bit Ubuntu 14.04
Wondering how best to proceed: wait, try a different approach?

My issue was on ram node - thing is you can't increase the swap. I just switched to using a python client (acme client) and it is up and running ok on cent os 6

Right, swap is fixed, can't be added even through a file.
Is there a howto somewhere on using the acme client?

Think I used this wrapper - then you only need Python and OpenSSL 

https://github.com/diafygi/acme-tiny/blob/master/README.md

Same issue. On a 512MB droplet. I have swap space enabled (I think) but I still have the issue.

Same issue on a 512MB Ubuntu droplet. I did the restart and it was able to run. The only things running on my server is a apache server, MySQL and irssi for IRC chats.

I have a DO of 512mb and it had had the same problem, I could solve it by creation of a swapfile

I was able to get is going by stopping (`sudo service apache2 stop`) the apache service and then running `letsencrypt-auto` on my debian wheezy. It really was due to memory. Hope this helps someone.

@DDecoene Worked for me.

+1 would really like to see this fixed. Right now we have to manually stop all running servers every time we need to renew in order to make it work, which defeats the automation

I upgraded to from OpenVZ 128M+64M to KVM 512M and no longer have this problem. It would be nice to have letsencrypt-auto work on low-memory though for the affected systems/distros.

@tifroz, this problem should only occur when installing or upgrading Certbot using `{certbot,letsencrypt}-auto`. If you're using one of the `auto` scripts, [we generally recommend](https://certbot.eff.org/#pip-other) to provide `--no-self-upgrade` on the command line which will prevent the script and Certbot from auto updating which both increases stability and avoids this problem.

@bmw Thank you for the guidance. I can't seem to get the `--no-self-upgrade` to have any effect however. When executing `./certbot-auto renew --no-self-upgrade --quiet --dry-run --staging > "/var/log/apps/letsencrypt.log"`, I am getting this output (pruned for brevity):

```
Bootstrapping dependencies for Debian-based OSes...
Hit http://security.ubuntu.com trusty-security InRelease
...
Reading package lists...
...
The following extra packages will be installed:
  libssl1.0.0
The following packages will be upgraded:
  libssl-dev libssl1.0.0
2 upgraded, 0 newly installed, 0 to remove and 34 not upgraded.
Need to get 1,904 kB of archives.
After this operation, 6,144 B of additional disk space will be used.
Do you want to continue? [Y/n]
```


Resolving patch:
diff --git a/letsencrypt-auto-source/pieces/pipstrap.py b/letsencrypt-auto-source/pieces/pipstrap.py
--- a/letsencrypt-auto-source/pieces/pipstrap.py
+++ b/letsencrypt-auto-source/pieces/pipstrap.py
@@ -21,6 +21,7 @@
 # The above copyright notice and this permission notice shall be included in
 # all copies or substantial portions of the Software.
 from __future__ import print_function
+from distutils.version import StrictVersion
 from hashlib import sha256
 from os.path import join
 from pipes import quote
@@ -55,12 +56,14 @@ def check_output(*popenargs, **kwargs):
     from urllib.parse import urlparse  # 3.4
 
 
-__version__ = 1, 1, 1
+__version__ = 1, 3, 0
+PIP_VERSION = '9.0.1'
 
 
 # wheel has a conditional dependency on argparse:
 maybe_argparse = (
-    [('https://pypi.python.org/packages/source/a/argparse/'
+    [('https://pypi.python.org/packages/18/dd/'
+      'e617cfc3f6210ae183374cd9f6a26b20514bbb5a792af97949c5aacddf0f/'
       'argparse-1.4.0.tar.gz',
       '62b089a55be1d8949cd2bc7e0df0bddb9e028faefc8c32038cc84862aefdd6e4')]
     if version_info < (2, 7, 0) else [])
@@ -68,13 +71,19 @@ def check_output(*popenargs, **kwargs):
 
 PACKAGES = maybe_argparse + [
     # Pip has no dependencies, as it vendors everything:
-    ('https://pypi.python.org/packages/source/p/pip/pip-8.0.3.tar.gz',
-     '30f98b66f3fe1069c529a491597d34a1c224a68640c82caf2ade5f88aa1405e8'),
+    ('https://pypi.python.org/packages/11/b6/'
+     'abcb525026a4be042b486df43905d6893fb04f05aac21c32c638e939e447/'
+     'pip-{0}.tar.gz'
+     .format(PIP_VERSION),
+     '09f243e1a7b461f654c26a725fa373211bb7ff17a9300058b205c61658ca940d'),
     # This version of setuptools has only optional dependencies:
-    ('https://pypi.python.org/packages/source/s/setuptools/'
+    ('https://pypi.python.org/packages/69/65/'
+     '4c544cde88d4d876cdf5cbc5f3f15d02646477756d89547e9a7ecd6afa76/'
      'setuptools-20.2.2.tar.gz',
      '24fcfc15364a9fe09a220f37d2dcedc849795e3de3e4b393ee988e66a9cbd85a'),
-    ('https://pypi.python.org/packages/source/w/wheel/wheel-0.29.0.tar.gz',
+    ('https://pypi.python.org/packages/c9/1d/'
+     'bd19e691fd4cfe908c76c429fe6e4436c9e83583c4414b54f6c85471954a/'
+     'wheel-0.29.0.tar.gz',
      '1ebb8ad7e26b448e9caa4773d2357849bf80ff9e313964bcaf79cbf0201a1648')
 ]
 
@@ -124,11 +133,21 @@ def read_chunks(response, chunk_size):
 
 
 def main():
+    pip_version = StrictVersion(check_output(['pip', '--version'])
+                                .decode('utf-8').split()[1])
+    min_pip_version = StrictVersion(PIP_VERSION)
+    if pip_version >= min_pip_version:
+        return 0
+    has_pip_cache = pip_version >= StrictVersion('6.0')
+
     temp = mkdtemp(prefix='pipstrap-')
     try:
         downloads = [hashed_download(url, temp, digest)
                      for url, digest in PACKAGES]
         check_output('pip install --no-index --no-deps -U ' +
+                     # Disable cache since we're not using it and it otherwise
+                     # sometimes throws permission warnings:
+                     ('--no-cache-dir ' if has_pip_cache else '') +
                      ' '.join(quote(d) for d in downloads),
                      shell=True)
     except HashError as exc:


Test patch: