Repository: certbot/certbot

Issue:
Certbot JWSObject structure is different from the acme spec (specifically re: header/protected/nonce))
If I'm not mistaken, here's the line where certbot generates the signature that is used on the server to verify integrity of a JWSObject:
https://github.com/certbot/certbot/blob/8011fb2879659008528e8b6ba5c04ac3de1e63a0/acme/acme/jose/jws.py#L233

Certbot seems to deviate from the ACME spec in terms of the structure of the requests. In the spec the nonce is in the header and there is just the protected, payload and signature fields. The requests from certbot however have only the nonce in the protected field and have the header attributes separately, not Base64Url encoded, as just plain JSON.

I'm working with the JOSE implementation from nimbusds.

Here's the line where the signing input is created:
https://bitbucket.org/connect2id/nimbus-jose-jwt/src/a730749c3831e429617d0cb4203303b36edfc6f2/src/main/java/com/nimbusds/jose/JWSObject.java?at=master&fileviewer=file-view-default#JWSObject.java-204

Here's where that method is called:
https://bitbucket.org/connect2id/nimbus-jose-jwt/src/a730749c3831e429617d0cb4203303b36edfc6f2/src/main/java/com/nimbusds/jose/JWSObject.java?at=master&fileviewer=file-view-default#JWSObject.java-165

This line of code is invoked just a few lines above that call:
https://bitbucket.org/connect2id/nimbus-jose-jwt/src/a730749c3831e429617d0cb4203303b36edfc6f2/src/main/java/com/nimbusds/jose/JWSObject.java?at=master&fileviewer=file-view-default#JWSObject.java-151

The JWSHeader.parse() method that's called validates that the header contains an "alg" attribute:
https://bitbucket.org/connect2id/nimbus-jose-jwt/src/a730749c3831e429617d0cb4203303b36edfc6f2/src/main/java/com/nimbusds/jose/JWSHeader.java?at=master&fileviewer=file-view-default#JWSHeader.java-631

Putting the pieces together:
- certbot generates a signature based on a protected field with only a nonce (no alg or other information)
- nimbus-jose-jwt checks the signature based on the header, which must contain the alg.
- based on the ACME spec (https://tools.ietf.org/html/draft-ietf-acme-acme-04#section-5.2), the protected field should contain all of the information that certbot puts in the header AND the nonce 

The net result is that verification doesn't work properly without some contortions on the server side.

I hope this all makes sense. Are there plans to bring certbot in line with the acme spec?


Discussion:
I see that this was called out in a recent version of the acme-divergences document:
https://github.com/letsencrypt/boulder/commit/db3a6d650774228515b6b0d869b442f9409661c7#diff-3d6d8baf462b78681424da8ce68634d9

But compatibility with other toolkits as a result is probably still relevant.
@jsha do you want to decide how urgent this is, and assign it to 0.15.0, 1.0 or Wishlist accordingly?
Yep, we should fix this. In short, JWS offers an unprotected header ("header") and a protected header ("protected"). Boulder originally allowed JWK to be in the unprotected header, because tampering with the JWK would just result in a failure to validate. However, we later decided it was silly to allow anything in the unprotected header. I think it's worth moving jwk to the protected header, and should make @laran's implementation easier.

Resolving patch:
diff --git a/acme/acme/jose/jws.py b/acme/acme/jose/jws.py
--- a/acme/acme/jose/jws.py
+++ b/acme/acme/jose/jws.py
@@ -222,7 +222,8 @@ def sign(cls, payload, key, alg, include_jwk=True,
 
         protected_params = {}
         for header in protect:
-            protected_params[header] = header_params.pop(header)
+            if header in header_params:
+                protected_params[header] = header_params.pop(header)
         if protected_params:
             # pylint: disable=star-args
             protected = cls.header_cls(**protected_params).json_dumps()
diff --git a/acme/acme/jws.py b/acme/acme/jws.py
--- a/acme/acme/jws.py
+++ b/acme/acme/jws.py
@@ -49,6 +49,6 @@ def sign(cls, payload, key, alg, nonce, url=None, kid=None):
         # jwk field if kid is not provided.
         include_jwk = kid is None
         return super(JWS, cls).sign(payload, key=key, alg=alg,
-                                    protect=frozenset(['nonce', 'url', 'kid']),
+                                    protect=frozenset(['nonce', 'url', 'kid', 'jwk', 'alg']),
                                     nonce=nonce, url=url, kid=kid,
                                     include_jwk=include_jwk)


Test patch: