Repository: certbot/certbot

Issue:
dns-cloudflare: Support API tokens
As per [this comment by Cloudflare staff](https://community.cloudflare.com/t/restricted-api-keys/13647/100?u=judge), the new API keys functionality is in open beta.  The feature allows creating multiple separate API tokens for each application with fine-grain permissions the user can apply. I recommend support for these new tokens be added to Certbot.

This feature uses a new authorization method: bearer tokens 
(specifically `Authorization: bearer :token`).

![](https://i.judge.sh/Tempest/qUYTibMu.png)

Due to the change in the authentication method, certbot will have to account for users who were using the global API key previously, and users who want to use the global API key when setting up dns-cloudflare for the first time (since it's still in beta, I recommend keeping Global API key support).

I made a PR for changing the API tokens link about a month ago - #7052 - and while that is still the correct URL (even now for the Global API key) more text or instruction is probably needed to have the user choose either the global API key or one of the new API tokens.


Discussion:
Cool! I hadn't heard about this new feature from Cloudflare yet.

I agree we should keep support for the global API key at least for now.

One possible complication is we probably can't depend on a super recent [cloudflare](https://github.com/certbot/certbot/blob/750d6a9686a98204c3ea46208999f6ae0ee15479/certbot-dns-cloudflare/setup.py#L12) package as it would prevent us from being able to backport new versions of our Cloudflare plugin to stable distros who won't update their packaged version of the cloudflare library. If support for API tokens is only in recent versions, we may have to conditionally support this feature based on the installed version of `cloudflare`.

Assuming that's not too difficult and this feature can be added without too much additional complexity to the plugin, we'd accept a well written PR for this!
Is there a way to get `certbot` to use my API token currently?  In my `--dns-cloudflare-credentials` file I've tried `dns_cloudflare_api_key = <token>` but I always get the 

```
Error determining zone_id: 6003 Invalid request headers. Please confirm that you have supplied valid Cloudflare API credentials. (Did you copy your entire API key?)
```

Error message.

(shameless plug: working on a [cetbot cookbook](https://github.com/qubitrenegade/certbot-exec) and wanting to [leverage cloudflare-dns plugin](https://github.com/qubitrenegade/certbot-cloudflare), which works!, But I'm not sharing my global API key with Travis CI... which hampers testing.)

Either way, we love `certbot`!  So Thanks!

(tangentially related: I hate to admit, but not really _that_ much... I actually started with my token and went "Wtf?! this always worked before!!" and had to google the error to determine I needed my "Cloudflare Global API key"... so maybe the docs would benefit from an update... if PRs are accepted?)
@qubitrenegade, thanks for pointing this out and the kind words about Certbot. I created #7280 to track making this clearer in our docs.
I don’t think this has been implemented upstream, given the documentation of the Cloudflare package. I created cloudflare/python-cloudflare#74 to track this.
Already the 6th most voted certbot feature-request. Without a doubt the first one in less than a month.
@drzraf Yes but this is blocked upstream. Please help pushing the Cloudflare team! 😄 
@FranklinYu It's a shame it's still blocked at this point too.

There was even a very helpful commenter on the CloudFlare repository's issue, along with a suggested pull request, but the pull request has no activity on it; no comments, no suggested code changes, nothing. There haven't been any other comments on the issue either. It's just sat there like that for over a month.

Like, come on CloudFlare.... a helpful community member gave them a solution and they're just sitting on it :/
FYI https://github.com/go-acme/lego/pull/937
Well cloudflare/python-cloudflare#76 was just merged thanks to a little prodding :joy:

For implementation here, the end result needs to be that the call to python-cloudflare on line 67 is not provided with an email when a token is used(however the email is still needed when using a global key).

https://github.com/certbot/certbot/blob/0cfedbc5f5443e94c6e802d1f8fb46cbc44b776c/certbot-dns-cloudflare/certbot_dns_cloudflare/dns_cloudflare.py#L67

The call [also supports named arguments](https://github.com/cloudflare/python-cloudflare#providing-cloudflare-username-and-api-key) if that makes things neater.

Other than that, I'll probably leave this to someone else as I'm not very familiar with the codebase, and I'm also not sure how to present the option to the user. Perhaps a `dns_cloudflare_api_token` option?
Not too fast - also need to wait for python-cloudflare to get a release.
Ah, yes. I assumed that they would put a release through shortly after merging, however they haven't yet. I shall investigate...
2.3.1 released! (after much poking behind the scenes 😂)
@Tugzrida Does this mean end users can start using non-global Cloudflare API keys to do Cloudflare DNS verification in the latest version of certbot or are there other things that still need to happen?

Apologies, I’m struggling to keep track of the various things that need to be updated, merged in, etc!
Only the pypi package `cloudflare` has been updated. https://pypi.org/project/cloudflare/#history The way it works now is by passing only a token with no email to use API tokens instead of the API Key.

https://github.com/cloudflare/python-cloudflare/blob/f71c3e221735ad50230918f3c9bee6d34471ae58/CloudFlare/cloudflare.py#L52-L58

Certbot does currently require an email to be present, so Certbot will need to be updated for this.

https://github.com/certbot/certbot/blob/4f3010ef3f245e66933d19595038535934376c30/certbot-dns-cloudflare/certbot_dns_cloudflare/dns_cloudflare.py#L67



Resolving patch:
diff --git a/certbot-dns-cloudflare/certbot_dns_cloudflare/__init__.py b/certbot-dns-cloudflare/certbot_dns_cloudflare/__init__.py
--- a/certbot-dns-cloudflare/certbot_dns_cloudflare/__init__.py
+++ b/certbot-dns-cloudflare/certbot_dns_cloudflare/__init__.py
@@ -22,17 +22,40 @@
 
 Use of this plugin requires a configuration file containing Cloudflare API
 credentials, obtained from your Cloudflare
-`account page <https://www.cloudflare.com/a/account/my-account>`_. This plugin
-does not currently support Cloudflare's "API Tokens", so please ensure you use
-the "Global API Key" for authentication.
+`account page <https://dash.cloudflare.com/profile/api-tokens>`_.
+
+Previously, Cloudflare's "Global API Key" was used for authentication, however
+this key can access the entire Cloudflare API for all domains in your account,
+meaning it could cause a lot of damage if leaked.
+
+Cloudflare's newer API Tokens can be restricted to specific domains and
+operations, and are therefore now the recommended authentication option.
+
+However, due to some shortcomings in Cloudflare's implementation of Tokens,
+Tokens created for Certbot currently require ``Zone:Zone:Read`` and ``Zone:DNS:Edit``
+permissions for **all** zones in your account. While this is not ideal, your Token
+will still have fewer permission than the Global key, so it's still worth doing.
+Hopefully Cloudflare will improve this in the future.
+
+Using Cloudflare Tokens also requires at least version 2.3.1 of the ``cloudflare``
+python module. If the version that automatically installed with this plugin is
+older than that, and you can't upgrade it on your system, you'll have to stick to
+the Global key.
+
+.. code-block:: ini
+   :name: certbot_cloudflare_token.ini
+   :caption: Example credentials file using restricted API Token (recommended):
+
+   # Cloudflare API token used by Certbot
+   dns_cloudflare_api_token = 0123456789abcdef0123456789abcdef01234567
 
 .. code-block:: ini
-   :name: credentials.ini
-   :caption: Example credentials file:
+   :name: certbot_cloudflare_key.ini
+   :caption: Example credentials file using Global API Key (not recommended):
 
    # Cloudflare API credentials used by Certbot
    dns_cloudflare_email = cloudflare@example.com
-   dns_cloudflare_api_key = 0123456789abcdef0123456789abcdef01234567
+   dns_cloudflare_api_key = 0123456789abcdef0123456789abcdef01234
 
 The path to this file can be provided interactively or using the
 ``--dns-cloudflare-credentials`` command-line argument. Certbot records the path
diff --git a/certbot-dns-cloudflare/certbot_dns_cloudflare/_internal/dns_cloudflare.py b/certbot-dns-cloudflare/certbot_dns_cloudflare/_internal/dns_cloudflare.py
--- a/certbot-dns-cloudflare/certbot_dns_cloudflare/_internal/dns_cloudflare.py
+++ b/certbot-dns-cloudflare/certbot_dns_cloudflare/_internal/dns_cloudflare.py
@@ -4,6 +4,10 @@
 import CloudFlare
 import zope.interface
 
+from acme.magic_typing import Any
+from acme.magic_typing import Dict
+from acme.magic_typing import List
+
 from certbot import errors
 from certbot import interfaces
 from certbot.plugins import dns_common
@@ -38,14 +42,35 @@ def more_info(self):  # pylint: disable=missing-docstring,no-self-use
         return 'This plugin configures a DNS TXT record to respond to a dns-01 challenge using ' + \
                'the Cloudflare API.'
 
+    def _validate_credentials(self, credentials):
+        token = credentials.conf('api-token')
+        email = credentials.conf('email')
+        key = credentials.conf('api-key')
+        if token:
+            if email or key:
+                raise errors.PluginError('{}: dns_cloudflare_email and dns_cloudflare_api_key are '
+                                         'not needed when using an API Token'
+                                         .format(credentials.confobj.filename))
+        elif email or key:
+            if not email:
+                raise errors.PluginError('{}: dns_cloudflare_email is required when using a Global '
+                                         'API Key. (should be email address associated with '
+                                         'Cloudflare account)'.format(credentials.confobj.filename))
+            if not key:
+                raise errors.PluginError('{}: dns_cloudflare_api_key is required when using a '
+                                         'Global API Key. (see {})'
+                                         .format(credentials.confobj.filename, ACCOUNT_URL))
+        else:
+            raise errors.PluginError('{}: Either dns_cloudflare_api_token (recommended), or '
+                                     'dns_cloudflare_email and dns_cloudflare_api_key are required.'
+                                     ' (see {})'.format(credentials.confobj.filename, ACCOUNT_URL))
+
     def _setup_credentials(self):
         self.credentials = self._configure_credentials(
             'credentials',
             'Cloudflare credentials INI file',
-            {
-                'email': 'email address associated with Cloudflare account',
-                'api-key': 'API key for Cloudflare account, obtained from {0}'.format(ACCOUNT_URL)
-            }
+            None,
+            self._validate_credentials
         )
 
     def _perform(self, domain, validation_name, validation):
@@ -55,6 +80,8 @@ def _cleanup(self, domain, validation_name, validation):
         self._get_cloudflare_client().del_txt_record(domain, validation_name, validation)
 
     def _get_cloudflare_client(self):
+        if self.credentials.conf('api-token'):
+            return _CloudflareClient(None, self.credentials.conf('api-token'))
         return _CloudflareClient(self.credentials.conf('email'), self.credentials.conf('api-key'))
 
 
@@ -88,8 +115,15 @@ def add_txt_record(self, domain, record_name, record_content, record_ttl):
             logger.debug('Attempting to add record to zone %s: %s', zone_id, data)
             self.cf.zones.dns_records.post(zone_id, data=data)  # zones | pylint: disable=no-member
         except CloudFlare.exceptions.CloudFlareAPIError as e:
+            code = int(e)
+            hint = None
+
+            if code == 9109:
+                hint = 'Does your API token have "Zone:DNS:Edit" permissions?'
+
             logger.error('Encountered CloudFlareAPIError adding TXT record: %d %s', e, e)
-            raise errors.PluginError('Error communicating with the Cloudflare API: {0}'.format(e))
+            raise errors.PluginError('Error communicating with the Cloudflare API: {0}{1}'
+                                     .format(e, ' ({0})'.format(hint) if hint else ''))
 
         record_id = self._find_txt_record_id(zone_id, record_name, record_content)
         logger.debug('Successfully added TXT record with record_id: %s', record_id)
@@ -139,6 +173,8 @@ def _find_zone_id(self, domain):
         """
 
         zone_name_guesses = dns_common.base_domain_name_guesses(domain)
+        zones = []  # type: List[Dict[str, Any]]
+        code = msg = None
 
         for zone_name in zone_name_guesses:
             params = {'name': zone_name,
@@ -148,16 +184,26 @@ def _find_zone_id(self, domain):
                 zones = self.cf.zones.get(params=params)  # zones | pylint: disable=no-member
             except CloudFlare.exceptions.CloudFlareAPIError as e:
                 code = int(e)
+                msg = str(e)
                 hint = None
 
                 if code == 6003:
-                    hint = 'Did you copy your entire API key?'
+                    hint = ('Did you copy your entire API token/key? To use Cloudflare tokens, '
+                            'you\'ll need the python package cloudflare>=2.3.1.{}'
+                    .format(' This certbot is running cloudflare ' + str(CloudFlare.__version__)
+                    if hasattr(CloudFlare, '__version__') else ''))
                 elif code == 9103:
-                    hint = 'Did you enter the correct email address?'
-
-                raise errors.PluginError('Error determining zone_id: {0} {1}. Please confirm that '
-                                         'you have supplied valid Cloudflare API credentials.{2}'
-                                         .format(code, e, ' ({0})'.format(hint) if hint else ''))
+                    hint = 'Did you enter the correct email address and Global key?'
+                elif code == 9109:
+                    hint = 'Did you enter a valid Cloudflare Token?'
+
+                if hint:
+                    raise errors.PluginError('Error determining zone_id: {0} {1}. Please confirm '
+                                  'that you have supplied valid Cloudflare API credentials. ({2})'
+                                                                         .format(code, msg, hint))
+                else:
+                    logger.debug('Unrecognised CloudFlareAPIError while finding zone_id: %d %s. '
+                                 'Continuing with next zone guess...', e, e)
 
             if zones:
                 zone_id = zones[0]['id']
@@ -165,9 +211,10 @@ def _find_zone_id(self, domain):
                 return zone_id
 
         raise errors.PluginError('Unable to determine zone_id for {0} using zone names: {1}. '
-                                 'Please confirm that the domain name has been entered correctly '
-                                 'and is already associated with the supplied Cloudflare account.'
-                                 .format(domain, zone_name_guesses))
+                                'Please confirm that the domain name has been entered correctly '
+                                'and is already associated with the supplied Cloudflare account.{2}'
+                                .format(domain, zone_name_guesses, ' The error from Cloudflare was:'
+                                ' {0} {1}'.format(code, msg) if code is not None else ''))
 
     def _find_txt_record_id(self, zone_id, record_name, record_content):
         """


Test patch:
diff --git a/certbot-dns-cloudflare/tests/dns_cloudflare_test.py b/certbot-dns-cloudflare/tests/dns_cloudflare_test.py
--- a/certbot-dns-cloudflare/tests/dns_cloudflare_test.py
+++ b/certbot-dns-cloudflare/tests/dns_cloudflare_test.py
@@ -12,6 +12,9 @@
 from certbot.tests import util as test_util
 
 API_ERROR = CloudFlare.exceptions.CloudFlareAPIError(1000, '', '')
+
+API_TOKEN = 'an-api-token'
+
 API_KEY = 'an-api-key'
 EMAIL = 'example@example.com'
 
@@ -49,6 +52,50 @@ def test_cleanup(self):
         expected = [mock.call.del_txt_record(DOMAIN, '_acme-challenge.'+DOMAIN, mock.ANY)]
         self.assertEqual(expected, self.mock_client.mock_calls)
 
+    def test_api_token(self):
+        dns_test_common.write({"cloudflare_api_token": API_TOKEN},
+                              self.config.cloudflare_credentials)
+        self.auth.perform([self.achall])
+
+        expected = [mock.call.add_txt_record(DOMAIN, '_acme-challenge.'+DOMAIN, mock.ANY, mock.ANY)]
+        self.assertEqual(expected, self.mock_client.mock_calls)
+
+    def test_no_creds(self):
+        dns_test_common.write({}, self.config.cloudflare_credentials)
+        self.assertRaises(errors.PluginError,
+                          self.auth.perform,
+                          [self.achall])
+
+    def test_missing_email_or_key(self):
+        dns_test_common.write({"cloudflare_api_key": API_KEY}, self.config.cloudflare_credentials)
+        self.assertRaises(errors.PluginError,
+                          self.auth.perform,
+                          [self.achall])
+
+        dns_test_common.write({"cloudflare_email": EMAIL}, self.config.cloudflare_credentials)
+        self.assertRaises(errors.PluginError,
+                          self.auth.perform,
+                          [self.achall])
+
+    def test_email_or_key_with_token(self):
+        dns_test_common.write({"cloudflare_api_token": API_TOKEN, "cloudflare_email": EMAIL},
+                              self.config.cloudflare_credentials)
+        self.assertRaises(errors.PluginError,
+                          self.auth.perform,
+                          [self.achall])
+
+        dns_test_common.write({"cloudflare_api_token": API_TOKEN, "cloudflare_api_key": API_KEY},
+                              self.config.cloudflare_credentials)
+        self.assertRaises(errors.PluginError,
+                          self.auth.perform,
+                          [self.achall])
+
+        dns_test_common.write({"cloudflare_api_token": API_TOKEN, "cloudflare_email": EMAIL,
+                               "cloudflare_api_key": API_KEY}, self.config.cloudflare_credentials)
+        self.assertRaises(errors.PluginError,
+                          self.auth.perform,
+                          [self.achall])
+
 
 class CloudflareClientTest(unittest.TestCase):
     record_name = "foo"
@@ -83,7 +130,7 @@ def test_add_txt_record(self):
     def test_add_txt_record_error(self):
         self.cf.zones.get.return_value = [{'id': self.zone_id}]
 
-        self.cf.zones.dns_records.post.side_effect = API_ERROR
+        self.cf.zones.dns_records.post.side_effect = CloudFlare.exceptions.CloudFlareAPIError(9109, '', '')
 
         self.assertRaises(
             errors.PluginError,
@@ -106,6 +153,25 @@ def test_add_txt_record_zone_not_found(self):
             self.cloudflare_client.add_txt_record,
             DOMAIN, self.record_name, self.record_content, self.record_ttl)
 
+    def test_add_txt_record_bad_creds(self):
+        self.cf.zones.get.side_effect = CloudFlare.exceptions.CloudFlareAPIError(6003, '', '')
+        self.assertRaises(
+            errors.PluginError,
+            self.cloudflare_client.add_txt_record,
+            DOMAIN, self.record_name, self.record_content, self.record_ttl)
+
+        self.cf.zones.get.side_effect = CloudFlare.exceptions.CloudFlareAPIError(9103, '', '')
+        self.assertRaises(
+            errors.PluginError,
+            self.cloudflare_client.add_txt_record,
+            DOMAIN, self.record_name, self.record_content, self.record_ttl)
+
+        self.cf.zones.get.side_effect = CloudFlare.exceptions.CloudFlareAPIError(9109, '', '')
+        self.assertRaises(
+            errors.PluginError,
+            self.cloudflare_client.add_txt_record,
+            DOMAIN, self.record_name, self.record_content, self.record_ttl)
+
     def test_del_txt_record(self):
         self.cf.zones.get.return_value = [{'id': self.zone_id}]
         self.cf.zones.dns_records.get.return_value = [{'id': self.record_id}]