Repository: ansible/ansible-modules-core

Issue:
"authorized_key" re-orders the keys and comments in the pre-existing authorized_keys file
<!--- Verify first that your issue/request is not already reported in GitHub -->
##### ISSUE TYPE

<!--- Pick one below and delete the rest: -->
- Bug Report
##### COMPONENT NAME

<!--- Name of the plugin/module/task -->

system/authorized_key
##### ANSIBLE VERSION

<!--- Paste verbatim output from “ansible --version” between quotes below -->

```
ansible 2.1.1.0
```
##### CONFIGURATION

<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->
##### OS / ENVIRONMENT

<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say “N/A” for anything that is not platform-specific.
-->

Managing from: LinuxMint
Managing: CentOS 6
##### SUMMARY

<!--- Explain the problem briefly -->

authorized_key is re-ordering the lines in a pre-existing authorized_keys file. This causes comments and the related lines to be re-ordered.
##### STEPS TO REPRODUCE

<!---
For bugs, show exactly how to reproduce the problem.
For new features, show how the feature would be used.
-->

The managed host (B) should have a authorized_keys file containing multiple keys and comments. Example could look like this:

```
# The following is allice's key
ssh-rsa <key content-removed> ssh-key-comment-allice
# The following is bob's key
ssh-rsa <key content-removed> ssh-key-comment-bob
# The following is eve's key
ssh-rsa <key content-removed> ssh-key-comment-eve

```

After adding a key from another host (A) to the host (B) using the "authorized_key" task, below the pre-existing authorized_keys file as shown above is completely reordered.

<!--- Paste example playbooks or commands between quotes below -->

```

---
- name: Generate ssh key on host (A)
  user: name=root generate_ssh_key=yes ssh_key_bits=4096 ssh_key_file=.ssh/id_rsa ssh_key_type=rsa

- name: Download generated ssh key from host (A)
  fetch: src=/root/.ssh/id_rsa.pub dest=/tmp/ansible/{{ ansible_hostname }}_id_rsa.pub 
         flat=yes validate_checksum=yes

- name: Upload/install the ssh-key from host (A) to host (B)
  authorized_key: user=root state=present 
      key="{{ lookup('file', '/tmp/ansible/{{ ansible_hostname }}_id_rsa.pub') }}"
  delegate_to: "{{ host_B }}"
```

<!--- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS

<!--- What did you expect to happen when running the steps above? -->

```
# The following is allice's key
ssh-rsa <key content-removed> ssh-key-comment-allice
# The following is bob's key
ssh-rsa <key content-removed> ssh-key-comment-bob
# The following is eve's key
ssh-rsa <key content-removed> ssh-key-comment-eve
ssh-rsa <key content-removed> ssh-key-comment-joe
```

As shown in the example, it would be expected to see the added key (in the example "joe") at the end of the file keeping the pre-existing lines unchanged.
##### ACTUAL RESULTS

<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

The result of the reordering might look like this containing the added key for "joe". The exact reason and ordering is not known at this point as of such the following example is just an example.

```
# The following is allice's key
ssh-rsa <key content-removed> ssh-key-comment-eve
# The following is eve's key
ssh-rsa <key content-removed> ssh-key-comment-joe
# The following is bob's key
ssh-rsa <key content-removed> ssh-key-comment-allice
ssh-rsa <key content-removed> ssh-key-comment-bob

```

<!--- Paste verbatim command output between quotes below -->



Discussion:
<!-- @gerhard-tinned: Thanks for filing this bug report! -->

@ansible ping, this issue is waiting for your response.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)

<!--- boilerplate: issue_bug_new --->

bot_broke 

What Information is missing? I provided all I know and also all the fields required according to the bot help. 

The bot would mark the issue needs info if it were waiting on you. In this case the ball is in ansible's court and the bot is pinging the whole org to respond 

Sent from my iPhone

> On Sep 13, 2016, at 7:15 AM, Gerhard notifications@github.com wrote:
> 
> bot_broke
> 
> What Information is missing? I provided all I know and also all the fields required according to the bot help.
> 
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub, or mute the thread.

Is this the wrong place for this ticket? Is this a dead project? What is the expected time until someone is at least responding? Other tickets wait since more the a month....

@ansible, ping. This issue is still waiting on your response.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)

<!--- boilerplate: issue_renotify_maintainer --->

@ansible, ping. This issue is still waiting on your response.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)

<!--- boilerplate: issue_renotify_maintainer --->

I am waiting too !!!

As a workaround you should be able to use lineinfile instead of authorized_keys... The present authorized_keys code treats the file something like a database of records and so it doesn't preserve order.  I'll take a look to see if there's an easy way to enhance it or not.


Resolving patch:
diff --git a/system/authorized_key.py b/system/authorized_key.py
--- a/system/authorized_key.py
+++ b/system/authorized_key.py
@@ -149,6 +149,7 @@
 import tempfile
 import re
 import shlex
+from operator import itemgetter
 
 from ansible.module_utils.basic import AnsibleModule
 from ansible.module_utils.pycompat24 import get_exception
@@ -303,10 +304,13 @@ def parseoptions(module, options):
 
     return options_dict
 
-def parsekey(module, raw_key):
+def parsekey(module, raw_key, rank=None):
     '''
     parses a key, which may or may not contain a list
     of ssh-key options at the beginning
+
+    rank indicates the keys original ordering, so that
+    it can be written out in the same order.
     '''
 
     VALID_SSH2_KEY_TYPES = [
@@ -333,6 +337,10 @@ def parsekey(module, raw_key):
     lex.whitespace_split = True
     key_parts = list(lex)
 
+    if key_parts and key_parts[0] == '#':
+        # comment line, invalid line, etc.
+        return (raw_key, 'skipped', None, None, rank)
+
     for i in range(0, len(key_parts)):
         if key_parts[i] in VALID_SSH2_KEY_TYPES:
             type_index = i
@@ -355,7 +363,7 @@ def parsekey(module, raw_key):
     if len(key_parts) > (type_index + 1):
         comment = " ".join(key_parts[(type_index + 2):])
 
-    return (key, key_type, options, comment)
+    return (key, key_type, options, comment, rank)
 
 def readkeys(module, filename):
 
@@ -364,15 +372,15 @@ def readkeys(module, filename):
 
     keys = {}
     f = open(filename)
-    for line in f.readlines():
-        key_data = parsekey(module, line)
+    for rank_index, line in enumerate(f.readlines()):
+        key_data = parsekey(module, line, rank=rank_index)
         if key_data:
             # use key as identifier
             keys[key_data[0]] = key_data
         else:
-            # for an invalid line, just append the line
-            # to the array so it will be re-output later
-            keys[line] = line
+            # for an invalid line, just set the line
+            # dict key to the line so it will be re-output later
+            keys[line] = (line, 'skipped', None, None, rank_index)
     f.close()
     return keys
 
@@ -380,10 +388,17 @@ def writekeys(module, filename, keys):
 
     fd, tmp_path = tempfile.mkstemp('', 'tmp', os.path.dirname(filename))
     f = open(tmp_path,"w")
+
+    # FIXME: only the f.writelines() needs to be in try clause
     try:
-        for index, key in keys.items():
+        new_keys = keys.values()
+        # order the new_keys by their original ordering, via the rank item in the tuple
+        ordered_new_keys = sorted(new_keys, key=itemgetter(4))
+
+        for key in ordered_new_keys:
             try:
-                (keyhash,type,options,comment) = key
+                (keyhash, key_type, options, comment, rank) = key
+
                 option_str = ""
                 if options:
                     option_strings = []
@@ -394,7 +409,15 @@ def writekeys(module, filename, keys):
                             option_strings.append("%s=%s" % (option_key, value))
                     option_str = ",".join(option_strings)
                     option_str += " "
-                key_line = "%s%s %s %s\n" % (option_str, type, keyhash, comment)
+
+                # comment line or invalid line, just leave it
+                if not key_type:
+                    key_line = key
+
+                if key_type == 'skipped':
+                    key_line = key[0]
+                else:
+                    key_line = "%s%s %s %s\n" % (option_str, key_type, keyhash, comment)
             except:
                 key_line = key
             f.writelines(key_line)
@@ -430,43 +453,47 @@ def enforce_state(module, params):
             module.fail_json(msg=error_msg % key)
 
     # extract individual keys into an array, skipping blank lines and comments
-    key = [s for s in key.splitlines() if s and not s.startswith('#')]
+    new_keys = [s for s in key.splitlines() if s and not s.startswith('#')]
 
     # check current state -- just get the filename, don't create file
     do_write = False
     params["keyfile"] = keyfile(module, user, do_write, path, manage_dir)
     existing_keys = readkeys(module, params["keyfile"])
-
     # Add a place holder for keys that should exist in the state=present and
     # exclusive=true case
     keys_to_exist = []
 
+    # we will order any non exclusive new keys higher than all the existing keys,
+    # resulting in the new keys being written to the key file after existing keys, but
+    # in the order of new_keys
+    max_rank_of_existing_keys = len(existing_keys)
+
     # Check our new keys, if any of them exist we'll continue.
-    for new_key in key:
-        parsed_new_key = parsekey(module, new_key)
+    for rank_index, new_key in enumerate(new_keys):
+        parsed_new_key = parsekey(module, new_key, rank=rank_index)
 
         if not parsed_new_key:
             module.fail_json(msg="invalid key specified: %s" % new_key)
 
         if key_options is not None:
             parsed_options = parseoptions(module, key_options)
-            parsed_new_key = (parsed_new_key[0], parsed_new_key[1], parsed_options, parsed_new_key[3])
+            # rank here is the rank in the provided new keys, which may be unrelated to rank in existing_keys
+            parsed_new_key = (parsed_new_key[0], parsed_new_key[1], parsed_options, parsed_new_key[3], parsed_new_key[4])
 
         matched = False
         non_matching_keys = []
 
         if parsed_new_key[0] in existing_keys:
-            # Then we check if everything matches, including
+            # Then we check if everything (except the rank at index 4) matches, including
             # the key type and options. If not, we append this
             # existing key to the non-matching list
             # We only want it to match everything when the state
             # is present
-            if parsed_new_key != existing_keys[parsed_new_key[0]] and state == "present":
+            if parsed_new_key[:4] != existing_keys[parsed_new_key[0]][:4] and state == "present":
                 non_matching_keys.append(existing_keys[parsed_new_key[0]])
             else:
                 matched = True
 
-
         # handle idempotent state=present
         if state=="present":
             keys_to_exist.append(parsed_new_key[0])
@@ -476,8 +503,12 @@ def enforce_state(module, params):
                         del existing_keys[non_matching_key[0]]
                         do_write = True
 
+            # new key that didn't exist before. Where should it go in the ordering?
             if not matched:
-                existing_keys[parsed_new_key[0]] = parsed_new_key
+                # We want the new key to be after existing keys if not exclusive (rank > max_rank_of_existing_keys)
+                total_rank = max_rank_of_existing_keys + parsed_new_key[4]
+                # replace existing key tuple with new parsed key with its total rank
+                existing_keys[parsed_new_key[0]] = (parsed_new_key[0], parsed_new_key[1], parsed_new_key[2], parsed_new_key[3], total_rank)
                 do_write = True
 
         elif state=="absent":
@@ -487,6 +518,7 @@ def enforce_state(module, params):
             do_write = True
 
     # remove all other keys to honor exclusive
+    # for 'exclusive', make sure keys are written in the order the new keys were
     if state == "present" and exclusive:
         to_remove = frozenset(existing_keys).difference(keys_to_exist)
         for key in to_remove:


Test patch: