Repository: ansible/ansible

Issue:
lineinfile regexp empty string replaces last line instead of inserting
From @rfwekker on 2016-12-05T14:38:56Z
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
lineinfile module
http://docs.ansible.com/ansible/lineinfile_module.html

##### ANSIBLE VERSION
ansible 2.1.0.0

##### ENVIRONMENT
Ansible Configuration: default
CentOS Linux release 7.2.1511 (Core) 3.10.0-327.36.1.el7.x86_64 

##### SUMMARY
BUG: The lastline is deleted and replaced, with lineinfile insertafter=EOF
Want to add a new string to the end of the file, but the result is that the lastline is deleted and replaced. Replace last line instead of insert after last line. 


##### STEPS TO REPRODUCE
/etc/crontab

```
*/45 *  *  *  *  root /sbin/blabla
*/45 *  *  *  *  root /sbin/tratra
```


Playbook section:

```yaml
 - name: add at the end of the file /etc/crontab rhn_check
   lineinfile: dest=/etc/crontab
               regexp=''
               insertafter=EOF
               line='*/60 *  *  *  *  root /sbin/rhn_chk'
```

##### EXPECTED RESULTS
Line added to the crontab

```
*/45 *  *  *  *  root /sbin/blabla
*/45 *  *  *  *  root /sbin/tratra
*/60 *  *  *  *  root /sbin/rhn_chk
```


##### ACTUAL RESULTS
/etc/crontab

```
*/45 *  *  *  *  root /sbin/blabla
*/60 *  *  *  *  root /sbin/rhn_chk
```
Last line in the file deleted and replaced

Copied from original issue: ansible/ansible-modules-core#5825



Discussion:
From @ansibot on 2016-12-05T14:38:56Z

@rfwekker, we need more information to resolve this issue. For that reason, we put this Issue into the 'needs_info' state.

Here are the required items we could not find in your description:
- issue type
 
When you have filled in the missing data, we will notify the module maintainer for further action.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: issue_needs_info --->
From @ioggstream on 2016-12-05T14:38:56Z

@rfwekker iiuc:

- if you don't want the last line to be replaced, you shouldn't use `regexp` there.

From the module documentation:
```
For state=present, the pattern to replace if found; only the last line found will be replaced
```

Let us know if this
```
  - name: add at the end of the file /etc/crontab rhn_check
    lineinfile: dest=/tmp/foo.txt
                insertafter=EOF
                line='*/60 *  *  *  *  root /sbin/rhn_chk'
```

or the `cron` module works http://docs.ansible.com/ansible/cron_module.html

Peace,
R:
cc @ahtik @dhozac
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
cc @samdoran
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
I am able to duplicate this in `devel`. The issue is not EOF but `regexp: ''`. If I omit `regexp: ''` from the reproducer, the issue goes away.

I need to determine if this is a bug of a byproduct of matching on an empty string `regexp` or both.
The tests in the code were testing against `None`. Since an empty string is different than `None`, it believed a regexp was defined and compiled a regexp object. Since an empty string regexp matches every line, the regexp would match the last line in the file and replace it with the line specified in the module.

Resolving patch:
diff --git a/lib/ansible/modules/files/lineinfile.py b/lib/ansible/modules/files/lineinfile.py
--- a/lib/ansible/modules/files/lineinfile.py
+++ b/lib/ansible/modules/files/lineinfile.py
@@ -193,9 +193,8 @@
 def write_changes(module, b_lines, dest):
 
     tmpfd, tmpfile = tempfile.mkstemp()
-    f = os.fdopen(tmpfd, 'wb')
-    f.writelines(b_lines)
-    f.close()
+    with open(tmpfile, 'wb') as f:
+        f.writelines(b_lines)
 
     validate = module.params.get('validate', None)
     valid = not validate
@@ -247,14 +246,13 @@ def present(module, dest, regexp, line, insertafter, insertbefore, create,
 
         b_lines = []
     else:
-        f = open(b_dest, 'rb')
-        b_lines = f.readlines()
-        f.close()
+        with open(b_dest, 'rb') as f:
+            b_lines = f.readlines()
 
     if module._diff:
         diff['before'] = to_native(b('').join(b_lines))
 
-    if regexp is not None:
+    if regexp:
         bre_m = re.compile(to_bytes(regexp, errors='surrogate_or_strict'))
 
     if insertafter not in (None, 'BOF', 'EOF'):
@@ -270,7 +268,7 @@ def present(module, dest, regexp, line, insertafter, insertbefore, create,
     m = None
     b_line = to_bytes(line, errors='surrogate_or_strict')
     for lineno, b_cur_line in enumerate(b_lines):
-        if regexp is not None:
+        if regexp:
             match_found = bre_m.search(b_cur_line)
         else:
             match_found = b_line == b_cur_line.rstrip(b('\r\n'))
@@ -406,21 +404,20 @@ def absent(module, dest, regexp, line, backup):
             'before_header': '%s (content)' % dest,
             'after_header': '%s (content)' % dest}
 
-    f = open(b_dest, 'rb')
-    b_lines = f.readlines()
-    f.close()
+    with open(b_dest, 'rb') as f:
+        b_lines = f.readlines()
 
     if module._diff:
         diff['before'] = to_native(b('').join(b_lines))
 
-    if regexp is not None:
+    if regexp:
         bre_c = re.compile(to_bytes(regexp, errors='surrogate_or_strict'))
     found = []
 
     b_line = to_bytes(line, errors='surrogate_or_strict')
 
     def matcher(b_cur_line):
-        if regexp is not None:
+        if regexp:
             match_found = bre_c.search(b_cur_line)
         else:
             match_found = b_line == b_cur_line.rstrip(b('\r\n'))
@@ -480,13 +477,15 @@ def main():
     backrefs = params['backrefs']
     path = params['path']
     firstmatch = params['firstmatch']
+    regexp = params['regexp']
+    line = params.get('line', None)
 
     b_path = to_bytes(path, errors='surrogate_or_strict')
     if os.path.isdir(b_path):
         module.fail_json(rc=256, msg='Path %s is a directory !' % path)
 
     if params['state'] == 'present':
-        if backrefs and params['regexp'] is None:
+        if backrefs and not regexp:
             module.fail_json(msg='regexp= is required with backrefs=true')
 
         if params.get('line', None) is None:
@@ -500,13 +499,13 @@ def main():
 
         line = params['line']
 
-        present(module, path, params['regexp'], line,
+        present(module, path, regexp, line,
                 ins_aft, ins_bef, create, backup, backrefs, firstmatch)
     else:
-        if params['regexp'] is None and params.get('line', None) is None:
+        if not regexp and line is None:
             module.fail_json(msg='one of line= or regexp= is required with state=absent')
 
-        absent(module, path, params['regexp'], params.get('line', None), backup)
+        absent(module, path, regexp, line, backup)
 
 
 if __name__ == '__main__':


Test patch:
diff --git a/test/integration/targets/lineinfile/tasks/main.yml b/test/integration/targets/lineinfile/tasks/main.yml
--- a/test/integration/targets/lineinfile/tasks/main.yml
+++ b/test/integration/targets/lineinfile/tasks/main.yml
@@ -717,3 +717,37 @@
   assert:
     that:
       - result.stat.checksum == 'eca8d8ea089d4ea57a3b87d4091599ca8b60dfd2'
+
+###################################################################
+# Issue 29443
+
+- name: Deploy the test file for lineinfile
+  copy:
+    src: test.txt
+    dest: "{{ output_dir }}/test.txt"
+  register: result
+
+- name: Assert that the test file was deployed
+  assert:
+    that:
+      - result is changed
+      - result.checksum == '5feac65e442c91f557fc90069ce6efc4d346ab51'
+      - result.state == 'file'
+
+- name: Insert a line in the file using an empty string as a regular expression
+  lineinfile:
+    path: "{{ output_dir }}/test.txt"
+    regexp: ''
+    line: This is line 6
+  register: insert_empty_regexp
+
+- name: Stat the file
+  stat:
+    path: "{{ output_dir }}/test.txt"
+  register: result
+
+- name: Assert that the file contents match what is expected
+  assert:
+    that:
+      - insert_empty_regexp is changed
+      - result.stat.checksum == '3baeade8eb2ecf4b01d70d541e9b8258b67c7f9f'
\ No newline at end of file