Repository: ansible/ansible

Issue:
Wrong line number reported for problems with INI-style inventory file
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
INI-style inventory

##### ANSIBLE VERSION
```
ansible 2.2.0.0
```

##### CONFIGURATION
N/A

##### OS / ENVIRONMENT
N/A

##### SUMMARY
When reporting errors in an INI-style inventory file, the reported line number is incorrect. This is because Ansible filters out lines beginning with `#` or `;` before parsing the data.

##### STEPS TO REPRODUCE
Broken inventory file:

```
# some comment
# another comment
1.2.3.4 foo ; bad line
```

Error message:

```
$ ansible -i hosts all -m ping
ERROR! Attempted to read "hosts" as YAML: 'AnsibleUnicode' object has no attribute 'keys'
Attempted to read "hosts" as ini file: hosts:1: Expected key=value host variable assignment, got: foo
```

##### EXPECTED RESULTS
Ansible should have reported the error as occurring on line 3 instead of line 1.

##### ACTUAL RESULTS
The error was reported as occurring on line 1 of the inventory file.



Discussion:
The stripping of comment lines happens in this block:

https://github.com/ansible/ansible/blob/devel/lib/ansible/inventory/ini.py#L60
This was probably "broken" by @abadger as part of https://github.com/ansible/ansible/issues/17593

Resolving patch:
diff --git a/lib/ansible/inventory/ini.py b/lib/ansible/inventory/ini.py
--- a/lib/ansible/inventory/ini.py
+++ b/lib/ansible/inventory/ini.py
@@ -38,6 +38,8 @@ class InventoryParser(object):
     Takes an INI-format inventory file and builds a list of groups and subgroups
     with their associated hosts and variable settings.
     """
+    _COMMENT_MARKERS = frozenset((u';', u'#'))
+    b_COMMENT_MARKERS = frozenset((b';', b'#'))
 
     def __init__(self, loader, groups, filename=C.DEFAULT_HOST_LIST):
         self.filename = filename
@@ -60,12 +62,19 @@ def __init__(self, loader, groups, filename=C.DEFAULT_HOST_LIST):
         try:
             # Faster to do to_text once on a long string than many
             # times on smaller strings
-            data = to_text(b_data, errors='surrogate_or_strict')
-            data = [line for line in data.splitlines() if not (line.startswith(u';') or line.startswith(u'#'))]
+            data = to_text(b_data, errors='surrogate_or_strict').splitlines()
         except UnicodeError:
-            # Skip comment lines here to avoid potential undecodable
-            # errors in comments: https://github.com/ansible/ansible/issues/17593
-            data = [to_text(line, errors='surrogate_or_strict') for line in b_data.splitlines() if not (line.startswith(b';') or line.startswith(b'#'))]
+            # Handle non-utf8 in comment lines: https://github.com/ansible/ansible/issues/17593
+            data = []
+            for line in b_data.splitlines():
+                if line and line[0] in self.b_COMMENT_MARKERS:
+                    # Replace is okay for comment lines
+                    #data.append(to_text(line, errors='surrogate_or_replace'))
+                    # Currently we only need these lines for accurate lineno in errors
+                    data.append(u'')
+                else:
+                    # Non-comment lines still have to be valid uf-8
+                    data.append(to_text(line, errors='surrogate_or_strict'))
 
         self._parse(data)
 
@@ -95,8 +104,8 @@ def _parse(self, lines):
 
             line = line.strip()
 
-            # Skip empty lines
-            if not line:
+            # Skip empty lines and comments
+            if not line or line[0] in self._COMMENT_MARKERS:
                 continue
 
             # Is this a [section] header? That tells us what group we're parsing


Test patch: