Issue 1:
From @water4u99 on 2016-11-18T22:28:43Z

ISSUE TYPE

Bug Report

COMPONENT NAME

template

ANSIBLE VERSION

2.0 and higher

CONFIGURATION

[ssh_connection]

control_path = %(directory)s/%%C

OS / ENVIRONMENT

Mac OS X 10.11.6

Centos 6.x, 7.x

SUMMARY

In the input .j2 file, we substitute a variable with an environment variable that has a line/string that contains a grok expression containing "(?m)\n" . The output generated by the template module in versions 2.0 and later, treats the \n as actual line break. Where as versions up to 1.9.6 retains the literal "(?m)\n" without replacing the \n with an actual line break. We see the line break after we upgraded the Ansible version to 2.x.

Any way we can work around this issue? Thank you for your help.

STEPS TO REPRODUCE

Our execution flow is probably not the nicest - we want to reengineer it soon. Basic steps:

Run a shell script with ansible-playbook command that pass in an env variable with (?m)\n literal.

Playbook calls a main yaml file and assigns shell environment var to a included task yaml file.

The task yaml file invokes the template module.

In the snippet below I stripped out other lines/vars for clarity.

main shell

set GROK_PATTERN_GENERAL_ERROR_PG="%{TIMESTAMP_ISO8601} ERROR \[%{USER:handlerName}\] %{USER:className}%{GREEDYDATA:errorline1}((?m)\n%{USER:logerror}%{GREEDYDATA})"

ansible-playbook -i ../common/host.inventory \
-${VERBOSE} \
t.yml \
${CHECK_ONLY} \
--extra-vars "hosts='${HOST}'
xlogstash_grok_general_error='${GROK_PATTERN_GENERAL_ERROR_PG}'
"

t.yml

---
- hosts: 127.0.0.1
connection: local

tasks:
- include_vars: ../common/defaults/main.yml
- name: generate logstash kafka logscan filter config file
include: tasks/t.yml
vars:
logstash_grok_general_error: "{{xlogstash_grok_general_error}}"

tasks/t.yml

---
- name: generate logstash kafka logscan filter config file
template: src=../common/templates/my.conf.j2
dest="./500-filter.conf"

my.conf.j2

grok {
break_on_match => "true"
match => [
"message", "{{logstash_grok_general_error}}"
]
}

Note the (?m)\n are still on the same line.

EXPECTED RESULTS

grok {
break_on_match => "true"
match => [
"message", "%{TIMESTAMP_ISO8601} ERROR \[%{USER:handlerName}\] %{USER:className}%{GREEDYDATA:errorline1}((?m)\n%{USER:logerror}%{GREEDYDATA})"
]
}

ACTUAL RESULTS

Note (?m)\n now has the \n as actual line break.

grok {
break_on_match => "true"
match => [
"message", "%{TIMESTAMP_ISO8601} ERROR \[%{USER:handlerName}\] %{USER:className}%{GREEDYDATA:errorline1}((?m)
%{USER:logerror}%{GREEDYDATA})"
]
}

Copied from original issue: ansible/ansible-modules-core#5674

Issue 2:
ISSUE TYPE

Bug Report

COMPONENT NAME

template

ANSIBLE VERSION

2.0 and higher

CONFIGURATION

[ssh_connection]
control_path = %(directory)s/%%C

OS / ENVIRONMENT

Mac OS X 10.11.6

Centos 6.x, 7.x

SUMMARY

In the input .j2 file, we substitute a variable with an environment variable that has a line/string that contains a grok expression containing (?m)\n . The output generated by the template module in versions 2.0 and later, treats the \n as actual line break. Where as versions up to 1.9.6 retains the literal (?m)\n without replacing the \n with an actual line break. We see the line break after we upgraded the Ansible version to 2.x.

Any way we can work around this issue? Thank you for your help.

STEPS TO REPRODUCE

Our execution flow is probably not the nicest - we want to reengineer it soon. Basic steps:

Run a shell script with ansible-playbook command that pass in an env variable with `(?m)\n` literal.
Playbook calls a main yaml file and assigns shell environment var to a included task yaml file.
The task yaml file invokes the template module.

In the snippet below I stripped out other lines/vars for clarity.

main shell

set GROK_PATTERN_GENERAL_ERROR_PG="%{TIMESTAMP_ISO8601} ERROR \[%{USER:handlerName}\] %{USER:className}%{GREEDYDATA:errorline1}((?m)\n%{USER:logerror}%{GREEDYDATA})"

ansible-playbook -i ../common/host.inventory \
-${VERBOSE} \
t.yml \
${CHECK_ONLY} \
--extra-vars "hosts='${HOST}'
xlogstash_grok_general_error='${GROK_PATTERN_GENERAL_ERROR_PG}'
"

t.yml

---
- hosts: 127.0.0.1
connection: local

tasks:
- include_vars: ../common/defaults/main.yml
- name: generate logstash kafka logscan filter config file
include: tasks/t.yml
vars:
logstash_grok_general_error: "{{xlogstash_grok_general_error}}"

tasks/t.yml

---
- name: generate logstash kafka logscan filter config file
template: src=../common/templates/my.conf.j2
dest="./500-filter.conf"

my.conf.j2

grok {
break_on_match => "true"
match => [
"message", "{{logstash_grok_general_error}}"
]
}

Note the (?m)\n are still on the same line.

EXPECTED RESULTS

grok {
break_on_match => "true"
match => [
"message", "%{TIMESTAMP_ISO8601} ERROR \[%{USER:handlerName}\] %{USER:className}%{GREEDYDATA:errorline1}((?m)\n%{USER:logerror}%{GREEDYDATA})"
]
}

ACTUAL RESULTS

Note (?m)\n now has the \n as actual line break.

grok {
break_on_match => "true"
match => [
"message", "%{TIMESTAMP_ISO8601} ERROR \[%{USER:handlerName}\] %{USER:className}%{GREEDYDATA:errorline1}((?m)
%{USER:logerror}%{GREEDYDATA})"
]
}