Repository: fail2ban/fail2ban

Issue:
ignoreip does not handle multiline values
```
==> /var/log/fail2ban.log <==
2016-05-19 02:00:29,517 fail2ban.filterpyinotify[28253]: DEBUG   Default Callback for Event: <Event dir=False mask=0x2 maskname=IN_MODIFY name='' path=/var/log/mail.log pathname=/var/log/mail.log wd=2 >
2016-05-19 02:00:29,518 fail2ban.datedetector   [28253]: DEBUG   Matched time template (?:DAY )?MON Day 24hour:Minute:Second(?:\.Microseconds)?(?: Year)?
2016-05-19 02:00:29,518 fail2ban.datedetector   [28253]: DEBUG   Got time 1463623229.000000 for "'May 19 02:00:29'" using template (?:DAY )?MON Day 24hour:Minute:Second(?:\.Microseconds)?(?: Year)?
2016-05-19 02:00:29,518 fail2ban.filter         [28253]: DEBUG   Processing line with time:1463623229.0 and ip:81.2.237.54
2016-05-19 02:00:29,518 fail2ban.filter         [28253]: INFO    [courier-smtp] Ignore 81.2.237.54 by ip
2016-05-19 02:00:29,521 fail2ban.datedetector   [28253]: DEBUG   Sorting the template list
2016-05-19 02:00:29,522 fail2ban.datedetector   [28253]: DEBUG   Winning template: (?:DAY )?MON Day 24hour:Minute:Second(?:\.Microseconds)?(?: Year)? with 2678 hits

# fail2ban-client get courier-smtp ignoreip
These IP addresses/networks are ignored:
`- 127.0.0.0/8
89.40.125.242
66.249.64.0/19
```

v0.9.4 backported to Debian jessie.

Could you help?



Discussion:
Is there a way to debug the ignore procedure?

Both `a` and `b` are zero:
https://github.com/fail2ban/fail2ban/blob/master/fail2ban/server/filter.py#L374-L375
IP-s are 81.2.237.54 and 127.0.0.0

Now it is OK:

```
# fail2ban-client get courier-smtp ignoreip
These IP addresses/networks are ignored:
|- 127.0.0.0/8
|- 89.40.125.242
`- 66.249.64.0/19
```

Previously the 3 addresses were in separate lines and they were handled as _one_ address.

Please support multi-line values for `ignoreip`.

I usually separate it like this

``` python
#          localhost
#          own IP
#          Googlebot
ignoreip = 127.0.0.0/8
           89.40.125.242
           66.249.64.0/19
```

On Wed, 18 May 2016, Viktor Szépe wrote:

>    Both a and b are zero:
>    https://github.com/fail2ban/fail2ban/blob/master/fail2ban/server/filter.py#L374-L375

and the values of i, ip, s?

I've found out that in case of a multiline value `ignoreip` gets this **one** value: (not 3)

```
ignoreip="127.0.0.0/8
89.40.125.242
66.249.64.0/19"
```

I think it is a bug.

On Wed, 18 May 2016, Viktor Szépe wrote:

>    Now it is OK:
> 
>  # fail2ban-client get courier-smtp ignoreip
>  These IP addresses/networks are ignored:
>  |- 127.0.0.0/8
>  |- 89.40.125.242
>  `- 66.249.64.0/19
> 
>    Previously the 3 addresses were in separate lines and they were handled as
>    one address.
> 
>    Please support multiline value for ignoreip.

I would say we better add support to crash if 'ignoreip' value
doesn't look like an IP address/mask at all, and then add support for
multiline ;)


Resolving patch:
diff --git a/fail2ban/client/jailreader.py b/fail2ban/client/jailreader.py
--- a/fail2ban/client/jailreader.py
+++ b/fail2ban/client/jailreader.py
@@ -34,7 +34,7 @@
 from .actionreader import ActionReader
 from ..version import version
 from ..helpers import getLogger
-from ..helpers import splitcommaspace
+from ..helpers import splitwords
 
 # Gets the instance of the logger.
 logSys = getLogger(__name__)
@@ -214,7 +214,7 @@ def convert(self, allow_no_files=False):
 			elif opt == "maxretry":
 				stream.append(["set", self.__name, "maxretry", self.__opts[opt]])
 			elif opt == "ignoreip":
-				for ip in splitcommaspace(self.__opts[opt]):
+				for ip in splitwords(self.__opts[opt]):
 					stream.append(["set", self.__name, "addignoreip", ip])
 			elif opt == "findtime":
 				stream.append(["set", self.__name, "findtime", self.__opts[opt]])
diff --git a/fail2ban/helpers.py b/fail2ban/helpers.py
--- a/fail2ban/helpers.py
+++ b/fail2ban/helpers.py
@@ -128,12 +128,12 @@ def excepthook(exctype, value, traceback):
 		"Unhandled exception in Fail2Ban:", exc_info=True)
 	return sys.__excepthook__(exctype, value, traceback)
 
-def splitcommaspace(s):
-	"""Helper to split on any comma or space
+def splitwords(s):
+	"""Helper to split words on any comma, space, or a new line
 
 	Returns empty list if input is empty (or None) and filters
 	out empty entries
 	"""
 	if not s:
 		return []
-	return filter(bool, re.split('[ ,]', s))
+	return filter(bool, map(str.strip, re.split('[ ,\n]', s)))


Test patch:
diff --git a/fail2ban/tests/misctestcase.py b/fail2ban/tests/misctestcase.py
--- a/fail2ban/tests/misctestcase.py
+++ b/fail2ban/tests/misctestcase.py
@@ -33,7 +33,7 @@
 from StringIO import StringIO
 
 from ..helpers import formatExceptionInfo, mbasename, TraceBack, FormatterWithTraceBack, getLogger
-from ..helpers import splitcommaspace
+from ..helpers import splitwords
 from ..server.datetemplate import DatePatternRegex
 
 
@@ -56,13 +56,15 @@ def testFormatExceptionConvertArgs(self):
 			# might be fragile due to ' vs "
 			self.assertEqual(args, "('Very bad', None)")
 
-	def testsplitcommaspace(self):
-		self.assertEqual(splitcommaspace(None), [])
-		self.assertEqual(splitcommaspace(''), [])
-		self.assertEqual(splitcommaspace('  '), [])
-		self.assertEqual(splitcommaspace('1'), ['1'])
-		self.assertEqual(splitcommaspace(' 1 2 '), ['1', '2'])
-		self.assertEqual(splitcommaspace(' 1, 2 , '), ['1', '2'])
+	def testsplitwords(self):
+		self.assertEqual(splitwords(None), [])
+		self.assertEqual(splitwords(''), [])
+		self.assertEqual(splitwords('  '), [])
+		self.assertEqual(splitwords('1'), ['1'])
+		self.assertEqual(splitwords(' 1 2 '), ['1', '2'])
+		self.assertEqual(splitwords(' 1, 2 , '), ['1', '2'])
+		self.assertEqual(splitwords(' 1\n  2'), ['1', '2'])
+		self.assertEqual(splitwords(' 1\n  2, 3'), ['1', '2', '3'])
 
 
 class SetupTest(unittest.TestCase):