Repository: pypi/warehouse

Issue:
Make tag display consistent
![tags-alt](https://cloud.githubusercontent.com/assets/3323703/11621054/3eb9941c-9cb0-11e5-8119-150e0a0648c0.png)

![tags](https://cloud.githubusercontent.com/assets/3323703/11621055/3edf45fe-9cb0-11e5-8e4e-f5c8973c44f9.png)

We need to make the output of tags consistent.  Each tag should be followed by a comma and space `,` i.e `tag1, tag2, tag3`



Discussion:
This will require some massaging of the data. Currently the tags field is just a raw text box that people can put whatever they want into. Some people do `tag,tag,tag` others do `tag, tag, tag` others do `tag tag tag` and I'm sure there are other variations as well.

How about:
1. Split the tags up with `/[, ]+/g`.
2. Split the tag icon and text into two separate columns, like:

``` html
<div class="tags">
  <div class="icon">TAG_ICON</div>
  <ul>
    <li>TAG_1</li>
    <li>TAG_2</li>
  </ul>
</div>
```

?

@shiroyuki Some tag fields have multiple-word tags separated by commas, e.g. `tagA,tagB,another tag`, this would need to be taken into consideration, as this should produce `['tagA', 'tagB', 'another tag']` and not `['tagA', 'tagB', 'another', 'tag']`

So... this means the splitter function has to be:

``` javascript
function splitTags(tagText) {
  var tags = [];
  return tagText.match(/,/)
    ? tagText.split(/\s*,\s*/g) // split with comma surrounding by any whitespaces
    : tagText.split(/\s+/g) // split with whitespace
  ;
}
```

@shiroyuki I found a few more example cases that we may want to be able to handle as well:
- Double-quoted, separated with commas: `"Record","speaking","Toefl","Rehearse","Educational","pickledb","Tkinter"`
- Double-quoted, separated with commas and spaces:
  `"consul", "backup", "yml", "key-value", "boto", "s3", "s3bucket", "bucket", "cli", "command"`
- Separated with semicolons and spaces:
  `natural language processing; thesaurus`

So probably:
- Split on comma or semicolon if present, if not, spaces;
- Strip whitespace, single quotes and double quotes from the split elements.

That sounds about right. Wow, I'm impressed. :D

On the HTML side, I'd prefer if we kept the existing tag structure and simply replace the content, returning a comma separated string.

@nlhkabu IMO, the only reason to wrap each tag in a HTML tag is so that we can control the white spaces. For example, instead of...

```
something this long
```

would not end up like

```
something <next line>
this long
```

.

In this case, each tag should be wrapped in a span with `display: inline-block` applied.

``` javascript
function stripTag(tag) {
  return tag.replace(/^["'\s]+|["'\s]+$/g, '');
}

function splitTags(tagText) {
  var tags = [];
  return tagText.match(/,/)
    ? tagText.split(/\s*,\s*/g) // split with comma surrounding by any whitespaces
    : tagText.split(/\s+/g) // split with whitespace
  ;
}

function formatTags(tagText) {
  return $.map(splitTags(tagText), stripTag);
}
```

Does this look like the right sort of thing? Or should the server split the tags (e.g. with a jinja2 filter, which could be called `format_tags`) then render them in the template? Something like:

``` html+jinja
{% if release.keywords %}
<p class="tags"><i class="fa fa-tags"></i>
  {% for keyword in release.keywords | format_tags %}
  <span class='keyword'>{{ keyword }}</span>
  {% endfor %}
</p>
{% endif %}
```

(disclaimer: this is my first comment on this project! so if I have been too unclear or done something obviously wrong please don't be afraid to tell me.)

I would prefer to have it handled inside of a jinja2 filter.

And no worries! You're doing fine :)

I think the Jinja2 approach is better. :D


Resolving patch:
diff --git a/warehouse/config.py b/warehouse/config.py
--- a/warehouse/config.py
+++ b/warehouse/config.py
@@ -236,6 +236,7 @@ def configure(settings=None):
 
     # We'll want to configure some filters for Jinja2 as well.
     filters = config.get_settings().setdefault("jinja2.filters", {})
+    filters.setdefault("format_tags", "warehouse.filters:format_tags")
     filters.setdefault("json", "warehouse.filters:tojson")
     filters.setdefault("readme", "warehouse.filters:readme")
     filters.setdefault("shorten_number", "warehouse.filters:shorten_number")
diff --git a/warehouse/filters.py b/warehouse/filters.py
--- a/warehouse/filters.py
+++ b/warehouse/filters.py
@@ -13,6 +13,7 @@
 import binascii
 import hmac
 import json
+import re
 import urllib.parse
 
 import html5lib
@@ -95,3 +96,21 @@ def tojson(value):
 
 def urlparse(value):
     return urllib.parse.urlparse(value)
+
+
+def format_tags(tags):
+    # split tags
+    if re.search(r',', tags):
+        split_tags = re.split(r'\s*,\s*', tags)
+    elif re.search(r';', tags):
+        split_tags = re.split(r'\s*;\s*', tags)
+    else:
+        split_tags = re.split(r'\s+', tags)
+
+    # strip whitespace, quotes, double quotes
+    stripped_tags = [re.sub(r'^["\'\s]+|["\'\s]+$', '', t) for t in split_tags]
+
+    # remove any empty tags
+    formatted_tags = [t for t in stripped_tags if t]
+
+    return formatted_tags


Test patch:
diff --git a/tests/unit/test_filters.py b/tests/unit/test_filters.py
--- a/tests/unit/test_filters.py
+++ b/tests/unit/test_filters.py
@@ -165,3 +165,28 @@ def test_urlparse():
     inp = "https://google.com/foo/bar?a=b"
     expected = urllib.parse.urlparse(inp)
     assert filters.urlparse(inp) == expected
+
+
+@pytest.mark.parametrize(
+    ("inp", "expected"),
+    [
+        (
+            "'python', finance, \"data\",        code    , test automation",
+            ["python", "finance", "data", "code", "test automation"]
+        ),
+        (
+            "'python'; finance; \"data\";        code    ; test automation",
+            ["python", "finance", "data", "code", "test automation"]
+        ),
+        (
+            "a \"b\" c   d  'e'",
+            ["a", "b", "c", "d", "e"]
+        ),
+        (
+            "      '    '   \"  \"",
+            []
+        )
+    ]
+)
+def test_format_tags(inp, expected):
+    assert filters.format_tags(inp) == expected