Repository: piskvorky/gensim

Issue:
Documentation of strip_punctuation vs strip_punctuation2 in gensim.parsing.preprocessing
Thanks for all the hard work on this fantastic library. I found a small quirk today, not really a bug, just a bit of a rough edge:

In `gensim.parsing` [preprocessing.py ](https://github.com/RaRe-Technologies/gensim/blob/e210f73c42c5df5a511ca27166cbc7d10970eab2/gensim/parsing/preprocessing.py#L121) `strip_punctuation2` is defined: `strip_punctuation2 = strip_punctuation`.

In the [documentation](https://radimrehurek.com/gensim/parsing/preprocessing.html) the description of [`strip_punctuation2`](https://radimrehurek.com/gensim/parsing/preprocessing.html#gensim.parsing.preprocessing.strip_punctuation2) is a duplication of [`strip_punctuation`](https://radimrehurek.com/gensim/parsing/preprocessing.html#gensim.parsing.preprocessing.strip_punctuation) rather than a statement of equality.

I noticed this while reading the documentation and, assuming I was missing an obvious distinction, attempting to hand diff the the docs for the two functions. When I gave up and flipped to the source it became obvious how the two functions are related.



Discussion:
I have no idea why we have two identical functions there. Would you be able to check the code and see which one is used where? Maybe we can keep just one? (which would solve the problem with duplicate documentation automatically)

Thanks!
To the uninformed eye on a first pass neither [strip_punctuation](https://github.com/RaRe-Technologies/gensim/search?q=strip_punctuation) nor [strip_punctuation2](https://github.com/RaRe-Technologies/gensim/search?q=strip_punctuation2) seem to be used widely within the library. Conceptually they seem to me to be on the user facing edge of the API so that is not surprising. 

Of the two, based on the linked results, `strip_punctuation` has it over `strip_punctuation2` 2:1.
Yeah looks useless. Can you open a PR to get rid of `strip_punctuation2`?
I don't know how to use any of the actual github functionality but I can try learn how to do so (assuming someone who knows doesn't deletes the line first). Thx again for the fantastic library.

Resolving patch:
diff --git a/gensim/parsing/__init__.py b/gensim/parsing/__init__.py
--- a/gensim/parsing/__init__.py
+++ b/gensim/parsing/__init__.py
@@ -1,8 +1,18 @@
 """This package contains functions to preprocess raw text"""
 
 from .porter import PorterStemmer  # noqa:F401
-from .preprocessing import (remove_stopwords, strip_punctuation, strip_punctuation2,  # noqa:F401
-                            strip_tags, strip_short, strip_numeric,
-                            strip_non_alphanum, strip_multiple_whitespaces,
-                            split_alphanum, stem_text, preprocess_string,
-                            preprocess_documents, read_file, read_files)
+from .preprocessing import (  # noqa:F401
+    preprocess_documents,
+    preprocess_string,
+    read_file,
+    read_files,
+    remove_stopwords,
+    split_alphanum,
+    stem_text,
+    strip_multiple_whitespaces,
+    strip_non_alphanum,
+    strip_numeric,
+    strip_punctuation,
+    strip_short,
+    strip_tags,
+)
diff --git a/gensim/parsing/preprocessing.py b/gensim/parsing/preprocessing.py
--- a/gensim/parsing/preprocessing.py
+++ b/gensim/parsing/preprocessing.py
@@ -118,9 +118,6 @@ def strip_punctuation(s):
     return RE_PUNCT.sub(" ", s)
 
 
-strip_punctuation2 = strip_punctuation
-
-
 def strip_tags(s):
     """Remove tags from `s` using :const:`~gensim.parsing.preprocessing.RE_TAGS`.
 


Test patch:
diff --git a/gensim/test/test_parsing.py b/gensim/test/test_parsing.py
--- a/gensim/test/test_parsing.py
+++ b/gensim/test/test_parsing.py
@@ -8,9 +8,18 @@
 import logging
 import unittest
 import numpy as np
-from gensim.parsing.preprocessing import \
-    remove_stopwords, strip_punctuation2, strip_tags, strip_short, strip_numeric, strip_non_alphanum, \
-    strip_multiple_whitespaces, split_alphanum, stem_text
+
+from gensim.parsing.preprocessing import (
+    remove_stopwords,
+    stem_text,
+    split_alphanum,
+    strip_multiple_whitespaces,
+    strip_non_alphanum,
+    strip_numeric,
+    strip_punctuation,
+    strip_short,
+    strip_tags,
+)
 
 
 # several documents
@@ -38,7 +47,7 @@
 for many searching purposes, a little fuzziness would help. """
 
 
-dataset = [strip_punctuation2(x.lower()) for x in [doc1, doc2, doc3, doc4]]
+dataset = [strip_punctuation(x.lower()) for x in [doc1, doc2, doc3, doc4]]
 # doc1 and doc2 have class 0, doc3 and doc4 avec class 1
 classes = np.array([[1, 0], [1, 0], [0, 1], [0, 1]])