Repository: nltk/nltk

Issue:
Adding Good turing and Add -n Smoothing
Apart from the existing 2 smoothing algorithms already present, I think the add -n smoothing and the Good turing smoothing can be added


Discussion:
Hmm, by "add-n smoothing" do you mean Lidstone/Laplace? or absolute discounting (of which Kneser-Ney is a particular case)?

The former is already present (should be importable...). The latter you are welcome to add, just like good-turing.

I personally don't have the time in the foreseeable future to do this. Unfortunately.

I'll gladly help review submissions!
Thanks @Copper-Head Will shortly create a pull request for that 
@Copper-Head For the GoodTuring Smoothing, i think we need to add a function which counts the no of n-grams which have a particular count. Should I add it to the smoothing function, or should I can I add it to the Language Model object itself, as a generalized func
 
I would say keep it more localized to the GT model, wouldn't wanna burden the LM object with too much functionality. If at some point we need to reuse it, we can consider factoring it out somehow


Resolving patch:
diff --git a/nltk/lm/api.py b/nltk/lm/api.py
--- a/nltk/lm/api.py
+++ b/nltk/lm/api.py
@@ -45,6 +45,8 @@ class Smoothing(object):
     Implements Chen & Goodman 1995's idea that all smoothing algorithms have
     certain features in common. This should ideally allow smoothing algoritms to
     work both with Backoff and Interpolation.
+
+    counter represents the number of counts for ngrams
     """
 
     def __init__(self, vocabulary, counter):
diff --git a/nltk/lm/smoothing.py b/nltk/lm/smoothing.py
--- a/nltk/lm/smoothing.py
+++ b/nltk/lm/smoothing.py
@@ -21,6 +21,10 @@ def _count_non_zero_vals(dictionary):
 class WittenBell(Smoothing):
     """Witten-Bell smoothing."""
 
+    def __init__(self, vocabulary, counter, discount=0.1, **kwargs):
+        super(WittenBell, self).__init__(vocabulary, counter, *kwargs)
+        self.counts = counter
+
     def alpha_gamma(self, word, context):
         gamma = self.gamma(context)
         return (1.0 - gamma) * self.alpha(word, context), gamma
@@ -42,9 +46,10 @@ class KneserNey(Smoothing):
     def __init__(self, vocabulary, counter, discount=0.1, **kwargs):
         super(KneserNey, self).__init__(vocabulary, counter, *kwargs)
         self.discount = discount
+        self.vocabulary = vocabulary
 
     def unigram_score(self, word):
-        return 1.0 / len(self.vocab)
+        return 1.0 / len(self.vocabulary)
 
     def alpha_gamma(self, word, context):
         prefix_counts = self.counts[context]
@@ -55,3 +60,19 @@ def alpha(self, word, prefix_counts):
 
     def gamma(self, prefix_counts):
         return self.discount * _count_non_zero_vals(prefix_counts) / prefix_counts.N()
+
+
+class GoodTuring(Smoothing):
+    """Good-Turing Smoothing"""
+    def __init__(self, vocabulary, counter, **kwargs):
+        super(GoodTuring, self).__init__(vocabulary, counter, *kwargs)
+        self.counts = counter
+        self.vocabulary = vocabulary
+
+    def unigram_score(self, word):
+        word_count = self.counts[word]
+        count_plus_1 = 0.
+        for everyContext in self.counts.keys():
+            if len(everyContext.split()) == word_count+1:
+                count_plus_1 += 1
+        return count_plus_1 / len(self.vocabulary)


Test patch: