Repository: fedora-infra/bodhi

Issue:
Update.has_negative_karma() counts karma from before new or removed builds
Bodhi's karma counting code resets the karma to 0 when builds are added or removed from an Update, and [a user's karma from before the reset is not supposed to be counted](https://github.com/fedora-infra/bodhi/blob/2.2.4/bodhi/server/models/models.py#L1426-L1430):

```
            # Take all comments since the previous karma reset
            reset_index = 0
            for i, comment in enumerate(self.comments):
                if (comment.user.name == u'bodhi' and ('New build' in
                    comment.text or 'Removed build' in comment.text)):
                        reset_index = i
```

Due to #1018 this code doesn't work quite as intended, but it also occurred to me that [`Update.has_negative_karma()`](https://github.com/fedora-infra/bodhi/blob/f515793beaf5d35d3918c808edd9dd153eeba2b0/bodhi/server/models/models.py#L1739-L1748) does count karma that was given before karma reset events:

```
    def has_negative_karma(self):
        """Check for negative karma, Returns True if the update has negative karma"""
        feedback = defaultdict(int)
        for comment in self.comments:
            if not comment.anonymous:
                feedback[comment.user.name] = comment.karma
        for karma in feedback.values():
            if karma < 0:
                return True
        return False
```

This means that critical path updates that add or remove builds after receiving negative karma will still automatically get disabled when receiving new karma comments after the reset event because the old negative karma is still affecting the test. The old karma will not contribute to the Update's karma, however.

This is an edge case scenario, and fails in a "safe" way because the consequence is simply that autopush will get disabled for the Update. Therefore, I consider this a low priority issue.



Discussion:
I am adding a new `karma()` method that counts the karma correctly in order to solve #1018. I think I will create a `karma_details()` method while I'm in there that returns the good and bad karma counts separately. Then `karma()` can just sum the two and `has_negative_karma()` can be removed in favor of just comparing the bad karma count returned by `karma_details()`. I'm killing three birds with this stone ☺


Resolving patch:
diff --git a/alembic/versions/06aa0e8aa5d2_drop_the_update_karma_column.py b/alembic/versions/06aa0e8aa5d2_drop_the_update_karma_column.py
new file mode 100644
--- /dev/null
+++ b/alembic/versions/06aa0e8aa5d2_drop_the_update_karma_column.py
@@ -0,0 +1,30 @@
+"""Drop the Update.karma column.
+
+Revision ID: 06aa0e8aa5d2
+Revises: 3cde3882442a
+Create Date: 2016-10-26 16:55:54.875994
+
+"""
+from alembic import op
+
+
+# revision identifiers, used by Alembic.
+revision = '06aa0e8aa5d2'
+down_revision = '3cde3882442a'
+
+
+def upgrade():
+    """
+    Drop the karma column from the updates table.
+    """
+    op.drop_column('updates', 'karma')
+
+
+def downgrade():
+    """
+    Downgrade is not supported. If we ever do want to do this for some reason, we can use the
+    code from the bodhi.server.models.models.Update.karma() property that was written in the same
+    commit that introduced this migration as a guide for how to calculate the karma column. As it is
+    highly unlikely that will ever be needed, this function simply raises an Exception for now.
+    """
+    raise NotImplemented('Downgrade not supported')
diff --git a/bodhi/server/models/models.py b/bodhi/server/models/models.py
--- a/bodhi/server/models/models.py
+++ b/bodhi/server/models/models.py
@@ -603,7 +603,6 @@ class Update(Base):
 
     title = Column(UnicodeText, unique=True, default=None, index=True)
 
-    karma = Column(Integer, default=0)
     autokarma = Column(Boolean, default=True)
     stable_karma = Column(Integer, nullable=True)
     unstable_karma = Column(Integer, nullable=True)
@@ -660,6 +659,51 @@ class Update(Base):
 
     user_id = Column(Integer, ForeignKey('users.id'))
 
+    @property
+    def karma(self):
+        """
+        Calculate and return the karma for the Update.
+
+        :return: The Update's current karma.
+        :rtype:  int
+        """
+        positive_karma, negative_karma = self._composite_karma
+        return positive_karma + negative_karma
+
+    @property
+    def _composite_karma(self):
+        """
+        Calculate and return a 2-tuple of the sum of the positive karma comments, and the sum of the
+        negative karma comments. The total karma is simply the sum of the two elements of this
+        2-tuple.
+
+        :return: a 2-tuple of (positive_karma, negative_karma)
+        :rtype:  tuple
+        """
+        positive_karma = 0
+        negative_karma = 0
+        users_counted = set()
+        # We want to traverse the comments in reverse order so we only consider the most recent
+        # comments from any given user, and only the comments since the most recent karma reset
+        # event.
+        for comment in reversed(self.comments):
+            if (comment.user.name == u'bodhi' and
+                    ('New build' in comment.text or 'Removed build' in comment.text)):
+                # We only want to consider comments since the most recent karma reset, which happens
+                # whenever a build is added or removed from an Update. Since we are traversing the
+                # comments in reverse order, once we find one of these comments we can simply exit
+                # this loop.
+                break
+            if not comment.anonymous and comment.user.name not in users_counted:
+                # Make sure we only count the last comment this user made
+                users_counted.add(comment.user.name)
+                if comment.karma > 0:
+                    positive_karma += comment.karma
+                else:
+                    negative_karma += comment.karma
+
+        return positive_karma, negative_karma
+
     @classmethod
     def new(cls, request, data):
         """ Create a new update """
@@ -838,7 +882,6 @@ def edit(cls, request, data):
         # And, updates with new or removed builds always get their karma reset.
         # https://github.com/fedora-infra/bodhi/issues/511
         if new_builds or removed_builds:
-            data['karma'] = 0
             data['karma_critpath'] = 0
 
         new_bugs = up.update_bugs(data['bugs'], db)
@@ -1419,11 +1462,9 @@ def obsolete_if_unstable(self, db):
     def comment(self, session, text, karma=0, author=None, anonymous=False,
                 karma_critpath=0, bug_feedback=None, testcase_feedback=None,
                 check_karma=True):
-        """ Add a comment to this update, adjusting the karma appropriately.
+        """Add a comment to this update.
 
-        Each user has the ability to comment as much as they want, but only
-        their last karma adjustment will be counted.  If the karma reaches
-        the 'stable_karma' value, then request that this update be marked
+        If the karma reaches the 'stable_karma' value, then request that this update be marked
         as stable.  If it reaches the 'unstable_karma', it is unpushed.
         """
         if not author:
@@ -1441,56 +1482,59 @@ def comment(self, session, text, karma=0, author=None, anonymous=False,
                 notice = 'You may not give karma to your own updates.'
                 caveats.append({'name': 'karma', 'description': notice})
 
+        comment = Comment(
+            text=text, anonymous=anonymous,
+            karma=karma, karma_critpath=karma_critpath)
+        session.add(comment)
+
+        if anonymous:
+            author = u'anonymous'
+        try:
+            user = session.query(User).filter_by(name=author).one()
+        except NoResultFound:
+            user = User(name=author)
+            session.add(user)
+
+        user.comments.append(comment)
+        self.comments.append(comment)
+        session.flush()
+
         if not anonymous and karma != 0:
-            # Take all comments since the previous karma reset
-            reset_index = 0
-            for i, comment in enumerate(self.comments):
-                if (comment.user.name == u'bodhi' and
-                        ('New build' in comment.text or 'Removed build' in comment.text)):
-                    reset_index = i
-
-            mycomments = [c.karma for c in self.comments if c.user.name == author][reset_index:]
-            if karma not in mycomments:
-                if karma == 1 and -1 in mycomments:
-                    self.karma += 2
-                    caveats.append({
-                        'name': 'karma',
-                        'description': 'Your karma standing was reversed.',
-                    })
-                elif karma == -1 and 1 in mycomments:
-                    self.karma -= 2
+            # Determine whether this user has already left karma, and if so what the most recent
+            # karma value they left was. We should examine all but the most recent comment, since
+            # that is the comment we just added.
+            previous_karma = None
+            for c in reversed(self.comments[:-1]):
+                if c.user.name == author and c.karma:
+                    previous_karma = c.karma
+                    break
+            if previous_karma and karma != previous_karma:
+                caveats.append({
+                    'name': 'karma',
+                    'description': 'Your karma standing was reversed.',
+                })
+            else:
+                log.debug('Ignoring duplicate %d karma from %s on %s' % (karma, author, self.title))
+
+            log.info("Updated %s karma to %d" % (self.title, self.karma))
+
+            if check_karma and author not in config.get('system_users').split():
+                try:
+                    self.check_karma_thresholds(session, 'bodhi')
+                except LockedUpdateException:
+                    pass
+                except BodhiException as e:
+                    # This gets thrown if the karma is pushed over the
+                    # threshold, but it is a critpath update that is not
+                    # critpath_approved. ... among other cases.
+                    log.exception('Problem checking the karma threshold.')
                     caveats.append({
-                        'name': 'karma',
-                        'description': 'Your karma standing was reversed.',
+                        'name': 'karma', 'description': str(e),
                     })
-                else:
-                    self.karma += karma
-
-                log.info("Updated %s karma to %d" % (self.title, self.karma))
-
-                if check_karma and author not in config.get('system_users').split():
-                    try:
-                        self.check_karma_thresholds(session, 'bodhi')
-                    except LockedUpdateException:
-                        pass
-                    except BodhiException as e:
-                        # This gets thrown if the karma is pushed over the
-                        # threshold, but it is a critpath update that is not
-                        # critpath_approved. ... among other cases.
-                        log.exception('Problem checking the karma threshold.')
-                        caveats.append({
-                            'name': 'karma', 'description': str(e),
-                        })
 
-                # Obsolete pending update if it reaches unstable karma threshold
-                self.obsolete_if_unstable(session)
-            else:
-                log.debug('Ignoring duplicate %d karma from %s on %s' % (karma, author, self.title))
+            # Obsolete pending update if it reaches unstable karma threshold
+            self.obsolete_if_unstable(session)
 
-        comment = Comment(
-            text=text, anonymous=anonymous,
-            karma=karma, karma_critpath=karma_critpath)
-        session.add(comment)
         session.flush()
 
         for feedback_dict in bug_feedback:
@@ -1505,19 +1549,6 @@ def comment(self, session, text, karma=0, author=None, anonymous=False,
 
         session.flush()
 
-        if anonymous:
-            author = u'anonymous'
-        try:
-            user = session.query(User).filter_by(name=author).one()
-        except NoResultFound:
-            user = User(name=author)
-            session.add(user)
-            session.flush()
-
-        user.comments.append(comment)
-        self.comments.append(comment)
-        session.flush()
-
         # Publish to fedmsg
         if author not in config.get('system_users').split():
             notifications.publish(topic='update.comment', msg=dict(
@@ -1678,7 +1709,7 @@ def check_karma_thresholds(self, db, agent):
         if not self.locked:
             if self.status is UpdateStatus.testing:
                 # If critical update receives negative karma disable autopush
-                if self.critpath and self.autokarma and self.has_negative_karma:
+                if self.critpath and self.autokarma and self._composite_karma[1] != 0:
                     log.info("Disabling Auto Push since the critical update has negative karma")
                     self.autokarma = False
                 elif self.stable_karma not in (0, None) and self.karma >= self.stable_karma:
@@ -1735,18 +1766,6 @@ def last_modified(self):
         possibilities.sort()  # Sort smallest to largest (oldest to newest)
         return possibilities[-1]  # Return the last one
 
-    @property
-    def has_negative_karma(self):
-        """Check for negative karma, Returns True if the update has negative karma"""
-        feedback = defaultdict(int)
-        for comment in self.comments:
-            if not comment.anonymous:
-                feedback[comment.user.name] = comment.karma
-        for karma in feedback.values():
-            if karma < 0:
-                return True
-        return False
-
     @property
     def critpath_approved(self):
         """ Return whether or not this critpath update has been approved """
@@ -1903,6 +1922,8 @@ def __json__(self, request=None, anonymize=False):
         # Also, put the update submitter's name in the same place we put
         # it for bodhi1 to make fedmsg.meta compat much more simple.
         result['submitter'] = result['user']['name']
+        # Include the karma total in the results
+        result['karma'] = self.karma
 
         # For https://github.com/fedora-infra/bodhi/issues/270, throw the JSON
         # of the test cases in our output as well but take extra care to
@@ -1978,6 +1999,12 @@ def __json__(self, *args, **kwargs):
 
         # Similarly, duplicate the update's title as update_title.
         result['update_title'] = result['update']['title']
+
+        # Updates used to have a karma column which would be included in result['update']. The
+        # column was replaced with a property, so we need to include it here for backwards
+        # compatibility.
+        result['update']['karma'] = self.update.karma
+
         return result
 
     def __str__(self):


Test patch:
diff --git a/bodhi/tests/server/__init__.py b/bodhi/tests/server/__init__.py
--- a/bodhi/tests/server/__init__.py
+++ b/bodhi/tests/server/__init__.py
@@ -72,7 +72,6 @@ def populate(db):
     db.add(comment)
     comment.user = user
     update.comments.append(comment)
-    update.karma = 1
 
     comment = Comment(karma=0, text=u"srsly.  pretty good.", anonymous=True)
     comment.user = anonymous
diff --git a/bodhi/tests/server/functional/test_updates.py b/bodhi/tests/server/functional/test_updates.py
--- a/bodhi/tests/server/functional/test_updates.py
+++ b/bodhi/tests/server/functional/test_updates.py
@@ -2549,8 +2549,10 @@ def test_edit_testing_update_reset_karma(self, publish, *args):
         upd = Update.get(nvr, self.db)
         upd.status = UpdateStatus.testing
         upd.request = None
-        upd.karma = 2
+        upd.comment(self.db, u'LGTM', author=u'bob', karma=1)
+        upd.comment(self.db, u'LGTM2ME2', author=u'other_bob', karma=1)
         self.db.flush()
+        self.assertEqual(upd.karma, 2)
 
         # Then.. edit it and change the builds!
         args['edited'] = args['builds']
@@ -2624,8 +2626,9 @@ def test_edit_testing_update_reset_karma_with_same_tester(self, publish, *args):
 
     @mock.patch(**mock_valid_requirements)
     @mock.patch('bodhi.server.notifications.publish')
-    def test_has_negative_karma(self, publish, *args):
-        """The test asserts that has_negative_karma returns True when an update receives negative karma"""
+    def test__composite_karma_with_one_negative(self, publish, *args):
+        """The test asserts that _composite_karma returns (0, -1) when an update receives one
+           negative karma"""
         user = User(name=u'bob')
         self.db.add(user)
         self.db.flush()
@@ -2643,13 +2646,13 @@ def test_has_negative_karma(self, publish, *args):
         # The user gives negative karma first
         up.comment(self.db, u'Failed to work', author=u'luke', karma=-1)
         up = self.db.query(Update).filter_by(title=nvr).one()
-        self.assertTrue(up.has_negative_karma)
+        self.assertEqual(up._composite_karma, (0, -1))
 
     @mock.patch(**mock_valid_requirements)
     @mock.patch('bodhi.server.notifications.publish')
-    def test_has_no_negative_karma_with_changed_karma(self, publish, *args):
+    def test__composite_karma_with_changed_karma(self, publish, *args):
         """
-        This test asserts that has_negative_karma returns False when a user posts negative karma 
+        This test asserts that _composite_karma returns (1, 0) when a user posts negative karma
         and then later posts positive karma.
         """
         user = User(name=u'bob')
@@ -2669,19 +2672,19 @@ def test_has_no_negative_karma_with_changed_karma(self, publish, *args):
         # The user gives negative karma first
         up.comment(self.db, u'Failed to work', author=u'ralph', karma=-1)
         up = self.db.query(Update).filter_by(title=nvr).one()
-        self.assertTrue(up.has_negative_karma)
+        self.assertEqual(up._composite_karma, (0, -1))
 
         # The same user gives positive karma later
         up.comment(self.db, u'wfm', author=u'ralph', karma=1)
         up = self.db.query(Update).filter_by(title=nvr).one()
-        self.assertFalse(up.has_negative_karma)
+        self.assertEqual(up._composite_karma, (1, 0))
         self.assertEquals(up.karma, 1)
 
     @mock.patch(**mock_valid_requirements)
     @mock.patch('bodhi.server.notifications.publish')
-    def test_has_negative_karma_with_positive_karma_first(self, publish, *args):
+    def test__composite_karma_with_positive_karma_first(self, publish, *args):
         """
-        This test asserts that has_negative_karma returns True when a user posts positive karma 
+        This test asserts that _composite_karma returns (1, -1) when one user posts positive karma
         and then another user posts negative karma.
         """
         user = User(name=u'bob')
@@ -2701,18 +2704,18 @@ def test_has_negative_karma_with_positive_karma_first(self, publish, *args):
         #  user gives positive karma first
         up.comment(self.db, u'Works for me', author=u'ralph', karma=1)
         up = self.db.query(Update).filter_by(title=nvr).one()
-        self.assertFalse(up.has_negative_karma)
+        self.assertEqual(up._composite_karma, (1, 0))
 
         # Another user gives negative karma later
         up.comment(self.db, u'Failed to work', author=u'bowlofeggs', karma=-1)
         up = self.db.query(Update).filter_by(title=nvr).one()
-        self.assertTrue(up.has_negative_karma)
+        self.assertEqual(up._composite_karma, (1, -1))
         self.assertEquals(up.karma, 0)
 
     @mock.patch(**mock_valid_requirements)
     @mock.patch('bodhi.server.notifications.publish')
-    def test_has_no_negative_karma(self, publish, *args):
-        """The test asserts that has_negative_karma returns False when there is no negative karma"""
+    def test__composite_karma_with_no_negative_karma(self, publish, *args):
+        """The test asserts that _composite_karma returns (*, 0) when there is no negative karma."""
         user = User(name=u'bob')
         self.db.add(user)
         self.db.flush()
@@ -2729,20 +2732,20 @@ def test_has_no_negative_karma(self, publish, *args):
 
         up.comment(self.db, u'LGTM', author=u'mac', karma=1)
         up = self.db.query(Update).filter_by(title=nvr).one()
-        self.assertFalse(up.has_negative_karma)
+        self.assertEqual(up._composite_karma, (1, 0))
 
         # Karma with no comment
         up.comment(self.db, u' ', author=u'bowlofeggs', karma=1)
         up = self.db.query(Update).filter_by(title=nvr).one()
-        self.assertFalse(up.has_negative_karma)
+        self.assertEqual(up._composite_karma, (2, 0))
         self.assertEquals(up.karma, 2)
 
     @mock.patch(**mock_valid_requirements)
     @mock.patch('bodhi.server.notifications.publish')
-    def test_has_negative_karma_with_Anonymous_karma(self, publish, *args):
+    def test__composite_karma_with_anonymous_comment(self, publish, *args):
         """
-        The test asserts that has_negative_karma returns False when an anonymous user
-        gives negative karma to an update
+        The test asserts that _composite_karma returns (0, 0) when an anonymous user
+        gives negative karma to an update.
         """
         user = User(name=u'bob')
         self.db.add(user)
@@ -2760,12 +2763,12 @@ def test_has_negative_karma_with_Anonymous_karma(self, publish, *args):
 
         up.comment(self.db, u'Not working', author='me', anonymous=True, karma=-1)
         up = self.db.query(Update).filter_by(title=nvr).one()
-        self.assertFalse(up.has_negative_karma)
+        self.assertEqual(up._composite_karma, (0, 0))
 
     @mock.patch(**mock_valid_requirements)
     @mock.patch('bodhi.server.notifications.publish')
-    def test_has_negative_karma_with_no_feedback(self, publish, *args):
-        """This test asserts that has_negative_karma return False when an update has no feedback"""
+    def test__composite_karma_with_no_feedback(self, publish, *args):
+        """This test asserts that _composite_karma returns (0, 0) when an update has no feedback."""
         user = User(name=u'bob')
         self.db.add(user)
         self.db.flush()
@@ -2781,7 +2784,7 @@ def test_has_negative_karma_with_no_feedback(self, publish, *args):
         self.db.flush()
 
         up = self.db.query(Update).filter_by(title=nvr).one()
-        self.assertFalse(up.has_negative_karma)
+        self.assertEqual(up._composite_karma, (0, 0))
 
     @mock.patch(**mock_valid_requirements)
     @mock.patch('bodhi.server.notifications.publish')
@@ -3110,7 +3113,9 @@ def test_submit_older_build_to_stable(self, publish, *args):
         update = Update(title=oldbuild, builds=[build], type=UpdateType.bugfix,
                         request=UpdateRequest.testing, notes=u'second update',
                         user=update.user, release=update.release)
-        update.karma = 3
+        update.comment(self.db, u"foo1", 1, u'foo1')
+        update.comment(self.db, u"foo2", 1, u'foo2')
+        update.comment(self.db, u"foo3", 1, u'foo3')
         self.db.add(update)
         self.db.flush()
 
diff --git a/bodhi/tests/server/models/test_models.py b/bodhi/tests/server/models/test_models.py
--- a/bodhi/tests/server/models/test_models.py
+++ b/bodhi/tests/server/models/test_models.py
@@ -194,7 +194,6 @@ class TestUpdate(ModelTest):
         unstable_karma=-3,
         close_bugs=True,
         notes=u'foobar',
-        karma=0,
         )
 
     def do_get_dependencies(self):
@@ -387,6 +386,96 @@ def test_dupe(self):
         self.get_update()
         self.get_update()
 
+    def test_karma_no_comments(self):
+        """Check that karma returns the correct value with one negative and two positive comments.
+        """
+        eq_(self.obj.karma, 0)
+
+    def test_karma_one_negative_two_positive(self):
+        """Check that karma returns the correct value with one negative and two positive comments.
+        """
+        self.obj.comment(self.db, u"foo", 1, u'foo')
+        self.obj.comment(self.db, u"foo", -1, u'bar')
+        self.obj.comment(self.db, u"foo", 1, u'biz')
+
+        eq_(self.obj.karma, 1)
+
+    def test_karma_two_negative_one_positive(self):
+        """Check that karma returns the correct value with two negative and one positive comments.
+        """
+        self.obj.comment(self.db, u"foo", -1, u'foo')
+        self.obj.comment(self.db, u"foo", -1, u'bar')
+        self.obj.comment(self.db, u"foo", 1, u'biz')
+
+        eq_(self.obj.karma, -1)
+
+    def test__composite_karma_ignores_anonymous_karma(self):
+        """Assert that _composite_karma ignores anonymous karma."""
+        self.obj.comment(self.db, u"foo", -1, u'foo')
+        self.obj.comment(self.db, u"foo", -1, u'bar')
+        # This one shouldn't get counted
+        self.obj.comment(self.db, u"foo", 1, u'biz', anonymous=True)
+
+        eq_(self.obj._composite_karma, (0, -2))
+
+    def test__composite_karma_ignores_comments_before_new_build(self):
+        """Assert that _composite_karma ignores karma from before a new build karma reset event."""
+        self.obj.comment(self.db, u"foo", -1, u'foo')
+        self.obj.comment(self.db, u"foo", -1, u'bar')
+        # This is a "karma reset event", so the above comments should not be counted in the karma.
+        self.obj.comment(self.db, u"New build", 0, u'bodhi')
+        self.obj.comment(self.db, u"foo", 1, u'biz')
+
+        eq_(self.obj._composite_karma, (1, 0))
+
+    def test__composite_karma_ignores_comments_before_removed_build(self):
+        """Assert that _composite_karma ignores karma from before a removed build karma reset event."""
+        self.obj.comment(self.db, u"foo", 1, u'foo')
+        self.obj.comment(self.db, u"foo", 1, u'bar')
+        # This is a "karma reset event", so the above comments should not be counted in the karma.
+        self.obj.comment(self.db, u"Removed build", 0, u'bodhi')
+        self.obj.comment(self.db, u"foo", -1, u'biz')
+
+        eq_(self.obj._composite_karma, (0, -1))
+
+    def test__composite_karma_ignores_old_comments(self):
+        """Assert that _composite_karma ignores karma from a user's previous responses."""
+        self.obj.comment(self.db, u"I", -1, u'foo')
+        self.obj.comment(self.db, u"can't", 1, u'foo')
+        self.obj.comment(self.db, u"make", -1, u'foo')
+        self.obj.comment(self.db, u"up", 1, u'foo')
+        self.obj.comment(self.db, u"my", -1, u'foo')
+        self.obj.comment(self.db, u"mind", 1, u'foo')
+        self.obj.comment(self.db, u".", -37, u'foo')
+
+        eq_(self.obj._composite_karma, (0, -37))
+
+    def test__composite_karma_mixed_case(self):
+        """Assert _composite_karma with mixed responses that hits a lot of the method."""
+        self.obj.comment(self.db, u"ignored", -1, u'foo1')
+        self.obj.comment(self.db, u"forgotten", -1, u'foo2')
+        # This is a "karma reset event", so the above comments should not be counted in the karma.
+        self.obj.comment(self.db, u"Removed build", 0, u'bodhi')
+        self.obj.comment(self.db, u"Nice job", -1, u'foo')
+        self.obj.comment(self.db, u"Whoops my last comment was wrong", 1, u'foo')
+        self.obj.comment(self.db, u"LGTM", 1, u'foo2')
+        self.obj.comment(self.db, u"Don't ignore me", -1, u'foo1')
+
+        eq_(self.obj._composite_karma, (2, -1))
+
+    def test__composite_karma_no_comments(self):
+        """Assert _composite_karma with no comments is (0, 0)."""
+        eq_(self.obj._composite_karma, (0, 0))
+
+    def test__composite_karma_one_negative_two_positive(self):
+        """Assert that _composite_karma returns (2, -1) with one negative and two positive comments.
+        """
+        self.obj.comment(self.db, u"foo", 1, u'foo')
+        self.obj.comment(self.db, u"foo", -1, u'bar')
+        self.obj.comment(self.db, u"foo", 1, u'biz')
+
+        eq_(self.obj._composite_karma, (2, -1))
+
     @mock.patch('bodhi.server.notifications.publish')
     def test_stable_karma(self, publish):
         update = self.obj