Patent Document ID: 9336310
Application ID: 12830927

Base Claim:
1. A computer-implemented method for identifying on-line comments as being legitimate or illegitimate, the method comprising: receiving from a user of a computing service a comment directed to a first piece of on-line content; identifying results of reviews of one or more comments by the user directed to other pieces of on-line content; identifying a measure of trust associated with the user that is determined based at least on the results of the reviews of the one or more comments by the user directed to the other pieces of on-line content; determining whether to provide for review of the first piece of on-line content by using a random selection that randomly selects particular pieces of on-line content, from among a plurality of pieces of on-line content, to provide for review, wherein the determining comprises: selecting a number at random; adjusting the selected number up or down using the measure of trust associated with the user, and determining, based on the adjusted number, whether to provide the first piece of on-line content for review to determine whether the on-line content is legitimate or not, wherein a likelihood that the first piece of on-line content is provided for review is influenced based at least on the measure of trust associated with the user; taking action on changing a status of the first piece of on-line content based on the determination of whether to provide for review of the first piece of on-line content; iteratively providing, in a plurality of rounds that each correspond to a submission of a comment from the user, for review of a series of comments by the user and further adjusting the adjusted number up or down based on whether the reviews indicate that the comments are accurate or inaccurate; and determining, for each round of review, a bias level for a decision to trust the comment and a bias level for a decision to review the comment, and determining whether to make active the decision to trust the comment or the decision to review the comment, where making active defines which of the decisions will be used to control review of a comment.

---

Claim 3:
3. The method of claim 1 , wherein the comment is a negative flag by the user that indicates that the first piece of on-line content is inappropriate.