Wikipedia talk:Version 1.0 Editorial Team/Wiki Sort

message: #Hammer out the specifics of the proposed rating system
#Request software modifications
#Hope and pray it gets implemented
#Start rating
#Decide on a verification method
#Wikipedia 1.0!
:: "Current" as of when? Who is the author? As I write this comment, the next comment section is from 2005: that suggests to me that "current" is not particularly current in 2014. hunterhogan (talk) 07:56, 4 January 2014 (UTC)
thread_title:  Current Tasks 

message: I would just like to make it known that I am deeply devoted to this project.  It has a LOT of potential, and I am willing to see it through to the end.  However, that does not mean I intend to hijack it!  Rather, we need to do this together.  Let's git 'er done!the1physicist 03:51, 24 October 2005 (UTC)
thread_title:  A Message 

message: I know this is going to come up so I thought it best to pre-emptively create this.  What range of numbers will this rating system cover?  Will it be 0 to 3, 1 to 10, or something in between?  Personally, I strongly favor a scale of 1 to 10.  Among other things, this would make it much easier on the software.  More importantly, I think people are used to rating things on a scale of 1 to 10, which jives with the goal of making rating as easy as possible.the1physicist 03:13, 21 October 2005 (UTC)
thread_title:  Rating Scale 

message: This will also come up, so we might as well address it now.  A disk version of Wikipedia will naturally contain less information than the online version, and a paper version will contain even less.  The wonderful thing about a rating system is by varying the selection criteria, it automagically allows for both.  For example, a disk version could include all articles with a rating of say 7 and up, whereas a paper version could limit itself to articles rated 9 or 10.  In contrast to our sister projects, a rating system will allow for any conceivable distribution method, without pulling from the editor pool.the1physicist 03:13, 21 October 2005 (UTC)
thread_title:  Online vs. CD/DVD vs. Paper 

message: While reading other folks' past suggestions, I saw that a few had the idea of additional rating categories, so I have added this to the project page.  I am definitely in favor of any and all additional categories (suggestions, please!), but feel we should limit them to the top 5 or 10.  I also think the scope and comprehensiveness ratings are most important, so they should be separate from the others.the1physicist 03:13, 21 October 2005 (UTC)
thread_title:  Categorized Rating 

message: An issue that may or may not happen is that people may (unintentionally) not reliably rate articles.  This will primarily be a problem with casual viewers who may not be well versed enough in wikipedia.  I personally don't think this will be a problem (due to averaging, etc), but for some articles it very well could be.  At the very least, an article with wildly varying ratings suggests an issue with the article itself.  As User:Stirling_Newberry put it, "even noise is data".the1physicist 20:21, 22 October 2005 (UTC)
thread_title:  Rating Accuracy 

message: This is particularly true of the scope rating, where only an expert in that topic will truly know how much an article fulfills it's scope.  The solution to this problem would be to weight experts' scope ratings (or possibly suggest an admin to override it), which of course requires a way to identify experts.the1physicist 20:21, 22 October 2005 (UTC)
thread_title:  Scope Rating 

message: ==== Expert Rating ====
An editor could select areas where he or she is knowledgeable.  Then other editors could rate that editor's level of expertise after reveiwing edit histories, etc.  It may even be possible for the software to determine the expert rating.  The software could compare an article's rating before and after an edit (assuming an improvement), and using it's category determine that the editor is knowledgeable in that subject.the1physicist 20:21, 22 October 2005 (UTC)
thread_title:  Expert Determination 

message: An editor could select areas where he or she is knowledgeable.  Then other editors could rate that editor's level of expertise after reveiwing edit histories, etc.  It may even be possible for the software to determine the expert rating.  The software could compare an article's rating before and after an edit (assuming an improvement), and using it's category determine that the editor is knowledgeable in that subject.the1physicist 20:21, 22 October 2005 (UTC)
thread_title:  Expert Rating 

message: Alternatively, if the (proposed) Wikiversity ever decides to include testing (which I believe they should), those scores could be used to determine an expert rating.the1physicist 20:21, 22 October 2005 (UTC)
thread_title:  Wikiversity 

message: Vandalism is going to be an issue.  We have two options.  The first would be to fork the 'pedia at the last minute and have editors do a final check for vandalism/copyvio.  The other option would be to verify each version of an article as free from vandalism, and have the software choose the latest vandalism-free version for release.  This option would require much more effort than a last minute fork.  More importantly, a continuous validation scheme would probably undergo final validation anyway, so we might as well do it all at once.the1physicist 02:10, 24 October 2005 (UTC)
replies: text: My vote is for continuous validation.  Academic studies show that the average act of vandalism is fixed with in  x \pm \sigma  minutes.  We can mine the database to get up-to-date and detailed values for x as a function of the number of users who list the article on their watchlist, the number of those users who are admins, etc.  Once we set a statistical confidence level (such as 95%), we can choose a length of time  x + k \sigma , tailored to the watchlist status of the article in question, and simply use the newest version which is older than that time interval.  In our example, k=2.  The figure from 2004 is x = 2.8 , but that's the aggregate, and I'm sure it's a bit smaller now, especially for well-watched articles.  An extremely simple version could be based on the old aggregate, though.--Joel 15:40, 4 November 2005 (UTC)
replies: text: Have you seen the latest version on the project page?  Note this sentence: "Perhaps the best solution would be to verify revisions of articles, fork at the last minute, and simply present to editors articles whose most recent revision isn't already verified."  I think it combines the best of both worlds.  Let me know what you think.the1physicist 16:31, 4 November 2005 (UTC)
thread_title:  Last Minute Fork vs. Continuous Validation 

message: On the specifics of the rating system: I think that users should be prompted to rate articles, and that a system should be devised which uses previous ratings and the size of the diff file to choose which artilce is brought to the users' attention.  I'm quite fond of the randomization trick that Slashdot uses to prevent the rating of things one already cares about; I think we should steal it.  Perhaps the developers involved would even donate their services to the Wikimedia team; failing this, their software is open source.  See Coase's Penguin (the paper itself, not the Wikipedia article) if you aren't familiar with the finer points of their rating system; a quick treatment (which omits my favorite part of it) can be found here.
On the implementation of software modifications: see Slashdot comment.  Also: the Unification church ("Moonies") is working on sorting out the best of Wikipedia, as well. Perhaps they'd help with financing the software development.  Of course, this won't affect whether or not it gets accepted into MediaWiki...
On the last-minute fork: If the article rating applies to the version itself (with, perhaps, a spillover if the edits are small), there is no need to fork or skim.  Only the articles that have been automatically rated would be included.--Joel 06:43, 20 October 2005 (UTC)
replies: text: On second thought, ratings of the scope of the topic should be kept through all versions, but ratings on quality and thoroughness within that scope should apply to particular versions.--Joel 21:43, 20 October 2005 (UTC)

text: I think that's a great idea.  I was thinking an external system like de.lirio.us, but your way sounds even better.  Go ahead and set up the categories yourself.  However, I would suggest using the pipe trick: 9.3, PageName will produce a synoptic "category" page ranked by rating and then sorted alphabetically.  I'd also suggest creating a template for the talk page that allows users to vote, and invites people to check that the average/consensus of votes is what the page is listed under.--Joel 17:22, 8 November 2005 (UTC)
replies: text: That would be a very good idea, except it wouldn't allow for all the nifty data-mining techniques discussed on the project page.  More importantly, I recently talked with User:Magnus_Manske, and according to him, most of the software changes have already been implemented.  I I understood him correctly, the only thing we're waiting on is the appropriate data mining algorithms (some of which I have outlined).the1physicist 17:51, 8 November 2005 (UTC)
thread_title:  General Discussion 

message: What is needed to provide safety in verification is an heuristic to determine how reliable an article's rating is.  I agree with the time-based method suggested above, but I also want to suggest an approach based on character counting.  Under this system, whenever an edit is made, the server notes the number of characters added + number removed.  With this as change:
new unknown score = net rating - [old article length / (change + old article length) * (net rating - old unknown score)]
(This would need special cases to prevent division by 0.)
Where "unknown score" refers to a rating value meaning that the server is agnostic as to the value of the new content.  This would roughly approximate it; someone changing punctuation wouldn't register much on the scores, while someone venturing a new paragraph or an outright page blank would cause the consensus to be automatically decimated.  The algorithm would guard against last-minute vandalism getting into printed copies, as a large change would edge a page out of a B-rating.  Just a suggestion. Alksub 01:14, 9 April 2006 (UTC)
replies: text: I'm having a hard time understanding exactly what that does, so you'll have to break it down for me.  It looks like it assigns an accuracy rating based on how different a revision is from the previous one.  Is that correct?the1physicist 14:28, 11 April 2006 (UTC)
replies: text: Yes, an accuracy rating of the page's rating. Alksub 01:48, 22 April 2006 (UTC)

text: Actually, the formula I gave above is just goofy. But you get the general idea. Alksub 14:03, 8 July 2006 (UTC)
thread_title:  Another mathematical approach 

message: Is this project still active? It could be good to put an update at Wikipedia:Version 1.0 Editorial Team. Thanks. Maurreen 19:06, 30 April 2006 (UTC)
replies: text: We're planning to publicise WP1.0 a lot in the coming months, maybe we can include this project in some of that. Walkerma 02:16, 1 May 2006 (UTC)
replies: text: Has anyone here gone through to CD burning ? Redlink cleanup, Category links that work, reduction of large pictures ? Wizzy…☎ 18:36, 20 June 2006 (UTC)
replies: text: Not that I know of, but someone could have slipped that past me.the1physicist 18:46, 20 June 2006 (UTC)
thread_title: Status

message: This is going to be nearly impossable unless we're ranking a specific export of the database.  An article can be rated 10 (1000 votes) and then some it10t can come along, make "valuable minor" changes, and blow the current rank.  If this has already been solved, I'm on the project.
-NickSentowski 18:09, 6 July 2006 (UTC)
thread_title:  Ranking Articles? 

message: I think it would be easier to use a bot + a script. The script would create a "rate" tab on each article and post the rating to a user subpage. The bot would look at the ratings on the user subpage and get the final rating somehow. This way, we don't have to wait for the developers to make changes to the software because the bot can be run by one user and the scripts can be posted on the scripts page. We could make the script more well known by posting it on the village pump. The bot doesn't need to be well known because only a single user has to run it. Eyu100 23:19, 24 August 2006 (UTC)
thread_title: Software suggestion

message: Wikipedia:Version 1.0 Editorial Team states that the Wiki Sort "project has been rendered obsolete by WikiProject-based assessments (see above)." 
Problems:
# The linked page directs the reader to "see above", but as of this writing it is not clear what the author is directing the reader to see. As best as I can tell, the author is referencing Work via WikiProjects.
# As best as I can tell, this page, Wiki Sort, contains information that is still relevant but not reproduced in other "active" projects. If this project is obsolete, editors should migrate the non-obsolete knowledge to non-obsolete projects.
# The Wiki Sort page does not include a warning that it is obsolete. Compare to Wikipedia:Article_assessment.
hunterhogan (talk) 08:13, 4 January 2014 (UTC)
thread_title:  Obsolete