Wikipedia talk:Content review/workshop/Archive 1

message: I don't quite see how that description fits this. And I don't really see the point of this to begin with -- not to say that there is no point, just that I don't understand it. You may want to offer some additional rationale at the top of this talk page, especially with regard to how this process compares with "good article" nominations. Equazcion • argue/improves • 10:08, 10/1/2007
:Ah, sorry. Insofar as it might supplant GA it's radical streamlining. Marskell 12:04, 1 October 2007 (UTC)
replies: text: No more review options, I agree. A.Z. 00:52, 2 October 2007 (UTC)
replies: text: Guys, I said insofar as it might supplant GA. No, of course, simply starting a third structure doesn't make sense. Call it "back to GA's roots", if you like. Marskell 11:15, 2 October 2007 (UTC)
thread_title:  Radical streamlining? 

message: How does this idea differ significantly from WP:GA? There is no point having a new process almost identical to an existing one. If you have ideas to improve GA, then suggest them over there, but starting a new process is a bad idea. --Tango 21:54, 1 October 2007 (UTC)
thread_title:  Good articles 

message: To this I'd add: "Participation is so low many submitted articles don't get any review. This has caused many editors to abandon PR altogether, causing some to use FAC as a substitute.". As currently written, the PR con list hints at this but doesn't actually say it.Rlevse 13:31, 4 October 2007 (UTC)
replies: text: Well, last time I looked, this is wikipedia, and "everyone can edit", so why don't you add it to the list? Dr. Cash 05:03, 5 October 2007 (UTC)
thread_title:  PR cons 

message: After some discussions with Marskell (visible on our respective talk pages, if anyone is interested), I would like to try adding some additional material to the top of the workshop page, with the goal of laying out a roadmap that tries to avoid some of the problems FA/GA/PR discussions have had.  I hope it will be self-explanatory what I'm trying to do.  A brief summary is that I'd like to start by getting folks to agree on goals and scope, and then work from there to identifying specific improvements that can be made.  There are some risks in this approach -- for example it's easy to make the scope too big, and easy to try to come up with ideas that will fix everything in one go.  However, if we can avoid those pitfalls, I think the benefit of working from goals first is that we might get some common understanding of what we're trying to accomplish, and build some trust among the participants in order to avoid the sense of "drive-by commentary".
I also am not particularly keen to advertise this page widely until we have some evidence that it's going to be useful.  These are contentious topics; I know we already have some habitues of GA and FA here, and if we see things starting to gel I suggest we post some notices to attract more attention at that point.  Right now, I feel like this is a workshop which needs to work, rather than be examined for effectiveness.  Obviously we should be glad of anybody who stops by and contributes, but I think we might be better off for a few days at least if we see what progress we can make before asking for validation. Mike Christie (talk) 21:48, 4 October 2007 (UTC)
thread_title:  Approach 

message: Malleus Fatuarum just changed "Produce well-written articles with accurate, complete, and neutral content" to ". . . broad, and neutral content", with the edit summary "Scope creep".  Could we get a bit more explanation?  I'm not sure I disagree with the change, but I don't really understand the edit summary. Mike Christie (talk) 13:21, 5 October 2007 (UTC)
replies: text: I meant that the goal posts for GA seem to be changing. WP:WIAGA says that good articles are broad in their coverage, not complete, or comprehensive, as required for FA. So the scope of GA seemed to be overlapping with FA, hence scope creep. --Malleus Fatuarum 14:04, 5 October 2007 (UTC)
thread_title:  Query on recent edit 

message: Howsabout trying to devolve PR to the various wikiprojects? Serves to promote importance of wikiprojects by hopefully encouraging article improvement there rather than on a general page, with a PR coordiantor who can leave PR to only contain articles not related to a psecific wikiproject  -the reduction in the queue might make for more reviews there. cheers,  Casliber (talk · contribs) 06:54, 7 October 2007 (UTC)
replies: text: In my view, wikiprojects are perennially "mentioned", to the point where a newcomer might get the impression that they are a fundamental organizational component of wikipedia—but really, on the whole, they barely exist, except in name (in my experience). I don't think PR could be channeled to projects for that reason. ¶ Since I'm on this page: I do like this general movement towards combining peer review and GA, and simply letting the nominator have the choice of saying, "please review this" or "please review this and add it to the list of good articles if you think it passes". I don't see how it couldn't be an improvement, with the only hurdle being the community that doesn't want GA to be touched. (And incidentally, I generally like GA, and I don't view it as duplication or inefficient.) –Outriggr § 08:27, 7 October 2007 (UTC)
replies: text: There already exist several topic-specific peer review processes, as well as Wikipedia:Scientific peer review, which appears to be marginally more successful that PR itself. Some WikiProjects are alive and well, and indeed very active. Others need encouragement. Geometry guy 10:34, 7 October 2007 (UTC)

text: That's a point. Maybe if PR just ceases to exist then it will all go to GA anyway. cheers,  Casliber (talk · contribs) 09:58, 7 October 2007 (UTC)
replies: text: Couple of comments: One is that I value 'vanilla' Peer Review because it gives me access (sometimes :)) to people from different areas. I can always ask for comments at my local WikiProject. Devolving PR to WikiProjects could lead to articles becoming 'specialist' rather than 'generalist'. That is, they would be reviewed by people who are familiar with all the jargon and concepts involved in the topic and would be less exposed to new thinking on the topic. Second one is to repeat the point made above: Not all Wikiprojects have the organisation or resources of MilHist. Some barely exist other than in name. Cheers. 4u1e 16:44, 12 October 2007 (UTC)
thread_title: Idea  - devolving PR to wikiprojects

message: Peer review
 
The Wikipedia Peer review is a centralized list of active review processes and users willing to help with reviewing. Reviews themselves are not listed here as a generalized peer review has proven uneven in producing content improvement. Rather, editors are either directed to a more specific review process or provided a list of users who are willing to be contacted on their user talk pages. In the latter case, reviews take place on article talk: start a thread and provide a link to the volunteer editor(s). If a subject area is not listed here, editors are strongly encouraged to seek out other users for a review either through the history of similar articles or at Wikiprojects. If there is a topic area that you are interested in, you may volunteer as a reviewer here.	
	
Marskell 13:18, 8 October 2007 (UTC)
replies: text: I'm not sure I like the tone of the above.  Check below for an alternate idea.   Plus, PR is a Good Thing (tm), but it needs fixing so it works better, not scrapping.--Jayron32|talk|contribs 16:36, 8 October 2007 (UTC)
thread_title: PR proposal

message: One last process question before we pick a problem and start brainstorming: how do we come to consensus on the Current Problem?  The brainstorming page's list is a source of candidate problems, though I think I'd generally like to see the problems phrased positively rather than negatively.  For example, I'd prefer "How do we improve the relationship between GA and FA reviewers in order to focus on actually reviewing?" to the current wording, which is "Constant infighting between FA & GA reviewers takes time away from actually reviewing articles".  (I'm not suggesting we pick that particular problem.)
Perhaps another front page section which lists problems?  Then put the Current Problem at or near the top of the project page until consensus is reached that it's time to change it.  Then a topic on the talk page to pick the next Current Problem: maybe straightforward straw polling/voting would be good enough.  If we can agree on this, we can go ahead and start building that list of problems now, drawing from the brainstorming page if we agree with those statements (and, as I said, perhaps rephrasing them). Mike Christie (talk) 22:26, 12 October 2007 (UTC)
replies: text: Well, in part, realizing that neither GA nor FA is going to go away would go a long way in improving relations. But maybe that's just me. Titoxd(?!? - cool stuff) 00:14, 13 October 2007 (UTC)
replies: text: Not just you, that would be me as well. --Malleus Fatuarum 01:14, 13 October 2007 (UTC)

text: Since the proposal as I understand it seems to be to concentrate on specific issues rather than the macroscopic process, it makes sense to me to have a to-do list, even if it starts out as only a few items. It's most likely going to be evolving anyway; sorting out one problem will probably throw up another two ;) A straw poll would be suitable IMO for choosing the 'issue of the week', and I think Mike's point re phrasing is important. This must be a constructive process, not an accusative one. I only have one question: should there be a time limit on discussing one problem before moving on to the next, and if so, what? EyeSereneTALK 10:07, 13 October 2007 (UTC)
thread_title:  Picking the Current Problem 

message: What is it that the community wants to achieve with these reviews? We cannot really go forward with any revisions, big or small, without coming to an agreement on what the goals of the reviews are. Awadewit | talk 02:30, 13 October 2007 (UTC)
replies: text: That's an easy one, better articles. The underlying problem seems to be an unwillingness to come to an agreement on how much "better" is "better" enough to get to each stage, and the ongoing FA/GA dispute is damaging in that respect. --Malleus Fatuarum 03:40, 13 October 2007 (UTC)
replies: text: Although Malleus is right, I think I'd be more specific and say that the goal is (or should be) to show what needs to be done to raise an article to the next level. Walkerma 04:19, 13 October 2007 (UTC)

text: I'd be even more specific and say that the goal is to do what needs to be done to raise an article to the next level. Not to sit in judgement over it, waiting for divine intervention. --Malleus Fatuarum 04:26, 13 October 2007 (UTC)
replies: text: I don't think moving an article to the next level is so simple - not everyone agrees on what the "levels" are, for example. Also, after having gone through FAC twelve times now, my impression of that process is not that it is cooperative, but that it is evaluative. While GA is evaluating against a set of criteria, it tends to feel more cooperative to me. This is just my personal impression. I think that this may be one reason why FAC is gaining a bad reputation. I wish FAC were either a place where an article passed or failed as it stood or a place where reviewers and editors worked together to improve articles. (What is this FA/GA dispute of which you speak? I've seen passing indications of it, but I didn't realize it was an out-and-out war or anything.) Awadewit | talk 04:33, 13 October 2007 (UTC)
replies: text: I try to make my reviews collaborative at FAC and get stuck in and help (if asked, or more recently even if not!) as Malleus is right - the aim is better articles not to sit in judgement of others. I used to list all errors, eg misspellings etc. now I try to fix uncontroversial glitches as I go as it saves a heck of alot of time. I realise some reviewers can be difficult at FAC but I have also seen the other where nominators rather than try to address issues raised get annoyed and/or uncooperative instead. cheers,  Casliber (talk · contribs) 10:57, 13 October 2007 (UTC)
replies: text: Yes and no...  I like to be collaborative when I do such a review, but then again that's partly why I don't now do GA or FA reviews (only ad hoc peer reviews) - the fixing up takes a lot of time.  Heck, I don't even find the time to fix up articles I'm trying to work on!.  Even when I do a quick review for 1.0, I often add a photo from Commons or rewrite some terrible text - and then I often end up getting one article reviewed instead of ten.  I think a "drive-by" review can still be very valuable if it's written as constructive criticism.  If everyone takes the time to fix up every article we review there would be a much bigger backlog at GAN and FAC.  If it's a lot of work I sometimes like to clean up one section thoroughly, to show what I mean, and let the main editor do the rest.  In short - a thorough review with feedback is good, but a thorough review with cleanup and feedback is better. Walkerma 18:13, 13 October 2007 (UTC)

text: If everyone took the time, then there would be no backlog. I agree that a collaborative review takes time, and a significant commitment from both the reviewer(s) and the nominator, but what's the point of any other kind of review? "Drive-by" reviews in particular are very demotivating. --Malleus Fatuarum 18:46, 13 October 2007 (UTC)

text: The intent of the goals section of this project page was to capture the answers to Awadewit's question, above.  Is there anything from the comments above that isn't listed there? Mike Christie (talk) 19:32, 13 October 2007 (UTC)
thread_title:  Query regarding goals of reviews 

message: Per some of the discussion above, I've put a candidate list of problems on the front page.  Please edit as necessary.  I got this list by going through Dr.Cash's original list of problems that was moved to the brainstorming page, and rephrasing them per the description I gave at the top of the list.  I think once we've got this list in reasonable shape we can pick a Current Problem, although there are other conversational threads recently started above we need to keep going as well. Mike Christie (talk) 14:04, 13 October 2007 (UTC)
thread_title:  List of problems on front page 

message: Change the locus of the review.  The Peer Review page still exists, but no actual reviews happen on that page.  Instead, it acts as a clearinghouse for articles requesting Peer Review.  Reviewers come to the Peer Review page to see articles that are requesting review.  Each article title will use the  template and that is it.  The reviewers are directed to make comments on the article's talk page, perhaps under a designated Peer Review section, which can be set aside on the talk page itself, perhaps by a border and color change, so as to indicate it is the official "peer review section".  That way, relevent improvement discussions are moved to the talk pages of the articles, where it is more appropriate and more likely to attract more attention; the PR page is shortend significantly (the page size is one hindrance of productive work there) and otherwise the process still exists.  Ideas?--Jayron32|talk|contribs 16:36, 8 October 2007 (UTC)
replies: text: Isn't that essentially what's posted in the above thread? Marskell 10:43, 9 October 2007 (UTC)
replies: text: No, because the articles are still listed at PR.  The above is essentially an historical tag telling people that since PR doesn't work, it has been trashed, and leaves them to go elsewhere without much guidance. --Jayron32|talk|contribs 03:13, 11 October 2007 (UTC)
replies: text: But it's precisely the leaving it on PR that's failing to work. And the above wouldn't be a dead page at all: it would have lists of people willing to review particular subject matter. I'll paraphrase qpb from the original village pump thread: at the end of the day, if you want people to look at your page you need to go to user talk and ask them to look at it. Marskell 12:20, 11 October 2007 (UTC)
replies: text: AH, so what you are saying is that PR will change from being a place where you go to "drop off" an article to be reviewed to being a place where you "pick up" a reviewer who would like to do the reviewing for you?  Interesting idea.  I think I like it.  Sorry I didn't pick that up before.  The text box left above spent so much time subtly (and not so subtly) trashing the old process that it was unclear what the new process was to be.  Not sure it will be enacted, but I dig it.  Its a neat idea. --Jayron32|talk|contribs 06:14, 12 October 2007 (UTC)

text: The text above was not meant to be placed immediately—we'd need transition wording introducing the change for about a month. Categories: yes, that would be the main idea. I think it should be much less over-specific than the GA list but slightly more specific than FA.Marskell 15:31, 15 October 2007 (UTC)
replies: text: Ummm, color me outdated, probably. I was in PR a long time ago.. I created a "desk" and I was supposed to wait for someone to drop a request in my "desk". And I waited. And i waited. And i quit in disgust at the passive nature of the system. Still the same? is that what we're discussing? Then i don't like it. :-) --Ling.Nut 15:34, 15 October 2007 (UTC)
replies: text: I don't know what you mean. Marskell 22:04, 15 October 2007 (UTC)
thread_title:  Another idea for how to handle Peer Review 

message: This something that LaraLove brought up to me as an option. Even the most experienced reviewers only have one pair of eyes, and can miss small (or pivotal) issues in an article through sheer accident. Just as no reputable news publication calls all done on a piece that only a single copyeditor has viewed, collaboration in reviewing might be an essential step in improving the quality of reviews. I think an optional way to review an article in two-man teams might be a good idea. Of course, implementation of this might best be started after we kill the current backlog. Probably the two problems to be dealt with are creating/updating templates for dual reviewing, and what to do in the case of a fundamental disagreement between reviewers. VanTucky Talk 22:21, 8 October 2007 (UTC)
replies: text: Thanks for posting this, VanTucky. I envisioned a subpage of the project page where there'd be a brief overview of how it works. Basically, anyone wanting to try out doing tandem reviewing could list their name and their strong points (for example, I'm great at nit-picking prose, style issues, references and the like, so I'd want to pair up with someone who's good at finding broadness and comprehensiveness issues). It would need to be clearly outlined that any issues/disagreements between the two reviewers should be handled on their respective talk pages and not the article's talk page. It's almost like the second opinion option, but it's a set team of two on all the reviews of those editors, for as long as they want to team up. It would also be very helpful to new reviewers. Lara❤Love 22:32, 8 October 2007 (UTC)
replies: text: I think the idea would only work if the two reviewers were really working together, in real time, simultaneously on the same review.  A bit like the XP approach to software development. I don't see much merit in the "I'll check grammar and you check everything else" approach. --Malleus Fatuarum 01:59, 9 October 2007 (UTC)
replies: text: I don't like the idea of "issues/disagreements between the two reviewers should be handled on their respective talk pages and not the article's talk page."  Even such debates should be transparent, and should occur on the article's talk page.  See my suggestion above for revamping PR.  If we can get PR to be a more widespread and utilized process (from the reviewer end.  Plenty of articles get nominated), and we can get discussions to occur on article talk pages, not on some obscure side page or user talk pages, we can get multiple eyes on an article.  Otherwise, however, the "team reviewing" idea is a very good idea.  --Jayron32|talk|contribs 04:22, 9 October 2007 (UTC)
replies: text: Regarding the disagreement discussions, I see this as a potential problem with tandem reviewing. There will inevitably be disagreements between reviewers in some cases. It would be unfortunate for one editor to undermine the recommendation of their review partner because they disagree with their interpretation of the criteria, for example, thus sparking an argument which could be an embarrassment to the project. I don't think every aspect has to be transparent; Not all disagreements have to be on display for everyone. It can be talked out on user talk pages, and any changes to the recommendations then made to the article talk page. The review discussion is not the proper place to work out such differences. Lara❤Love 15:54, 15 October 2007 (UTC)
thread_title:  Optional team reviewing? 

message: Extend the quick fail criteria;  for a short time I got involved in doing GA reviews (have since been busy building a new article up, offline), and many times I encountered articles that, on the very first reading, I knew weren't eligible for GA.  But, none of them met the quick fail criteria, so I felt obliged to do a full, and time consuming, review, even though I knew from the start that the end result would be a fail.  I think that basic prose requirements, as well as MoS strictures, should be brought into the quick fail criteria so that the GA list can be cut down quickly.  There will still be borderline cases where on-hold would be applicable, but so many of those nominations should be failed on first sight!
What is frustrating to contributors to wikipedia, who nominate articles for review, whether it be GA, PR or FA is the sheer time it takes to get a response;  by being able to quickly cut out the dross, the  quality articles will be able to get the attention they deserve, with the result of happy contributors, a better WP, and smarties all round! 
Just my tuppence (or 1 cent, to the Americans) worth. Carre 23:26, 9 October 2007 (UTC)
replies: text: Just a personal view, but I don't actually mind reviewing these sort of articles. Taking the time to point out MoS, style etc deficiencies is beneficial in that it educates editors; the personal touch works better than expecting all editors to read and understand the entire MoS for themselves. The net result is higher-quality submissions next time around, and an overall improvement in article quality. I agree that this may not help with baclogs, but IMO it pays off in the long run. EyeSereneTALK 08:07, 10 October 2007 (UTC)
replies: text: I am gonna agree with EyeSerene there.  I have often left some rather extensive reviews at some really crappy articles.  I find the idea of "quick-fail" criteria an invitation to laziness.  I have always felt that it is my responsibility as a GA reviewer to provide enough information about a failed article's shortcomings to actually make it a GA if my recommendations are followed.  Quick-failing an article does not actually lead to much improvement.  --Jayron32|talk|contribs 03:16, 11 October 2007 (UTC)
replies: text: Going along with what EyeSerene & Jayron32 have said, I don't think expanding the quick fail criteria is the right way at all. I just reviewed Auckland, and considered a quick fail at first (partially because it was simultaneously nominated for FAC & PR & GAN. all at once; and partially because it just fell short of the criteria in many ways). But in the end I did a full review, because I thought the editors were inexperienced and I was able to suggest many ways in which to review it, and lots of resources for them to look at. So hopefully the editors will improve upon this and come back. Dr. Cash 05:22, 11 October 2007 (UTC)
replies: text: Yep. Agree with Dr Cash and Jayron. If it's gonna fail it's gonna fail - a quick fail would have precluded the helpful comments above. cheers,  Casliber (talk · contribs) 06:21, 11 October 2007 (UTC)
replies: text: So true. We don't want to increase the number of quick-fails. That's very discouraging to article writers. We have the fail option, we need to use it. Articles that clearly fail to meet the criteria can always be brought up to standards in time. If it appears within a few days, then there's the on hold option. Otherwise, it can be failed. But the custodians will, ideally, take those recommendations as tools to build a better article, then renominate it. To expand the quick-fail criteria would be detrimental to the process. It would effectively stop some writers in their tracks because they'd be left with no building materials, so to speak. Lara❤Love 16:19, 15 October 2007 (UTC)
thread_title:  Want to reduce the GA backlog? 

message: This is not really relevant to the content review process specifically, but is so important to the entire mission of Wikipedia that I must mention it here to counter the plainly incorrect assertions made above that "Wikipedia is an encyclopedia for a general readership, not a specialised audience", "If a reasonably knowledgeable reviewer can't understand the article then it's probably not suitable for the encyclopedia", and that it is a problem if some articles are "not written for even the educated layman".
Wrong wrong wrong. And I am not expressing a point of view here.
:"Imagine a world in which every single person on the planet is given free access to the sum of all human knowledge. That's what we're doing." (Jimmy Wales)
Note that he does not say "all human knowledge which is understandable by the average reader".
The "Five Pillars" make this utterly explicit.
:"Wikipedia is an encyclopedia written for the benefit of its readers. It incorporates elements of general encyclopedias, specialized encyclopedias, and almanacs." (Pillar One)
In WP:NOT and WP:N, the meanings of "knowledge" and "encyclopedia" are clarified by excluding certain types of information, such as directories, and non-notable content, but nowhere is "knowledge" limited to that which is understandable by the general reader or educated layman.
I suspect that those who think otherwise are mentally comparing Wikipedia to Encyclopedia Britannica, but Wikipedia is not paper: EB is 20 times smaller than WP, with only about 70000 articles on 1/2 million topics. I find it more helpful to think of WP as a nested family of overlapping encyclopedias, rather than a single monolithic encyclopedia.
Of course, we should strive to make every article as widely accessible as possible, but this is not a criterion for inclusion. Geometry guy 13:35, 14 October 2007 (UTC)
replies: text: Point taken; Wikipedia is not Wikiversity, although I do think technical/specialised articles should at least include enough non-technical explanation and background so that the layman can understand why they are notable. My argument is not so much about content, but the way in which it is presented. EyeSereneTALK 14:37, 14 October 2007 (UTC)
replies: text: Quite. Geometry guy confuses "knowledge" with its presentation. And no amount of "Wrong, wrong, wrong" statements such as the rather patronising one above will persuade me otherwise. --Malleus Fatuarum 16:45, 14 October 2007 (UTC)
replies: text: I agree entirely with EyeSerene, and it was precisely this point I was alluding to in the last sentence of my post. I'm sorry that you found the emphatic tone of my argument patronising. That was not my intention. I spoke stridently because this is one aspect of Wikipedia that I am passionate about. However, I assure you, I am absolutely not confusing knowledge with presentation. Every article should be presented in as widely accessible way as possible, but that does not mean that material which cannot be presented in a way that is accessible to the general readership should be omitted. It would not be omitted in a specialist encyclopedia, so it should not be omitted here. Geometry guy 18:45, 14 October 2007 (UTC)

text: Yes, I agree.  The beauty of Wikipedia is that there is room for an article on persistent carbene as well as one on atom.  I also agree that for a specialized article, the introduction at least should be accessible to the lay reader; there's no way you can have a meaningful article that is completely accessible, though.  That's why we need both types of reviewers - specialists and generalists - and ideally they should agree that the level of writing is right.  Personally, when I write a specialised chem article like gold(III) chloride I try to write an intro at the high school level, then some basic content at the undergrad level, but then add some valuable recent material for the professional chemist.  And think about how many non-chemists will go searching for an article on gold(III) chloride or persistent carbene...! Walkerma 16:40, 14 October 2007 (UTC)
replies: text: I largely agree, except for one thing: WP:LEAD. The lead should summarize the article, which means that for some articles the lead will not be entirely accessible to the lay reader. However, often this can be componsated by having introductory material in other sections. Geometry guy 18:45, 14 October 2007 (UTC)
thread_title: A very important aside

message: I'd like to go ahead with a straw poll to pick a topic for us to focus on, per the discussions above.  I didn't hear anyone suggest that a focus topic was a bad idea, so let's try it.
My suggested format would be: paste in the list of problems/topics on the project front page, and simply put Support followed by your name underneath the one you think we should focus on next.  Sign under more than one if you'd be happy to focus on either one; I suggest we pick only one to focus on at once, though.  Discussion (e.g. "This one is a bad choice because we can't fix unless problem X is fixed first") can take place under each one too, with Comment prefixes if necessary.
Here's the list, with my support in place. Mike Christie (talk) 13:02, 14 October 2007 (UTC)
thread_title:  Straw poll for Current Topic 

message: ===WikiProject Biography peer reviews===
Would anyone object to me editing the bottom of the Peer review instructions as below? It would enable me to stop transcluding the WikiProject Biographies to this page, as they would show up in the announcement list instead. DrKiernan 08:12, 17 October 2007 (UTC)
replies: text: If I understand the change, you're removing the Biography peer reviews from the list of specialized peer reviews and transcluding them directly into the list of reviews; is that correct?  If so, the only comment I'd have is that some of the ideas being discussed here might conflict with this approach.  But the real issue is that I would think this is a question for PR, not for this group. Mike Christie (talk) 11:55, 17 October 2007 (UTC)
replies: text: I'm suggesting removing the Wikipedia:WikiProject Biography/Peer reviews currently transcluded to the Wikipedia:Peer review page (e.g. ) and instead transcluding this list: Wikipedia:WikiProject Biography/Peer review/Announce. DrKiernan 12:50, 17 October 2007 (UTC)
thread_title:  Tweak suggested 

message: Would anyone object to me editing the bottom of the Peer review instructions as below? It would enable me to stop transcluding the WikiProject Biographies to this page, as they would show up in the announcement list instead. DrKiernan 08:12, 17 October 2007 (UTC)
replies: text: If I understand the change, you're removing the Biography peer reviews from the list of specialized peer reviews and transcluding them directly into the list of reviews; is that correct?  If so, the only comment I'd have is that some of the ideas being discussed here might conflict with this approach.  But the real issue is that I would think this is a question for PR, not for this group. Mike Christie (talk) 11:55, 17 October 2007 (UTC)
replies: text: I'm suggesting removing the Wikipedia:WikiProject Biography/Peer reviews currently transcluded to the Wikipedia:Peer review page (e.g. ) and instead transcluding this list: Wikipedia:WikiProject Biography/Peer review/Announce. DrKiernan 12:50, 17 October 2007 (UTC)
thread_title: WikiProject Biography peer reviews

message: um, I just finished !voting... but I'm not sure if my non-votes were in the right place.. this page is so out of hand! Why did we split the P1 list etc into Dueling Lists, with one called "nitty gritty details"? err... uhhh.... mmmmm --Ling.Nut 17:33, 19 October 2007 (UTC)
replies: text: I don't have a map either, but I don't think we're in Kansas any more... EyeSereneTALK 17:50, 19 October 2007 (UTC)
replies: text: The second list is a follow-up of the first one. We all seem to be agreed in principle, so we need to start working on the details: that was the idea of the second list. Marskell has proposed (and I agree with him) that we consider them one by one, and so started a thread on the first one just above this. Geometry guy 18:04, 19 October 2007 (UTC)
replies: text: Supp... er... oh! Right! That does help a lot, actually - thanks Gguy ;) EyeSereneTALK 18:13, 19 October 2007 (UTC)
thread_title:  help! 

message: Looks like forward progress is taking place, thanks to many contributors... but hey.. let's not forget there are other major areas of discussion that we have left largely untouched (GA being the primary example). Let's not get too bogged down in PR. I know no one will agree, since this is wikipedia, but I favor setting a time-limit on this PR discussion. One more week would be the most unimaginably vast expanse of time I could possibly bring myself to suggest. Three more days would be better. GA needs a training program, needs... and... and... the task is never completed;  only stopped at some reasonable point --Ling.Nut 14:20, 21 October 2007 (UTC)
thread_title:  bogged down in PR..... 

message: :*Ban all bots at PR
:*Create a script that could be run at PR that would highlight MOS errors interactively
:*:On this topic, take a look at this diff showing Brighterorange's script at work on a current FA candidate.  This is quite impressive.  It looks like it would take a lot of tedium out of the necessary business of having a house style.  More capabilities along these lines would be great. Mike Christie (talk) 15:28, 16 October 2007 (UTC)
thread_title: Augment or modify PR's tools or features

message: No, think not on the word "bureaucracy"; I have not come to establish levels of reviewers. Instead I come bearing the idea that there could be 5 or so levels of reviews labeled simply A through E (nothing too descriptive.. that would be a headache).
The point is to make the process of reviewing more well-defined & less intimidating.
Level E, the easiest, would be just three or four thingies.. spellcheck, I dunno what else. That way newbies could swim in the kiddie pool, non-initimidating, etc. Some points:
* I don't think articles should be allowed to request a review level; everyone will ask for A so that they can get as much bang for the buck as possible. 
* Different people can offer different levels of review on the same article: I could do an E review, then Awadewit or LaraLove or SandyGeorgia could swoop in and do an A on that same article, if they desire.
There are only 2 requirements placed upon reviewers:
#When you review, you explicitly state the level offered.
# For any level of review offered by a reviewer for a given article, you must make a good faith effort to provide all the reviewing aspects required for that level.
The only trouble, of course,  would be coming up with the right mix of clearly spelled-out requirements for each level.. but y'all are smart... trust me, you can make it (that line gets laughs in Taiwan; long story). Cheers! --Ling.Nut
:Interesting idea but I'd prefer to see reviewee and reviewers actually working on the article together to improve it. Thus it becomes less of a review and more of a seminar. My take is that the influx of new editors would iron out wrinkles swiftly. This already happens informally of course but PR could be were refocused (repurposed) more specifically towards cooperative effort. I'm not sure I like the idea of Wiki gate-keepers, by the way. --ROGER DAVIES TALK 15:23, 16 October 2007 (UTC)
replies: text: Yeah of couse. That would be part of the process (wondering why anyone would think it wouldn't). Moreover, where are the wiki-gatekeepers in my idea? I must have made it sound completely different than the way it appears in my head. --Ling.Nut 15:29, 16 October 2007 (UTC)
replies: text: Sorry if I misunderstood you. I have seen many peer reviews which aren't hands on. They're a list of tasks for the reviewee to complete. I'm advocating writing hands on specifically into the process.--ROGER DAVIES TALK 15:49, 16 October 2007 (UTC)

text: I think a move towards a more cooperative type of review sounds great.  I think trying to put a level (A-E) adds an unnecessary extra level of (yes!) bureaucracy to it.  One of the nice things about informal review is that it starts with one person fixing a couple of paragraphs, and it ends up being three or four people doing a wholesale rewrite - you never know where it's going!  But the general approach sounds good. Walkerma 03:27, 17 October 2007 (UTC)
thread_title: Multi-tiered Peer Review

message: Proposal: Peer reviews should take place on the talk pages of articles.
Comments:
*I haven't seen any disagreement with this proposal yet (but there is a lot of discussion going on). It seems like an excellent idea, as it would promote involved reviewing. Awadewit | talk 06:30, 18 October 2007 (UTC)
thread_title: P1

message: Proposal: Peer review requests should be categorized on the peer review page, but the names of the articles should still be visible (much like GAC is now).
Comments:
*I also think that this is a good idea - there needs to be some organization! (But then, I am a compulsive organizer. I have all of my books entered into a database.) Awadewit | talk 06:32, 18 October 2007 (UTC)
replies: text: *I think that it is parallel at this point. Awadewit | talk 10:54, 18 October 2007 (UTC)
replies: text: *Parallel in the good sense, I think. We can do both. Marskell 15:26, 18 October 2007 (UTC)
replies: text: *Thanks Awadewit and Marskell for the clarification ;) I'll add my support too (might be worth linking the nomination category to the appropriate section of the reviewer list if the list ends up being organised that way). EyeSereneTALK 16:32, 18 October 2007 (UTC)

text: *PS, there's been a lot of movement with no input from Gimmetrow; are we prepared to undo a lot of  ?  GAC and GAR can be passed by one editor and have no file in articlehistory, which is frustrating;  peer review currently does have a file and can always be found.  SandyGeorgia (Talk) 02:19, 25 October 2007 (UTC)
replies: text: Huh? Last time I looked, GAC & GAR were both options in the  template. Dr. Cash 02:25, 25 October 2007 (UTC)
replies: text: They have no archived page; sometimes they may have a talk page link, wihch is most often hard to find.  *Every* FAC and FAR has a page, which is archived, as does *every* PR.  There is no such thing on GAC and GAR, and when trying to correct and rebuild articlehistory GA errors using Dr pda's script, one often has to spend a long time fiddling through history and diffs to find the GA event.  This is taking a weakness of GA relative to other processes (no separate archivable page) and implementing it in a process which has always had an archived PR page. SandyGeorgia (Talk) 03:05, 25 October 2007 (UTC)
thread_title: P2

message: Proposal: There should be a list of peer reviewers, listing their interests, that editors can contact.
Comments:
*I, personally, would love such a list. I think it make it easier to connect reviewers with editors, but I understand the reservations others have. I wonder if separating the list from the peer review page itself a little, as GAC has done (they also have such a list), might work. Awadewit | talk 06:29, 18 October 2007 (UTC)
replies: text: *Like the idea, and like Awadewit's addition to it to have the list on a separate page.  It almost goes without saying, too, that the list should be voluntary and optional, and that anyone at anytime should be able to review an article.--Jayron32|talk|contribs 06:43, 18 October 2007 (UTC)
replies: text: *Yes, and these could be included on the main PR page (if not, new editors may miss it). Point not just to reviewers but clearly active projects, as well. Marskell 08:23, 18 October 2007 (UTC)
thread_title: P3

message: Proposal: There should be some sort of automated process for adding and removing peer review requests as well as peer reviewers.
Comments:
*Maybe, but I think there should NOT be bot-generate or script generated reviews.  These are never helpful, and somewhat rude.  If you don't have the time to review, IMHO, it is better to say nothing than to leave an unhelpful, script generated generic review.  However, certain basic automated tasks, like clearing reviews from the page after 1 month may be a good idea.  Also, this may be able to be combined with the above, in that we can have a bot that buzzes people's talk page when a review appears that matches their interest.  For example, lets say that I have an interest in reviewing sports articles.  When a new article gets added to the sports list, a bot could (if I request that it does) drop a note on my talk page asking me to review the article.  How does that sound? --Jayron32|talk|contribs 06:39, 18 October 2007 (UTC)
thread_title: P4

message: Proposal: There should be some sort of reward system for peer reviewing. Options include a list of top reviewers, such as WP:WBFAN; barnstars for the reviewers who review the greatest number of articles; barnstars for the reviewers who offer the most thorough reviews (as counted by kb); or just a general encouragement of personal barnstar giving.
Comments:
*I think that personal barnstars should be the limit. There are too many problems with counting the numbers of reviews (encourages short, uninvolved reviews) and counting the bits/bytes (what would be counted, exactly?). Awadewit | talk 06:28, 18 October 2007 (UTC)
replies: text: *Again, I agree with Awadewit.  If someone notices that another editor is doing a lot of work at PR, a personal barnstar may be a good idea. Also, if there is a short-duration drive, like a backlog elimination drive, with moderators watching it, and there are "barnstar prizes" given for that one-off event, that may be OK, but having a quid-pro-quo system of automatically awarding barnstars for reaching certain benchmarks seems like a bad idea.  --Jayron32|talk|contribs 06:42, 18 October 2007 (UTC)
thread_title: P5

message: It looks like we are really getting somewhere! I think it is time to try and iron out exactly what our proposal is.
===P 1===
We want peer reviews of articles at any stage to be done on the talk pages of the articles.
===P 1 and P 2===
We want peer reviews to be listed at a centralized PR page.
===P 2===
We want peer reviews to be categorized at the centralized PR page. What is not clear is what categorization scheme should be used. Possibilities include: FA, and GA top-level (or equivalently Wikipedia 1.0).
More discussion is needed here:
*I think the GA-top level categories are a good choice. Since not all articles can become FA, this might provide some uniformity. I know less about the 1.0 categories. Awadewit | talk 20:47, 18 October 2007 (UTC)
replies: text: *This word "hierarchical" is really distracting to me for some reason! Perhaps "evaluative"? I actually think that reviewers' knowledge of the subject area is quite relevant. There are certain subjects that I would never review, because I have so little knowledge of them that I could not provide much help at all. Sports comes to mind immediately. I would have no idea if the page was well-researched (what are best sources for soccer players? I have no idea), comprehensive, or well-organized. Well-written I might know something about, but even that is difficult, because each discipline has its own language. Copy editing sports pages is a nightmare for me because I don't know the jargon. I agree that reviewers should not necessarily be tied to only one category and not always tied to that category, but I think that many of us would find such a list helpful. What do others think? Awadewit | talk 21:09, 18 October 2007 (UTC)

text: That word "hierarchical" is at the root of the problem with PR. Euphamism or neologism don't solve that problem. --Malleus Fatuarum 21:18, 18 October 2007 (UTC)
replies: text: *I'm sorry, I should have been clearer. I didn't mean for PR to use that word, I meant for you to use that word. I think it would be easier to follow your criticisms of the current PR process if you didn't use the word "hierarchical" and used the word "evaluative" or some such synonym. I must say that I don't really see this as a problem as PR at all. Most editors asking for reviews post comments like "This article is 'start' and I would like to get it to GA - please help me" or "This article is GA, please help me get it to FA" or some such. I have just never gotten the feeling that PR has the kinds of problem you are outlining. That does not mean we shouldn't try to make it as open as possible, but I think we might be trying to solve a non-problem. Could you maybe outline in a little more detail what you see as the exact problem? Thanks. Awadewit | talk 21:35, 18 October 2007 (UTC)
replies: text: *I have attempted to outline what I see as the problem with PR on a number of occasions now. I have no intention of continuing to shout into the wind. We clearly have to agree to disagree. --Malleus Fatuarum 21:44, 18 October 2007 (UTC)
replies: text: *Actually, it is not clear to me at all what you see as the major problems at PR at all and I just spent time rereading every single one of your posts on this page. I'm not really sure I'm disagreeing with you, because I'm not really sure what you are saying. I'm sorry if you think I'm being dense, but in an environment where everything is written, it is best to spell out one's ideas as clearly as possible, in my experience. Awadewit | talk 21:55, 18 October 2007 (UTC)

text: If I can attempt to translate again, I think Malleus means that if PR is not part of a hierarchy of review processes, then its reviewers should be generalists. If that is the case, then I disagree. For me, the watchword for PR is flexibility. A peer reviewer can approach the article as a generalist or an expert: all input is welcome. There is no particular goal other than improving the article. I think it would be useful to list reviewers and their interests, but I agree with Malleus that it would be a bad idea to pigeonhole reviewers into the same categories as articles. Geometry guy 22:32, 18 October 2007 (UTC)
replies: text: *I don't think anybody was suggesting that peer reviewers should only review one category of articles, were they? I too, agree, that reviewers should be able to review as generalists or as specialists. The list would help those interested in specialist-reviews. Awadewit | talk 22:36, 18 October 2007 (UTC)
replies: text: Agreed. Geometry guy 22:41, 18 October 2007 (UTC)

text: *I don't think it would be a masquerade to have people with some knowledge of a topic reviewing articles about those topics - it would be a benefit to the article. If you have ever had someone review an article about which they know nothing, you will realize why this can lead to disaster. Please try to imagine reviewing an article on a topic about which you know nothing. Could you really do a good job? This doesn't mean there shouldn't be reviewers who provide a "lay" perspective (I often do this for science articles myself), but articles must be reviewed at some point by people who know what they are doing, otherwise there is no real way to tell how good the article is. For example, if I tell you that I have used the best Mary Wollstonecraft scholarship to write the nine articles on her and her works that I wrote, how do you know that is true? You could easily find some article or book that says something different about Wollstonecraft than the articles I wrote. However, I spent a long time culling the best sources for what major scholars agree upon. Only someone else who has done this would really know that. Otherwise you have to trust me. In some ways, this is one of the fundamental problems of wikipedia. It is often very difficult to know how good someone's research is, if you yourself do not know the field. That is one reason why it is good to have reviewers who know something about the subject, who can say, "wait! you're missing this major theory!" Awadewit | talk 21:47, 18 October 2007 (UTC)
replies: text: *I'm afraid that your reply simply confirms to me that you have not taken on board the idea that PR is not a peer review. Hence, I suppose, your earlier confusion about "hieracrchy". --Malleus Fatuarum 00:15, 19 October 2007 (UTC)
replies: text: *It is not an academic peer review, but that is something very different entirely, which relies on credentialed experts. When you have submitted articles to peer review, what has been your experience with generalized vs. specialist reviewers? Awadewit | talk 00:23, 19 October 2007 (UTC)
replies: text: *Quite. So my access to your sources of knowledge about what you consider to be the "best" scholarship on a subject such as Mary Wollstencraft is not relevant to my ability to conduct a PR review. Unless you consider PR to be a part of the article ranking hierarchy of course. --Malleus Fatuarum 00:33, 19 October 2007 (UTC)
replies: text: *Your use of the word "hierarchy" here is continually confusing and I can barely parse your sentence. I can't see that you would have any access to "sources of knowledge about what I consider...." There are no sources about what I personally think. I used sources that are considered the best, demonstrably so. I simply do not understand why you think PR is "hierarchical" in any way. Articles gain nothing in the wikipedia "hierarchy" of grades by going through it, unlike GAC, FAC, or A-review. Awadewit | talk 01:09, 19 October 2007 (UTC)

text: *Sorry, I forgot to answer the last part of your question; "When you have submitted articles to peer review, what has been your experience with generalized vs. specialist reviewers?" Are you asking me about real life or on wikipedia?--Malleus Fatuarum 00:37, 19 October 2007 (UTC)
replies: text: *I am only talking about wikipedia PR. Awadewit | talk 01:09, 19 October 2007 (UTC)

text: I agree with Ling here (but then, he is my bad hand sock, so of course I would :-) that PR can offer both generalist and specialist reviews (and anything inbetween); it should be flexible. In my view, Malleus is using the word "hierarchy" in two different senses: the Wikipedia article quality hierarchy, and the hierarchy of knowledge. This is confusing. We seem to be agreed that PR should be separate from the article quality hierarchy. For me that strengthens the case that peer reviews can be as general or as specialized as the reviewer chooses. I can elaborate if necessary, but this topic has already received plenty of discussion. Geometry guy 17:45, 19 October 2007 (UTC)

text: Why? I agree that removing editors from lists is a bad idea, but it makes no difference whether a bot or another editor does it: it is still a bad idea. I don't see any bot craziness here, and you make no arguments to support your position. Ah... I get it now: you are setting up a straw man to make it easier for me to make my case :-) No, seriously, I have in mind that there would only be one bot: the question is, what tasks would it perform. In my view, the main work-saving task, and the one which seems to receive the most support elsewhere, is to maintain (and in my view, entirely autogenerate) the PR main page. I suggest starting from that position, and then applying your principle to other tasks that we might consider automating. Geometry guy 17:53, 19 October 2007 (UTC)

text: PS. I hope you don't mind me breaking your comment to restart the thread.
*(undent). I don't mind anything, except the crazy schizophrenic organization of this page.grumble grumble
* I don't like bots 'cause they're cool. I'm not joking. Cool things scare me. Again, I'm not joking. They scare me 'cause everybody wants to be the bright girl or boy who comes up with a new bot to organize this, delete that, soon we'll have a barnstar bot counting bytes and awarding barnstars, then we'll have a bot adding people to the membership list (resistance is futile -  you will be assimilated), then we'll have bots making reviews (which already happens.. I send out lotsa wiki-love to whoever invented that bot and whoever has used it, but stop now please, 'kay?)... I said it earlier.. and I meant it.. use bots if and only if they make a meaningful contribution in a way that is really really really a pain to do manually. Plus i have seen some folks express concerns that bots gobble up resources... --Ling.Nut 18:21, 19 October 2007 (UTC)

text: Well, I agree that bots should be used only where necessary, and that bots should be simple. Bots have a control setting which ensures that they don't use up resources, but the kind of bot I am proposing will not even need that, because: (1) It will only write one or two pages, perhaps once or twice each, per day; (2) It will never, ever, download a page!! Geometry guy 18:47, 19 October 2007 (UTC)

text: I'm lost. (Though your attempts to organize have been great, Awadewit.) Can we go through P1 to P5, one-by-one? Marskell 21:10, 18 October 2007 (UTC)
replies: text: *Sure, why don't you lay it out. Awadewit | talk 21:47, 18 October 2007 (UTC)
replies: text: OK! Marskell 21:57, 18 October 2007 (UTC)
replies: text: This already is laid out I think: try this. Geometry guy 22:04, 18 October 2007 (UTC)
replies: text: No. The original poll was edifying—this sub-thread is just unspecifying what had been specified. Marskell 22:11, 18 October 2007 (UTC)
replies: text: Okay, I've sub-subthreaded it, in case it is still useful, but we can move on if you like. Geometry Marskell 22:50, 18 October 2007 (UTC)
replies: text: I think this thread is officially nuts. (Though the original poll was a great act of clarification.) Marskell 22:21, 18 October 2007 (UTC)

text: But it was a good thing it did happen. We should not be anti-poll. We should be anti too many discussions at once. Marskell 22:32, 18 October 2007 (UTC)
replies: text: I think Awadawit did a great job there. In my view, the reason people started writing "support" was because of this: the essence of our common view had been distilled. This is a great place to move forward from. Let us make the best of it! Geometry guy 22:35, 18 October 2007 (UTC)
replies: text: Yes, let's—material available. This thread now has thirteen headlines and would be impossible to understand other than by the ten odd people who have already commented. Scrub what I wrote in the next thread, but still come up with something focused on a given point of agreement and how it might be implemented. Marskell 22:40, 18 October 2007 (UTC)
replies: text: That's actually exactly what I was trying to do, hence the title "nitty-gritty details". Sorry I wasn't more clear about that. Awadewit | talk 22:44, 18 October 2007 (UTC)
replies: text: No, it's all of the details at once that I'm not understanding. We're just going to get lost in talk. We winnowed and now, weirdly, we're broadening again. But whatever. Bedtime here. Marskell 22:50, 18 October 2007 (UTC)
replies: text: I don't think it is broadening, but isolating the precise proposal we want to make takes time and, because it is all written out, painstaking work. Good night! Awadewit | talk 22:52, 18 October 2007 (UTC)

text: I see I came down rather hard on this thread! Sorry. I was a little bleary-eyed. Yes, it does contain useful info already, so we can consider it as we move to each point. Marskell 06:47, 19 October 2007 (UTC)
replies: text: Well it is rather fragmented and has some irrelevant digressions, and I think it is a good idea to proceed point by point. Unfortunately, the thread below is developing digressions as well. I guess there is so much consensus around, that we have to digress to find something to argue about ;-) Geometry guy 17:57, 19 October 2007 (UTC)
thread_title:  Nitty-gritty details 

message: We want peer reviews to be categorized at the centralized PR page. What is not clear is what categorization scheme should be used. Possibilities include: FA, and GA top-level (or equivalently Wikipedia 1.0).
More discussion is needed here:
*I think the GA-top level categories are a good choice. Since not all articles can become FA, this might provide some uniformity. I know less about the 1.0 categories. Awadewit | talk 20:47, 18 October 2007 (UTC)
thread_title: P 2

message: We want a list of peer reviewers, by interest. It is not clear whether or not this should be included on the same page as the peer review requests themselves.
More discussion is needed here:
*I would suggest a prominent link, otherwise the page will be too long. Look at GAC already. Awadewit | talk 20:47, 18 October 2007 (UTC)
replies: text: *This word "hierarchical" is really distracting to me for some reason! Perhaps "evaluative"? I actually think that reviewers' knowledge of the subject area is quite relevant. There are certain subjects that I would never review, because I have so little knowledge of them that I could not provide much help at all. Sports comes to mind immediately. I would have no idea if the page was well-researched (what are best sources for soccer players? I have no idea), comprehensive, or well-organized. Well-written I might know something about, but even that is difficult, because each discipline has its own language. Copy editing sports pages is a nightmare for me because I don't know the jargon. I agree that reviewers should not necessarily be tied to only one category and not always tied to that category, but I think that many of us would find such a list helpful. What do others think? Awadewit | talk 21:09, 18 October 2007 (UTC)

text: That word "hierarchical" is at the root of the problem with PR. Euphamism or neologism don't solve that problem. --Malleus Fatuarum 21:18, 18 October 2007 (UTC)
replies: text: *I'm sorry, I should have been clearer. I didn't mean for PR to use that word, I meant for you to use that word. I think it would be easier to follow your criticisms of the current PR process if you didn't use the word "hierarchical" and used the word "evaluative" or some such synonym. I must say that I don't really see this as a problem as PR at all. Most editors asking for reviews post comments like "This article is 'start' and I would like to get it to GA - please help me" or "This article is GA, please help me get it to FA" or some such. I have just never gotten the feeling that PR has the kinds of problem you are outlining. That does not mean we shouldn't try to make it as open as possible, but I think we might be trying to solve a non-problem. Could you maybe outline in a little more detail what you see as the exact problem? Thanks. Awadewit | talk 21:35, 18 October 2007 (UTC)
replies: text: *I have attempted to outline what I see as the problem with PR on a number of occasions now. I have no intention of continuing to shout into the wind. We clearly have to agree to disagree. --Malleus Fatuarum 21:44, 18 October 2007 (UTC)
replies: text: *Actually, it is not clear to me at all what you see as the major problems at PR at all and I just spent time rereading every single one of your posts on this page. I'm not really sure I'm disagreeing with you, because I'm not really sure what you are saying. I'm sorry if you think I'm being dense, but in an environment where everything is written, it is best to spell out one's ideas as clearly as possible, in my experience. Awadewit | talk 21:55, 18 October 2007 (UTC)

text: If I can attempt to translate again, I think Malleus means that if PR is not part of a hierarchy of review processes, then its reviewers should be generalists. If that is the case, then I disagree. For me, the watchword for PR is flexibility. A peer reviewer can approach the article as a generalist or an expert: all input is welcome. There is no particular goal other than improving the article. I think it would be useful to list reviewers and their interests, but I agree with Malleus that it would be a bad idea to pigeonhole reviewers into the same categories as articles. Geometry guy 22:32, 18 October 2007 (UTC)
replies: text: *I don't think anybody was suggesting that peer reviewers should only review one category of articles, were they? I too, agree, that reviewers should be able to review as generalists or as specialists. The list would help those interested in specialist-reviews. Awadewit | talk 22:36, 18 October 2007 (UTC)
replies: text: Agreed. Geometry guy 22:41, 18 October 2007 (UTC)
thread_title: P 3

message: We want automation of some sort. Possibilities include: bots that alert subject area reviewers when new articles appear in their categories; removing inactive reviewers; removing PR requests that haven't been filled; some sort of bot that reduces the work of the GA-system.
More discussion is needed here:
*I like the idea of bots alerting reviewers, if that is possible. I don't like the idea of being removed (unless the user has been indefinitely banned). I think maybe a 30-day waiting period is more realistic than a 14-day waiting period at this point. Whatever automation reduces the clicking around, I am in favor of. :) Awadewit | talk 20:49, 18 October 2007 (UTC)
replies: text: *I don't think it would be a masquerade to have people with some knowledge of a topic reviewing articles about those topics - it would be a benefit to the article. If you have ever had someone review an article about which they know nothing, you will realize why this can lead to disaster. Please try to imagine reviewing an article on a topic about which you know nothing. Could you really do a good job? This doesn't mean there shouldn't be reviewers who provide a "lay" perspective (I often do this for science articles myself), but articles must be reviewed at some point by people who know what they are doing, otherwise there is no real way to tell how good the article is. For example, if I tell you that I have used the best Mary Wollstonecraft scholarship to write the nine articles on her and her works that I wrote, how do you know that is true? You could easily find some article or book that says something different about Wollstonecraft than the articles I wrote. However, I spent a long time culling the best sources for what major scholars agree upon. Only someone else who has done this would really know that. Otherwise you have to trust me. In some ways, this is one of the fundamental problems of wikipedia. It is often very difficult to know how good someone's research is, if you yourself do not know the field. That is one reason why it is good to have reviewers who know something about the subject, who can say, "wait! you're missing this major theory!" Awadewit | talk 21:47, 18 October 2007 (UTC)
replies: text: *I'm afraid that your reply simply confirms to me that you have not taken on board the idea that PR is not a peer review. Hence, I suppose, your earlier confusion about "hieracrchy". --Malleus Fatuarum 00:15, 19 October 2007 (UTC)
replies: text: *It is not an academic peer review, but that is something very different entirely, which relies on credentialed experts. When you have submitted articles to peer review, what has been your experience with generalized vs. specialist reviewers? Awadewit | talk 00:23, 19 October 2007 (UTC)
replies: text: *Quite. So my access to your sources of knowledge about what you consider to be the "best" scholarship on a subject such as Mary Wollstencraft is not relevant to my ability to conduct a PR review. Unless you consider PR to be a part of the article ranking hierarchy of course. --Malleus Fatuarum 00:33, 19 October 2007 (UTC)
replies: text: *Your use of the word "hierarchy" here is continually confusing and I can barely parse your sentence. I can't see that you would have any access to "sources of knowledge about what I consider...." There are no sources about what I personally think. I used sources that are considered the best, demonstrably so. I simply do not understand why you think PR is "hierarchical" in any way. Articles gain nothing in the wikipedia "hierarchy" of grades by going through it, unlike GAC, FAC, or A-review. Awadewit | talk 01:09, 19 October 2007 (UTC)

text: *Sorry, I forgot to answer the last part of your question; "When you have submitted articles to peer review, what has been your experience with generalized vs. specialist reviewers?" Are you asking me about real life or on wikipedia?--Malleus Fatuarum 00:37, 19 October 2007 (UTC)
replies: text: *I am only talking about wikipedia PR. Awadewit | talk 01:09, 19 October 2007 (UTC)

text: I agree with Ling here (but then, he is my bad hand sock, so of course I would :-) that PR can offer both generalist and specialist reviews (and anything inbetween); it should be flexible. In my view, Malleus is using the word "hierarchy" in two different senses: the Wikipedia article quality hierarchy, and the hierarchy of knowledge. This is confusing. We seem to be agreed that PR should be separate from the article quality hierarchy. For me that strengthens the case that peer reviews can be as general or as specialized as the reviewer chooses. I can elaborate if necessary, but this topic has already received plenty of discussion. Geometry guy 17:45, 19 October 2007 (UTC)

text: Why? I agree that removing editors from lists is a bad idea, but it makes no difference whether a bot or another editor does it: it is still a bad idea. I don't see any bot craziness here, and you make no arguments to support your position. Ah... I get it now: you are setting up a straw man to make it easier for me to make my case :-) No, seriously, I have in mind that there would only be one bot: the question is, what tasks would it perform. In my view, the main work-saving task, and the one which seems to receive the most support elsewhere, is to maintain (and in my view, entirely autogenerate) the PR main page. I suggest starting from that position, and then applying your principle to other tasks that we might consider automating. Geometry guy 17:53, 19 October 2007 (UTC)

text: PS. I hope you don't mind me breaking your comment to restart the thread.
*(undent). I don't mind anything, except the crazy schizophrenic organization of this page.grumble grumble
* I don't like bots 'cause they're cool. I'm not joking. Cool things scare me. Again, I'm not joking. They scare me 'cause everybody wants to be the bright girl or boy who comes up with a new bot to organize this, delete that, soon we'll have a barnstar bot counting bytes and awarding barnstars, then we'll have a bot adding people to the membership list (resistance is futile -  you will be assimilated), then we'll have bots making reviews (which already happens.. I send out lotsa wiki-love to whoever invented that bot and whoever has used it, but stop now please, 'kay?)... I said it earlier.. and I meant it.. use bots if and only if they make a meaningful contribution in a way that is really really really a pain to do manually. Plus i have seen some folks express concerns that bots gobble up resources... --Ling.Nut 18:21, 19 October 2007 (UTC)

text: Well, I agree that bots should be used only where necessary, and that bots should be simple. Bots have a control setting which ensures that they don't use up resources, but the kind of bot I am proposing will not even need that, because: (1) It will only write one or two pages, perhaps once or twice each, per day; (2) It will never, ever, download a page!! Geometry guy 18:47, 19 October 2007 (UTC)
thread_title: P 4

message: *...I don't see the need for subject area reviewers to be notified, or indeed the need for there to be subject area reviewers at all for that matter. Unless we want PR to masquerade as something that it doesn't claim to be. --Malleus Fatuarum 21:13, 18 October 2007 (UTC)
replies: text: *I don't think it would be a masquerade to have people with some knowledge of a topic reviewing articles about those topics - it would be a benefit to the article. If you have ever had someone review an article about which they know nothing, you will realize why this can lead to disaster. Please try to imagine reviewing an article on a topic about which you know nothing. Could you really do a good job? This doesn't mean there shouldn't be reviewers who provide a "lay" perspective (I often do this for science articles myself), but articles must be reviewed at some point by people who know what they are doing, otherwise there is no real way to tell how good the article is. For example, if I tell you that I have used the best Mary Wollstonecraft scholarship to write the nine articles on her and her works that I wrote, how do you know that is true? You could easily find some article or book that says something different about Wollstonecraft than the articles I wrote. However, I spent a long time culling the best sources for what major scholars agree upon. Only someone else who has done this would really know that. Otherwise you have to trust me. In some ways, this is one of the fundamental problems of wikipedia. It is often very difficult to know how good someone's research is, if you yourself do not know the field. That is one reason why it is good to have reviewers who know something about the subject, who can say, "wait! you're missing this major theory!" Awadewit | talk 21:47, 18 October 2007 (UTC)
replies: text: *I'm afraid that your reply simply confirms to me that you have not taken on board the idea that PR is not a peer review. Hence, I suppose, your earlier confusion about "hieracrchy". --Malleus Fatuarum 00:15, 19 October 2007 (UTC)
replies: text: *It is not an academic peer review, but that is something very different entirely, which relies on credentialed experts. When you have submitted articles to peer review, what has been your experience with generalized vs. specialist reviewers? Awadewit | talk 00:23, 19 October 2007 (UTC)
replies: text: *Quite. So my access to your sources of knowledge about what you consider to be the "best" scholarship on a subject such as Mary Wollstencraft is not relevant to my ability to conduct a PR review. Unless you consider PR to be a part of the article ranking hierarchy of course. --Malleus Fatuarum 00:33, 19 October 2007 (UTC)
replies: text: *Your use of the word "hierarchy" here is continually confusing and I can barely parse your sentence. I can't see that you would have any access to "sources of knowledge about what I consider...." There are no sources about what I personally think. I used sources that are considered the best, demonstrably so. I simply do not understand why you think PR is "hierarchical" in any way. Articles gain nothing in the wikipedia "hierarchy" of grades by going through it, unlike GAC, FAC, or A-review. Awadewit | talk 01:09, 19 October 2007 (UTC)

text: *Sorry, I forgot to answer the last part of your question; "When you have submitted articles to peer review, what has been your experience with generalized vs. specialist reviewers?" Are you asking me about real life or on wikipedia?--Malleus Fatuarum 00:37, 19 October 2007 (UTC)
replies: text: *I am only talking about wikipedia PR. Awadewit | talk 01:09, 19 October 2007 (UTC)

text: I agree with Ling here (but then, he is my bad hand sock, so of course I would :-) that PR can offer both generalist and specialist reviews (and anything inbetween); it should be flexible. In my view, Malleus is using the word "hierarchy" in two different senses: the Wikipedia article quality hierarchy, and the hierarchy of knowledge. This is confusing. We seem to be agreed that PR should be separate from the article quality hierarchy. For me that strengthens the case that peer reviews can be as general or as specialized as the reviewer chooses. I can elaborate if necessary, but this topic has already received plenty of discussion. Geometry guy 17:45, 19 October 2007 (UTC)
thread_title: Digression on specialist vs generalist peer review

message: * I'm very wary of automation. I especially strongly dislike the idea of removing inactive editors from some mebership list via bot. Membership lists are always and everywhere voluntary anyhow!! Plus.. I've argued above... I don't see the bot-craziness as a good thing in most cases. Let's start with the idea tha that "The fewer bots, the better!" and have only those functions tthat are really necessary and really better performed by bots be done by them. Done talking. Coffee time! --Ling.Nut 03:09, 19 October 2007 (UTC)
replies: text: Why? I agree that removing editors from lists is a bad idea, but it makes no difference whether a bot or another editor does it: it is still a bad idea. I don't see any bot craziness here, and you make no arguments to support your position. Ah... I get it now: you are setting up a straw man to make it easier for me to make my case :-) No, seriously, I have in mind that there would only be one bot: the question is, what tasks would it perform. In my view, the main work-saving task, and the one which seems to receive the most support elsewhere, is to maintain (and in my view, entirely autogenerate) the PR main page. I suggest starting from that position, and then applying your principle to other tasks that we might consider automating. Geometry guy 17:53, 19 October 2007 (UTC)
replies: text: PS. I hope you don't mind me breaking your comment to restart the thread.
*(undent). I don't mind anything, except the crazy schizophrenic organization of this page.grumble grumble
* I don't like bots 'cause they're cool. I'm not joking. Cool things scare me. Again, I'm not joking. They scare me 'cause everybody wants to be the bright girl or boy who comes up with a new bot to organize this, delete that, soon we'll have a barnstar bot counting bytes and awarding barnstars, then we'll have a bot adding people to the membership list (resistance is futile -  you will be assimilated), then we'll have bots making reviews (which already happens.. I send out lotsa wiki-love to whoever invented that bot and whoever has used it, but stop now please, 'kay?)... I said it earlier.. and I meant it.. use bots if and only if they make a meaningful contribution in a way that is really really really a pain to do manually. Plus i have seen some folks express concerns that bots gobble up resources... --Ling.Nut 18:21, 19 October 2007 (UTC)

text: Well, I agree that bots should be used only where necessary, and that bots should be simple. Bots have a control setting which ensures that they don't use up resources, but the kind of bot I am proposing will not even need that, because: (1) It will only write one or two pages, perhaps once or twice each, per day; (2) It will never, ever, download a page!! Geometry guy 18:47, 19 October 2007 (UTC)
thread_title: Discussion of automation continues below

message: We want some sort of reward system, including perhaps all of the following: personal barnstars and barnstars awarded by quantity of reviews (to be determined by a bot somehow - either number of reviews or kb).
More discussion is needed here:
*Do we want to try all of the above barnstars and see what works best? Awadewit | talk 20:49, 18 October 
2007 (UTC)
*These discussions have ended up rather fragmented, but I'll shove this in here ;) Personal barnstars are a great idea - it might be nice to have more than one type (eg an 'official' PR project award, a 'slap on the back' type award from other reviewers, and one that can be awarded by article nominators to show appreciation to particularly helpful reviewers). I don't believe awarding by number of reviews is the best way to go though. I believe that anything that even inadvertently puts quantity over quality is not good. However, if kudos and a sense of friendly competition do need to be generated, listing the number of reviews along with the reveiwer name (as I proposed somewhere above) could meet this need (maybe a bot could handle this). As an aside, it would also help nominators pick an active reviewer from the list. If consensus goes the way of award per review count though, it might be fairer to do this by hand rather than based on a bot, as then quality can also be taken into account. EyeSereneTALK 17:46, 19 October 2007 (UTC)
thread_title: P 5

message: :I'm lost. (Though your attempts to organize have been great, Awadewit.) Can we go through P1 to P5, one-by-one? Marskell 21:10, 18 October 2007 (UTC)
replies: text: *Sure, why don't you lay it out. Awadewit | talk 21:47, 18 October 2007 (UTC)
replies: text: OK! Marskell 21:57, 18 October 2007 (UTC)
replies: text: This already is laid out I think: try this. Geometry guy 22:04, 18 October 2007 (UTC)
replies: text: No. The original poll was edifying—this sub-thread is just unspecifying what had been specified. Marskell 22:11, 18 October 2007 (UTC)
replies: text: Okay, I've sub-subthreaded it, in case it is still useful, but we can move on if you like. Geometry Marskell 22:50, 18 October 2007 (UTC)
replies: text: I think this thread is officially nuts. (Though the original poll was a great act of clarification.) Marskell 22:21, 18 October 2007 (UTC)

text: But it was a good thing it did happen. We should not be anti-poll. We should be anti too many discussions at once. Marskell 22:32, 18 October 2007 (UTC)
replies: text: I think Awadawit did a great job there. In my view, the reason people started writing "support" was because of this: the essence of our common view had been distilled. This is a great place to move forward from. Let us make the best of it! Geometry guy 22:35, 18 October 2007 (UTC)
replies: text: Yes, let's—material available. This thread now has thirteen headlines and would be impossible to understand other than by the ten odd people who have already commented. Scrub what I wrote in the next thread, but still come up with something focused on a given point of agreement and how it might be implemented. Marskell 22:40, 18 October 2007 (UTC)
replies: text: That's actually exactly what I was trying to do, hence the title "nitty-gritty details". Sorry I wasn't more clear about that. Awadewit | talk 22:44, 18 October 2007 (UTC)
replies: text: No, it's all of the details at once that I'm not understanding. We're just going to get lost in talk. We winnowed and now, weirdly, we're broadening again. But whatever. Bedtime here. Marskell 22:50, 18 October 2007 (UTC)
replies: text: I don't think it is broadening, but isolating the precise proposal we want to make takes time and, because it is all written out, painstaking work. Good night! Awadewit | talk 22:52, 18 October 2007 (UTC)

text: I see I came down rather hard on this thread! Sorry. I was a little bleary-eyed. Yes, it does contain useful info already, so we can consider it as we move to each point. Marskell 06:47, 19 October 2007 (UTC)
replies: text: Well it is rather fragmented and has some irrelevant digressions, and I think it is a good idea to proceed point by point. Unfortunately, the thread below is developing digressions as well. I guess there is so much consensus around, that we have to digress to find something to argue about ;-) Geometry guy 17:57, 19 October 2007 (UTC)
thread_title: Question and comments

message: No templates: Start a thread on article talk, note the thread on PR under your preferred subject category.
*Do two things? There must be a way to nominate an article for PR with one stroke. Awadewit | talk 20:50, 21 October 2007 (UTC)
thread_title: Pr1

message: A single topic specific template on article talk: Same as last; avoids the categories, but not the bot.
:Am I right in thinking that if we implement Pr3, Pr4 becomes almost trivial, in that  could be called (if that's the verb I need) by ?  If so, Pr3 permits Pr4 for those who prefer it. Mike Christie (talk) 21:20, 21 October 2007 (UTC)
replies: text: Well, they don't permit eachother insofar as 3 is a single template that allows for different categorization, while 4 is multiple templates that would cover every category. They would accomplish the same thing, though. Marskell 21:33, 21 October 2007 (UTC)
replies: text: If we implement Pr3, then yes, it is perfectly possible to provide  as a synonym for , but there isn't a lot of point in doing so: this wouldn't really be Pr4, just syntactic sugar for those who can't find the pipe character on their keyboard ;-)  Geometry guy 21:46, 21 October 2007 (UTC)
thread_title: Pr4

message: A template each on PR and on article talk: GAN structure.
*I find this tedious - there must be a better way. Awadewit | talk 20:40, 21 October 2007 (UTC)
thread_title: Pr5

message: I have a general idea which I think could help solve the lack of reviewers problem, at least for PR and FAC, as well as some general comments. I may have missed it, but I did not see anything that really addressed the lack of reviewers above. 
At WP:RfC, any new request for comment has to get two users to agree to it before it goes forward. I propose that new peer reviews or Featured Article candidates first enter a wait list. The nominator then has to go and review three other articles (I think three would work better than two, but it should be at least two). They then add the names of the articles they reviewed to their nomination, and once they have met the minimum number requirement, their nomination moves into the actual PR or FAC queue and can be reviewed by others. Outside reviewers would still be welcome, but they could not add comments to any article that was still waiting to enter the PR or FAC queue. This would combine the two main problems (lots of requests, lack of reviewers) and help solve both. I believe it would encourage only serious requests for PR and cut down on multiple simultaneous PR requests on closely related articles (as someone asking for 5 articles to be reviewed would have to first review 15 others). I also think this idea would help, as it is good to look at the work of others and this would force nominators to look at other articles. 
For GAC there is just one review per article, so there a GAC queue could be established and to leave it they could either do one GAC review, OR some combination of three reviews for FAC or PR. 
I think the easiest way to promote a nomination from the wait queue to the review queue would be to have any other editor besides the nominator be allowed to do this. They could look at the three articles and make sure there were substantive reviews by the nominator there. People receiving reviews could also comment on them in the waiting queue. This way if someone was a slacker and their three reviews were just "good job" or something similar, the recipients could note this and the request could go back into the wait queue, or not be promoted.  
I think this system would actually work best for FAC, as the level of experience of editors nominating articles there is highest. On PR, perhaps it would be helpful to have a listing of what the person is asking for. I think three categories would be useful, the next step after this PR is either FAC, GAC, or general improvement. This could help all reviewers, but someone looking for general improvement could review other general improvement articles, etc. This would help match level of reviewers too (so a newbie would not have to review a future FAC).
Finally, I worry about making PR just a list. Now it is easy to see what articles have received no reviews, but I am not sure how easy this would be to see on a list only format. I hope this helps, Ruhrfisch  ><>°° 14:05, 23 October 2007 (UTC)
replies: text: These are all interesting thoughts, but I will just comment on the last point for the moment. It would be possible to show, in a list, which articles have received a review, provided the first reviewer modified the talk page template after making the review. I don't have a particular view on whther this is worthwhile or not. Geometry guy 19:25, 23 October 2007 (UTC)
replies: text: Some sort of check mark, etc. could appear on the PR page. This is not an insurmountable problem, in my opinion, but Ruhrfisch is right to point it out. Awadewit | talk 22:13, 23 October 2007 (UTC)

text:  I don't know what I think about requiring people to do peer reviews. I have a sinking feeling that it might generate poor reviews just so one's own article moves up in the queue rather than the opposite effect. However, I could be wrong - I'm just a pessimistic graduate student. :) Also, isn't it kind of against wiki-philosophy? All of these requirements? It doesn't sound as cooperative as I would like somehow. Awadewit | talk 22:13, 23 October 2007 (UTC)
replies: text: I think the number one problem here is too many requests and not enough reviewers in PR, GAC, and FAC. I am also not thrilled with introducing a sort of wikitax or corvee, but really do not see any other alternative to get more people involved (nor have any been suggested above, as far as I can tell). I know for a while there was a suggestion at the top of the PR page that all editors submitting a PR request also review at least one other article. That did not work, and PR is barely working now (my best results are from asking people I know and some times work with to please look at my PR request, but that can be a bit incestuous). I guess I see this idea as a way of dealing with the free rider problem, or sort of forcing users to follow the golden rule (review unto others as you would have them review unto you). Since the goal of almost everyone requesting a peer review is to get the article to GA or FA, they are fine with dealing with those requirements, so what's the problem with also having PR requirements or requirements to enter the GA or FA process? As I pointed out, there is a precedent of sorts at RfC. Finally, I think it is much more against the wiki spirit to propose banning the semi automated Peer Reviews (and yes, I do them at PR so I am biased). How many helpful activities on Wikipedia are banned? Ruhrfisch  ><>°° 15:06, 24 October 2007 (UTC)
replies: text: I understand these points. I, too, have to solicit peer reviews. You have presented two different analogies here: a tax and the golden rule. Which do you think we should follow? Your initial proposal was more of a tax - review 2 or 3 articles and get your article reviewed. Your second article was more like an enforced golden rule - 1 for 1. (Perhaps banning bots is against the wiki-spirit, but I also think that their automated analysis of the articles isn't up to snuff yet. When it is, I am all for the bots. It isn't the bots per se I am against - it is the fact that they do not quite work well enough yet. They do feel impersonal, but perhaps I could get over that if they offered good suggestions!) Oftentimes I want to try proposals such as your 1 for 1 idea and see if they work, but in a wiki-environment, I worry that they might become entrenched and hard to remove. I am asking myself right now: if we try this and it becomes entrenched, what is the likelihood that it will make things worse? what is the likelihood that it will perpetuate drive-by reviewing? is it better to start with smaller reforms and see if those work first or is it better to revolutionize the whole system? are we France in the 1780s? (Thanks for the compliment.) Awadewit | talk 17:30, 24 October 2007 (UTC)
replies: text: I much prefer voluntary measures (as, I would imagine, do almost all editors here). My concern is more that voluntary means are not producing the number of reviewers needed, and this was the only way I could think of to solve that for everyone. I am fine with trying smaller measures first (and even if we introduced a "wikitax" method, I think PR would be a good place to try it out initially). I have made my views on the use of the semi automated peer reviews (it is a script used by a human, not a bot) above (under A Dissenting view). Thanks, Ruhrfisch  ><>°° 17:50, 24 October 2007 (UTC)
thread_title:  New thoughts 

message: I've been exchanging notes with Sandy while she's been posting here; you can read our exchanges on our talk pages if you are interested.  I think Sandy has raised some points we haven't considered; conversely, I think the group has good reasons for some of the suggestions she has raised objections to.
It strikes me that we have an obligation to engage with interested editors who come by with opinions about the work in process here, but we can't always restart from scratch when someone shows up.  What I propose to do to address this is have a conversation with Sandy on our talk pages which will try to condense her concerns to a short thread, and perhaps resolve some of them.  Once we're down to some condensed version, I'll raise a single thread here to discuss them.  That should reduce the impact on ongoing discussions, while giving appropriate effort to integrating a new participant.  I also think that as facilitator it's my job to worry about the integration of new participants in the workshop.
Any thoughts on whether this is the right way to go?  Mike Christie (talk) 03:22, 25 October 2007 (UTC)
replies: text: It works for me; I'm sorry I came so late to the process and "made more work".  I'd also like to hear from Gimmetrow and avoid losing pieces of articlehistory to miscellaneous talk page archives, but think we should gel this down off of this page, so as not to slow you all down.  SandyGeorgia (Talk) 03:25, 25 October 2007 (UTC)
thread_title:  Comments from SandyGeorgia 

message: Because most articles only receive one or two reviews at PR, I think that it is to their benefit to go through GA, PR, and FA. The more people who read an article and who comment on it, the theory goes, the more mistakes will be caught and the better the article will become (this rests on the assumption that the reviewers know what they are doing, of course). I think that a well-organized GAC, PR, and FAC are all good ideas. A rigorous and continuous review system slows down the drive to FA, catching much that would have been missed and informing editors of large problems before they reach FAC. GAC seems to be working better now, although there are not enough reviewers. PR needs an organizational structure in my opinion. I've been sure why it isn't divided by subject matter. I feel completely unqualified to review certain topics while I have to scroll through a list of 100 articles to find those that I do feel qualified to review.
One of the most frustrating experiences for me at FAC is MOS-compliance. I cannot keep up with its changes. Once, months ago, I proposed a sort of "MOS-prep-room" for articles going to FAC. Articles that wanted to be nominated for FAC (and were not obvious WP:SNOW candidates) could go through a process of MOS polishing by those who are experts in it. That way, the only discussions at FAC would be related to prose and substance, and not dashes, dates, etc. Another easy way to solve this problem is to choose a common book of style, such as Chicago. Then there would be no more debate. That seems like a dim possibility, though.
*Dividing the articles by topic might encourage more people to do peer reviews.
*Providing some guidance on how to do a good peer review might help new editors.
*Separating MOS compliance from FAC would encourage good content editors to submit more articles. Awadewit | talk 02:22, 13 October 2007 (UTC)
replies: text: It'd also be helpful (and save 75% of time) if reviewers identifying MOS issues addressed them at the time rather than spend an equivalent amount of time pointing them out (unless there are loads I guess..)cheers,  Casliber (talk · contribs) 10:58, 13 October 2007 (UTC)
replies: text:  If by "addressed them" you mean fix them, I agree 100%. Whenever I read the discussion of a candidate, & find someone insisting that the wrong dash is being used (or any other trivial point from the MoS), I get a little angry; it's hard not to take this as anything other than the commenter has nothing useful to say about this article -- yet is so compelled to say something that they have no shame over saying something silly. -- llywrch 16:40, 15 October 2007 (UTC)
replies: text: This is an ongoing debate, and I see little chance of it ending. MoS concerns are legitimate, and reviewers can review quite a few articles if they are not expected to make changes themselves to each article they review. I'd rather have several articles reviewed by a competent MoS reviewer than to have one copy edited and MoS-fixed by an editor while nine others go unreviewed. — Brian (talk) 07:20, 16 October 2007 (UTC)
replies: text: It is one thing to point out that someone has not taken the time to format their references or dates consistently (tiring as this is) but it is another thing entirely to type out "you have a 'the the' in the fourth paragraph of the second section". Couldn't the editor just have removed the extra "the"? Such comments make FAC feel hostile rather than cooperative, in my opinion. Awadewit | talk 07:33, 16 October 2007 (UTC)

text: Although full MoS compliance isn't one of the GA criteria, when conducting a GA assessment I certainly look for obvious MoS issues. I can't pretend to catch or even know them all (a disadvantage of a single-reviewer system), but IMO ideally there should be no significant MoS problems in any article that passes GA. Perhaps this function should officially devolve to GA, as it's a time-consuming process that GA—with wider participation than FA—may be able to handle better. EyeSereneTALK 11:55, 25 October 2007 (UTC)
replies: text: EyeSerene, your comment got me thinking; I'm going to type up a longer analysis/proposal later today as I have time.  I suspect the PR and the FAC backlogs and issues could be solved if, as you say, there were "no siginificant MOS problems in any article that passes GA".  As of now, that's not the situation, and it causes a lot of disappointment to editors who show up at FAC thinking their article is almost there.  The sooner editors can learn Wiki's inhouse Manual of Style, referencing requirments, etc., the more effective *all* of their edits to all articles they work on will be.  If articles coming to FAC were prepared for FAC, editors might stop using FAC as a proxy for PR and FAC can get back to serious content review for prose, neutrality and comprehensiveness.  The problem now is that FAC has become a dumping ground for unprepared articles as PR is backlogged and GA doesn't necessarily encompass a complete review for MOS or reliable sources (the biggest shortcoming of all review processes IMO).   SandyGeorgia (Talk) 14:58, 25 October 2007 (UTC)
replies: text: I think it's definitely worth considering. I've often thought that some sort of MoS-based GA reviewer checklist would be useful, and would educate us reviewers as well as those editors whose articles we review (and since the GA process invites reviews from anyone, would be essential to new or inexperienced reviewers anyway). It would mean expanding the GA criteria slightly, but even if only the most common MoS issues seen at FA were addressed it should help. Incidentally, since FA reviewers are perhaps more MoS-aware than those at GA, and such a proposal would benefit from their input, closer cooperation may help to clear out some of the misconceptions between the two processes. EyeSereneTALK 19:19, 25 October 2007 (UTC)
replies: text: You are thinking exactly the way I'm thinking.  Or I should say, you influenced me to think that way :-)  I'm working on it, but to be honest ... ahem ... Awadewit proposed this idea a long time ago, but I resisted because I know I'll get blasted by critics as soon as I put something to paper.  This idea that there is some schism between GA and FA just has to end; some of my closest Wiki friends are GA folk. And, I can so easily put together a list of the most common deficiencies that show up at FAC, but as soon as I do that, I'll get blasted as someone who overly focuses on certain aspects.  Ah, heck.  All you can do is try :-)  Just so no one is caught unawares, I'm putting together ideas that would move GA in the direction of picking up extra work in terms of more comprehensive reviews of a couple of items, and extra credibility and recognition hopefully to go along with it :-)  SandyGeorgia (Talk) 19:33, 25 October 2007 (UTC)
replies: text: I also think it would be helpful to clearly indicate what the next step for the article after PR is (within the Peer Review request). I think there would be three possible next steps: FAC, GAC, or general improvement of the article. If this were clearly shown (and some editors do indicate this already), then I would give priority to FAC, followed by GAC, and then just improvement. I also think the level of detail in the review would be highest for FAC, medium for GAC, and "Get some references and fix your spelling" might work for some improvement requests. If we go to a new system, I would hope it would indicate the next step after PR clearly. Ruhrfisch  ><>°° 20:00, 25 October 2007 (UTC)
replies: text: I'm not sure the process of review can be standardized; it really depends on the article and the editor.  Many experienced FA writers, for example, go straight to FA without even a peer review and without GA.  Some get their peer review extra-officially, by approaching editors they know.  In one case, even though an article I was helping to prepare for FAC got a solid peer review and was ready for FAC and really had no reason to get a GA rubber stamp in between, I recommended to the main author that he get that GA stamp to show broad consensus existed for the article, as it is subjected to disruptive and POV editing.  So, I'm not sure there's a natural or logical progression that all articles or editors must follow.  I'm focusing on what we're missing; getting some fundamental issues in place before unprepared articles show up at FAC, resulting in shock and disappointment to the nominators, so that FAC can better focus on in-depth content review.  I understand clearly the purpose of peer review; it's for whatever the editor needs at the time PR is approached.  I understand the purpose of FAC; it's to assure that articles represent our best work, with no known omissions.  I think we can better carve out a more clear niche for GA, since it's no longer viewed as only for short, good articles.  SandyGeorgia (Talk) 20:13, 25 October 2007 (UTC)

text: *Just to clarify, I don't want to standardize Peer Review or make it some lockstep process. My thought was more that it would be helpful to always specify what the submitter's next goal for the article is after PR, so that reviewers could do their job better. I have taken 3 articles to FA and always went through PR, but skipped GA on one. Ruhrfisch  ><>°° 01:17, 26 October 2007 (UTC)
replies: text: There was quite a recent discussion about including more MOS issues in the GA criteria here. Epbr123 20:22, 25 October 2007 (UTC)
replies: text: Thanks for that lead, Epbr; I think there's a more collaborative spirit now, thanks to this effort.  SandyGeorgia (Talk) 21:27, 25 October 2007 (UTC)

text: (edit conflict) I'm still pretty new to the whole review process myself really—certainly compared to some of the editors posting here—and still have much to learn. I'm only aware of this antagonism because I've read various comments in discussions I've stayed out of, but it only seems to be certain editors that have problems. From what I've seen, many others work across all quality processes anyway... and anyone who's served their time has my respect, regardless of where they've served it ;) I'm looking forward to seeing your ideas... EyeSereneTALK 20:31, 25 October 2007 (UTC)
replies: text:  Would it make sense, then, for me to put together a list of the most common shortcomings that show up at FAC (they are not confined to MOS issues) to get a sense of whether GAC would be interested in expanding to cover some of those issues?  I think that would get some pressure off of PR and FAC, and PR could go back to being a one-stop for whatever you need while FAC could focus more deeply on content.  SandyGeorgia (Talk) 21:12, 25 October 2007 (UTC)

text: I think it would be great too, and, in this collaborative spirit, once the list is established, I'm sure this workshop can find someone to present the idea in such a way that it won't get shot down immediately. Geometry guy 23:06, 25 October 2007 (UTC)
replies: text: I hope you're right, Gguy, but I'm not holding my breath.  I've started putting it together; it will take me some time, and I fully expect to be blasted once I put it out.  I'd love to be wrong :-)  SandyGeorgia (Talk) 23:23, 25 October 2007 (UTC)
replies: text: I also think that could be really helpful. I do take your point, in particular about taking some of the pressure off FA. --Malleus Fatuarum 23:21, 25 October 2007 (UTC)
thread_title:  Some thoughts 

message: Here's where I think we are in this discussion, with a suggested synthesis.
:Update: I have split the categorization question in two based on Awadewit's comments.  Please revert if I've mischaracterized. Mike Christie (talk) 21:39, 24 October 2007 (UTC)
replies: text: *I did not support the WP 1.0 categories for reviewers, only for peer review requests for articles. I am in favor of "free-form" text for the list of peer reviewers, as my example of how I would describe myself was supposed to demonstrate: "Eighteenth- and nineteenth-century literature and European history and literature; children's literature; lay reader of science articles". However, I do see the benefits of categories if the list becomes too long - over 100. Awadewit | talk 17:22, 24 October 2007 (UTC)
replies: text: One of those funny text-communication issues: I interpreted your earlier comments to mean that you didn't see the point of reviewer lists unless reviewers are listed by expertise/category! Anyway, I'm also in favour of free-form descriptions, but I think we should be aiming for a longish list, hence subdividing such lists by categories would be a useful search aid. Geometry guy 18:30, 24 October 2007 (UTC)

text: I don't know if Marskell's comments would apply here, because his objection related to the reviewer categorization.  Other than Marskell I have seen no objections to the use of WP 1.0 for this. Mike Christie (talk) 21:39, 24 October 2007 (UTC)

text: Here I saw two concerns: Marskell felt the WP 1.0 categories were too broad to be used for this purpose; Awadewit preferred not to have any categorization of the reviewer listings, but allow reviewers to express their interests with a text field. Mike Christie (talk) 21:39, 24 October 2007 (UTC)
replies: text: So the obvious comprimise would be to have interests expressed in a text field, but use the 1.0 categories to make searching for a reviewer easier. Geometry guy 21:47, 24 October 2007 (UTC)

text: *Without a peer reviewer listing some category of interest, I don't really see the point in having the list, frankly. What would be the point? Please explain. Awadewit | talk 17:22, 24 October 2007 (UTC)
replies: text: Ideally, all reviewers would list categories and free-form interests, but it is also useful to know which editors are actively involved in PR: for many articles, the nominator may just want to find an active general reviewer. Geometry guy 18:30, 24 October 2007 (UTC)
replies: text: But if someone posts a request for peer review, isn't that pretty much soliciting a general review? Are we saying the editors have to individual peer reviewers every time they post an article? That would be a terrible idea, in my opinion. Awadewit | talk 20:33, 24 October 2007 (UTC)
replies: text: I agree, it would be a terrible idea. Rules are generally bad, but information is generally good. For example, a nominator might recognise a name on the list of reviewers, and post a message on their talk page. Geometry guy 20:55, 24 October 2007 (UTC)
replies: text: Yes; to clarify, I think what we've agreed so far is that there should be two lists: articles requesting reviews, and reviewers willing to do reviews.  So there would be two methods of getting a review -- you can ask someone from the reviewer list, or post the article to the article list and wait.  Similarly there are two ways of reviewing -- pick something that is waiting for a review, or wait for a reviewer to contact you.  I would assume that there's no obligation to post your article to the list if you're just going to contact reviewers; and there's no obligation to post your name to the reviewer list if you're just going to pick things on the article list to review and don't want to be asked directly on your talk page. Mike Christie (talk) 21:54, 24 October 2007 (UTC)
replies: text: No to one point! Your assumption is wrong (i.e., has not been expressed anywhere on this page). All articles for peer review must be added to the PR list (but this is done simply by putting a template on the talk page).  Contacting reviewers is then an additional optional step. A private arrangement between individual editors can happen, but does not need a process. The point of PR is to get as much input into each article as possible. For this, the article must be listed. Geometry guy 22:20, 24 October 2007 (UTC)
replies: text: Well, I don't have a strong opinion either way.  I think it's realistic to acknowledge that if a reviewer lists their name and interests on the PR page, someone may just contact them directly without submitting their article to PR.  That's not in the scope of our process, true:  our scope is increasing input to "articles that are submitted to PR".  But it'll probably happen.  Or do you think it shouldn't even be mentioned in our notes, because it relates to something that is not PR?  Mike Christie (talk) 22:31, 24 October 2007 (UTC)
replies: text: Actually, on reflection, I think you have a point. PR can also be a dating service, and once the date has been made, the agent does not attend! I have expressed a similar view about GA in the past, but it might fit PR better. I'd be interested to hear other editor's views on this. Geometry guy 22:36, 24 October 2007 (UTC)
replies: text: *I think that your idea of a "dating service" fits perfectly with the idea that I've stated several times already, that PR ought not to be seen as a process within any kind of a hierarchy. So I'd be quite in favour of it. --Malleus Fatuarum 23:15, 24 October 2007 (UTC)
thread_title: Summary of the discussion so far

message: There are several interesting comments in the above, but I think the core question can be summarized as:
* Should the peer review reside on a single permanent page so that it can have a permanent link to it? Or is it OK for it to be on the main article talk page, where it will eventually be archived away?
Please correct this if I have it wrong.
The argument for a single permanent page is that a permanent link is valuable for people who wish to read the key reviews in the article's history.  The ArticleHistory template is designed to permit this.  The argument for article talk (which I didn't see recapped, but I believe I recall from earlier discussions) is that article talk is more likely to encourage interactive work on the article.
The solution proposed by Geometry Guy is that we use a subpage of the article talk page.  I have an additional question I think might clarify things:
* If we use a permanent page with a link that will never change, does it matter where it is?  Would it matter to a bot that might create it, or to the links from the PR listings page, or to the ArticleHistory template?
If the answer is no, it doesn't matter, then are there any other considerations?  A separate page won't appear in article talk without either transclusion or cut and paste (either of which could be done by the bot).  So is there some reason it matters where the permanent page is, if we have one?  The only reason I can think of is that if articles are moved, their subpages would require moving too.  On some PR subpage, they would not.
So did I miss any key points?  Are those the questions to be answered?  If so, let's see if we can get to consensus on this. Mike Christie (talk) 01:09, 26 October 2007 (UTC)
replies: text: From a  bot point of view, I'm almost certain it doesn't matter what the parking place is named (but I will ask Gimmetrow).  From a practical point of view, given that there is already a long-standing naming convention and procedure for archiving peer reviews that works, why change the existing naming conventions (which may create unneeded confusion)?  Also, if you change the naming convention now, future bots have to account for two different naming sytems; the old one and the new one.  SandyGeorgia (Talk) 01:25, 26 October 2007 (UTC)
replies: text: Which is more important? Making the bot's life easier or encouraging more participation in the review process? --Malleus Fatuarum 02:04, 26 October 2007 (UTC)
replies: text: Can someone please tell me (or give a diff) how putting peer reviews on a new page, whatever it is called, is going to encourage more participation? I came late to this discussion, I have tried to read it all, but I just don't follow the logic. As I see it, the problem is too many requests for reviews, and not enough reviewers. How does moving where PR takes place decrease requests or increase the number of reviewers or the number of reviews done per reviewer? I made a suggestionabove which is not perfect, but at least addresses this issue. Ruhrfisch  ><>°° 02:42, 26 October 2007 (UTC)
replies: text: I've reviewed a lot of articles, and I don't see that either. I certainly am *less* inclined to get tangled up in an article talk page, while I am inclined to pop in to the independent PR page.  SandyGeorgia (Talk) 02:45, 26 October 2007 (UTC)
replies: text: In my opinion, as both a frequently peer reviewer and peer reviewer-receiver, it is easier to have the review on the talk page because it encourages participation by the reviewer in the article editing process (something mentioned somewhere above that many of us desired). Moreover, more editors from the article are likely to get involved in the reviewing process (which is usually good). While I agree to an extent with what Sandy is saying about not getting involved in petty disputes, I also agree that it is good for reviewers to have an idea of the history of a page before making suggestions. I always try and read the talk pages first as a way to avoid re-starting old debates. This would be facilitated by having the PR on the talk page. I don't think that the only problem is too many requests and not enough reviewers, by the way. Awadewit | talk 03:06, 26 October 2007 (UTC)

text: What are editors of an article more likely to look at? The talk page or a separate review page? --Malleus Fatuarum 02:54, 26 October 2007 (UTC)
replies: text: Regular editors who have an interest in the article will look at either; the advantage of a separate PR page is that it brings in independent editors for a fresh look and new input, and editors who don't want to get tangled up in article talk page politics <raising my hand often on that one>.  SandyGeorgia (Talk) 02:59, 26 October 2007 (UTC)
replies: text: I don't think there can be a definitive answer here.  The argument for the article talk page is quite reasonable, as Malleus Fatuarum implies: casual editors will encounter a peer review written on the article talk page without the need to click on a link.  The argument for a permanent link is also reasonable; anyone who understands the ArticleHistory template would look at a peer review for an article they were interested in.  Article talk page peer reviews will eventually scroll to archives; separate peer review pages are never immediately visible on article talk.  So there are pros and cons on both sides.  Does the ability to link directly to a permanent page in ArticleHistory make a bg difference?  It's been argued that it does.  Is it useful or necessary to have that peer review linked? Mike Christie (talk) 03:09, 26 October 2007 (UTC)
thread_title: Key points from the above

message: I'm raising this issue in a separate section so that it doesn't get lost in the noise.
I would like to know why it's important to have an n-year old review available to an editor without that same editor also having available the n-year old criteria that article was assessed against. History always has to be viewed in context after all. --Malleus Fatuarum 02:10, 27 October 2007 (UTC)
replies: text: I´m not entirely convinced that it is important. I don´t see much use to an archived peer review after two years. However, you can argue that it´s always better to be able to find something than not, so the separate page has an advantage. I haven´t thought that where it occurs is the critical issue. In terms of driving reviewers, having the list of people to contact on user talk has been the innovation being argued for. Marskell 14:03, 28 October 2007 (UTC)
thread_title:  Don't want to interrupt the flow, but ... 

message: I don't really see how moving a peer review to some other page will improve participation. At least, it's not obvious that it will, and if the argument is that it's "worth a try", that's a mighty big change to the PR system just to try something marginally different.
All the featured processes use WP: subpages. The oldest were PR and FA, the rest (FL, FP, FT, FPO) probably copied them. I'm not really sure why they use the archive system for subsequent entries rather than the AfD numbering system. They could have just as well used permalink archiving and kept it all on one page. Guess that just indicates common ancestry.
The benefit of WP: subpages is that if an article is renamed, the associated pages in WP: space don't need to be moved. WP1.0 used a subpage of the article (at /Comments) for reviews. This is hardly ever used, but if it were, these pages would probably need to be moved with the article.
It doesn't really matter where something is as far as linking is concerned. A bot can deal with most anything. Let's not get bogged down in ArticleHistory - that template evolved in ways not originally intended.
If you want the PR page to "just be a list", then why not just use :Category:Requests for peer review? That's a list of pages with requests for peer review, no bots involved. If you wanted to categorize them into 10 groups, you could have the template do that too. Gimmetrow 02:53, 27 October 2007 (UTC)
replies: text: It's not "some other page", it's that article's talk page. Big difference. --Malleus Fatuarum 08:30, 27 October 2007 (UTC)
thread_title:  On review locations, subpages, bots and categories 

message: People were talking about the need for an award, so I made :Template:ContentReviewMedal. It's not only for PR; can be used for FA GA PR etc. The image probably needs tweaking (soften the edges of the silver wiki...?]. The template probably needs tweaking too. Tweak away; I shouldn't be doing this anyhow... should be working on dissertation...  ;-) Note to self: back to work! Mush! Yah mule! --Ling.Nut 07:54, 28 October 2007 (UTC)
replies: text: Hey, well that´s one thing done. The magnifying glass seems rather oddly tacked on, but otherwise I think it fine. Marskell 13:59, 28 October 2007 (UTC)
replies: text: Yeah, nice one - I'll take carrots over sticks any day. cheers,  Casliber (talk · contribs) 14:10, 28 October 2007 (UTC)
replies: text: Magnifying glass: I had a pretty hard time thinking of anything to symbolize "reviewing". It wouldn't fit nicely on the silver wiki medal (looked too cluttered/busy), so I had to stick it on the ribbon ;-) Thanks for the kind words  --Ling.Nut 14:48, 28 October 2007 (UTC)
replies: text: Very nice idea; I often have to choose the Working Man or Diligence barnstars to award folk who have done a lot for FAC or FAR and don't feel those awards sum up the effort; this would be much better.   Now, BACK to work on the dissertation, Ling.nut!!  SandyGeorgia (Talk) 15:32, 28 October 2007 (UTC)
replies: text: Hey Ling nut, coulda done a nice image of a masseur - the masseur medal for prose massage, or a wikimicroscope for minute copyediting...cheers,  Casliber (talk · contribs) 20:26, 28 October 2007 (UTC)
replies: text: Although...as mentioned, we already have a Reviewer's Award. I notice that it's only ever been given to five people. One was me, and two of the others were people I gave it to... So the problem perhaps is not the lack of the award but the lack of propagation. Marskell 16:14, 29 October 2007 (UTC)
thread_title:  [[:Template:ContentReviewMedal]] 

message: For a first proposal to help improve the peer review system initially, I would suggest banning the use of all of the automated peer review bots. I don't think anything truly useful really comes out of them, as they mostly suggest manual of style and really basic grammatical changes, many of which don't even apply to the article in question. They're also very useful. When people come to peer review, they want an analysis of the article's content, and any information that is potentially missing in order for the article to be improved, as well as suggestions how to actually improve the article's prose. None of the bots really go into these details all that much, and mainly just concentrate on the mundane grammatical and wikifying issues -- in which case, a better solution for the users of the bots is to take the information that is fed out from the bot and just make these minor updates themselves.
I also think that the high number of automated reviews at WP:PR can also be discouraging human reviewers. Many reviewers might see the automated bot review, and say to themselves, "well, a bot has already checked this out, so there's no point in me reviewing it, since I'll probably just be rehashing stuff that the bot has already said." Which is really a bogus excuse to not review an article, but an excuse nonetheless. The human reviews are the most useful information here. The only real use of a bot might be to reconfigure it to strictly check for manual of style issues, and to run that at the conclusion of a major review for something like WP:FAC, just to make one final check that the article is in good shape. Dr. Cash 01:38, 12 October 2007 (UTC)
replies: text: I have no problem with stopping automated peer review bots, but I disagree with most of the rest of your argument. Peer review involves looking at an article's neutrality, comprehensiveness, etc., but it also includes all the little manual of style things that are easy to overlook. I see no problem with individual editors running the peer review script on articles and then posting the results. Often, the bots find lots of errors, and it's unfair to expect an uninvolved editor to make the changes themself. The time I have to devote to Wikipedia is in short supply, and it's frankly better spent elsewhere. — Brian (talk) 01:46, 12 October 2007 (UTC)
replies: text: I'd pretty much go along with Dr. Cash here: any quality review process needs itself to be producing reviews of a decent quality - for reasons of credibilty if nothing else. Bot reviews all generally seem to say the same things regardless of the article under review, and come over as impersonal, unhelpful and give the impression that the article hasn't actually been looked at by an interested reviewer. Seeing these banned across the project would be only be a 'good thing' IMO (unless perhaps retained for use by editors themselves to check articles before submission). EyeSereneTALK 10:28, 12 October 2007 (UTC)
replies: text: I think the automated reviews can be very useful, if used sensibly and in the knowledge that it's not infallible. That is only the case for a mind that is genuinely looking for ways to improve an article, though. Given the amount of advice given at peer review that is ignored by those who requested it, I suspect that not all editors are looking for opportunities to do more work! That being the case, we might as well just have a note at the top of the peer review and/or MoS pages noting that there is a tool that can automatically review your article. I see it as along the lines of a spell or grammar checker. (Hey - maybe we should require editors to run their articles through a combined spell, grammar and MoS checker before they submit to FAC!) 4u1e 16:20, 12 October 2007 (UTC)
replies: text: I've never really found the bots that useful. I can always see why they have made the mistakes they have, but that doesn't help me improve the article, and I am dedicated to improving articles. I also find the impersonal nature of the bot off-putting. Editors who put hours into articles are, generally, I find, not happy to find a bot telling them not to use "weasel words" or whatever. Those decisions are context-based and our bots have not reached that level of sophistication. I am in favor of banning the bots. Awadewit | talk 02:27, 13 October 2007 (UTC)
replies: text: The benefit I do get from them is as a reminder of various MoS points I may have forgotten about. As I say, along the same lines as spelling/grammar - it's not that I karnt spel, but a tool to catch my goofs is useful. I agree that it's probably better not to use the bot on the Peer Review page. 4u1e 15:48, 13 October 2007 (UTC)

text: Eye wood bee won off those inn favour if banning bots. --Malleus Fatuarum 16:11, 13 October 2007 (UTC)

text: That's a great idea, I think. It would allow a lot of those minor issues to be corrected prior to nomination. The better the articles are when they come to us, then faster the review process can go. Lara❤Love 14:04, 15 October 2007 (UTC)

text: In support of the above, but also kind of in support of the previous views:  I use the semi-auto reviewer personally, on articles I've written, to check various MoS issues.  Generally, I've found it helpful for spotting things I've missed, or wasn't aware of.  I've also used it to check articles I've GA reviewed, but I've never just posted its findings;  in all cases, I've checked what it came up with and either ignored its findings, or found the specific problems and either fixed or mentioned them in the review section on talk page.  Of course it makes mistakes, and I wouldn't ever dream of posting its results as a GA/FA or any other kind of review, and so I fully endorse the banning of it in formal peer reviews, but I'd hate to see it denigrated to the point it's no longer used at all.  Carre 17:49, 23 October 2007 (UTC)
replies: text: What do you mean by "banning of it in formal peer reviews"? Just in FAC and GAC, or in PR too? While anyone can  add the script to their monobook, a lot of users don't know how to do that or don't want to, and I get some requests to run the script from people who know I have it installed. Ruhrfisch  ><>°° 15:08, 24 October 2007 (UTC)

text: I agree with you: I think semi-automatic reviews are fine as long as they are stored on a separate page. Concerning the final question, the idea is to keep the bot pretty simple, so that a bot request will be quickly answered, although I can also think of a couple of editors whom I could ask. Geometry guy 16:42, 24 October 2007 (UTC)

text: So if I understand your point of view, you think the semi-automated suggestions are useful if run by the editor themself, but not if run by me or another editor? With all due respect, I do not think the semi-automated peer reviews are to be confused with regular peer reviews, which is why they are just one line linked to the actual suggestions on a separate page (WP:PR/A). I would love to read all the articles and make comments on them, but I do not have time for that. There were 109 semi-automated reviews in September (on WP:PR/A, plus others that were left in the review itself by different editors). In October so far (one week to go in the month) it is already up to 112. Ruhrfisch  ><>°° 19:03, 24 October 2007 (UTC)

text: I suspect this can be resolved by a simple rewording, such as "A script has been used to generate a semi-automated review of the article for minor issues of grammar and house style. If you would find such a review helpful, please click here." Geometry guy 18:57, 24 October 2007 (UTC)
replies: text: Was taking a day off WP yesterday, so didn't spot this lot.  I think the above bits have pretty much explained my opinion.  The semi-auto thing is useful, but it does make mistakes.  Simply taking the time to check its results is much more personal and helpful than just pointing an editor at its output, especially as it does have known issues.  So, it's useful whoever uses it, but taking the extra step of checking it and humanising is so much better.  I know that if I submitted an article for PR and only got the bot's output, I'd feel pretty disheartened by the whole process. Carre 11:19, 25 October 2007 (UTC)

text: I support and approve the automated peer review bots; when I was a Wiki newbie, the info it provided was a helpful starting place.  Many newcomers first encounter with any sort of outside input may be peer review, and the automated information may be helpful to them.  SandyGeorgia (Talk) 01:11, 25 October 2007 (UTC)
replies: text: I just run the script (and do not have access to it, or the expertise to modify it), but I will see if AndyZ can modify the text left on the PR page to be more user friendly as suggested above. Unfortunately there are some review requests that only get the semi-automated PR comments. If the script were not run, then they would presumably get nothing, which would be even worse. I do try to look at the oldest PR requests and make comments on the ones that received nothing but the scripted note. As for hand checking the scripted replies, the number done (well over 100 per month) and the nature of the script (when I run it as AZPR) both make that difficult to do. I would be fine with putting a note somewhere in the instructions that if a semi-automated peer review is not wanted, just say so. I do read the requests and skip those that ask to be left out. Ruhrfisch  ><>°° 15:20, 25 October 2007 (UTC)
replies: text: AndyZ has changed the wording of the message left in peer review to " "A script has been used to generate a semi-automated review of the article for minor issues of grammar and house style. If you would find such a review helpful, please click here." as suggested by Geometry Guy above. Thanks, Ruhrfisch  ><>°° 13:49, 1 November 2007 (UTC)
replies: text: I'd like to see "for minor issues of grammar and house style" either removed or changed. It doesn't just pick up minor issues, it also picks up very major ones such as lack of inline citations or references, and unsupported claims like "Some people say..." and "It has been suggested that..." DrKiernan 14:07, 1 November 2007 (UTC)
replies: text: OK, how about just taking out "minor" and linking "house style" to WP:MOS? Ruhrfisch  ><>°° 14:48, 1 November 2007 (UTC)
replies: text: OK. Thanks, DrKiernan 14:52, 1 November 2007 (UTC)
thread_title:  Proposal: Ban automated peer review bots 

message: I currently run the User:AZPR account (with User:AndyZ's blessing and permission). It is not a bot, it is a semi-automated tool that still needs a human user. I go in and open each file and cut and paste the suggestions into a file, which I then paste into the current WP:PR/A page. I then go in and add the automated peer review notice to each article on the WP:PR page, which is one line if done this way ("Please see automated peer review suggestions here. Thanks, APR t 02:29, 23 October 2007 (UTC)
replies: text: In support of the above, but also kind of in support of the previous views:  I use the semi-auto reviewer personally, on articles I've written, to check various MoS issues.  Generally, I've found it helpful for spotting things I've missed, or wasn't aware of.  I've also used it to check articles I've GA reviewed, but I've never just posted its findings;  in all cases, I've checked what it came up with and either ignored its findings, or found the specific problems and either fixed or mentioned them in the review section on talk page.  Of course it makes mistakes, and I wouldn't ever dream of posting its results as a GA/FA or any other kind of review, and so I fully endorse the banning of it in formal peer reviews, but I'd hate to see it denigrated to the point it's no longer used at all.  Carre 17:49, 23 October 2007 (UTC)
replies: text: What do you mean by "banning of it in formal peer reviews"? Just in FAC and GAC, or in PR too? While anyone can  add the script to their monobook, a lot of users don't know how to do that or don't want to, and I get some requests to run the script from people who know I have it installed. Ruhrfisch  ><>°° 15:08, 24 October 2007 (UTC)

text: I agree with you: I think semi-automatic reviews are fine as long as they are stored on a separate page. Concerning the final question, the idea is to keep the bot pretty simple, so that a bot request will be quickly answered, although I can also think of a couple of editors whom I could ask. Geometry guy 16:42, 24 October 2007 (UTC)

text: So if I understand your point of view, you think the semi-automated suggestions are useful if run by the editor themself, but not if run by me or another editor? With all due respect, I do not think the semi-automated peer reviews are to be confused with regular peer reviews, which is why they are just one line linked to the actual suggestions on a separate page (WP:PR/A). I would love to read all the articles and make comments on them, but I do not have time for that. There were 109 semi-automated reviews in September (on WP:PR/A, plus others that were left in the review itself by different editors). In October so far (one week to go in the month) it is already up to 112. Ruhrfisch  ><>°° 19:03, 24 October 2007 (UTC)

text: I suspect this can be resolved by a simple rewording, such as "A script has been used to generate a semi-automated review of the article for minor issues of grammar and house style. If you would find such a review helpful, please click here." Geometry guy 18:57, 24 October 2007 (UTC)
replies: text: Was taking a day off WP yesterday, so didn't spot this lot.  I think the above bits have pretty much explained my opinion.  The semi-auto thing is useful, but it does make mistakes.  Simply taking the time to check its results is much more personal and helpful than just pointing an editor at its output, especially as it does have known issues.  So, it's useful whoever uses it, but taking the extra step of checking it and humanising is so much better.  I know that if I submitted an article for PR and only got the bot's output, I'd feel pretty disheartened by the whole process. Carre 11:19, 25 October 2007 (UTC)

text: I support and approve the automated peer review bots; when I was a Wiki newbie, the info it provided was a helpful starting place.  Many newcomers first encounter with any sort of outside input may be peer review, and the automated information may be helpful to them.  SandyGeorgia (Talk) 01:11, 25 October 2007 (UTC)
replies: text: I just run the script (and do not have access to it, or the expertise to modify it), but I will see if AndyZ can modify the text left on the PR page to be more user friendly as suggested above. Unfortunately there are some review requests that only get the semi-automated PR comments. If the script were not run, then they would presumably get nothing, which would be even worse. I do try to look at the oldest PR requests and make comments on the ones that received nothing but the scripted note. As for hand checking the scripted replies, the number done (well over 100 per month) and the nature of the script (when I run it as AZPR) both make that difficult to do. I would be fine with putting a note somewhere in the instructions that if a semi-automated peer review is not wanted, just say so. I do read the requests and skip those that ask to be left out. Ruhrfisch  ><>°° 15:20, 25 October 2007 (UTC)
replies: text: AndyZ has changed the wording of the message left in peer review to " "A script has been used to generate a semi-automated review of the article for minor issues of grammar and house style. If you would find such a review helpful, please click here." as suggested by Geometry Guy above. Thanks, Ruhrfisch  ><>°° 13:49, 1 November 2007 (UTC)
replies: text: I'd like to see "for minor issues of grammar and house style" either removed or changed. It doesn't just pick up minor issues, it also picks up very major ones such as lack of inline citations or references, and unsupported claims like "Some people say..." and "It has been suggested that..." DrKiernan 14:07, 1 November 2007 (UTC)
replies: text: OK, how about just taking out "minor" and linking "house style" to WP:MOS? Ruhrfisch  ><>°° 14:48, 1 November 2007 (UTC)
replies: text: OK. Thanks, DrKiernan 14:52, 1 November 2007 (UTC)
thread_title: A dissenting view

message: :Touche. Marskell 09:16, 1 November 2007 (UTC)
thread_title:  An apposite quotation, to refocus minds 

message: Sorry I've been away from the page; not a good start for someone who is supposed to be the facilitator.  I've been very busy off-wiki and I felt I had to devote the limited time I had to a FAC I had going.
I've tried to read through and catch up, and I've looked at the PR thread and Allen3's comments.  Is this an accurate summary of where we are?
*There's agreement that categories are worth trying, but Allen3's comments indicate it's pointless unless a bot does the work.
*We don't have anyone volunteering to do the bot work yet.
*The remaining disagreement seems to be about whether we should do more design work and come to PR with more ideas, or whether to go ahead and try to implement just the categories.
If that's accurate, my suggestion would be something like this:
#Find a bot volunteer.  Nothing can be done without that.
#Work on a more detailed design of a minimum implementation of categories, and demonstrate it running on a separate copy of a PR page so that it can be evaluated.
#Repropose the idea at PR using that as the model, so that people like Allen3 with real experience can comment with more precision.
I take Geometry Guy's points about a full redesign, but I think a small win, and demonstration that this workshop can achieve something, is worth a lot at this point.  Mike Christie (talk) 23:23, 2 November 2007 (UTC)
replies: text: A small win is definitely needed now. Moving the review to the talk page - although I remain convinced that's the way to go - clearly carries some baggage. So I'd suggest that trying out a categorised version of the PR review page might give a quick win. Or not, but either way, things will have been seen to be done. Not just jaw-jaw. --Malleus Fatuarum 23:33, 2 November 2007 (UTC)
replies: text:  That seems like a good summary to me, and I'm not completely against cashing in on a win idea, but I think we need to build up the bank too. Actually, the prerequisite for categorization of the PR page - automatic archiving - would already make it a big win, not a small one. As Marskell mentioned above, this has been on the PR agenda for some time. Also, it may translate easily to GA: I have raised the idea of automation of GAN and GA already, and the response was generally positive. As for the talk page question, I can see in my mind a "best of both worlds" situation in which reviews are on a stable page, but are transcluded to the article talk page. However moving either GA or PR in this direction means getting the automation working first, so that the technicalities are invisible and do not take up editor time. Geometry guy 00:10, 3 November 2007 (UTC)
replies: text: OK, it seems that a bot writer is the next step then.  I'll draft a note to leave at WP:BOTREQ and put it here for a quick review before posting.  Other than the three who've been suggested -- Gimmetrow, Dr pda and Rick Block, the only other bot writer I am familiar with is Daniel Vandersluis, who wrote the GAC page statistics bot.  I think Daniel is ramping down his activity, and so perhaps wouldn't be the best candidate.  We should post at BOTREQ, and drop a note to the other three as a courtesy since they clearly have the knowledge and capability -- if they are indeed interested and have the time, that would be great. Mike Christie (talk) 00:58, 3 November 2007 (UTC)
replies: text: Asking one of them first may pan out, and they are familiar with GimmeBot and articlehistory, so can integrate them.  SandyGeorgia (Talk) 01:10, 3 November 2007 (UTC)
replies: text: OK, I posted a note to the three of them.  Will draft something for BOTREQ in the next day or so and post it here for review. Mike Christie (talk) 04:04, 3 November 2007 (UTC)
replies: text: I posted to user Misza13, whose bot is archiving this page, here. S/he said it might be tricky because of the transclusion. Marskell 15:38, 3 November 2007 (UTC)
thread_title:  Recap 

message: Apologies for the recent silence; it's not a sign of flagging interest (I've been away for a week's holiday). It's great to see the progress so far, and increased input from PR veterans. The mock-up looks good. I had envisaged maybe linking the article categories to the corresponding sections of a separate reviewer-list page, but having this list on the same page is even easier to navigate (providing it doesn't get too crowded). Anyhow, nice work all ;) EyeSereneTALK 17:32, 5 November 2007 (UTC)
thread_title:  Good progress 

message: I had said above that the list of volunteers before the categories might be the cart before the horse. But it occurred to me that having a free form list on PR itself might make the page awfully muddled and that a sub-page, prominently displayed in the PR instructions, might work better. Rather than starting a thread about whether we should start the page, I have started a page: Wikipedia:Peer review/volunteers. Per the "let's get a small victory" I think we can run with this. It's a matter of propagation, I think. Ask everyone you know to sign up. When it gets to, say, fifty people I think we can debut it on the main PR. Sound good? Marskell 12:32, 9 November 2007 (UTC)
replies: text: Yes. I agree. It is an easy and helpful addition to make to the PR page. Geometry guy 18:41, 9 November 2007 (UTC)
replies: text: Agreed and signed. Will this page also track the numbers of reviews undertaken and/or active/inactive reviewers? (I can't remember what consensus was about this, or even if there was one!) EyeSereneTALK 22:11, 9 November 2007 (UTC)
replies: text: When it grows we would need to keep track of who is active. That should probably be in the instructions. Whether it's working or not should be clear from the reviews themselves. We can do random audits. Marskell 11:23, 10 November 2007 (UTC)

text: Yes, I agree. Geometry guy 21:04, 21 November 2007 (UTC)
replies: text: I think when we get to discussion of direct changes to other pages it's time to move the discussion to the relevant talk pages, so that we can be sure everyone actively involved in the content review process has plenty of time to review the change.  I've posted a suggested way to add the volunteer page to the PR instructions in a thread at PR.  The particular wording I suggested is just a strawman to get the conversation started.  Please comment over there, and with luck we can make the change in a day or two. Mike Christie (talk) 13:20, 23 November 2007 (UTC)
replies: text: When I started a thread there it received very few comments. I'll try to write up a formatted example tonight. Marskell (talk) 10:28, 25 November 2007 (UTC)
replies: text: I think the lack of comments also indicates a lack of opposition.  I was planning to change the instructions template later today per the note I posted, but if you have a better format I suggest you just boldly implement it and let people modify it if they think it can be improved. Mike Christie (talk) 11:39, 25 November 2007 (UTC)
thread_title:  There is one thing we can do... 

message: While greater minds discuss the bot issues, I'm wondering if it's time to develop new wording for the PR page to incorporate the volunteers list. Marskell (talk) 07:23, 21 November 2007 (UTC)
replies: text: Yes, I agree. Geometry guy 21:04, 21 November 2007 (UTC)
replies: text: I think when we get to discussion of direct changes to other pages it's time to move the discussion to the relevant talk pages, so that we can be sure everyone actively involved in the content review process has plenty of time to review the change.  I've posted a suggested way to add the volunteer page to the PR instructions in a thread at PR.  The particular wording I suggested is just a strawman to get the conversation started.  Please comment over there, and with luck we can make the change in a day or two. Mike Christie (talk) 13:20, 23 November 2007 (UTC)
replies: text: When I started a thread there it received very few comments. I'll try to write up a formatted example tonight. Marskell (talk) 10:28, 25 November 2007 (UTC)
replies: text: I think the lack of comments also indicates a lack of opposition.  I was planning to change the instructions template later today per the note I posted, but if you have a better format I suggest you just boldly implement it and let people modify it if they think it can be improved. Mike Christie (talk) 11:39, 25 November 2007 (UTC)
thread_title: Moving on with volunteers

message: The discussion seems to have progressed significantly. I thought I would chime in. There are places where FA and GA work well, and places where they hardly work at all. I have seen similar discussions but I thought I would post these thoughts here. Basically, the place where FA and GA lack is determining the accuracy of the article. They are really fine at detecting bad grammar, MOS violations etc. but when it comes to content, both places, like much of the wiki, sufferes from lack of experts, especially on more obscure topics. My only experience at FAC is with Rock Springs massacre, which was an almost useless endeavour. The article easily passed, probably because I spent so long on it, but an article like that is unlikely to have a leading expert on Wikipedia, heck, comparitively, almost nothing has been published on it, I may indeed be one of the world's leading experts on that topic now. ;) I am not sure how this can be addressed other than by targeted volunteer drives on wiki for processed like PR, GA and FA. IvoShandor (talk) 14:33, 25 November 2007 (UTC)
replies: text: Glad to see you back from the dead. :-) Ling.Nut (talk) 14:35, 25 November 2007 (UTC)
thread_title:  Comments 

message: Here is an idea that may be useful for increasing participation, but would require a bot. 
What if every WikiProject had a "Reviewers needed" section on its main page, with subsections for WP:FAC (and WP:FLC), WP:GAC, and WP:PR, and others if needed? The bot would go through the nominees at each (FAC, FLC, GAC, PR) page, check the talk page of each nominated article for WikiProject tags, then list the articles in the appropriate section of each WikiProject's page. 
I assume the bot would update the lists every few days, and would both add new requests and remove old ones (that were no longer on FAC, FLC, GAC or PR). The assumption is that the notices would attract more reviewers - I already do this for my FAC articles (though on the WikiProject's talk page). 
I like the idea and figure the details would need to be worked out, but wanted to see what others thought here, Ruhrfisch  ><>°° 02:48, 26 November 2007 (UTC)
replies: text: At WikiProject Mathematics, there is a subpage called Wikipedia:WikiProject Mathematics/Current activity, which is mainly autogenerated by Jitse's bot. However, I'm not convinced that very many people check it, and the WikiProject Mathematics page itself isn't particularly active. If you want to attract the attention of a bunch of mathematicians, the place to post is on the project talk page, and there, a personal message from a nominator is more likely to get a response than something automatic. I imagine other WikiProjects are similar. Geometry guy 17:45, 26 November 2007 (UTC)
replies: text: It does depend on active Wikiprojects to work - I know the ones I am involved in are decent for feedback on specific questions, less so on requests (on the talk page) for PR or GAC or FAC feedback. Oh well, thanks for the feedback, Ruhrfisch  ><>°° 21:16, 26 November 2007 (UTC)
thread_title:  Another bot idea 

message: I'd like to request a bot writer to work with the Wikipedia talk:Content review/workshop on a suggested change to Peer Review.  There has been quite a bit of talk at the workshop about possible changes to PR and the current plan is to implement categorization on it. Because the page is currently manually archived, this would require a bot -- the manual process is already tedious and would become unmanageable without automation if categorization were introduced.  We'd also like to change the page to list only links, rather than the whole existing review. Here's a mock-up of how the peer review page might look: Wikipedia talk:Content review/workshop/Peer Review mockup.
We are hopeful that this is the first step in a reinvigoration of Peer Review.  If this works, we hope to come up with more ideas to help improve not only peer review but other content review processes.  However, this first step is in some ways the most important: peer review is a key part of Wikipedia, and it's not working as well as it could.  We believe organizing the page will really help participation, and we'd like to work with someone who would be interested in continuing to participate and who can help us improve our ideas and make them implementable.
If you're interested, post a note either here or at the workshop page, and we can talk about implementation details.  Thanks. Mike Christie (talk) 02:13, 4 November 2007 (UTC)
thread_title: Peer review automation bot

message: Further to the above discussion, Carl has now fixed the bug, and has also, at my request, made the VeblenBot's category listing more robust by keeping a record of the timestamps of articles in the category. This means that if an article is removed from the peer review category for a short time, and then re-added, the timestamp remains unchanged, and so the position of the article in the peer review list is unchanged. In particular, mistakes, such as the one I made previously with the peer review template, will no longer cause problems. Also, this prevents editors from moving their article to the top of the list by removing it briefly from the peer review category. Not that anyone would try that ;-)
The auto-generated peer review page is now strikingly similar to the hand-made version. There are one or two minor differences in ordering (due to delays in adding listed articles to the category), but the only substantial difference is that a WikiProject highways article is listed on the PR page, but not on the automated mock-up. Actually, the highways review is closed, so this should probably be archived anyway. Anyway, my feeling is that we could try rolling out this technology soon. I'll start a thread on the PR talk page in a day or two, but any comments here are welcome. Geometry guy 20:19, 8 December 2007 (UTC)
thread_title: Another update (and good news)

message: Just spotted this at LoCE.  Interesting parallels to some of our discussions.  EyeSerene has invited some LoCE participants over to this workshop; I don't know if LoCE really falls within our remit, but I thought I'd point it out.  Mike Christie (talk) 01:29, 6 December 2007 (UTC)
replies: text: The copyediting thing is very interesting, and I think very much ought to be in our remit. Almost by definition, the only stable points in an article's life are when it's nominated for either GA or FA. So surely that's when the limited resources of the LoCE could  be best deployed. Why would anyone want to invest the effort to copyedit an article that may change out of all recogniton in a couple of weeks and need to be copyedited all over again? --Malleus Fatuorum (talk) 01:52, 6 December 2007 (UTC)
replies: text: Been there, done that, got no T-shirts. Have pretty much decided not to copy-edit any articles except GAC and FAC, for exactly those reasons. Unimaginative Username (talk) 04:03, 6 December 2007 (UTC)

text: Having just read this, I'm encouraged to think that I'm not alone in my belief. --Malleus Fatuorum (talk) 02:16, 6 December 2007 (UTC)
replies: text: As all the sci-fi movies about extra-terrestial life say, "You are not alone". Unimaginative Username (talk) 04:03, 6 December 2007 (UTC)
replies: text: I'll second (sorry, third) this.....FA and GA are great as reference points...cheers,  Casliber (talk · contribs) 06:02, 6 December 2007 (UTC)
replies: text: I'm in 100% agreement with all the above; it was partly the discussions Mike linked to above that prompted me to alert the LoCE, as the same things we're discussing seem to be cropping up elsewhere too. It makes sense to collaborate with like-minded editors, and poor article prose is almost invariably mentioned in reviews. To take Malleus's point, all the projects have limited resources, and unless we can find ways of deploying them more intelligently we're all going to drown ;) EyeSereneTALK 11:53, 6 December 2007 (UTC)
thread_title:  A side note on LoCE