Wikipedia talk:Bots/Archive 5

message: * en:User:NetBot - pywikipediabot, no objections given for bot flag on en:Wikipedia, see en:Wikipedia talk:Bots#NetBot request. -- Netoholic @ 14:41, 24 Jan 2005 (UTC)
replies: text: Yeah, the objections were only made after I posted the request on Meta: (which itself was done after giving one week here).  Screw it.  If admins have to sink to low levels and sandbag my attempt to help perform these maintenance tasks, then I don't wanna be involved.  You guys argue it out.  If you all decide this is worthwhile, you let me know. -- Netoholic @ 04:30, 2005 Jan 26 (UTC)
More from m:Requests for permissions
* en:User:NetBot moved to en:Wikipedia talk:Bots#NetBot request since there are two objections to this bot. Angela 03:01, 26 Jan 2005 (UTC)
thread_title: From [[m:Requests for permissions]]

message: I will shortly demo a new capability of Pearle, to add  tags to a list of specified categories.  Please post if you have any qualms about her doing this on a routine basis to any categories mentioned on CFD, especially long lists of badly-named categories. -- Beland 07:11, 1 Dec 2004 (UTC)
thread_title: Adding CFD tags

message: I have used a new capability to fix the intro text in first two subcategories of :Category:Japanese prefectures.  The problem was when there is a long list of categories to be renamed, she was able to move articles and subcategories, but not intro text.  The problem with intro text is that there might be some on both the source and destination pages, and a simple combination might be the wrong thing to do.  Human intervention is needed to verify these moves and merges.  I run Pearle in two steps.  First, a TRANSFER_TEXT_CHECK command makes some suggestions about what the source and destination categories should be changed to.  It automatically deals with  tags, merges lists of categories, and trims excess whitespace.  Where the suggestion is wrong, I fix the text manually.  Where the suggestion is correct, I simply run TRANSFER_TEXT_ACTUALLY with the same arguments, and the suggestion is automatically implemented.
Pearle is currently renaming subcategories :Category:Japanese prefectures so that "prefecture" is capitalized where appropriate.  I have scheduled a TRANSFER_TEXT_ACTUALLY run immediately thereafter, which will fix intro texts (which should also create the correct category hierarchy).  Please let me know if you have any qualms about Pearle doing this on a routine basis to categories approved by WP:CFD for deletion. -- Beland 00:23, 5 Dec 2004 (UTC)
thread_title: Moving category intro text for CFD

message: # The Pearle bot ws approved for use to "For each member, replace all instances of :Category:Name_of_A, with :Category:Name_of_B, preserving sort fields. Members that contain any nowiki or pre tags in the wikisource will be skipped."
# The bot operated as in this report , carrying out the moves not in the manner described, an efficient change from one category to the other, but by making two edits in rapid succession, the first to remove a category, the second to add the new category to the same article.
#This mode of operation unnecessarily increased the server load required for the operation and introduced the permanent need to store an unnecessary article revision in history.
This bot appears to be
# operating in a manner inconsistent with its authorisation
# operating contrary to the bot policy that edits should be about ten seconds apart
# operating contrary to the policy that a bot should not be a server hog, by making edits at an excessive rate and making two edits where one is sufficient for the job.
Per the bot policy, please demonstrate that this bot is harmless, not a server hog and operating as approved when operating as described above. Jamesday 22:58, 10 Jan 2005 (UTC)
replies: text: Thanks for the improvements - I expect the humans appreciate the greater efficiency.:) All bot operations are probably also benefitting from some much improved saving code put into service a few weeks ago - if you notice your bots taking less time per edit, that'll be why. The most human visible symptom is far fewer lock wait timeouts. Jamesday 21:57, 3 Feb 2005 (UTC)
thread_title:  Pearle bot conduct 

message: Unfortunately this bot is contravening policy.  Categories should be placed before interwiki links, and not after, as per Wikipedia:Interlanguage links#Syntax and Wikipedia:Categorization#How to create categories.  This has been reported on the bot talk page.  I request that the bot be stopped until this has been remedied.  
It would be nice if the bot could go and correct all the articles that have been affected.  Noisy | Talk 19:50, Jan 16, 2005 (UTC)
I will be replying on User talk:Pearle. -- Beland 22:43, 17 Jan 2005 (UTC)
thread_title: Original complaint

message: OK, so there are 2353 articles* that offline tests show need fixing, out of the 17000 or so with categories that Pearle has edited.  I am going to run the first 22 and then wait for 24 hours to see if there are any complaints or comments.  If there are none, I will run the remainder, then resume normal operations. -- Beland 02:38, 10 Mar 2005 (UTC)
thread_title: Demo in progress

message: Personally, I would remove/not add HTML comments. Besides, I feel it would be more productive if the bot would also add additional interwikis, rather just HTML comments. Many of pearle's recent contributions ("Minor category/interwiki code cleanup" on 05:00-06:00, 2005 Mar 12 ) seem to add just comments and re-arrange links. -- User:Docu
:Well, it was requested that Pearle go back and rearrange category and interwiki links in cases where she failed to follow the established style.  I'm restricting the current run to articles she's edited before, but there are also minor changes from the "recommended" style now.
:If you're saying it's not particularly worthwhile to tidy things up in this way, I would agree.  (As opposed to "please don't add HTML comments, ever" - if you mean that, then perhaps we would need to reconcile that opinion with the opinion of the editors who tend to leave these random comments lying around.)  After the self-cleanup, such tidying will only be done in conjuction with a more important edit.  Adding interwiki links is outside the scope of what I'm trying to do; I assume that could be done just by comparing what interwiki links other wikis have?  That does sound like a good idea. -- Beland 00:09, 13 Mar 2005 (UTC)
thread_title: Comments 

message: Sometimes a large number of categories which have been tagged by Pearle don't get deleted, and have to be untagged.  This command would simply remove , , , etc.  Will do a first run in a few days, then integrate into normal operations.  Comment/complain as needed. -- Beland 06:31, 11 Mar 2005 (UTC)
replies: text: OK, it looks like the last good edit was at 02:13, 15 Mar 2005 (Auxerre), and the first mistake was at 02:42, 15 Mar 2005 (National_Movement_for_Simeon_II).  Fortunately, everything after that before Iron were political party articles, which have sparse interwiki links.  I will be able to fix those manually in the next day or two. ::sigh:: -- Beland 06:53, 18 Mar 2005 (UTC)
replies: text:  I rollbacked 98% of them.  Some of the articles have been edited so there are at least 1% to 2% that I missed. -- AllyUnion (talk) 07:26, 18 Mar 2005 (UTC)

text: I think I have manually repaired the rest.  Thanks for your help, and apologies to anyone who was inconvenienced. I found the source of the problem; I commented out a whole line when I should have commented out only part of it.  (This was when I was implementing the request not to add the "The below are interlanguage links." comment.)  Mental note: Make sure "good/evil" switch is set to "good" before unleashing creation into the world. -- Beland 20:44, 19 Mar 2005 (UTC)
thread_title:  Pearle REMOVE_CFD_TAG 

message: Something has gone wrong with Pearle's interwiki style cleanup logic.  (She was just wholesale deleting all interwiki links.)  All of her edits from 02:23, 18 Mar 2005 (Iron) to 06:02, 18 Mar 2005 (Lockheed L-1011) need to be reverted.  I will need to figure out when this started happening and see if there are any more reverts, and of course to fix the underlying problem.  I'm hoping someone has a vandalism revert tool handy that can do this before human editors spend too much time trying to clean up the mess.  If not, I will try to create one as soon as possible, which may be Saturday, US West Coast time. -- Beland 06:15, 18 Mar 2005 (UTC)
replies: text: OK, it looks like the last good edit was at 02:13, 15 Mar 2005 (Auxerre), and the first mistake was at 02:42, 15 Mar 2005 (National_Movement_for_Simeon_II).  Fortunately, everything after that before Iron were political party articles, which have sparse interwiki links.  I will be able to fix those manually in the next day or two. ::sigh:: -- Beland 06:53, 18 Mar 2005 (UTC)
replies: text:  I rollbacked 98% of them.  Some of the articles have been edited so there are at least 1% to 2% that I missed. -- AllyUnion (talk) 07:26, 18 Mar 2005 (UTC)

text: I think I have manually repaired the rest.  Thanks for your help, and apologies to anyone who was inconvenienced. I found the source of the problem; I commented out a whole line when I should have commented out only part of it.  (This was when I was implementing the request not to add the "The below are interlanguage links." comment.)  Mental note: Make sure "good/evil" switch is set to "good" before unleashing creation into the world. -- Beland 20:44, 19 Mar 2005 (UTC)
thread_title: Revert request

message: Originally posted at Wikipedia:Village pump
Hey all, I was thinking about making a bot to browse through Wikipedia and generally help out in any way possible.  I've recently learned how to access the internet using Java (which I know well) and I wanted to make a little program for practice.  My question is, what should this bot do?  I've thought about it, and right now I'm leaning towards having it make links more efficient by changing them so they directly link to a page if they're currently linked to a redirect.  I figure it can't hurt, right?  Anyways, what other miscellaneous chores should this bot do?  Any suggestions? --pie4all88 06:49, 13 Dec 2004 (UTC)
replies: text: There's a downside in that piped links make the wikicode slightly harder to read for an editor, and, especially because editors are not all techy-types, we need to make it as readable as possible. For example, the wikitext could be either ...NSA... or ...NSA... The latter is more readable. — Matt Crypto 14:44, 14 Dec 2004 (UTC)
replies: text: Ok.  Someone else had mentioned at the village pump that he or she thought that there are some good reasons for redirects, too.  Maybe that's not the best thing to have a bot do...I'm still open to suggestions, though :D --pie4all88 20:59, 14 Dec 2004 (UTC)

text: Have a look at Wikipedia:Bot_requests. RedWolf 01:39, Dec 17, 2004 (UTC)
Hi pie4all88, Here's a possibility - don't know if you'll like it or not, but I'll just put it out there and you can decide what you think .... : Where A is a redirect to B (e.g. "Hamsters" → "Hamster"), and the word A is a plural of the word B (as defined here), and where A does not contain the string "" (e.g.  this plural redirect does contain that string, but most currently don't), then add the string " " to A, and save it. (Basically, this would put all the plural redirects into the right category.) If you don't like it then let me know, and if I think of anything else I'll let you know. All the best, -- Nickj 22:52, 16 Dec 2004 (UTC)

text: Categories with thousands of entries help a denial of service attack on the servers. To display a category page currently requires retrieving the full text of every article in the category, making a category with 5,000 entries cause the same work as 5,000 seldom-visited page views (not 5,000 normal page views because normal views tend to end up in cache). For that reason I was recently forced to remove the category (and image) from the GFDL template after it tripped the denial of service attack emergency response: one view of the category took the master server from 8 to over 700 pending queries in less than 30 seconds. And down again 30 seconds later after that page view. The damage can be significantly limited, for suitably small categories, by using the first letter or two letters of the page to create sub-categories, each of which is tolerably small. The sub-pages of allpages are currently set at 500, which is too high for comfort, but I've been reluctant to ask for that count to be reduced to 100, mostly because I know that there are improvements planned for MediaWiki version 1.5. Please refrain from using a bot to expand the number of such denial of service opportunities. We can in an emergency disable any of these things (by turning off category display, for example) but I'd much rather not have the opportunities created in the first place. Also see bug 1058, which is why I'm not asking a bot operator to split Category:GFDL or doing it myself on the servers, and bug 973. Jamesday 04:38, 17 Dec 2004 (UTC)
replies: text: Whoops... Ah... OK... How do I put this... Maybe "Houston, we (may) have a problem" ? You see, I had no way of realising (until now) that large categories caused a problem, and there's a small project I'm running for adding Missing Redirects, and as part of that, well, new plural redirects are being added to the exact category described above. Just to be clear though, this is humans doing the adding — bots are in no way doing any of the adding (it's just that I'm interested in bots, and I'm doing this other thing as well, so me seeing your message on this topic is basically just blind luck). Anyway, when redirects are plurals, they're getting put into this category. Currently there are 239 redirects in this category. By the time we're done, there will be (I estimate) 5000 redirects in this category - so in a few weeks that example 5000 item category will be, as it were, more of an actual than an example ... I guess this is all just proving your point about the joys of coping with exponential growth ... So, I guess my question is this: is it OK to keep adding stuff into this category, or do we need to stop doing that (irrelevant of the category being appropriate)? All the best, -- Nickj 06:35, 17 Dec 2004 (UTC)
replies: text: You should also estimate exponential growth with doubling time of 8-12 weeks. That's where it gets interesting trying to keep up in the race of capability against growth.:) User:Jamesday/report/letter frequency gives the first letter frequency for en Wikipedia, showing that about 9% of articles begin with S. I call it 10% for convenience and a safety margin. So 5000 redirects by letter and you'd end up with the uncomfortably large 500 size for the worst case  page. 12 months and four to six doublings and it's at 2000-8000. Oops.:) I haven't checked the actual article doubling time - I'm using the capacity doubling period. For 8000 10% 800 but something like Sc is likely to be the most popular and that's likely to be closer to 500. So: might try going to two letters initially, if you want to think a year ahead. Can skip two for letters with less than 1% frequency or any which show a worst case 12 month prediction under 50. MediaWiki 1.5 is planned to chop the article text out of the cur table and make just getting a title much less slow. But it's a bad idea to assume any delivery time for software.:) For now, anything which needs titles needs to have a target result of under 500 titles. Jamesday 13:31, 17 Dec 2004 (UTC)
replies: text: Why not just use lists instead of categories?  Categories should have never been turned on, because the design was completely flawed (and very little was done to get input from the community before throwing them in).  So just pretend we don't have categories, and use lists.  Lists can do everything categories can do, plus more, plus they're more efficient (they're basically designed the way categories should have been). anthony 警告 14:22, 17 Dec 2004 (UTC)

text: That's a great idea--it's perfect for what I want to do.  I've been thinking about it, and I don't see any downside to it, so...I'll work on it pretty soon.  Right now I'm juggling finals, World of Warcraft, and other programming ventures :S  Anyways, thanks for the suggestion, Nickj! --pie4all88 01:45, 17 Dec 2004 (UTC)
thread_title: Creating a Wikipedia Bot

message: It will be running pywikipediabot and it will follow robots.txt and your speed limit
AxyJo 23:43, 1 Jan 2005 (UTC)
replies: text: What will the bot do? --Carnildo 01:05, 3 Jan 2005 (UTC)
thread_title:  Permission for a bot 

message: Hello. I posted this comment to Wikipedia:Bot requests but it didn't attract any attention so I'm trying again. I'm thinking about running a read-only robot to get random pages for the purpose of doing statistical analysis on them. Probably the edit history will be just as interesting as the page text. I don't think I could ever get more than 1 page per second, which iirc is below the limit placed on spiders (can't find the rules governing spiders at the moment). Does that seem OK in general? If anyone has any comments I'd be interested to hear them. Regards & happy editing, Wile E. Heresiarch 22:16, 8 Jan 2005 (UTC)
replies: text: Could you get the same results by downloading a snapshot of the database and running queries on it? --Carnildo 02:30, 9 Jan 2005 (UTC)
replies: text:  Well, maybe so. Maybe someone is selling snapshots on dvd? I'm not able to download snapshots, unfortunately. On the other hand, I'm not sure that the entire db would be interesting -- I was planning to sample just a fraction of the articles. On the other other hand, maybe if all goes well, it would turn out to be interesting to analyze the whole db. As you can see nothing has been decided yet. Best, Wile E. Heresiarch 03:43, 9 Jan 2005 (UTC)
thread_title:  Read-only random-page getter for statistical stuff 

message: Hi, I'm writing this here because I'm active on this wiki and I wasn't sure where else to put it. The request I'm making is for (what I'm assuming would be) a simple bot for the bosnian language wikipedia. Basically we made several mistakes in writing articles for all the years, and fixing them now would mean tediously editing hundreds of articles. Namely, our articles for decades would be named, for example, "1960te" when they should have been "1960e", so we need to change the links to all of those on the pages for the years and centuries, redirect the articles for the decades, etc. Also, we in the end decided that we would use different words for "century" and "decade". If it's possible, could someone help us make a bot that would fix this problem? Asim Led 19:25, 1 Apr 2005 (UTC)
thread_title: Request for a bot

message: Hi folks! I've been working on a WML gateway so that I can read wikipedia on my moblie phone while waiting for public transport. The gateway is read only, and I've got no plans to change that. I think the bot policy is geared towards robots that edit, but I'm asking for permission here to be on the safe side. The gateway is apache/mod_perl/libwww-perl based. The gateway uses Special:Export as the backend, and then parses the wikimarkup to make a fairly basic WML page. Long term, I would like to see the functions of this gateway moved closer to the database, but I'm happy to keep it running as long as it is is needed (and I'm happy to switch it off as soon as it is not needed). There is a testing version avaiible on my DSL line (so it runs very slow when my flatmates are filesharing), however I will move it to a better hosted site once I've got permission. (I don't want to put the link anywhere too popular untill testing is finished and I've got permission to run a bot, so you will need to look at the Talk page for Wikipedia:WAP_access if you have a WML browser and want to try it). Thanks. Osric 10:12, 3 Feb 2005 (UTC)
thread_title: WML Gateway

message: A summary of its tasks can be found on its user page and whether or not it is working at this time: Allyunion -- AllyUnion (talk) 05:36, 8 Feb 2005 (UTC)
thread_title:  Summary of tasks