Wikipedia talk:WikiProject Vandalism studies/Study2/Archive2

message: Alright, as it looks like the Random Edits study one the vote above by 2:1 (see above), lets get the blueprints laid out. I was thinking this time we should follow a breakdown using the scientific method. Let's start filling in areas we can and set things up to start work. (omitted until work is done: Abstract, Data, Results and Discussion, Conclusion, References). JoeSmack Talk 20:06, 6 April 2007 (UTC)
replies: text: I went ahead and added this to the main page.  I think we can start working on stuff there now unless anyone objects. Remember 16:45, 7 April 2007 (UTC)
replies: text: I was WP:BOLD myself and proposed an outline procedure.  Make any changes and comments as need, including scrapping the whole thing.  This is just one idea. --Jayron32|talk|contribs 17:51, 7 April 2007 (UTC)
replies: text: Some have talked about a recent changes study, but I think the consensus was to get a truely random sampling we need to look at randomly selected edits (which would be selected by a number gneerator and then going straight to that edit).  So I have moved your suggestions to below for further comments. Any other feelings? Remember 18:19, 7 April 2007 (UTC)
replies: text: My understanding is that we're going to go forward with the random edit study, not the recent changes version.  That said, I'd like to see the scope of this study to be massive.  We need to look at far more edits than we will be able to look at ourselves.  I think we should proceed by (1) Figuring out what data we want to look at for each edit.  See the working list above.  (2) Spelling out a step-by-step procedure for examining an edit.  This way we can easily harness the work of new volunteers. (3) Generating the list of random edit numbers that we need data for.  (4) Start filling in the data.  I've talked this over with another contributor, so I'm going to make to further specific proposals.  First, that we examine 5000 edits.  That sounds like a huge number, but it will give the study credibility and attention.  That, coupled with a significant amount of information about each edit will also make this study useful.  Economist dig massive amounts of information, and our study can help provide that (along with whatever our initial conclusions are).  Which leads me to my second proposal, that we collect the data into a comma separated value (CSV) chart.  CSV is a common denominator for spreadsheets and databases and will make it easy to load for sophisticated analysis.  I believe that is the way to add additional value to the study. Martschink 00:29, 8 April 2007 (UTC)
replies: text:  I concur. Pick a date range, generate a large set of random edit numbers within that range (after determining what edit #s correspond to that date range). The set should include more than the x edits we want to study, because not all edits will be in article space. Once the measurements are decided on, then decide on a format for recording the results, ensure all contributors understand what their task is (do a test round perhaps), divide the set of random edits among those involved, and off we go. (Still not sure how much I will participate, but thought I'd contribute here.) –Outriggr § 23:39, 20 April 2007 (UTC)
thread_title:  Random Edits study, formulation and structure 

message: I thought I would create a section for possible tools we could use to collect the relevant data. Remember 17:56, 9 April 2007 (UTC)
thread_title:  Possible tools 

message: Is the decision to analyze "random edits" in a given time frame, not "recent changes"?
If the goal is a large data set, we need specific designed programmed tools to gather data faster.  Without that help 5000 edits is to much for 8 or 9 volunteers.  It would be easy feed in random numbers for the edits to programs which could plug fields like article name, hyperlink, edit date, user name, +/- 10 edit date, and creation date.  Maybe size of edit to.  If we work from such generated data set, we can click link to each edit, code its edit type and edit notes, and also other data I do not know could be gathered well by programs because of judgement involved such as catalog count, # links to article and revert information, in fraction of the time.  Also we can put more time on coding atypical edits, for example edits filtered because there are not 10 edits prior.Venado 15:43, 23 April 2007 (UTC)
replies: text: Yes the idea is to do random edits and not recent changes. I totally agree with the second idea that a program would do this much better than we could and we should probably just focus on doing the stuff that humans can do well (like saying what is vandalism and what isn't).  Any help in anyone figuring out how to set this up would be most appreciated. Remember 16:39, 23 April 2007 (UTC)
replies: text: We have two bot coder volunteers up there, perhaps tap them on the shoulder via their talk pages? JoeSmack Talk 17:30, 23 April 2007 (UTC)
replies: text: I did that with one, feel free to tap the other. Remember 17:51, 23 April 2007 (UTC)
replies: text: Finito! :) JoeSmack Talk 18:06, 23 April 2007 (UTC)
replies: text: Yes, if you have a set of edits we can probably get you some information - user, edit summary, time, article name, net defference in size. Do you have a list of edits? I can give you a list of random numbers as oldids, and use that as the bot's input. LMK. ST47Talk 14:22, 24 April 2007 (UTC)
replies: text: Thank you for doing this.  I can develop random number list if needed.  We have not chosen the time frame yet but I can get the list when we do. Venado 17:06, 25 April 2007 (UTC)
replies: text:  I would suggest we study a recent discrete time period (e.g., all the edits in 2006 or if that is too big, all the edits from January through March 2007).  User ST47, could you just provide an example of what your information your bot could provide on a random edit.  For example, could you give a demonstration of all the info your bot can pull for edit number 87309971, which can be found here . Remember 17:22, 26 April 2007 (UTC)
thread_title: Clarification

message: First, you guys are doing a wonderful job. Results of your first study will be cited in many places, I am sure. Second - I'd like to invite all to meta:General User Survey. Studying vandalism is important. But studying all of Wikipedians would be extremly useful... the project, unfortunatly, is stalled due to lack of interests among developers (the question part is mostly done, but nobody has a workable idea to carry out a survey (this being the best one...).-- Piotr Konieczny aka Prokonsul Piotrus | talk  07:30, 24 April 2007 (UTC)
thread_title: Think big

message: I was thinking now that the article's history displays how large an article is at each point in its history (by stating how many bytes it is in parentheses on the history page), we could limit our study to a time period that just has this feature incorporated into the wiki software and this might prove to add a lot of useful data easily gatherable data to our study.  For instance, this would let us know the average size that a vandalized article was at the time of vandalism and how much information was added by a vandal when he vandalizes it.  So I was thinking for this next study, we should just do a random sampling of all of the edits from May 1 to May 31, 2007 (the change to the wiki software took place on April 19).  We could use the rest of this month to prepare and start the study on June 1 2007.  What do people think of this? Remember 20:46, 6 May 2007 (UTC)
replies: text: There's certainly a strong argument for doing it that way.  We'd lose seasonal effects (like April Fools' Day) and we would be able to show changes over long periods of time, but otherwise I don't think the overall quality of the study would be compromised.  Good plan.  Martschink 15:44, 7 May 2007 (UTC)
replies: text: Yes, I don't see any reason to use older dates that contain less data. So we'd be gathering data in May 1 - May 31 from random edits, and start analyzing data in June? If so, lets focus on gathering data right now, namely how? JoeSmack Talk 16:25, 7 May 2007 (UTC)

text: We would have to wait until May ends so that we could randomly select edits from all of May.  Therefore, we can't start the study until after May 31.  But we could get everything ready for the study. Remember 17:02, 7 May 2007 (UTC)
replies: text: Agreed, we'd have to wait until the end of May to gather the data.  Do we need to talk about numbers?  Also, do we know how we're going to generate the non-repeating list of random edit numbers?  And I'm still of the opinion that we should do a trial run.  Maybe we should try a 100-edit micro-study of April so we can work any kinks out of the system?  Martschink 19:03, 7 May 2007 (UTC)
replies: text: Test run sounds good to me.  As for a random number generator, how about this: 
Wait... this is all very silly... I already collect the size of the article at the time of the edit regardless... I shall go ahead and add in the difference in size between the selected revision and the prior one (if it exists) for the 2006 samples which I collected the first 1,000 of last night. --Auto(talk / contribs) 13:15, 4 June 2007 (UTC)
thread_title:  New revision to planned study proposed 

message: Notes: Some sampled revision numbers are deleted revisions... I decided that sampling those would be valuable, so I keep the data (examples #5, #12). Redirects behave weirdly (#9). I don't have contrib history for IP address (#1, #3, #4). I can format in colums to edit later... it is probably best if input for determining whether an edit is vandalism is put into an actual SQL database as we go along to prevent conflicts and allow us to read the data easily. It's probably more likely, though, that I'll just end up breaking everything into 100 sample sections and recombining with a bot. --Auto(talk / contribs) 21:16, 17 May 2007 (UTC)
replies: text: This is great! Finally we're going to have some raw chunks of data to work with! Thanks for all the work on this! :D JoeSmack Talk 16:43, 18 May 2007 (UTC)
thread_title: Sample Output