Checking in changes prior to tagging of version 2.52.

Changelog diff is:

diff --git a/CHANGES b/CHANGES
index df4564d..54166a5 100644
--- a/CHANGES
+++ b/CHANGES
@@ -1,3 +1,17 @@
+2011-07-30: Release version 2.52
+
+   * Plugins (such as FilePaths) might want to be run in info/debug commands (Michael Baehr <mbaehr@plos.org>)
+
+   * schema version 14; support devices > 16TB (dormando <dormando@rydia.net>)
+
+   * Make FSCK worker handle dead devices (dormando <dormando@rydia.net>)
+
+   * Throw out old HTTPFile->size code and use LWP (dormando <dormando@rydia.net>)
+
+   * Remove forced loading of DBD::mysql (dormando <dormando@rydia.net>)
+
+   * Don't test write on readonly devices (Akzhan Abdulin <akzhan.abdulin@gmail.com>)
+
 2011-06-27: Release version 2.51

    * Add minor backcompat for ZoneLocal and friends (dormando <dormando@rydia.net>)
Fix host status not updating

New internal monitoring was ignoring host updates due to an early return
error. This bug is semi serious, as it only effects hosts due to them often
having many NULL fields in the database.
Checking in changes prior to tagging of version 2.53.

Changelog diff is:

diff --git a/CHANGES b/CHANGES
index 54166a5..d992720 100644
--- a/CHANGES
+++ b/CHANGES
@@ -1,3 +1,7 @@
+2011-08-12: Release version 2.53
+
+   * Fix host status not updating for trackers (important)
+
 2011-07-30: Release version 2.52

    * Plugins (such as FilePaths) might want to be run in info/debug commands (Michael Baehr <mbaehr@plos.org>)
Allow chunked PUT's of unlimited size

For some reason the Perlbal PUT default is 0 (unlimited), but chunked PUT is
200M. Lets flip the default to 0 in mogstored, so chunked PUT's will work
reliably.

Thanks to victori for pointing out that this setting exists and has a silly
default :)
Fix various issues in the new mogilefs server query worker.

1) Error: Can't locate object method get_by_id via package Mgd::domain_factory (perhaps you forgot to load Mgd::domain_factory?) at /usr/share/perl5/MogileFS/Worker/Query.pm line 639

Fixed by using MogileFS::Server everywhere that calls Mgd::xxxx functions

2) :: [queryworker(10076)] Error running command 'list_fids': Can't locate object method get_by_id via package Mgd::domain_factory (perhaps you forgot to load Mgd::domain_factory?) at /usr/share/perl5/MogileFS/Worker/Query.pm line 639.

Fixed by changing Mgd::domain_factory->foo to Mgd::domain_factory()->foo
Fix broken logging
fid size workaround for lighttpd 1.4.x

Fixes Issue 39. Lighttpd doesn't return a Content-Length header for zero byte
files. So we add a specific workaround for it. If it's not lighttpd, we should
still throw errors if that header is missing.
sqlite: fix mogdbsetup failure on new setups

upgrade_modify_device_size() should not fail on new setups.
SQLite won't enforce MEDIUMINT vs INT anyways, so it's safe
to always continue and store larger INT values in MEDIUMINT
columns.
Fix mb_free() counting empty device as missing
the exist method no longer exists

Update the if condition to match the if condition of the similar code blocks in the Delete Worker
Allow default class to be overridden again.

Also makes its support slightly more DWIM. If you ask to modify the default
class, it will be autoadded to the database if it hasn't yet. Add works as it
should.

Delete *doesn't work*. Once you add a default class to the database, you're
stuck with it. You can continue to change it. We could "delete" it and walk it
back to an original form, but it seems more correct to tell people to not do
that.
Issue 36: Further insulated from autoinc resets

If MySQL is restarted and tempfile loses its autoinc counter, normally
mogilefs will detect that a new fid insertion is a duplicate and resets it to
MAX(fid) FROM file. However this doesn't work as well when your low fids may
be deleted.

So now, trackers check MAX(fid) on start and refuse to go below that. Needs
further work but this will go a long way.
Checking in changes prior to tagging of version 2.54.

Changelog diff is:

diff --git a/CHANGES b/CHANGES
index 5e4e8d2..79e8a7c 100644
--- a/CHANGES
+++ b/CHANGES
@@ -1,3 +1,21 @@
+2011-10-29: Release version 2.54
+
+   * Issue 36: Further insulate from mysql restarts resetting the fid counter (dormando <dormando@rydia.net>)
+
+   * Allow default class to be overridden again. (dormando <dormando@rydia.net>)
+
+   * Fix crash in old delete worker code (Daniel Frett <daniel.frett@ccci.org>)
+
+   * Fix mb_free() counting empty device as missing (git.user <git.user@gmail.com>)
+
+   * sqlite: fix mogdbsetup failure on new setups (Eric Wong <normalperson@yhbt.net>)
+
+   * fid size workaround for lighttpd 1.4.x (dormando <dormando@rydia.net>)
+
+   * Fix various issues in the new mogilefs server query worker. (Tomas Doran <bobtfish@bobtfish.net>)
+
+   * Allow chunked PUT's of unlimited size (dormando <dormando@rydia.net>)
+
    * Fix listing FIDs.
    * Fix fsck fixing a file from logging a message which appears to be
      two commands (due to spurious \n). Also fix debug function to
Wrap the queue tx with a mutex lock under mysql

MySQL versions are very deadlocky with this pattern, unfortunately. Worse yet,
under 5.1+ INSERT's can block forever so long as the SELECT's overlap.

Wrapping it in a lock to serialize the transaction (which is fine for
performance, anyway) stops the INSERT's from blocking and reduces the
deadlocks a little.
restore new file upload replication priority

now that the queues are locked uniquely, and MySQL 5.1+ does better at
avoiding deadlocks, return to scheduling new file uploads as "0" (immediate)
so reaper work doesn't gum them up.
stop enforcing a source id on new file upload

file_on rows have been added ahead of the replication call since... forever?
so just use it. Over-replication bugs on new upload get jammed up due to this.

I just can't think of a justification for keeping it the way it was.
schedule fsck replications for the future.
return 1 from condthrow if all is well

many functions just return from condthrow, but we often check their return
values :/

Thanks to Eric Wong for the tip.
More stats to help troubleshoot rebalance

Need to be able to see more easily if a rebalance is jammed. Also fix a random
warning.
fixed sources in replication are now a suggestion

Code used to hard error if a fixed source was missing. That would be useful
if, say, FSCK decides that some copies are bad and it should be re-sourced
from a particular device. However FSCK can just nuke or mask bad rows and
re-run.

So... attempt, then give up. We can always revert the behavior if some feature
requires it. New file uploads no longer use fixed sources, which gets rid of
most of the issue. This change should let existing clusters clean themselves
up.