[Flipside General] gongo2k1:
EDIT: The contents of this post are now deprecated and remain here for historical purposes only! Please see the posts below for the "proper" scripts.

So after playing with it some more, mounting a vfat filesystem on /data simply doesn't work because permissions and ownership can't be set. Even setting the most relaxed permissions possible I found bizarre side effects like apps won't install from the Market and wifi can't be toggled on/off. I found an ext2.ko compiled for the Milestone stock rom (I've attached it here for convenience, but I found it on android-hilfe.de) works just fine though, so now we can use a proper filesystem.

Right now I'm mounting a 512 MB ext2 filesystem onto /data and everything works great. The biggest problem I'm having though is that switching to USB Mass Storage mode forcibly unmounts my filesystem so that obviously causes problems, but I don't really see a way around this since UMS mode offers up the entire MMC device so there's no way to just pick one partition to expose...

I'm going to test more things to make sure nothing has broken before I post a full write up on this, but if you know what you're doing, it shouldn't be hard to figure it out on your own. I know most of the Apps2SD projects are using an application to manage data then setting up a separate partition on the SD card, but it usually relies on some custom recovery image and the goal for me is to do this against a stock rom. I'm also using a loop-mounted filesystem because it makes it easier to migrate to a new SD card without having to worry about partitions and what-not.

EDIT: Here's a quick-n-dirty version for those of you who would like to try it. This will show the proof of concept and let you see if it works for you without breaking anything major. All changes to /data will be undone on the next reboot. If you know what you're doing, then I would recommend making the changes permanent by mounting /data in install-recovery.sh. This is assuming you have rooted your phone (I used z4root) and have Busybox installed (I used 1.18 from the Market) and at least 512 MB free on your SD card.

adb push ext2.ko /sdcard
adb shell
su -
/system/bin/busybox mount -o rw,remount /system
/system/bin/busybox cp /sdcard/ext2.ko /system/lib/modules
/system/bin/busybox chown 0.0 /system/lib/modules/ext2.ko
/system/bin/busybox chmod 0644 /system/lib/modules/ext2.ko
/system/bin/busybox mount -o ro,remount /system
/system/bin/busybox insmod /system/lib/modules/ext2.ko
/system/bin/busybox dd if=/dev/zero of=/sdcard/userdata.ext2 bs=1M count=512
/system/bin/busybox losetup /dev/block/loop0 /sdcard/userdata.ext2
/system/bin/busybox mkfs.ext2 /dev/block/loop0
/system/bin/busybox mount -o rw,remount /
/system/bin/busybox mkdir /userdata
/system/bin/busybox mount /dev/block/loop0 /userdata
/system/bin/busybox chown 1000.1000 /userdata
/system/bin/busybox chmod 0771 /userdata
cd /data
/system/bin/busybox cp -r -f -H -p . /userdata
cd /
/system/bin/busybox umount /userdata
/system/bin/busybox rmdir /userdata
/system/bin/busybox mount -o ro,remount /
/system/bin/busybox umount -l /data
/system/bin/busybox mount /dev/block/loop0 /data -o rw,sync,nosuid,nodev,noatime,nodiratime

If you've done everything right, "lsmod" and "cat /proc/filesystems" will show "ext2", "mount" will show /dev/block/loop0 on /data, and "df" will show more space available on /data. You should also see the extra space in Settings -> SD card & phone storage -> Internal phone storage.

My install-recovery.sh currently looks like this:

#!/system/bin/sh

if [ -e /system/lib/modules/ext2.ko ] && [ "`/system/bin/busybox lsmod | /system/bin/busybox grep -c ext2`" -eq "0" ]
then
/system/bin/busybox insmod /system/lib/modules/ext2.ko
fi

# wait 30 seconds, /sdcard is being re-mounted by vold...
/system/bin/busybox sleep 30

if [ -s /sdcard/userdata.ext2 ] && [ "`/system/bin/busybox cat /proc/filesystems | /system/bin/busybox grep -c ext2`" -eq "1" ]
then
/system/bin/busybox losetup -d /dev/block/loop0
/system/bin/busybox losetup /dev/block/loop0 /sdcard/userdata.ext2
/system/bin/busybox umount -l /data
/system/bin/busybox mount /dev/block/loop0 /data -o rw,sync,nosuid,nodev,noatime,nodiratime
fi

You don't have to use this as-is, but it's at least a starting point.

UPDATE: Seems like we need a Busybox with a proper tune2fs utility as the one in the Market is pretty useless (only changes volume labels...). You'll want to run "tune2fs -c 0 -i 0 /dev/block/loop0" to disable mandatory fsck'ing, otherwise the kernel will reject the mount after a few times because it's dirty. Something funky is happening because any apps that I install once the image is mounted will not show up after reboot, but the files are there and they continue to take space. Maybe I'm breaking something with the lazy umount?