yashsingh1/xdaforums huggingface dataset:
[Epic 4G Android Development] codest3r:
Assuming the recovery kernel has the drivers loaded and can mount the mtd partition flashing filesystem changes and overwrites should be the same, as I understand theming pushes work.

Other. Neither the ROM or kernel would set the initial size. The partition map would. It's not clear to me if the values (for size, location, and order) are set in a config file or script during conversion from BML to MTD.

The best visual tool I could use as an analogy would be gparted in linux or disk management (microsoft management console snapin) in windoze. Once you set partitions with said tools of various sizes, start points, end point, format types (ext, fat, ntfs, etc), you can load any number of OS's on each respectively (or ROM's in our case) assuming they are compatible with the filesystem types and have a common unifying bootloader that understands them all (that can read each fs and call the respective bootloader). Sorry for the poor anaology but trying to put it all in more familiar terms.

Sent via Decadent Epic 4G CM7
[Epic 4G Android Development] toadlife:
In testing with my ROM, I modified the updater script to check the /system and /data partition sizes and if they don't match up, flash the kernel and reboots before flashing the ROM.

A standard partition size would be easiest, but a checking partition sizes would also work.

---------- Post added at 02:51 PM ---------- Previous post was at 02:44 PM ----------

I don't think that's true. I'm pretty sure the partition map is defined in the kernel...period. So if you write data to your partitions to the nand and then load a kernel with a different partition map, the device will not boot because it won't be able to read the partitions.

I'm not 100% sure about this...only 98%, so if someone wants to correct me, please do.
[Epic 4G Android Development] codest3r:
You do need the proper drivers in the kernel to interact with the OneNAND, it's partitions, and data formats (e.g. the storage memory or ROM in this case). However, the kernel itself doesn't define the partition map. Without the proper kernel (and associated drivers) we can't mount the filesystems in storage memory, that's for sure!

With BML the partitions are sized and created from the Partition Information Table during flash (e.g. ODIN, Samsung Upgrade Tool, Heimdall). More information on PIT specifics can be found (with good links too) in a post from the SGS i9000 forum: [GUIDE] [INFO] The reality of PIT files

Notice in his post the PIT is not part of the kernel - it has it's own parition. Which is likely why you don't have to specify the PIT file when flashing BML ROM's on our Epic because it just uses the PIT on the device! Pretty genius idea to keep it stored here in it's own space.

START

PBL (256KB: Primary Boot Loader: boot.bin)

PIT (*256KB: .pit )

EFS (5.9MB: efs.rfs )

SBL (1.25MB: Secondary Boot Loader: Sbl.bin)

SBL2 (1.25MB: Backup Secondary Boot Loader: Sbl.bin)

PARAM (*640KB: param.lfs)

KERNEL (7.5MB: Primary Kernel: zImage)

RECOVERY (7.5MB: Backup Kernel: zImage)

FACTORYFS (276.3MB: factoryfs.rfs)

DBDATAFS (126.7MB, 117.2MB, 107.2MB (depends on pit file): dbdata.rfs)

CACHE (30.1MB: cache.rfs)

MODEM (12MB: modem.bin)

END

*=Assumed

I'm not sure what they are using with MTD to set the partition map. It could be set in the conversion script or in a configuration file. I couldn't find it after a cursory skim of the contents of the CWM flashable zip. That has eluded me so far. :)
[Epic 4G Android Development] toadlife:
The reason you couldn't find it is because it's defined by the kernel. Specifically, the partition layout is defined in the file Kernel/drivers/mtd/onenand/samsung_epic.h

See: https://github.com/toadlife/CleanGB-Kernel/blob/mtd/Kernel/drivers/mtd/onenand/samsung_epic.h
[Epic 4G Android Development] codest3r:
Ahhh... ha. Thanks for that! I'm surprised to see that there. :o

Leads me to more questions! :eek:

If you were to change the mapping (i.e. modify offset or size of a few partitions in samsung_epic.h) - compile and flash your kernel - what would happen? Assuming you didn't change ROM's - just kernels. My guess is nothing if your phone is already converted to MTD. The kernel should mount the partitions just fine without the map, right? :confused:

However, if your device wasn't converted to MTD yet the flashable ZIP for a ROM (with kernel) would contain the script instruction to check and if not converted -> execute conversion... at which point it would look for these values in the kernel. Right?

Taking that a step further, each ROM maker could choose whether to check the sizes of MTD partitions and if in tolerance (some predefined range of acceptable) it would flash ROM package on existing partition sizes, OR if not acceptable it could wipe out the partition (and all data on it) and start over with preferred sizes. Right?

Debunk my guesses. Thanks for the learning opportunity. Please excuse the ignorance and dumb questions - I'm trying to figure out how this works and missed the instruction manual. :D
[Epic 4G Android Development] codest3r:
First, thanks for taking the time to run this through the gamut and test it out! Also appreciate you sharing the results and code. Helps me understand more of whats happening. Thanks man. :D

Fascinating to me. Seems like a risky approach to map the partitions in the kernel and not have it fixed elsewhere, but I assume this is best solution for employing MTD. Wonder if any other approaches are available or were considered? Frankly, I don't want a kernel modifying the partition sizes, locations, etc. -- at least without me being fully aware of it before flashing. Hopefully all the kernel devs are aware of this!

And I agree with your conclusion. If you had modified the start, stop, or size parameters of the system partition, or even tried to swap the /data and /system locations, it likely might have prevented it from even booting up.

That said, it should be possible to build on your code that recognizes the different sizes (as you did) and if changes were warranted to partition layout it could first verify the contents of each partition would fit in the new partition layout and if so backup the system, data, and cache (easiest to just straight up copy contents to /sdcard instead of nandroid?), then execute the partition layout changes, then after changes are made to the partition sizes restore files from backup. I suppose... ;-)
[Epic 4G Android Development] noobnl:
Maybe you shouldn't poke around with partition map.

What wrong with the one we have now? Please don't mess around with our layout. Why do you need 50 MB of Cache space?

market fix: https://github.com/EpicCM/android_d...mmit/a7b74d44929150c24ad61daa3f7f8a93e954d5ce

https://github.com/EpicCM/android_v...m-rebase/prebuilt/common/etc/init.d/06mountdl

Hope that helps!
[Epic 4G Android Development] toadlife:
Yeah I know of that fix. More space for /data is actually what I might be interested in. Good luck with your partition size mandate. Honestly there is NO point to using MTD/Yaffs2 if we don't make use of the ability to change partition sizes.
[Epic 4G Android Development] noobnl:
I think you should keep cache the same size but only change data and system because it is better to change only one offset instead of two offset.
[Epic 4G Android Development] hazridi:
There's plenty of reason to move to MTD without altering the current MTD partition layout, like getting rid of the crappy BML abstraction layer and replacing it with one actually built to reduce issues with flash filesystems, so we can use yaffs2 and have more advanced wear levelling. And reduce the reliance on a closed source filesystem and block layer.

Agree with noob here, you're really not creating anything but a huge problem for users.
[Epic 4G Android Development] toadlife:
You are right that this will cause issues for users, especially those who love the kernel/ROM hop, but how will those issues be avoided when ICS/CM9 rolls out and more space is needed in system? Will another official partition layout be shouted down from the mountain top and those of us who might want to continue to build TW based ROMs be forced to use massive system partitions which will go only half-used?
[Epic 4G Android Development] codest3r:
Would it be possible to merge system and data on a common root partition? No semi-arbitrary boundaries. Anything not used by ROM/system is avail for data. Still leave something tiny for cache like you have now.

I know this can cause problems with problem/offending apps bugging out and hogging space beyond reason or debugging logs running unchecked.... But that's unlikely to be an issue for us right?

Sorry if it's a dumb question but thought it would eliminate any differences of opinion (or need) for different ROM's partition sizing allocations on MTD. Probably some dependencies I'm not aware of... ;-)
[Epic 4G Android Development] toadlife:
I don't see that as a viable option. It would require a radical change in the way partitions are arranged and it would mean /system has to be mounted read+write, whereas it's typically mounted read-only.

I think I'm going to stick with the CM7 defaults for now. With the way people like to hop from kernel to kernel, and the way people don't read or follow directions, it would cause too much havoc for users. It would be nice to have the partition layouts defined somewhere else, so the kernel wouldn't have to have them baked in.
[Epic 4G Android Development] mkasick:
There is an on-flash partition table, bml2 (not mapped in mtd), and it would certainly make sense to support it under MTD kernels to allow for different partition sizes. It's not done simply becuase there's no MTD-pathway code for reading the on-flash partition table. Partition sizes are hardcoded in the Samsung MTD driver on the Nexus S, and, since that's how the driver was written, was kept for Galaxy S MTD kernels.

It's possible to add on-flash partition table support for MTD, but would require some amount of code. I'd guess it's a reasonably-sized project for someone sufficiently ambitious. The format is known for the relevant portion of the partition table, plus there's code in the tfsr driver for reading it also--which I wouldn't copy, but might help in making sense of it.

To be honest, the main benefit of MTD vs BML is maintenance of the CM port. Since all other supported SGS devices use MTD, for the Epic to use MTD reduces the amount of our device-specific configutation. While a BML-based CM7 port is certainly doable (an has been done), being the sole BML device means there's more configuration junk that we have to maintain.

The other major benefit of MTD would be the ability to forward port victory (Epic) support to a Nexus S-derived (S5PV210) ICS kernel, i.e., for CM9. With Samsung's proprietary storage stack, we're stuck on our present kernel version unless newer modules leak out somewhere. Given that Samsung appears uninterested in porting ICS to the SGS, those might be hard to come by, especially timely.

Personally I don't think MTD is particularly advantageous to TW kernels, aside from where basing on the EpicCM kernel makes such kernels easier to maintain.
[Epic 4G Android Development] toadlife:
Thanks for the insight. The other advantage to MTD on TW is the ability to jump between TW and CM releases without having to go through ODIN to get back to TW, so I think MTD is the way to go even for TW ROMs.

The ability to store the partition map to bml2 as you described would be a worthy project for the future. That kind of thing is over my head though, so I'll just have to pray to the dev gods for that one.
[Epic 4G Android Development] mkasick:
Yes, you're right, that's definitely an advantage.
[Epic 4G Android Development] The Root:
I think he made my brain grow again.
[Epic 4G Android Development] RandomKing:
I'm just confused by this statement. Correct me if I'm wrong, but last I checked all four of those phones in all their touchwizzy glory still come on BML. Is this just referring to us getting a CM7 port a long time after the others? :D
[Epic 4G Android Development] haas599:
Does switching to a MTD ROM effect the restoration of Titanium Backups made from a BML ROM?
[Epic 4G Android Development] zanderman112:
No.

Sent from my CM7 Touchpad.
[Epic 4G Android Development] codest3r:
Zanderman is right on. Though I'd recommend just using Titanium Backup to restore apps/data. And TB can restore from a nandroid too. To provide more detail on your original question (stolen from Toadlife thread):

To restore data from a nandroid backup (a BML backup restoring to MTD FS):

Mount your sdcard as USB storage

Navigate to clockworkmod/backups/<LAST BACKUP>/

Rename system.rfs.tar to system.yaffs2.tar, data.rfs.tar to data.yaffs2.tar

Open the nandroid.md5 file, replace all rfs in the filenames with yaffs2

Save/close, unmount USB, and reboot into recovery

Advanced restore data (And system if you really want). Only advanced restore, or else it will give you a BML kernel and recovery, which wont boot

Reboot!

(If you were on ext4, just replace rfs with ext4)

If you want your previous Rom, advanced restore system too.

From: [ROM] [EI22] [CWM5] [MTD] CleanGB-MTD Post #3
[Epic 4G Android Development] AproSamurai:
No my good sir, you are correct. :D We're a step closer now than we've been for a long time especially being that we're official. MTD is just a nice addon I'd suppose haha.
[Epic 4G Android Development] Ragnar:
Actually it was Tortel1210 thread http://xdaforums.com/showthread.php?p=20486368

I did this process to convert a non mtd rom into mtd, it works perfect. Except that besides renaming the data and system file in the nandroid file you need to rename the cache file also.
[Epic 4G Android Development] christophocles:
If I install the new CM 7.2 RC0 MTD on my phone, then is there any way I can still dual-boot a TW rom from my SD card? I want to use CM as my daily driver because I have absolutely no use for 4G where I live, but when I go to the city on the weekends I need to be able to use 4G. I have been using kexec to boot from the SD card to do this in the past, but I am still using a BML rom/kernel. Will this still be possible after I move on to MTD, or will I need to odin back to a TW rom every weekend? :eek:
[Epic 4G Android Development] Ragnar:
Kexec doesn't exist on the mtd kernel so you can't dual boot. You can load a tw mtd rom and create a nandroid and just restore that when you need to. I am currently doing that. I love cm7 but where I work the only way to get any data is on 4g so I revert to my favorite tw type rom.
[Epic 4G Android Development] christophocles:
Thanks, I guess that will work.. Seems a bit more time-consuming but perhaps easier to maintain. I wonder why there's no kexec support in the MTD kernel. I hadn't seen that mentioned anywhere...
[Epic 4G Android Development] AproSamurai:
It's because it naturally doesn't exist, as far as I'm aware. However I for personal preference put the same kexec patches into my own boot.img for the sake of science and taking one for the team(Not the CM Team) or just curiosity lol. I triple boot looped from the Samsung Logo to the Hacksung Logo and to the Hacksung logo immersed in voodoo color. And prevented myself from booting recovery(onenand brick), that may not have been kexec in itself though so I don't blame it more on me, but kexecing a boot.img is rather different than a zImage. So as a precautionary tale, don't be an Apro.:D
[Epic 4G Android Development] mkasick:
Yes, although I don't think anyone's actually tested it. You'll need a modified initramfs for the TW kernel to mount partitions from the SD card, but otherwise any kernel should work.

I prepared an EI22 almost-stock kernel (as in, bug-compatible with stock) for Warren and he posted some notes on his blog (see "Multiboot Android for Debugging/Testing" section) for getting a TW ROM on SD. The patch archive has a README with some better instructions, although I threw the whole thing together rather quickly.

The tricky part is you'll have to flash back to EI22 first and make tar of /system (a .tar system dump, not an Odin-flashable .tar). Otherwise it works essentially same as the CM7 SD ROM I posted for the 1022 version a while back.

The EpicCM (now CM7 victory) kernel was forked from official GB sources very early on before the kexec patches were well-tested, and they just haven't been folded back in. That can be done, folks have just been very busy getting CM7 out the door.

I'm guessing other MTD kernels are based off the CM7 one and so are missing the patches for that reason. There's nothing inherent about MTD that precludes the ability to patch in kexec support.

I was looking into this last night. You should be able to kexec a boot.img as long as you extract boot.cpio.gz first and pass it into kexec with the --initrd parameter.

However, I don't recommend using unmodified boot.img kernels as they contain stage1 in their embedded initramfs which, while shouldn't be executed, it's never removed either, and so takes up extra memory. For whichever boot.img kernel you're using, I recommend rebuilding it from source with CONFIG_INITRAMFS_SOURCE="" to force it to build with a stub embedded initramfs, then use boot.cpio.gz or recovery.cpio.gz with --initrd to avoid stage1 entirely.

If you can't rebuild the boot.img kernel from source, you can repack it with a stub initramfs, but that's almost always harder to do.
[Epic 4G Android Development] Ragnar:
^ yes what he said lol
[Epic 4G Android Development] christophocles:
Warren's blog is great reading. I obviously don't spend enough time on IRC with you guys.. :o I will mull over this and try it out when I get home. I have installed the CM7.2 nightly and it runs beautifully, thanks so much for your efforts!
[Epic 4G Android Development] zanderman112:
My hopes and dreams are now gone....

Sent from my CM7 Touchpad.
[Epic 4G Android Development] scooterman:
On all of the MTD builds I've tried, Android says I'm out of storage space when the app manager says there is still about 220MB left. What's going on here?
[Epic 4G Android Development] zanderman112:
Reboot, then you can install more apps.

Sent from my Legendary 2.2 infused device.
[Epic 4G Android Development] KennyG123:
Is it possible to swap some files from the MTD CWM5 with the update zip the ROM Manager loads to fool it into working on MTD?
[Epic 4G Android Development] AproSamurai:
I personally am unsure, that's a bit of a grey the world is flat area haha. I wouldn't know what to expect as any possible repurcussions to that, however it may work just fine, I'm not too versed to give a proper answer than, be careful.:D
[Epic 4G Android Development] KennyG123:
Does anyone know which files are different between the MTD CWM and the BML version?
[Epic 4G Android Development] celsius0010:
Ive lost 2 Epics to the stock cable.. mine are almost useless they barely charge my phone and have almost no usage.. NEVER EVER use the stock cable.. I use a nice thick Blackberry cable, the Blackberry cable is better by larger margin than a new Samsung phone compared to a new Blackberry phone.. People will probably be using ODIN more since MTD is the standard and buying a new cable will pay for itself with gas going to the Sprint store.. Luckily my exchanges were at no cost..
[Epic 4G Android Development] HeartGold95:
I was wondering about MTD. Since I'm in the old Clockworkmod and upgrade (3.1 blahhh->5 blahh) ;) and I want to flash say the newest MIUI, being MTD and all, would Clockworkmod work fine?

Sent From My Gingerbread MIUI Epic 4G
[Epic 4G Android Development] AproSamurai:
If you have CWM 5 and you want to move from BML to MTD, yes you should be completely fine.
[Epic 4G Android Development] bbelos:
I actually got this working on monday thanks to your post in the kexec thread and some extra tips from ugothakd. Compiled my tw mtd shadowkernel as an sdcard kernel with the initramfs patches, and was able to boot a tw rom off the sdcard, while having CM7 as the primary. Although, now that CM has working WiMAX, i don't know how much interest people will still have.
[Epic 4G Android Development] orlzzt:
Sorry for the redundancy; I lost a contact so reading is difficult today. Does anyone have the steps (I think Apro or someone had posted, or maybe Marcus) that you need to follow to rename the data files in Nan when you are switching from BML to MTD?
[Epic 4G Android Development] KennyG123:
They are in this thread:

http://xdaforums.com/showthread.php?t=1420584
[Epic 4G Android Development] orlzzt:
Duh, it is in Step 4! I looked right at this! Time to call the optometrist and get that contact lens right now.

Thanks as always to Mr Glass, who is a savior when it comes to Q&A...
[Epic 4G Android Development] christophocles:
Yeah.. I was going to try this but we got WiMax support before I even got a chance. Never would have imagined WiMax would come so suddenly :D
[Epic 4G Android Development] codest3r:
CWM Recovery 3.x.x.x for Epic 4G with MTD Support

Pvt. JoKeR tipped me to this here-

There is a new choice was available in ROM Manager - now there are 3 choices when you select flash recovery: a.) Epic 4G b.) Epic 4G MTD & c.) i9000

Once you select the MTD version and acknowledge MTD version is unsupported, select 3.x.x.x, reboot to recovery works again!

:D
[Epic 4G Android Development] KennyG123:
Not in mine. Just downloaded the newest version and there is no Epic MTD version.
[Epic 4G Android Development] bbelos:
There needed to be an update to the kernel before it shows up. I've released Shadow TW MTD v0.2 which allows you to see the option. Although I was able to reboot to recovery using both the Power Menu and the Reboot option in ROM Manager without installing CWM again through ROM Manager.
[Epic 4G Android Development] AproSamurai:
I'm on nightly 8, and it still pops up for me, although I don't truly use Rom Manager other than just renaming backups.
[Epic 4G Android Development] KennyG123:
You are the man B! As usual! You made me like voodoo color and now will keep me on MTD! I was almost ready to go back to BML...lol! This still won't allow renaming of backups in ROM Manager will it?

Edit: ROM Manager works perfectly again. Just did a renamed backup and restored to verify it! Thanks guys!!!!!!!
[Epic 4G Android Development] Paiyne:
Not sure if this has been asked already or not, but I'm jumping back into testing and was wondering if mobile odin will work to go back to BML or if I absolutely have to use Odin on a pc?

I'm ok either way....but since my laptop was stolen, mobile odin has kept me in the game.

Thanks for any light you can shed on this situation for me :D
[Epic 4G Android Development] sonarchist:
Essentially, once on MTD, you would only be using ODIN to go back to stock EC05 (BML), + up the chain to EI22, right?..........presumably, same as you do at the moment w/ mobile..........but the chances are, once on MTD, you really will not *want* to Odin back, though you should be able to do so same as on pc (theoretically?)

............since converting from Legendary2.2_ext4 to MTD, I have ODIN'd back to BML only once, and it was only to Update PRL.......then right back to MTD ......... will try to ODIN via mobile at my next opportunity as I am most interested in knowing with certainty + be able to give you an exact answer.

In the meantime, anyone else w/ mobile ODIN?
[Epic 4G Android Development] KennyG123:
If you needed it to go back to BML it has to have the ability to repartition the phone with the PIT file. That is why you have to Odin. So if it has that capability then you can. But as stated only if you are looking to go back to BML. There are plenty of MTD ROMs out now.