yashsingh1/xdaforums huggingface dataset:
[Nexus S General] steven676:
Modifying an existing ROM to use LVM

(This is a condensed version of the full guide: https://raw.github.com/steven676/android-lvm-mod/master/HOWTO-MOD)

You will first want to grab a copy of the binaries and config files by cloning my git repository: https://github.com/steven676/android-lvm-mod.

Unpack the ramdisks from your boot and recovery images. Make the following changes:

Copy the contents of lvm-bin/ in the git repository to lvm/ in the ramdisks.

Copy devices/crespo/lvm.conf to lvm/etc/lvm.conf in the ramdisks.

For the boot image, integrate the changes in devices/crespo/boot/init.herring.rc-changes.diff into your init.herring.rc.

For the recovery image, integrate the changes in devices/crespo/recovery/init.rc-changes.diff into your init.rc.

For the recovery image, copy devices/crespo/recovery/recovery.fstab to etc/recovery.fstab.

Repack the ramdisks and create new boot and recovery images with them (the original kernels are fine to use).

If creating a flashable zip to install your ROM, you will also want to modify the installer script to set up the LVM volume group and logical volumes (see the previous post).

If you're building your own kernel, consider enabling CONFIG_DM_SNAPSHOT in your kernel's build configuration, to allow taking snapshots of logical volumes.

Remember that LVM is released under the terms of the GPL v2 (same as the Linux kernel), so you are required to comply with the GPL's source distribution requirements with respect to LVM.

More extensive modifications (such as changing the devices used to form the volume group) are possible; see the full HOWTO guide for details.
[Nexus S General] steven676:
Building LVM from source for Android

The LVM source was written for standard glibc-based GNU/Linux systems, and doesn't build agains Android libc. We could invest time and effort into porting it to Android, but it's easier to just statically link the LVM binary against glibc (we already have to use a statically linked binary, since the boot image doesn't ship shared libraries).

To build LVM with glibc, you will need either an actual GNU/Linux ARM system with a development environment (gcc, make, and glibc header files) set up, or a GNU/Linux ARM cross-compiler toolchain (more difficult to set up). The binaries in lvm-bin/ were compiled with an Emdebian squeeze/armel toolchain running on a Debian squeeze/amd64 machine; this procedure has also been tested with a Nexus S running a Debian squeeze chroot (though any recent Linux distribution and any device should do). The Linaro prebuilt GCC toolchain and the CodeSourcery CodeBench Lite toolchain for ARM GNU/Linux should work; the Android NDK toolchain will not.

Getting the source

Clone the lvm-mod git repository. Then, in the newly cloned repository, do

git submodule init
git submodule update

You will have a copy of the LVM source (as used to build the binaries in lvm-bin/) in lvm-src/.

(Alternately, clone https://github.com/steven676/lvm2 and checkout branch android-lvm-mod. You can also use upstream's git repository or source tarballs; see their website for details.)

Building the source

(This is a condensed version of the full guide: https://raw.github.com/steven676/android-lvm-mod/master/HOWTO-BUILD)

These instructions are written for a native build (building in a Debian/Ubuntu/Fedora chroot on an actual ARM device).

Configure the LVM source:

$ ./configure --prefix=/lvm --enable-static_link --disable-readline \
--disable-selinux --with-pool=none --with-cluster=none \
--with-confdir=/lvm/etc --with-default-run-dir=/data/lvm/run \
--with-default-system-dir=/lvm/etc \
--with-default-locking-dir=/data/lvm/lock \
--with-optimisation="-Os -march=armv5te -mtune=cortex-a8 -mthumb"

For a cross build, you need to set up the system to run ARM binaries using QEMU (see the full HOWTO for details). (For extra credit, fix the build so that it works with the configuration produced by running configure without qemu-arm enabled.) Then configure the source with:

$ export CC=arm-linux-gnueabi-gcc	# replace with your cross compiler
$ ./configure --host=arm-linux-gnueabi \
--prefix=/lvm --enable-static_link --disable-readline \
--disable-selinux --with-pool=none --with-cluster=none \
--with-confdir=/lvm/etc --with-default-run-dir=/data/lvm/run \
--with-default-system-dir=/lvm/etc \
--with-default-locking-dir=/data/lvm/lock \
--with-optimisation="-Os -march=armv5te -mtune=cortex-a8 -mthumb"

Build the source:

$ make

Grab the pieces you need: the statically linked, unstripped lvm binary is located in tools/lvm.static, and a sample configuration file is in doc/example.conf.
[Nexus S General] steven676:
(reserved)
[Nexus S General] solarispika:
Hello steven676,

This post is helpful to me!

But I have some questions on LVM as I am new to it.

1. Is it possible to use LVM only on one pv, say /sdcard?

2. If 1. yes, will lv's on a pv be linear mapped, that is, lv's in a pv are placed one after one?

3. now that removing lvm is possible, is it also possible to backup lvm setting, and switching between non-lvm and lvm environment?

Thanks :D
[Nexus S General] steven676:
Sure. Just modify the arguments to pvcreate and vgcreate accordingly:

# /lvm/sbin/lvm pvcreate /dev/block/mmcblk0p3
# /lvm/sbin/lvm vgcreate lvpool /dev/block/mmcblk0p3

Probably, but in general it is not safe to assume this. Imagine the following sequence of commands:

# /lvm/sbin/lvm lvcreate -L 1G -n vola lvpool
# /lvm/sbin/lvm lvcreate -L 1G -n volb lvpool
# /lvm/sbin/lvm lvresize -L +1G /dev/lvpool/vola

What's likely to happen here is that vola is going to be split into two segments on disk, like this:

[ 1GB vola ][ 1 GB volb ][ 1 GB vola ][ remaining free space ]

There are ways in which you can give LVM hints as to what on-disk layout you want, and the LVM tools will generate information about the current layout in /data/lvm/backup if that directory's available (and mounted read-write) at the time, but in general, it's best to treat the on-disk layout of a logical volume as unknown. In practice, unless you have physical volumes with widely varying characteristics in the same volume group (say, an SSD and some spinning-platter hard disks), you shouldn't need to worry about the actual layout.

If /data is mounted read-write, running

# /lvm/sbin/lvm vgcfgbackup

will create a human-readable backup of the current LVM volume group metadata in /data/lvm/backup (if that directory is available and mounted read-write).

I'm not sure what you mean by "switching" between non-LVM and LVM environments. If you mean going between an unmodified stock boot image and an LVM boot image, the answer is yes, but you won't have access to any of the data in your LVM logical volumes while running an image not configured for LVM, and you need to be careful not to accidentally format the LVM physical volume.

If you're asking about having two or more ROMs on the phone, one installed to the normal /data and /system eMMC partitions, and another on LVM logical volumes, the answer is also yes, but both ROMs will need modified ramdisks. The one for the eMMC-installed ROM should only mount /sdcard from LVM, while the others should be set up to mount /data and /system from LVM as well.
[Nexus S General] wizardslovak:
Good write up;-)

wonder if lvm swap could be connected as extra memory .
[Nexus S General] steven676:
Sure, of course. From my phone:

$ cat /proc/swaps
Filename	Type	Size	Used	Priority
/dev/mapper/lvpool-swap partition	65532	28196	-1

It works just fine, even if it's not a setup I'd use on a mission-critical server.
[Nexus S General] mandaman2k:
Hey Guys,

Thanks for this awesome guide.

I just have a quick question, what are the advantages of using LVM instead of the normal partitioning?

I just want to know cause i might added to my rom.

Thanks
[Nexus S General] steven676:
Primarily flexibility in deciding the number and layout of partitions -- see the first post.
[Nexus S General] wizardslovak:
Sounds good . Ill try it once home ;-)
[Nexus S General] steven676:
I just pushed a couple of changes:

Updated prebuilt binaries and LVM source to 2.02.96.

Fixed the instructions for cross compiling LVM.

The boot and recovery images attached to the second post have been updated accordingly.
[Nexus S General] eviloar:
Is this the only way to make the device function like HC/ICS/JB devices that ship with combined storage?

Are the images from Google partitioned with internal/external storage or are the latest images partitioned with a layout similar to the Galaxy Nexus?
[Nexus S General] steven676:
If you want a setup that's exactly as on a 3.0+ device with "SD card" data stored in /data, then yes, this is the only way I know of. If you just want MTP functionality, then no.

Official Google software builds use separate userdata and media partitions and continue to offer the 2.x-style USB mass storage access. I'm not sure the partition layout is adjustable in software -- the eMMC presents an EFI partition layout which no one seems to have figured out how to change.
[Nexus S General] eviloar:
Cool, thanks. Yea, I'm not too interested in the MTP functionality, I just don't want to deal with internal/external storage after using the Galaxy Nexus.
[Nexus S General] xxmurdocxx:
Swap partiton

Hi..

how i can to create a swap partition of 1G and added to system??

I had create a 1G partition "swap" but system no activated this partition.
[Nexus S General] steven676:
You'd need to format the swap partition:

# mkswap /dev/lvpool/swap

Also, you need to make sure that your kernel has swap support (CONFIG_SWAP=y -- the default kernel does not have this) and that your init scripts are enabling the swap partition (swapon /dev/lvpool/swap).

One other note: you might find lag to be somewhat severe with a 1 GB swap partition -- I'd suggest smaller.
[Nexus S General] eviloar:
I'm finally getting around to messing with my NS. From what I understand I can't just modify an existing rom(stock JB in this case) to use the larger partitions without compiling an entire new rom, right?

Which would mean I'd be at the mercy of whoever created the ROM to update when an OTA rolls around?
[Nexus S General] steven676:
No, you can modify an existing ROM yourself. It's most definitely not the simplest process around, though -- see the documentation at the top of the thread for instructions. As I say there ... read thoroughly and understand before attempting.
[Nexus S General] :
I skimmed through it, but I'll give it another look. What about when an OTA rolls around? Will that be an issue or do OTAs not look at the partition schemes?

Thanks for the response.
[Nexus S General] steven676:
The updater script uses a reference to the MMC block device to find /system, so you cannot flash an unmodified OTA. Changing the script is really easy (one-line change at the top of the script), but the result has to be flashed from custom recovery, since the modified OTA won't pass signature verification.
[Nexus S General] DerTeufel1980:
Hi,

Reading this thread I got very excited, until I saw, that mtd partitions are not really supported.

Do you know of a "simple" way for resizing/creating additional mtd partitions? I could compile a kernel and recovery if needed. I just don't know where to start.

If it is too off topic here, i am sorry.

edit: nevermind...i am too stupid :p the partition i want to change is mmcblk :)
[Nexus S General] DerTeufel1980:
Ok, now I have read it all several times.

Only one question left.

By flashing the modified rom onto an unmodified one, the lvm partitions need to be created.

Does this mean, if there is no lvm, the script has to flash the new kernel, including the modified ramdisks, then reboot itself to recovery. Then the zip has to be flashed again, and this time, it creates the new partitions and then installs the rom. Is this correct?

Edit: would it be better, to place the needed lvm files within the rom zip, and create the partitions using these files, and after this, reboot to recovery (the modified one) to finish install? If I got your instructions correct, a reboot is needed after formating. So, the script had to do this:

Check for lvm. If not present:

Flash kernel with modified ramdisks.

Create logical partitions, using the files from rom zip.

Reboot recovery

Finish install of the rom

Gesendet von meinem GT-I9000 mit Tapatalk 2
[Nexus S General] steven676:
Close, but not quite. Assuming your recovery uses a kernel from Froyo or later, the kernel-level support for basic LVM is already there (the needed device-mapper feature is also used for installing apps to the SD card). You actually don't need to reboot to activate the LVM partitions or anything -- they should be usable as soon as you've run lvcreate. If you ship the LVM tools in your zip file, then, you can just skip the "reboot" step in your list and go straight to installing your ROM.

EDIT: If you're wondering about the multiple reboots I have in the instructions above, they're there out of an abundance of caution. If you're flashing from a zip, you should not need them.

That said, you must tell your users to flash your LVM-aware recovery *before* installing your zip, or have your zip *immediately* reboot the system after installing itself. If an LVM-unaware recovery attempts to mount block devices underlying an LVM volume group, it could damage the LVM setup.
[Nexus S General] DerTeufel1980:
Great. This should be possible :)

One question left : you were writing about swap at the nand (at least this is how I got you) . Doesn't this cause the chip to get weared out very fast?

edit: wasn't the last ;)

i have to mount the devices before init.device.rc is executed. it seems to be working, but i get this output:

is it something i have to worry about?
[Nexus S General] steven676:
Depends on the quality of the underlying MMC controller. In theory, wear leveling should mitigate this problem; in practice, the wear leveling algorithm implemented by the controller may not be sufficient to prevent part of the flash from wearing out prematurely in this use case.

That's strange (that would have to be before ueventd starts up, which is really, really, really early in the init process), but probably harmless at that stage. I bet the program is looking for entropy to use in ASLR.
[Nexus S General] DerTeufel1980:
ah ok. it is early early in the init process ;)

idk how the initramfs of your device is working. for our sgs1, the init file is executed at first, and checks for example, if there is a build.prop at /system. if there isn't it boots to recovery. so, this gets executed before init binary and before any of the *.rc files.
[Nexus S General] blade1bambach:
Neat

Awesome, thank you.
[Nexus S General] adwinp:
I have lvm actually setup with cryptsetup (with my key file on a secret partition which gets mounted on early-init.

You can get a binary here http://ltrconsulting.com/howto/android/encrypteduserprofile/cryptsetup

and a source is available here:

http://nemesis2.qx.net/mobiledl.php?filename=cryptsetup_source.tar.bz2
[Nexus S General] hp420:
I just need a little clarification. It seems to me that after doing this I would need to edit my rom to allow it to be installed, is that correct? I can't just do this one time then leave it and install roms the same way I've always done in the past?
[Nexus S General] DerTeufel1980:
AW: [HOWTO] "Partitioning" your Nexus S using LVM

You got it.

The updater script is calling the partitions to mount by its path, specified in crespo device tree, in the fstab. You would always need to modify the updater script (or the device tree, if you want to publish the rom), and you need to include the lvm files.

By using this great guide here, I started to implement lvm at my hellybean i9000 rom. And now cm10 and almost all roms are using it.

Gesendet von meinem GT-I9000 mit Tapatalk 2
[Nexus S General] steven676:
Thanks for this -- I honestly had no idea this was getting any use! I thought I was just doing a cute little hack that wasn't going to attract much interest ...
[Nexus S General] DerTeufel1980:
AW: [HOWTO] "Partitioning" your Nexus S using LVM

We have to thank you. This is one of the most useful guides I have seen at xda. It has revolutionized the way our devices are working. There always has been a lack of space or speed because of stupid Samsung partition layout. Codeworkx solution was good, but yours is much better

Gesendet von meinem GT-I9000 mit Tapatalk 2
[Nexus S General] steven676:
I pushed a couple of changes to the repository:

Update LVM to 2.02.98. The prebuilt binary is now built with a different set of compiler flags ("-Os -march=armv5te -mtune=cortex-a8 -mthumb" instead of just "-O2"), which shaves ~300 KB off the resulting binary.

Update docs -- most notably, there are now instructions on how to retrieve the eraseblock size an eMMC/SD card reports to the system.

Added sample LVM configuration for maguro (GSM Galaxy Nexus).
[Nexus S General] hp420:
ok, I understand...so is it safe to assume that any cm-based rom should also be safe to use then since cm has implemented it at the source level?
[Nexus S General] steven676:
I think he means that GT-i9000 (Galaxy S) ROMs use LVM, not that all CM-based ROMs for any device use it.
[Nexus S General] nazklenknani:
Merge External Storage to Internal Storage

using this, can i merge external storage on nexus s to internal storage like nexus 4? if it can, would you please help me with the instructions?
[Nexus S General] steven676:
You could, but it'll require more than just LVM -- you also need to modify the ROM to provide the emulated external storage. I can't provide exact instructions, though if you have specific questions I may be able to help.
[Nexus S General] nazklenknani:
thanks for your reply. those low level modifications are beyond my knowledge :D i just wait someone post some instruction to do it then.. really bad that nexus s separate its storage to internal and external. its a bit confusing.. way better to have an integrated storage like nexus 4 does. thanks!
[Nexus S General] hp420:
this is yet one more reason I can't wait to get one :)
[Nexus S General] tatemz:
How to maximize storage flexibility?

Love the post and all the help you've been giving people.

I'm on a Sprint Galaxy S4 variant with a slot for an external_sd running PAC-Man and other CM based ROMs. I have a 64gb SD card that I've been mounting in place of my sdcard using

mount -o bind /external_sd/ /sdcard/

in a userinit.sh script. That has worked in giving me 64gb of Spotify offline music and other various data. Also it has helped me have enough space for dual booting Roms etc...

The problem is that I am looking to use my 64gb external sd AND 8gb internal SD as if they were a single 72gb storage. Im scared to play around with your tutorial for fear of bricking, but it seems that I may not have to do much since I'm running CM10.2-based ROMs.

Any direction/thoughts?
[Nexus S General] steven676:
That's certainly doable, but it's non-trivial. Unless I'm wildly off base, the Galaxy S4 (along with most other new Android phones) no longer has a separate media storage partition -- the internal media storage now lives on the /data partition (with some sleight of hand involved). If you want to join your microSD card with the internal storage, then, you'd need to create a new logical volume that holds all of /data, not just /sdcard. (You could also split the internal storage into the old-style separate /data and /sdcard layout, but that would require more extensive ROM modifications.)

As I understand it, LVM isn't part of CM builds other than those for the original Galaxy S family, so you don't get help from that front.
[Nexus S General] DerTeufel1980:
There's almost nothing more to say, besides, if you ever would remove your external sd, your data on /data would get corrupted

Gesendet von meinem GT-N7105 mit Tapatalk 2
[Nexus S General] newwind3:
Problem during power on/off test

Hello steven676

I'm developing to change two internal memorys into one internal memory as using your lvm

and trying to test power on/off after merged your lvm

but sometimes I have two type problems when device are going to boot

1. Frist problem

Sometimes, it takes a long time after "fs_mgr: Running /system/bin/e2fsck on /dev/lvpool/userdata" as below kernel log

-normal state kernel log-

[ 4.600000] fs_mgr: Running /system/bin/e2fsck on /dev/lvpool/userdata

[ 4.870000] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: nomblk_io_submit,discard,errors=panic

-wrong state kenel log

[ 4.600000] fs_mgr: Running /system/bin/e2fsck on /dev/lvpool/userdata

==>take a long time 1~15 minutes

[ 74.590000] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: nomblk_io_submit,discard,errors=panic

2. Second problem

Although I add "wait /dev/block/mmcblk1" in init.xxx.rc, but sometimes lvm can't mount /dev/lvpool/userdata, so can't boot completyl

Would you advice How to fix it?

-init.xxx.rc

wait /dev/block/mmcblk0 <== first interanl emmc

wait /dev/block/mmcblk1 <== second interanl emmc

start lvm_vgscan

start lvm_vgchange
[Nexus S General] mathuaerknedam:
I'm still pretty new to Android (and smartphones), though I have a fair bit of experience with Linux. My Nexus S 4g is running an unofficial Omnirom 4.4.2, and I'm curious if the repartitioning my phone could complicate future ROM updates by pushing me too far outside of the norm. Anyone know?

Thanks!
[Nexus S General] steven676:
Do not expect updates unless you plan on modifying the updates yourself to adapt to the repartitioning scheme.
[Nexus S General] mathuaerknedam:
I won't be doing that anytime soon. Maybe eventually, but by then I may have a phone where it isn't necessary. :)

Thanks!
[Nexus S General] alienmind:
Hi

I'm trying to follow your tutorial on a Oppo Find7a + TWRP

This kind of works, but I'm kind of stuck in a point where, regardless of the changes I am doing to my recovery fstab.qcom, the recovery does not update /etc/fstab and it still reflects the real partitions as the /data /system /sdcard devices

A more detailed description can be found here.

Do you have any idea? I understand this is for a different device and a different recovery (clockworkmod) but may be you could add some light...

Sorry if the question sounds stupid this is my first time modding the recovery

EDIT: Solved! etc/twrp.fstab is the fstab file for twrp
[Nexus S General] demosf:
CWM 6

I'm trying to make this work on cwm 6 for the nexus s, so I can than modify and flash kitkat, I add the lvm folder and make the modifications of the files, mkbootimg ....

when I flash the new recovery to the nexus and try to boot to recovery after the google logo I get a black screen, fastboot and adb don't react

I noticed that the size of CWM is 6.1 MB the original CWM is 5.5MB, can this be a problem or i'm doing something wrong?
[Nexus S General] steven676:
The lvm binary is statically linked and includes all of glibc -- the size inflation is expected.

The recovery's changed quite a bit since CWM 5, so the instructions might need a little updating. That said, the recovery is extremely resilient -- if it's not even getting to the main recovery menu, something is probably wrong with your reassembled recovery image. (Easy way to test whether you're doing that right: unpack a recovery image, reassemble it without making any changes, see if the result boots.)
[Nexus S General] demosf:
I tried what you suggested and the the phone booted in to recovery with no problem, with fastboot and flash, so I can say that my steps for creating the recovery.img is OK

I will continue to compare the init.rc from the CWM 5 and CWM 6, because if i'm not mistaking the adb service is started at the end of init.rc so because it doesn't start i guess the problem is somewhere in init.rc