yashsingh1/xdaforums huggingface dataset:
[HD2 Android NAND Development] MarkAtHome:
Well, Android/Linux has a FAT32 filesize limit of about 4G.

I have over 1000 installed (mostly unused) apps, so one or more of the NAND Backup's images will tend to exceed that filesize limit.

This is the reason I looked into TWRP when v2.2 came out. This version introduced the capability to split those large image files during NAND Backup (I have since discovered that the splitting is done automatically, as needed - 2.2.2 fails at its attempt), and handles them upon restore. Brilliant!

So, that is why I am anxious to get the port properly working.

____________________
[HD2 Android NAND Development] NYLimited:
TWRP folders

Just in case you were curious...

Apparently, that folder is created as an attempt to identify the device (though I am not sure what purpose that might serve). It is not HD2 specific although you are right that it has been there all along. The function being called is "get_device_id":

extern "C"
{
#include "common.h"
#include "data.h"
#include "ddftw.h"

int get_battery_level(void);
void get_device_id(void);

extern char device_id[15];

void gui_notifyVarChange(const char *name, const char* value);
}

and looking at the code shows how the directory is created:

get_device_id();
str = "/sdcard/TWRP/backups/";
str = "/sdcard/TWRP/BACKUPS/";
str += device_id;

Seems that the HD2 returns nulls or zeros on that function call.
[HD2 Android NAND Development] Dark_Avenger:
Xylograph, arif-ali,

could you port the latest 2.3.2.0 version to our lovely HD2?

Thanks in advance
[HD2 Android NAND Development] MarkAtHome:
I believe that Arif-Ali now has an HD2 to work with. He has already indicated that he plans to get back to this project, so when he does, you can be certain that we will get whatever is current.

____________________

Sent from my HD2 JB-CM10 with XDA Premium
[HD2 Android NAND Development] arif-ali:
Unfortunately, my 3rd hd2 is now also faulty. Now after spending monies for the last year on 3 different faulty hd2's I have given up. I have now spent around £300 for nothing. So unfortunately I will not be able to continue. Whoever wants to continue this project, I will be happy to get the mods to change ownership
[HD2 Android NAND Development] kokotas:
Hey,

I have compiled the latest TWRP successfully but I didn't keep it. I have been working on it, in order to add some extra features. Will post soon in this thread the first result.

Actually from what I saw the max size is set even lower, to 1.5G. I'll have to get a second sdcard around 8G to be able to test this feature.

You got it right! TWRP tries to get the device id first from the /proc/cmdline using the serial number ("androidboot.serialno=") and if that fails, it uses the /proc/cpuinfo. It looks there first for "Serial" and then for "Hardware".

In HD2's case the "Serial" is 0s and that's why we get that directory inside BACKUPS. By just removing the "Serial" stuff from the code it works ok and we get htcleo as device id. By removing all the device id stuff, we don't get any subdir under BACKUPS(which I personally prefer since I won't be using the same sdcard into different devices).

Third in a row!? That's really unfortunate.:( I was hoping you were back in the game..

Regards!
[HD2 Android NAND Development] Burag:
​

:laugh:
[HD2 Android NAND Development] kokotas:
Ok, how did you know I'm allergic to cats ? :p

Well...it worked:silly: I found an image in my Trash that should be the TWRP 2.3.2.3 untouched (meaning it is the first build I did, before I changed the code).

I just flashed it to see if there are any things I added and it seems clean. Wanna test it?
[HD2 Android NAND Development] NYLimited:
I tend to agree, for what little that may be worth. :)

Whatever you do don't look at my avatar!! Oh no, now you did it!

Test? Test is a four letter word.. Does it write the boot partition on restore? :eek:

---

Lately it occurs to me what a long, strange trip it's been... (via Tapatalk)
[HD2 Android NAND Development] cajunflavoredbob:
If you want one of my 16GB microSDs, let me know. I've got a few, so you can have it. Also have an 8GB one if you need it to specifically be 8.
[HD2 Android NAND Development] Dark_Avenger:
Thank you, kokotas! Your build works fine for me - recovery boots ok, did two backups, wipe cache, dalvik cache without problem. Keep it your great work. Looking forward your next builds of TWRP and cLK Black.

Thanks again
[HD2 Android NAND Development] kokotas:
Since I found the cause of the creation of the 000000000000 folder and we could now have a proper name for it, I first thought about including it in the version I'm working on, but then I found it to be rather redundant.

Let me rephrase that: Please do not use the provided recovery without backing up your backups;)

As for the boot partition, I guess you're referring to how recovery will handle the boot partition if MAGLDR is the bootloader, right?

I'm not using MAGLDR, but from what I see in the code

IF your recovery.fstab is set properly (boot yaffs2 /boot) then

1.Backup_Method == FILES

2.DataManager::SetValue("tw_boot_is_mountable", 1);

Thank you very much for the offer. I already ordered a new 8G card (only 6,6 euro).

Thank you for trying it, but I shouldn't be credited for the work of others. This is Dees_Troy's work. I only compiled it for our (king)Leo.

For the next version I'm preparing you may blame me;)

Regards!
[HD2 Android NAND Development] MarkAtHome:
That is sad to hear, arif-ali. I appreciate all you have done and tried to do. Thank you. Stay well.

____________________

Sent from my HD2 JB-CM10 with XDA Premium

---------- Post added at 06:56 AM ---------- Previous post was at 06:46 AM ----------

I am, too, but found their healing abilities override their downside, so I took allergy shots and pills for over 20 years! No regrets. :)

Does this mean you have accepted the torch? ;)

____________________

Sent from my HD2 JB-CM10 with XDA Premium
[HD2 Android NAND Development] NYLimited:
Thanks - I have learned a long time ago about Murphy who said: “Everything that can go wrong will, and at the most inopportune moment.”

Then came O’Toole’s Corollary: “Murphy was an optimist.”

Seems, however, that they both ignored a key segment of the empirical data. Not only will things go wrong. That implies that uncontrollable forces will act against me, which is true.

Unfortunately, controllable forces will also act against me. I will, in fact, screw up! Count on it.

So, yes, I always backup my backups. :D

The last version that Xylograph kindly made available worked fine till one tried to restore but it had issues putting the boot files back. It did work fine with cLK I understand. Having quietly followed some your stuff I am just shocked you are not using Magldr! :eek:

Okay, I'll play and see what it does.

Hey, Murphy, here we go again!!
[HD2 Android NAND Development] MarkAtHome:
Xylo's version's image split module crashed and burned... :(
[HD2 Android NAND Development] NYLimited:
I don't have enough data to test the image splitting function - that is your baby, sorry. I only know about the boot partition.

I just did a backup with the 2.3.2.3 posted here and am in the process of restoring. Will have to put CWR back before booting (remember Murphy?) on the chance that I may not boot and we shall see.. As I said - I always backup my backup's backup! :D

Nope - no go! Same problem that Xylograph had. Using v2.3.2.3 posted here,

after restoring the HD2 has no valid boot image

in the boot partition. This same process works fine with CWM Recovery. I did not have enough time today to examine what, if anything, it wrote to the boot partition but rebooting dies on error and stays in magldr.

If it'll help, Xylograph's explanation of the problem is

here

.

Incidentally, the install option works fine - I installed CWM Recovery using TWRP before rebooting and had no problem getting into CWM to restore my device.

Will have to restore CWM backup to boot... :'(
[HD2 Android NAND Development] kokotas:
That was expected. No!Don't blame Murphy. Blame me, for not thinking of it earlier. Since the recovery img I posted here is just the default TWRP built for HD2 then it would have the same old recovery.fstab where /boot is mtd , while you want it to be yaffs2. I think Xylo got it right in the link you provided. You could see for yourself if you dump the img, unpack initrd.gz , edit recovery.fstab, repack initrd.gz, recompile the img and flash it back. I think it will work ok then.

I wouldn't say that. I ended up building TWRP recovery from source without realizing it. And I always had a couple ideas for some extra recovery features, so I thought that this could be a chance to see if my ideas were possible.

I will just try to implement them* and try to fix any existing bugs* and I'm done.

* Extra features :

Detection of bootloader

Nilfs2 support for sdcard's ext partition

Option for converting file system of sdcard's ext partition

[ext2 - ext3 - ext4 - nilfs2]

In-built NativeSD manager

[RomBackup - RomRestore - RomDelete - FixPerms - WipeData - WipeDalvik - (cLK)KernelRestore]

Option for adding a 2nd ext partition(mmcblk0p3)

(cLK)Direct rebooting to selected boot partition

* Bug fixes :

Archive splitting

(MAGLDR)Boot restoring

what else...

Regards!
[HD2 Android NAND Development] Dark_Avenger:
I'm really interesting in feature #3 - convert sd-ext partition file system. Thank you for your great work.
[HD2 Android NAND Development] NYLimited:
Since I really don't know the app as well as I perhaps should the only thing I can think of is recognition of DataOnExt setup although this is only wording to me. From my point of view it matters not if the backup calls it "sd ext" or "data" as long as it is backed up and restored but some might be really confused by the terminology. :)

Either way, your efforts are appreciated!
[HD2 Android NAND Development] Dark_Avenger:
Thank you kokotas. Could you provide the source of the default skin (ui.xml) in order to update mine with the new UI elements (for the new features)?
[HD2 Android NAND Development] NYLimited:
The story of my life...

Guess we must have met before - you know me pretty well! :D

I think I will wait for the "XXX" version...

One question - which may be premature since I have not yet installed it..

With the relatively new DataOnExt option CWM Recovery can have difficulties with the fix permissions module since what it considers to be the data partition is actually empty. With the changes you made can the fix permissions code now also be aware of the data partition's location and work accordingly?
[HD2 Android NAND Development] MarkAtHome:
I like what you have done so far, prior to this release. Nice going.

Can the file system conversion take place with no data loss? Also, since Tytung already supports NTFS in his ROM (that is what I use), how about also adding that along with nilfs2? Would certainly be great to easily access the SD-Ext partition from a Windows-based PC...

As for file splitting, would it be possible to create the NAND backup first on the SD-Ext partition, and then, if any of the the backup files exceeds Android's interpretation of how large a file can be in a FAT32 partition, only then split the backup into parts, just below Android's FAT32 filesize limit (a way to avoid so many small backup files). Then move the backup to the SDCard.
[HD2 Android NAND Development] kokotas:
Updated the previous post with some info. Check it out.

Attached the file above. You can also take it from the recovery itself with file manager : nav to /res => select ui.xml => Copy File => nav to /sdcard => Select => Done;)

That would require great-grand-parental advisory :p

I knew I forgot to take something into account. Added to my todo list:good:

Except for ext2/3 to ext4 conversion, I think not.

Didn't know about it. Will search and see what's possible.

I have to say that your idea sounds interesting, but let's see first if backup/restore with archive splitting is working in its current state.

EDIT:

Waiting feedback mostly regarding the bug fixes.

EDIT2:

Attached DAF related files.

Regards!
[HD2 Android NAND Development] NYLimited:
Did a factory wipe and a full restore. All is back to normal!

Question - what exactly does this mean:

I:Single storage only.
I:No internal storage defined.
I:No storage defined, defaulting to /sdcard.

---

Lately it occurs to me what a long, strange trip it's been... (via Tapatalk)
[HD2 Android NAND Development] NYLimited:
When restoring sd-ext the partition is formatted (my choice, I know) and a rather large value is specified first tune2fs. I always change it to a lower # but have to remember to do so. If I wasn't looking at the log I wouldn't have remembered.

Any chance of saving the # of mounts before a check is forced in the twrp config and applying it if/when the partition is formatted instead of the default?

Certainly not vital but I figured I would ask. :o

---

Lately it occurs to me what a long, strange trip it's been... (via Tapatalk)
[HD2 Android NAND Development] famewolf:
My recovery partition needs to be increased as it's currently 5mb...short of a full backup of nand, repartitioning then reloading is there a simpler alternative to increase the size so the new recovery will fit?

This is going to be an issue for anyone using the hd2 toolkits I assume since they appear to default to 5mb.
[HD2 Android NAND Development] NYLimited:
Don't know. Whatever you use would have to move all your other partitions and content. When I did it a ways back I did a backup, partition, restore.

Yes, a lot of people have 5 MB (required for newer CWM versions) and 3 MB for the old CWM stuff.

---

Lately it occurs to me what a long, strange trip it's been... (via Tapatalk)
[HD2 Android NAND Development] kokotas:
Yes I know... it reminds me of shoot first, ask questions later:rolleyes:

When I make a flashable zip package I'll include a script for recovery's partition size. Thanks for the suggestion.

For this version, that is the expected behavior. Whenever you press the "DataOnExt" button the data-path is reset to default "/data".

But if you set it to "/sd-ext" (and it is a valid path)

that path will be used

during backup.

It is working (well from what I tested so far). I just haven't made any modifications to the sizes shown, so don't expect to see any change there. I believe it is possible to make it show the new /data size after setting the path. We'll see in the next version...

While considering a way to add something useful for ph03n!x's DataOnExt I kept thinking "How does DataOnExt affects the recovery?".

And I came up only with the backup process. (Forgot about the permissions till you mentioned it). That's why I added that button under "Backup" and I didn't make the path stick. I mean if you reboot recovery you have to set again that path.

So now I'm thinking of moving the option about "DataOnExt" to "Settings" and make the entered path stick.

Just to save time. The operations take less time by setting it off.

That was one of my main concerns! Glad we nailed it :victory: and big thanks to Xylograph for the initial idea.

I totally agree with your point here and I think it is quite simple to change that in the upcoming version.

I think someone (it wasn't me:angel:) forgot to change the completion msg of the backup process and it is the same as the restore one. I mean when you restore a backup it is reasonable to show the "Reboot" msg, but when do a backup it is likely that you want to do something else next(like installing something).

(If you check the post where I attached the file) I changed the max size to 512MB so that most of us can test the archive splitting feature.

This was my second concern! So archive splitting works:victory:

Since it is working I will make it again 1.5GB in the next version.

TWRP has support for devices with internal (emmc) or external (sdcard) or dual storage. So when it checks our system it finds just our external sdcard. Normal debug msg.

Off topic: I really like your quotes:good:

I just can't understand what you're referring to.:o

Unfortunately there is no alternative cause by increasing the recovery, all the boundaries of the rest partitions change.

Regards!
[HD2 Android NAND Development] arif-ali:
kokotas, you are now OP of this thread

good luck with your work on TWRP
[HD2 Android NAND Development] NYLimited:
Got it. Will have to experiment more with it. I didn't actually try it (from my point of view it matters not what we call the partition as long as it is backed up but I understand it would be confusing to others).

I don't know how many actually use it but he now has the DalvikOnNAND option as well (in case TWRP wants to wipe dalvik) ...

I like to be safe. Call me chicken. :)

I assume you will be looking at the setup switch to decide if the file should be split or not, yes? I noted in the log that you correctly assesed the size of the data. I don't think there is any point breaking up the file unless 1) the switch is set and 2) the file is too big to store.

Tytung, like ph03n!x, has added the running of e2fsck on boot to check the integrity of the ext4 partition. e2fsck checks the partition on every Nth boot - the N being set by tune2fs. When TWRP formats the sd-ext partition it also sets this value - the TWRP log shows:

Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 34 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
Done.

As I said, this is no big deal since this can be easily changed in terminal or adb by entering a different value, such as:

tune2fs -c 10 /dev/block/mmcblk0p1

which happens to be my personal preference instead of the above shown 34 and 180 days. Again, just an idea, not a big deal at all. :)
[HD2 Android NAND Development] Dark_Avenger:
Hi kokotas,

I've tested you new EXT FS-type feature, but I got a strange result. First I have my sd-ext in ext3 format. I did a backup of it with TWRP (sd-ext.ext3.win). Than I changed the FS-type to ext4 through the TWRP and restored the backup. At the end my sd-ext is still in ext3 format. Is it possible backup restore to revert the FS-type back to ext3 as backup is made on ext3? Is it possible to implement ext2/ext3 <--> ext4 convertion without data lost (need for backup/restore)?
[HD2 Android NAND Development] famewolf:
Recommended Recovery size

What is the new "suggested" size for the recovery partition to allow for current versions and future versions... 10mb....15mb?

and while I'm asking...what size are you guys setting your /system to? if I have to do a full backup, repartition and restore I'd like to do it ONCE and not have to do it again anytime soon.

Went through this when CWM went from 3mb to 5mb so this time I'd like to plan growth ahead of time.
[HD2 Android NAND Development] zexmunze:
Size of IMG is 5.85~ you can try at least 6mb,mine is 8mb.
[HD2 Android NAND Development] NYLimited:
Keep in mind that if using DataOnExt you have a lot of NAND space to play with which otherwise would be totally wasted.

---

No one can make you feel inferior without your consent... (via Tapatalk)
[HD2 Android NAND Development] MarkAtHome:
I would agree with you. I also believe that anyone involved in a development thread has no business riding a tricycle, with all its limitations, and should kick that toolkit to the curb. :D
[HD2 Android NAND Development] NYLimited:
A tricycle?!? :eek:
[HD2 Android NAND Development] MarkAtHome:
Pretty please make that optional. :D

---------- Post added at 08:47 PM ---------- Previous post was at 08:43 PM ----------

My bad. I meant the use of NTFS instead of FAT32 (not SD-Ext filesystems). That would also eliminate the need for any splits.
[HD2 Android NAND Development] NYLimited:
Why would you want to make recovery partition size checking optional? If it doesn't fit, it doesn't fit.. Confused.

---

No one can make you feel inferior without your consent... (via Tapatalk)
[HD2 Android NAND Development] famewolf:
I have 7 devices I actively USE from a logitech revue to phones, tablets (one is even an htpc remote)....I'll take the simplest approach to reliably get things repartitioned and recovery flashed....YOU have time to tinker. ;)
[HD2 Android NAND Development] Dark_Avenger:
I found why my sd-ext is converted back to ext3. After discussion in TWRP IRC channel I understand that when you restore a partition, TWRP format the partition based on file system specified in the backup file name. If the name is "sd-ext.ext3.win", TWRP format the sd-ext in ext3 before restoring the data. An easy conversion from ext3 to ext4 without data lost is simply to rename the "sd-ext.ext3.win" to "sd-ext.ext4.win" (same for MD5 file and the content of MD5 file) and restore it.
[HD2 Android NAND Development] NYLimited:
I haven't tried this yet but how about the format / rm switch? In theory, you could stop twrp from formatting that way, no?

---

No one can make you feel inferior without your consent... (via Tapatalk)
[HD2 Android NAND Development] NYLimited:
The latest version from the Play Store recognizes the TWRP folder and backup files. If I select a backup file to extract from TB dies immediately.

I have not tried this before so I cannot compare to previous behavior. No idea if the problem is with TB or the TWRP files created by this version.

---

No one can make you feel inferior without your consent... (via Tapatalk)
[HD2 Android NAND Development] Dark_Avenger:
I think it's because of missing intermediate folder with zeros. Just for a test you could move your backups in another folder "0000000000000000" under "TWRT/Backups" and try again.
[HD2 Android NAND Development] MarkAtHome:
I have no problem with the check, but I might have incorrectly interpreted it to mean check and, if needed, fix. I will go back and re-read later when I am more awake. :)

____________________

Sent from my HD2 JB-CM10 with XDA Premium
[HD2 Android NAND Development] NYLimited:
I don't think "fix". I am thinking check partition size and warn before aborting install (which would have died on "error writing block.." messages anyway.

​

Interesting idea.. I never thought of it since TB showed all the files in the folder and flagged them as "TWRP" files. I figured if the missing folder was a problem it wuold not even find/show the files. Odd.. will check it later.
[HD2 Android NAND Development] NYLimited:
Has anyone gotten Titanium Backup to extract files from a Nandroid backup of either CWM or TWRP?

As I noted earlier, Titanium Backup recognized the TWRP v2.3.2.3 backups and correctly marked them as "TWRP" on the display. Clicking on them to open one terminates Titanium Backup unexpectedly.

I went back to two different TWRP versions (thanks MarkAtHome) and created backups with each one. Titanium Backup recognized the folders ("TWRP/Backup/000000000000/") but could not find any files in them. Of course, the files were there as evidenced by ES File Explorer.

Lastly, I pointed Titanium Backup to a CWM backup which it also recognized. It managed to open the file successfully but saw nothing inside it.

Not sure where all this leaves us. My personal preference is not to have the extra folder with the zeros in it and, Titanium Backup seems to find the files just fine although it cannot open them successfully. I have never tried to recover individual files from a Nandroid backup so it isn't all that important to me.

Still, I must wonder if this is an issue with the latest Titanium Backup. I did not restore older Titanium Backup versions for more tests. :'(
[HD2 Android NAND Development] MarkAtHome:
There's no issue about making things easiest for oneself. The toolkit provides limited choices (e.g. partition sizes), requiring hand-on afterwards, if your needs don't fall into one of its offerings.