yashsingh1/xdaforums huggingface dataset:
[HD2 Android NAND Development] sunitknandi:
The thing is, I am seeing a weird log. Initially /sdcard has problems mounting but later mounts fine.

I have enabled DataOnExt but the recovery does not mount /sd-ext at start to deal with it. So fix_permissions fail. After I manually mount /sd-ext using mount menu, the thing works fine.

Here's my log with rev 6

kokotas, please see what's the issue here. :) And keep up the good work, man! :D

Sent with love from me to you via HTC HD2
[HD2 Android NAND Development] NYLimited:
Exactly. I noticed that on code load twrp attempts to blindly mount some things, seemingly before checking anything. Not certain what the idea is or if it is coming from twrp mainline code.

Note one of the odd things from the log:

I:Actual block device: ' ', current file system: 'vfat'

I doubt that any device could be mounted with a blank name - assuming the single space was supposed to be the name...

When the mounts eventually succeed twrp was testing the card (advanced section - this code was added by kokotas I believe). I am not really sure why it fails again later.

Oh well, we should type softly so we don't wake kokotas! We want him fresh and properly rested later! :D

---

I am so clever that sometimes I don't understand a single word of what I am saying... (via Tapatalk)
[HD2 Android NAND Development] dcervi:
Now it works with MAGLDR without issues (I'm using NativeSD and I have WP7 on NAND).

Thank you very much, kokotas!
[HD2 Android NAND Development] kokotas:
I didn't change anything related to "Fix Permissions". That's strange. I mean if it was working ok before it should continue to work as expected.

Glad to hear that.

You are not the only one. None of the SD versions really worked.

Firstly, the NAND version supports MAGLDR. If you can't boot with your translated twrp then you probably forgot something.

Secondly, the SD version of rev6 posted a couple of posts back is another attempt to handle magldr and haret issues. See :

When TWRP loads it reads the recovery.fstab and creates the partition table.

The postrecoveryboot.sh (a script that corrects the filesystems in the recovery.fstab) is executed after that.

Then TWRP re-reads the corrected fstab and creates the right partition table. [Only in HD2's Extended version]

That's why after running boot script everything is fine:

Finished running boot script.
I:Processing '/boot'
I:Processing '/cache'
I:Processing '/data'
I:Processing '/misc'
I:Processing '/recovery'
I:Processing '/sdcard'
I:Created '/and-sec' folder.
I:Processing '/system'
I:Processing '/sd-ext'
I:Processing '/sdext2'
Updating partition details...
I:Unmounting main partitions...

The mainline code doesn't have a second parsing of fstab for various reasons. The HD2 is a special case. Just consider that TWRP has to adjust according to the installed bootloader.

By enabling the DataOnExt option in TWRP you don't actually mount anything at start.

You just tell the recovery to use the ext partition for /data mount point when ever you (a)backup and (b)fix permissions.

It fails probably because it can't automatically mount the mmcblk0p2 to /data.

Hmm, that's why it was so quite last night..:p

Anyways..

Will take a look at the logs and see what's wrong. Thank you all for the help:)

Regards!
[HD2 Android NAND Development] o82239:
Hi,

here are the results of my first quick try with WM6.5 and haret.exe (NativeSD) with the last version "HD2_Extended_SD_TWRP_2.3.3.0.rev6(extract_to_sdcard).zip" ( http://xdaforums.com/showthread.php?p=36198017 ):

1) it boots and all relevant partitions are mounted

2) I started the tytung-NativeSD-Backup-tool and all installed Android-Versions are visible...

looks great... other haret-user could now test all other funktions...

Thank you very mutch. Best greetings

Klaus
[HD2 Android NAND Development] Nixda99:
Best recovery ever

What a wonderful recovery, NativeSD/DataOnExt support, file manager, fix permissions for NativeSD and DataOnExt, selectable backup/restore partitions, CWM backup restore ... Will replace my old CWM beta 8 touch with this one after increase my recovery size.

Thanks you very much kokotas!!!:D

Had tested also backup with DataOnExt, in v5 you had to mount sd-ext before backup (he show size (0) after data in backup).

In v6 it was not nessesary, it works without manual mounting.

Fix permissions works in v6 like in v5 without problems. Checked the log, no issues found.

Found 1 little bug. If you had a path in DataOnExt saved and open the DataOnExt path selection, change nothing and go back with the back arrow, he will reset the path to "sd-ext". You had to return always with the return button on the keyboard!!!

Had tested the sd versions via Magldr-Boot from SD via NativeSD subfolder, v5 and v6 started without problems, but I didn't have WP7 in NAND!
[HD2 Android NAND Development] NYLimited:
Sorry, I didn't have a chance to try Fix Permissions since the early days (perhaps rev 2) when it did not work. If it worked between 2 and 6 and I have no idea. :crying:

Well, I heard people claim that your city was rather noisy, especially around the center of the city (around the Gate Hotel) so I figured you could use a good night's sleep if you were anywhere nearby. :laugh:

​

What did you do differently? When I ran it it simply said "failed" within about half a second...
[HD2 Android NAND Development] pathologo:
I can confirm that Fix Permissions still works in rev6.

Your log says:

Can't check permissions
after Factory Reset.
Please boot rom and try
again after you reboot into
recovery.

although I don't know why, because as I see it you only ran a SD Card check before that.
[HD2 Android NAND Development] NYLimited:
Precisely. I have not done a factory reset in probably close to a year. No idea what the app looks at to determine "factory reset" - this has to be an assumption based on something failing or not being found. I did run the original test immediately after installing TWRP and rebooting (to get the new version running). I tried it again today, just to be certain - same result. Still thinks I did a factory reset.

There are some other oddities in the log which I do not understand.

Even though I am set for "DataOnExt" and under the DataOnExt tab it is showing me that "/sd-ext" is valid, if I go to the backup tab the display indicates

0 MB for /data and also 0 MB for sd-ext

. Refresh does not change this.

Just before "Check permissions" fails the log is reporting:

umount: can't umount /data: Invalid argument
mount: mounting /sd-ext on /data failed: Invalid argument

No idea what the invalid argument is but my sd-ext does exist - really! ;)
[HD2 Android NAND Development] jwchips:
Thanks for the kind words - hope I didn't annoy you too much! And thank you for giving me something to test :)

Anyway, in rev6 the DataOnExt box is there! Ran the sd check anyway, you can grab the log here.

Edit: tried fixing permissions to see what would happen and I get the 'cannot fix after factory reset' error too. I haven't wiped anything. Will upload log shortly.

Edit2: fix permission log here

Edit3: just realised that selecting DataOnExt has an impact on fix permissions. It should be noted I tried to run it without the DataOnExt box checked (I think). Tried with DataOnExt enabled - same error.
[HD2 Android NAND Development] Dark_Avenger:
When you are in recovery, the original /data partition is empty (DataOnExt is used). Probably this is an indication for TWRP that you did a factory reset before. Maybe kokotas could confirm this assumption from the code.
[HD2 Android NAND Development] jwchips:
By /data do you mean the userdata partition you set up in clk/magldr? I'm using DalvikOnNAND so there will be some files in userdata but I get the same error. Sorry if I've misunderstood you!
[HD2 Android NAND Development] Dark_Avenger:
Yes... that's exactly what I mean... but if your /userdata (/data) partition is not empty, maybe my assumption is not true.

Just looked at the code - fixPermissions.cpp:

int fixPermissions::fixPerms(bool enable_debug, bool remove_data_for_missing_apps) {
packageFile = "/data/system/packages.xml";
debug = enable_debug;
remove_data = remove_data_for_missing_apps;
multi_user = TWFunc::Path_Exists("/data/user");

if (!(TWFunc::Path_Exists(packageFile))) {
ui_print("Can't check permissions\n");
ui_print("after Factory Reset.\n");
ui_print("Please boot rom and try\n");
ui_print("again after you reboot into\n");
ui_print("recovery.\n");
return -1;
}
.....
.....

TWRP is looking for "/data/system/packages.xml" :)
[HD2 Android NAND Development] NYLimited:
Good find!

---

I am so clever that sometimes I don't understand a single word of what I am saying... (via Tapatalk)
[HD2 Android NAND Development] shanman-2:
I had an issue with fix permissions with dataonext.!

It seems that the default path /sdext/data shows as valid even though the proper path should have been /sdext/NexusHD2-JellyBean/data.

Just thought it might help somebody out!

Cracking Recovery BTW.

Must update my sig, it is rev6 I am using!

Edit: Also forgot to mention that for it to work I also need to manually mount sdext partition!
[HD2 Android NAND Development] Nixda99:
DataOnExt restore format whole sd-ext partition

Found a big issue. Had done a restore of a DataOnExt backup (choosed data, not sd-ext in backup screen) and it deleted my JB NativeSD rom on sd-ext. Had a seperate folder for DataOnExt (Nexus ICS 2.8). Looks like he format sd-ext instead of delete DataOnExt folder.

Today he didn't show the size of the data partition in backup screen again, yesterday it was OK (DataOnExt). Had to mount sd-ext manually and press the refresh size button in backup screen like in v5.
[HD2 Android NAND Development] kokotas:
Hi guys,

Attached below is rev7 of Extended TWRP 2.3.3.0.

Changes:

Removed the second parsing of recovery.fstab (the one after "postrecoveryboot.sh" run). Instead the script "prerecoveryboot.sh" will run before TWRP parses the recovery.fstab file.

This should also eliminate any weird errors you've been seeing in the logs.

Attempt to resolve problems relevant to "Fix Permissions" when DataOnExt mode is ON.

Added a second step/check in UnMount() in case it fails the first time. Just like I did for the Mount() in previous rev.

Added option to skip saving "dalvik-cache" during backup (under "Backup" menu) in order to minimize the size of the backup file. After-all dalvik-cache is recreated. It will be like you've wiped it when you restore that backup.

Changed the way Backup & Restore work for DataOnExt:

If you have DataOnExt checked and you go ahead and start a backup process then

a (hidden) file called .dataonext will be created in the backup folder.

The path for your /data mount point will be stored inside that file.

Do not delete that file cause you'll have to recreate it manually!

Regardless of the settings you might have(DataOnExt checked or unchecked) if you go ahead and start a restore process of a backup with DataOnExt

(so it has the .dataonext file inside - if not it's easy to manually create it)

the option for "Setup recovery for DataOnExt" will be checked and

the path (DataOnExt button) will be reset to the path you had when the backup was done.

If you were using the entire /sd-ext for /data (original DataOnExt) when you made your backup then

when you restore that backup, the sd-ext will be formatted.

If you were using a subdir of /sd-ext for /data (modified DataOnExt like "/sdext/NexusHD2-JellyBean/data") when you made your backup then

when you restore that backup, the sd-ext will not be formatted - instead only that subdir will be deleted.

Ported Tasssadar's "clicked" effect to all buttons as feedback.

Been experimenting and added some code to be able to take a screenshot (will be saved as /sdcard/TWRPScr.png).

It is done by touching the top LOGO. To enable it, there is a relevant checkbox under "Settings".

It doesn't work as expected yet! The pics taken are crap:( but it's a wip...

Regards!
[HD2 Android NAND Development] NYLimited:
Once again, whatever you do here is fine with me - I am easy. But, you should keep in mind that many of us use Tytung's DataOnExt which creates the folder automatically even though that may be the sole use for the sd-ext partition. This is true in my case. I was originally using the method as outlined by ph03n!x and am now using Tytung's method. As far as I am concerned the partition could be formatted before restore.

I guess this could easily be accomplished via a manual format before restoring. It must get confusing with the many different ways of setting up these ROMs now..

Thanks for sticking with it!

​

Sorry to say but rev 7 does not fix the problem.

Unless I should have done something new, checking the sd card went fine, as previously.

Fixing permissions failed, as previously. :(

Can't check permissions
after Factory Reset.
Please boot rom and try
again after you reboot into
recovery.
I:Unmounting main partitions...

Incidentally, I keep seeing this in the log after checking the card.

Even though it is fixed each time, the next check seems to find it again.

I have no explanation or even a guess.

Next free cluster in FSInfo block (218125) not free
Fix? yes

Recovery

log is here

.
[HD2 Android NAND Development] Dark_Avenger:
Updated theme for rev7:

- Header with logo is now a button (for taking a screenshot)

- Added checkbox "Enable screenshot taking (LOGO touch)" in Settings

- Added checkbox "Don't save davlik-cache..." in Backup
[HD2 Android NAND Development] kokotas:
Initially I had the same opinion, but instead of doing it automatically I thought that the wiping could be done...oh wait you read my mind:

Could you try it without doing anything before?

Just reboot to recovery and fix perms.

Don't worry. fsck_msdos is just complaining about the data in the FSInfo block not being up-to-date.

The FSInfo block includes "hints about where the operating system can allocate new clusters". Those "hints" are not required and it is OK to be out-dated or even missing.

Regards!
[HD2 Android NAND Development] NYLimited:
The log is shorter but the end-result is the same:

http://db.tt/K4MWCMhT

---

I am so clever that sometimes I don't understand a single word of what I am saying... (via Tapatalk)
[HD2 Android NAND Development] kokotas:
Just did it 4 times on my device. It worked.

Long shot but are you sure that the path is correct?

Remember..in case you're using a modified DataOnExt you have to set the path manually. And as someone noticed in a previous post a path may be valid(=existing) but not the correct one.

In any case I will make a version with more detailed logging for you to try in order to find the solution.

[EDIT]Attached new file[/EDIT]

[EDIT2]Removed attachment.[/EDIT]

Regards!
[HD2 Android NAND Development] shanman-2:
"Fix permissions" with DataOnExt working fine here on rev7!
[HD2 Android NAND Development] sunitknandi:
Everything working fine on rev7.

Fix permissions successful. No crazy logs either.

I noticed a big battery drain. Or maybe my /sd-ext is too large and takes a long time to back up.

Here's the log: http://ompldr.org/vZ3k5Ng/recovery.log

My battery fell from 53% to 18%.

Sent with love from me to you via HTC HD2
[HD2 Android NAND Development] Dark_Avenger:
kokotas,

I would suggest to place the screenshots in "/sdcard/TWRP/screenshots" folder.

Best
[HD2 Android NAND Development] NYLimited:
Yes, when the system is running I am sure. Using ES File explorer I can definitely find "/data/system/packages.xml".

My install is, as best as I can determine, a pretty clean Tytung install. Looking at the ext4 partition through AromaFM I have:

/sd-ext/NexusHD2-JellyBean/data/system/packages.xml

I will try the new version later today (it is 4 AM here currently).

[ EDIT ] Okay, here you go. I didn't see anything noteworthy but I am tired.

http://db.tt/bagGRKjX

Have fun, I am off to sleep a bit.. G'night!

​

A couple of other small things so you don't get bored. :D Please remember that these are purely cosmetic notes and I have no vested interest in them but you may want things perfect..

I suspect these little changes, if implemented, should be merged in the main code of TWRP as they probably impact all versions.

When using DataOnExt and going to Backups the partitions and their size is displayed. The line showing "data" (which is actually on my ext4 partition) showed 0 bytes. I tapped refresh but it continued to show 0 bytes. I went on to do something else and after a little while I noticed that the size was now populated correctly. It seems that the check for files on a larger partition takes a while before the result is displayed. Perhaps a small moving circle or some indication of "working.." should be added here.​

Similarly when I toggle the backing up of Dalvik the amount of data to be backed up is not changed up or down as Daliv is removed from or added to the backup. Again, purely cosmetic.​
[HD2 Android NAND Development] Dark_Avenger:
Maybe this is the problem:

I: DataOnExt detected. Path : /sd-ext

Did you set the path manually to "/sd-ext/NexusHD2-JellyBean/data" in the TWRP DataOnExt page?

PS. If you are using the skin, make sure that it is updated to the rev7 version (DataOnExt path is not reset when leaving DataOnExt page).
[HD2 Android NAND Development] g.andy:
I have try restore sdext4 with ver7, and all ok

Inviato dal mio multiboot HD2
[HD2 Android NAND Development] NYLimited:
All I did was install 7b, reboot into recovery, run fix permissions and save the log. I didn't set path, I have no skins currently - basically I did nothing. The ROM is working flawlessly as us twrp mostly. Except for this. :mad:

---

Sometimes I read because I am too lazy to think... (via Tapatalk)
[HD2 Android NAND Development] Dark_Avenger:
Because you are using a modified DataOnExt (NexusHD2-JellyBean) you have to set the path manually as kokotas mentioned. Set the path and try again.
[HD2 Android NAND Development] NYLimited:
Sorry, I am not sure what you mean by "modified DataOnExt". I only installed the ROM as far as I can tell and have made no changes to anything. The ROM name is assigned during the installation as:

##
# Initializing Rom Information
##
ini_set("rom_name", "NexusHD2-JellyBean");
ini_set("rom_version", "v1.3");
ini_set("rom_author", "tytung");
ini_set("rom_device", "HTC HD2");
ini_set("rom_date", "Dec. 8, 2012");

Tytung's SD-EXT_mounter.sh script during the installation simply does:

#!/sbin/sh
#
# NativeSD SD-EXT Mounter v1.0
# Author: tytung - xda-developers

if [ ! -d "/NativeSD" ]; then
mkdir "/NativeSD"
fi

if [ -f "/tmp/mount.prop" ]; then
rm "/tmp/mount.prop"
fi

# Mount the SD-EXT partition
mounted=`mount | grep '/dev/block/mmcblk0p2 on /NativeSD'`
if [ "$mounted" = "" ]; then
#mount -t ext4 /dev/block/mmcblk0p2 /NativeSD
mount -t auto /dev/block/mmcblk0p2 /NativeSD
fi

# `echo $?` will return 0 if mount is executed successfully
if [ `echo $?` = 0 ]; then
echo 'status=mounted' > "/tmp/mount.prop"
#else
# redirect mount failed messages to aroma-config
fi

exit 0

and lastly he mounts the sd-ext partition as:

#!/sbin/sh

# Get $ROM_NAME from ROM_NAME.prop
ROM_NAME=`cat '/tmp/aroma/ROM_NAME.prop' | sed 's/ROM_NAME=//g'`

# Mount the SD-EXT partition
# Done by SD-EXT_mounter.sh
STATUS=`cat '/tmp/mount.prop' | sed 's/status=//g'`
if [ "$STATUS" = "mounted" ]; then

mkdir -p /NativeSD/$ROM_NAME/data
mount --bind /NativeSD/$ROM_NAME/data /data

echo "DataOnEXT=true" >> /tmp/nfo.prop

else

# no SD-EXT partition found
echo "DataOnEXT=false" >> /tmp/nfo.prop

fi

Tytung also modifies the build.prop file to indicate DataOnExt being used:

Modify /system/build.prop and append " [DataOnEXT]" to the end of the line of ro.build.display.id

output=`grep -i "^ro\.build\.display\.id.*\[DataOnEXT\]$" /system/build.prop`
if [ -z "$output" ]; then
sed -i '/ro\.build\.display\.id/s/$/ \[DataOnEXT\]/' /system/build.prop
fi

which, in theory, should make things much simpler to identify.

Anyway, I must admit that I am confused by what here is not standard or what has been modified and just what the manual entry should reflect - sorry... :confused:
[HD2 Android NAND Development] Dark_Avenger:
To my knowledge, the "original" implementation of DataToExt is to mount /sd-ext partition directly as /data. In NexusHD roms, tytung modified it by introducing a Rom name as a segment in the path:

/sd-ext/NexusHD-JellyBean/data

That's why you have to specify this path in TWRP DataToExt page under Settings.
[HD2 Android NAND Development] NYLimited:
Yes, you are correct. That was the method used with the manual mods that ph03n!x came up with. And, yes, Tytung did change - further qualified - that method. Both are fairly well documented as the scripts indicate.

OK, I will experiment with changing the path and see where that leads..

[ EDIT ] Thinking more about this I am still confused everyone. Sorry to be slow today but.. The question was:

The answer was, no, I did not, BUT backup worked fine and prior to backup the "/data" partition was correctly identified and the correct bytes reported. So,

twrp found and correctly identified the location of the data folders

for backup without a manual entry.

How/why is Fix Permissions different?

It should also be noted that as far as I know Tytung's DataOnExt naming convention is used by a number of ROMs now, such as PA-HD2 and others, hence it is hardly unique. It seems that this has become the "standard" now. Am I wrong here?
[HD2 Android NAND Development] kokotas:
That was my intention too. I just wanted some time to try and fix the quality of the screenshots and I would change the storing path eventually. But I can change that before.

99% that's the cause!

Ok, now I'd say that the wrong path is 99.9% the cause.

Instead of /sd-ext/NexusHD2-JellyBean/data the path is left to be /sd-ext (original DataOnExt method).

Enter the correct path, press [ENTER] on keyboard and try fixing perms again..

[EDIT]

Of-course it did, because the location of the data folders was set to "/sd-ext". And since it wasn't an empty dir the backup worked. However this way the entire sd-ext is backed up. Not only your /sd-ext/NexusHD2-JellyBean/data which is the goal.

In order to fix the permissions for DataOnExt, I made TWRP to bind the path the user enters to /data mount point.

Since you left the path unchanged then TWRP searches for "/data/system/packages.xml" but it can't find it, because your xml is at "/data/NexusHD2-JellyBean/data/system/packages.xml".

[/EDIT]

Regards!

PS: rev8 is on the way...
[HD2 Android NAND Development] Dark_Avenger:
When you choose to fix permission TWRP first is checking for existence of "system/packages.xml" in your /data partition. With the original path ( /sd-ext ) TWRP is looking for "/sd-ext/system/packages.xml" and because it is not there it shows you an error. If you change the path in DataToExt page to "/sd-ext/NexusHD2-JellyBean/data", TWRP find the packages.xml under "/sd-ext/NexusHD2-JellyBean/data/system/packages.xml" and fix permission succeed. For making a backup there is no difference which one will backup: "/sd-ext" or "/sd-ext/NexusHD2-JellyBean/data".
[HD2 Android NAND Development] NYLimited:
Agreed - that worked as you expected. The only slightly odd thing was that the display showed "Done fixing permissions" while it was still doing it but that is rather minor. For all the installations which were not modified by the user (like mine) this may turn out to be a confusing issue. "I didn't think I had to specify the path since I never modified it" will be a common user response.

Question: Is it possible to run a script in twrp? I suspect that the use of DataOnExt could be parsed from ro.build.display.id and the actual path name could also be ascertained there..
[HD2 Android NAND Development] Dark_Avenger:
Kokotas,

maybe it will be better to verify the DataOnExt path entered by the user by appending the "/system/packages.xml" and checking for existence.
[HD2 Android NAND Development] NYLimited:
Got it, thanks!

VBulleting supports a "/edit" ? :D