yashsingh1/xdaforums huggingface dataset:
[Android Software/Hacking General [Developers Only]] domicq:
http://pastebin.com/npWMTUYz

After first reboot issues with maps, optimalizing 7(or 8) apps, force cloae of com.Google.Process.gapps

Second and every other still same issue with maps, force close and sometimes gms core too, optalizing of 13 apps everytime I reboot

Gesendet von meinem Nexus 5X mit Tapatalk
[Android Software/Hacking General [Developers Only]] slimcyril:
[MENTION=4544860]osm0sis[/MENTION] I know this is out of scope for this thread but I also know you're a great scripter. I finally found Cron script that works 200%

http://xdaforums.com/showthread.php?t=2700146
[Android Software/Hacking General [Developers Only]] osm0sis:
/data/app/com.google.android.gm-1 /system/app/PrebuiltGmail /system/priv-app/PrebuiltGmsCore	(58082481 > )
/data/app/com.google.android.gms-1

Aha! Here's one issue.. Looks like it's crossing over GMail and GMS (Google Play services) because of the similar package names. I'll see what I can figure out. I think I can fix that. :good:
[Android Software/Hacking General [Developers Only]] domicq:
And another force close of maps and photos?

It's problem just with permissions?

Gesendet von meinem Nexus 5X mit Tapatalk
[Android Software/Hacking General [Developers Only]] osm0sis:
No, like I said it was integrating GMail and GMS wrong.

Change:

sysapk=`grep "updated-package name=\"$package" $xml | grep -o 'codePath=.*$' | cut -d\" -f2`;

To:

sysapk=`grep "updated-package name=\"$package\"" $xml | grep -o 'codePath=.*$' | cut -d\" -f2`;

And try again. Let me know if it helps with any of what you were seeing and please post another log. Thanks in advance [MENTION=5040989]domicq[/MENTION]. :)
[Android Software/Hacking General [Developers Only]] domicq:
oh :D it messed up my phone again, systemUI force closes, restoring of system backup doesn't work

## GApps Auto-Integration Script Log

Sun Feb 7 08:32:58 CET 2016
/data/app/com.android.vending.billing.InAppBillingService.LACK-1
/data/app/com.google.android.apps.authenticator2-1
/data/app/com.google.android.apps.books-1
/data/app/com.google.android.apps.docs.editors.docs-1
/data/app/com.google.android.apps.docs.editors.sheets-1
/data/app/com.google.android.apps.docs.editors.slides-1
/data/app/com.google.android.apps.fitness-1
/data/app/com.google.android.apps.genie.geniewidget-1
/data/app/com.google.android.apps.inbox-1
/data/app/com.google.android.apps.maps-1 /system/app/Maps
/data/app/com.google.android.apps.plus-2
/data/app/com.google.android.apps.translate-1
/data/app/com.google.android.contacts-1 /system/priv-app/GoogleContacts
/data/app/com.google.android.contacts-2 /system/priv-app/GoogleContacts	(10307 > 10200)
/data/app/com.google.android.gms-1 /system/priv-app/PrebuiltGmsCore
/data/app/com.google.android.gms-2 /system/priv-app/PrebuiltGmsCore	(8489440 > 8186448)
/data/app/com.google.android.keep-1
/data/app/com.google.android.music-1 /system/app/Music2
/data/app/com.google.android.music-2 /system/app/Music2	(2317 > 1985)
/data/app/com.google.android.talk-1 /system/app/Hangouts
/data/app/com.google.android.talk-2 /system/app/Hangouts	(22901117 > 22582810)
/data/app/com.google.android.tts-1
/data/app/com.google.android.tts-2
/data/app/com.google.android.videos-1 /system/app/Videos
/data/app/com.google.android.videos-2 /system/app/Videos	(31105001 > 38129)
/data/app/com.google.android.webview-1 /system/app/WebViewGoogle
/data/app/com.google.android.webview-2 /system/app/WebViewGoogle	(262303450 > 246011750)
---

Sun Feb 7 08:41:13 CET 2016
/data/app/com.android.vending.billing.InAppBillingService.LACK-1
/data/app/com.google.android.apps.authenticator2-1
/data/app/com.google.android.apps.books-1
/data/app/com.google.android.apps.docs.editors.docs-1
/data/app/com.google.android.apps.docs.editors.sheets-1
/data/app/com.google.android.apps.docs.editors.slides-1
/data/app/com.google.android.apps.fitness-1
/data/app/com.google.android.apps.genie.geniewidget-1
/data/app/com.google.android.apps.inbox-1
/data/app/com.google.android.apps.maps-1 /system/app/Maps
/data/app/com.google.android.apps.plus-2
/data/app/com.google.android.apps.translate-1
/data/app/com.google.android.contacts-1
/data/app/com.google.android.contacts-2
/data/app/com.google.android.gms-1
/data/app/com.google.android.gms-2
/data/app/com.google.android.keep-1
/data/app/com.google.android.music-1
/data/app/com.google.android.music-2
/data/app/com.google.android.talk-1
/data/app/com.google.android.talk-2
/data/app/com.google.android.tts-1
/data/app/com.google.android.tts-2
/data/app/com.google.android.videos-1
/data/app/com.google.android.videos-2
/data/app/com.google.android.webview-1
/data/app/com.google.android.webview-2
---
[Android Software/Hacking General [Developers Only]] osm0sis:
Judging by all the duplicates in that log I'd say you hadn't properly cleaned up /data after the last time.. definitely not a clean test. :eek:
[Android Software/Hacking General [Developers Only]] domicq:
Sure it wasn't. ? I did fresh install and I'll try it again

Gesendet von meinem Nexus 5X mit Tapatalk
[Android Software/Hacking General [Developers Only]] remedy1419:
Noticed a very strange issue with your busy box zip and titanium backup. Not sure if it's your zip or a problem with how tibu interacts with your zip.

So, after flashing your busybox zip, most of my app backups are not restorable, only the data is. I have forced system busy box in the tibu preferences. I couldn't figure out why I was unable to restore apps and data, so I ran stericsons busy box and installed, Rebooted, and then I could restore everything just fine. This was all after a clean flash of PureNexus on the 6p. To verify it was your zip I clean flashed again, installed stericsons and my restorable were apps there. Clean flash again but just flashed your zip, could only restore app data.

Hopefully this might help, if you need more info let me know!
[Android Software/Hacking General [Developers Only]] osm0sis:
Hmm 6P so arm64. Can you verify that's what my zip installed? Install location from mine? Install location for Stericson?

I can't imagine it using more than mount (maybe), tar, cp and chown/chmod and all those applets definitely work fine on my N5.
[Android Software/Hacking General [Developers Only]] Surge1223:
I don't use your busybox (No offense I just compile my own)

But, just thinking out loud here, tibu probs has to use selinux commands, like restorecon, chcon, setcon, etc. Does your busybox include selinux support?
[Android Software/Hacking General [Developers Only]] osm0sis:
Good thinking, but Stericson doesn't have those either. Titanium appears to have finally updated their included busybox to v1.19.4-Titanium judging from my N5, but they've got very few applets indeed:

[, [[, basename, blkid, blockdev, bunzip2, bzcat, bzip2, cat, chgrp,
chmod, chown, chrt, cksum, cmp, cp, dd, dirname, du, echo, fdisk, find,
findfs, flock, fsync, fuser, gunzip, gzip, id, kill, killall, killall5,
ln, losetup, ls, lzop, lzopcat, md5sum, mkdir, mkdosfs, mke2fs, mkfifo,
mkfs.ext2, mkfs.vfat, mknod, mkswap, mktemp, mount, mountpoint, mv,
nice, pidof, ps, pwd, readlink, realpath, renice, rm, rmdir, sha1sum,
sha256sum, sha512sum, sort, stat, sync, tar, tee, test, timeout, touch,
umount, unlzop, wc, which, whoami, xargs, zcat

Edit: Of those I'm missing only sync.. but Stericson doesn't have that one either. Once we rule out user error here (installing Stericson to /su/bin or /system/bin would override ROM binaries, whereas mine only installs to the safer /su/xbin and /system/xbin but TiBU could fail due to /system/bin binaries still having precedence in that case), I guess I might have to sit down and test the other 76 applets in common..

Also, I'm pretty sure Stericson doesn't even support Android x64 yet, so his working means it's using the compatibility layer and a regular arm binary, in which case TiBU's included busybox should also work just fine. There's a lot that doesn't make sense in these TiBU+system busybox error reports. :eek:

I was going to release an updated busybox zip with the latest official patches (fixes truncate.. no biggie, and I had the other 2 already), since I compiled them for today's AIK-mobile release, but I'm going to hold off until we figure this out better.
[Android Software/Hacking General [Developers Only]] remedy1419:
Both yours and stericsons install to /system/xbin arm64. The included tibu busybox works. If forcing system, meefiks and stericsonsboth work. If I flash yours, back to just data and a select few apps are restorable.

I tested it 4 times last night with clean flashes each with different busy boxes. Meefiks and stericsons were fine.

Edit: spoke too soon! A bunch of apps are missing in tibu after flashing your script. Reinstalled stericsons, and everything is back.
[Android Software/Hacking General [Developers Only]] osm0sis:
Okay. Install mine, then manually extract the arm (not 64) binary from my zip, rename it to just busybox and then put it in /system/xbin, overwriting the arm64 binary. Let me know if that works better.
[Android Software/Hacking General [Developers Only]] remedy1419:
Done, but TiBu just errors out saying it can't acquire root privileges now. Other root apps work, was I supposed to do anything with the permissions for the busybox file?
[Android Software/Hacking General [Developers Only]] osm0sis:
Yeah make them match what they were before. chmod 755. Or rwxrwxr-x if you're using a root/file explorer app. Thanks for testing.
[Android Software/Hacking General [Developers Only]] remedy1419:
Done. Now all user apps show up in TiBu but there is still a ton of system data/apps missing. Screenshots of what I mean. 8:44PM is with stericsons, 914pm is yours just a minute ago after changing permissions etc.
[Android Software/Hacking General [Developers Only]] osm0sis:
Okay, so the arm build works better than arm64... There are a few applet differences I had to make there to get it to compile, but again pretty sure none of which TiBU has in its own compile. And my arm64 busybox appears to run just fine in my limited testing in my girlfriend's unrooted N5X and testing others have done, not to mention the busybox commands AIK-mobile utilizes.

Also, even with arm it doesn't make sense why TiBU would have some apps and not others. This is truly bizarre stuff.
[Android Software/Hacking General [Developers Only]] remedy1419:
I haven't the faintest idea, if you need a logcat or anything let me know. Not a huge deal, I guess my phone is just special.
[Android Software/Hacking General [Developers Only]] osm0sis:
Not sure about Meefiks but I seem to recall hearing Stericson uses a custom Android glibc (or uclibc?) toolchain instead of bionic/NDK so he doesn't have to hack as much sh*t to cross-compile correctly for Android.

My main goal was to get the "android-busybox-ndk" project up to date again so someone could easily cross-compile busybox with the NDK toolchains again, and my primary purpose in doing so: providing the applet functionality I needed for AIK-mobile natively on all Android architectures. I've done all that, and I'm pretty happy with what I've accomplished, but if there are issues with the applets after I've added in the known patches, debugging busybox itself is a bit out of my league unfortunately; it's a monster.
[Android Software/Hacking General [Developers Only]] bgiesing:
[MENTION=4544860]osm0sis[/MENTION] How hard would it be to add Chrome Beta and Dev support to the GApps integrator script since Chrome's stupid crazy libs.

They use com.chrome.(beta|dev) package names.

I think the main issue would be what the folder name is. Deltadroid's add on uses GoogleChromeDev and GoogleChromeBeta for example (but I'm sure somebody else uses a different name.)
[Android Software/Hacking General [Developers Only]] osm0sis:
Using the fix I posted for similar package names a couple pages back it should otherwise work I'd think.
[Android Software/Hacking General [Developers Only]] bgiesing:
Tried that and it did integrate the app but it was broken because since the folder is not named "Chrome", it doesn't do the Crazy Libs stuff to prevent Chrome from breaking
[Android Software/Hacking General [Developers Only]] osm0sis:
Ah of course. Yep, we just need to switch that part to a fuzzy match using case instead of if.. Try this out:

# workaround for Chrome not playing by the usual rules (per usual)
case $(basename $sysapk) in
*Chrome*)
mount -o remount,rw /system;
rm -f $sysapk/lib/arm*/libchrome.so;;
*)

cp -fp $datapk $gtmp/preopt-$i.apk;
zip -d $gtmp/preopt-$i.apk lib/*;
zipalign -v 4 $gtmp/preopt-$i.apk $gtmp/$i.apk;
datapk=$gtmp/$i.apk;;

esac;
[Android Software/Hacking General [Developers Only]] bgiesing:
If the directory is GoogleChromeDev doesn't it need to be *Chrome* instead of Chrome*
[Android Software/Hacking General [Developers Only]] osm0sis:
Thanks, revised in my previous post.

This stuff only happens when you're using zipalign and zip though, so otherwise things should work already, unless there's something else going on (entirely possible).
[Android Software/Hacking General [Developers Only]] bgiesing:
Yeah, I'm using the binaries. The modified code worked! Thanks.
[Android Software/Hacking General [Developers Only]] osm0sis:
Judging by the logcat I just took on my trusty old Galaxy Nexus running Marshmallow and the latest gappsintegrator script it appears the reason for all the errors is it looking for libraries included inside those APKs and being unable to find them for some reason, despite it being able to find them inside the base GApps APKs (as is the new way on Marshmallow).

As a proof, I've successfully managed to work around all the issues after the fact by extracting the erroring APK libs Lollipop-style into the system APK folders, and following a reboot things start to work normally again. I'm going to try some things to see if I can make just the APKs function as expected, but either way I should be able to fix up some form of Marshmallow support for the next version. :)

Edit

: zipaligning the Play Store originated APKs didn't do the trick like I'd hoped, so I guess I'll implement the workaround as I mentioned above; I'll see if we can optionally delete the libs and zipalign like on Lollipop to save the space too now that the internal APK libs are apparently useless for our purposes.. :good:

Edit 2

: Damnit, started running into the SystemUI FC issue.. First with my zipaligning test, and now with my automated lib workaround.. Weird that it didn't happen the first time, going to see if I can figure it out..
[Android Software/Hacking General [Developers Only]] domicq:
You're great! I didn't have time so far and still haven't, but I'll try it ASAP. ☺

Gesendet von meinem Nexus 5X mit Tapatalk
[Android Software/Hacking General [Developers Only]] osm0sis:
The only theme I can see is that things appear to work (better) when the files are dexopt'd in smaller batches (1-6), but when ~10 or so get processed at once we get SystemUI FCs. These seem to stem from both file-copy errors (e.g. libwebviewchromium.so was incomplete, despite having room on /system for the rest of the file) and, probably most importantly, permissions problems:

02-21 14:53:03.726 13978 13978 D AndroidRuntime: Shutting down VM
02-21 14:53:03.728 13978 13978 E AndroidRuntime: FATAL EXCEPTION: main
02-21 14:53:03.728 13978 13978 E AndroidRuntime: Process: com.android.systemui, PID: 13978
02-21 14:53:03.728 13978 13978 E AndroidRuntime: android.view.InflateException: Binary XML file line #87: uid=10018 needs permission android.permission.READ_CONTACTS to read lock_screen_owner_info_enabled for user 0

Why this doesn't occur with a slower/smaller integration I'm not sure; perhaps too many changes to packages.xml mysteriously corrupts it. Anyway, working on more possible solutions now. :confused: :cowboy:

Edit

:

pm grant com.android.systemui android.permission.READ_CONTACTS
pm grant com.google.android.gms android.permission.ACCESS_FINE_LOCATION

and correcting the truncated libwebviewchromium.so seems to resolve all the SystemUI+GPs+GApp stuff.

When things go bad it looks like the UserIds in packages.xml get renumbered, but everything corresponding (/data/data dir, other entries, etc.) seem to also get appropriately renumbered so I don't see how that should be a problem.

Edit 2

: I think it might actually be all-at-once running out of space on /system that screws things up. If I install all but WebView and GoogleTTS things work fine. Those are what would put it over the limit in /system and I was seeing that truncation in the WebView lib. Hmm.. Looks like I may have to try and implement that free space check after all just to save myself headaches in the future..
[Android Software/Hacking General [Developers Only]] domicq:
I have 1gb free and I got that systemUI force close. 1gb is enough for integrating, isn't it?

Gesendet von meinem Nexus 5X mit Tapatalk
[Android Software/Hacking General [Developers Only]] danerrickcuba@gmail.com:
Hello, looking for some help, how can I add or enable VIRTUAL TERMINAL Support on a given kernel, I'm a note 10.1 2014 lte user on 5.1.1 with a permissive kernel, I need VT-support enabled on this kernel...any ideas how I xud approach this?

Thank you

<<<New World Order (NWO)>>>
[Android Software/Hacking General [Developers Only]] osm0sis:
No idea. If it's a kernel feature then it needs to be added to the kernel source and compiled.

Yup. The truncation I was seeing was due to free space since I wasn't accounting for command order in my size estimates, should have that all fixed now with my new free space check.

The Lollipop-style external lib workaround for Marshmallow is completely ready; the SystemUI FCs are from GoogleTTS being incompatible with integration (at least in that form), so I've added an exclusion for it. Storing the libs properly in the APKs so they can continue to function Marshmallow-style (thanks to Open GApps and [MENTION=6404745]MastahF[/MENTION] for details on the process) requires a newer arm zipalign binary, so that's what I'm going to see about hunting down or building today. :)
[Android Software/Hacking General [Developers Only]] osm0sis:
No luck on the hunt. I'll see about building it natively on my Nexus 7, but it has a number of dependencies so I'm not super excited about it. [MENTION=4939386]Surge1223[/MENTION], you wouldn't by-chance already have enough of a native build environment set up to whip a static zipalign up, would you? :)

So complete Marshmallow-style internal lib support won't work until we have a new zipalign, but it's otherwise ready to go, and the Lollipop-style external lib fallback works perfectly in the meantime, so...

gappsintegrator: v4 - fix Marshmallow support, add free space check, many fixes/improvements:

https://github.com/osm0sis/gappsintegrator/commit/a472da164f82d7fa81fccd31a3f9bc083d994890

Updated in the OP as well. :D :cowboy:

This took a fair amount of my time off this week, so if you like the progress I'm making on this project, or enjoy anything else I've done on xda, please do hit the donate link from my profile.

Last (and pretty much only) "donation" I think I've received was someone's free Starbucks coupon well over a year ago. :p
[Android Software/Hacking General [Developers Only]] Paget96:
[MENTION=4544860]osm0sis[/MENTION] what about this zipalign?

Please, check it
[Android Software/Hacking General [Developers Only]] osm0sis:
If that's the same one as in your latest APK it's a dynamically linked binary, and still the older one, lacking the -p commandline option, so my older statically linked binary is still preferable until we get a newer static binary. :good:
[Android Software/Hacking General [Developers Only]] Paget96:
[MENTION=4544860]osm0sis[/MENTION] it's not from my apk.
[Android Software/Hacking General [Developers Only]] osm0sis:
Well the attachment is broken until xda sorts out their business, so I can't check if it has dependencies, but maybe you can just tell me if it has -p?

Edit: [MENTION=5514152]Paget96[/MENTION], looks like attachments and mentions have caught up. That binary appears to be from Linux or x86? Test running it on-device shows this:

sush: <stdin>[2]: /data/local/tmp/zipalign: not executable: 32-bit ELF file

We need a static arm binary. Until then, at least I figured out a fallback in my script for Marshmallow support. :)
[Android Software/Hacking General [Developers Only]] Surge1223:
Hope arm64 is OK? If not I can recompile for armeabi.

https://www.androidfilehost.com/?fid=24421527759886844
[Android Software/Hacking General [Developers Only]] osm0sis:
Amazing! SO many thanks! Can you do static arm as well? Widest compatibility that way (usually also works on x86 and arm64 due to compatibility layers). :D
[Android Software/Hacking General [Developers Only]] Surge1223:
https://www.androidfilehost.com/?fid=24421527759887630

let me know if the ARM version doesnt work properly..have never dealt with the from E? message before. Works fine on my angler though so it should be ok.\

Also included appt and appt2 since their dependencies were similar enough to compile :)

aapt: ELF 64-bit LSB executable, version 1 (SYSV), statically linked, stripped
aapt2: ELF 64-bit LSB executable, version 1 (SYSV), statically linked, stripped
zipalign: ELF 32-bit LSB executable, ARM, version 1 (SYSV), statically linked, from 'E?', stripped
zipalign64: ELF 64-bit LSB executable, version 1 (SYSV), statically linked, stripped
[Android Software/Hacking General [Developers Only]] osm0sis:
The arm zipalign binary executes fine on my N5 and N7, but weirdly a no-go on my GN test device, throwing an "Illegal instruction" error, even with no arguments. :confused:

Here's the strace:

execve("./zipalign", ["./zipalign"], [/* 21 vars */]) = 0
set_tid_address(0x5f518) = 7922
set_tls(0x5f558, 0, 0x24, 0x5fbe0, 0x5f510) = 0
mmap2(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x400a1000
madvise(0x400a1000, 12288, MADV_MERGEABLE) = 0
mprotect(0x400a1000, 4096, PROT_NONE) = 0
sigaltstack({ss_sp=0x400a2000, ss_flags=0, ss_size=8192}, NULL) = 0
prctl(0x53564d41 /* PR_??? */, 0, 0x400a2000, 0x2000, 0x53d58) = -1 EINVAL (Invalid argument)
personality(0xffffffff /* PER_??? */) = 8388608
personality(0x800008 /* PER_??? */) = 8388608
openat(AT_FDCWD, "/dev/__properties__", O_RDONLY|O_LARGEFILE|O_NOFOLLOW|O_CLOEXEC) = 4
fstat64(4, {st_mode=S_IFREG|0444, st_size=131072, ...}) = 0
mmap2(NULL, 131072, PROT_READ, MAP_SHARED, 4, 0) = 0x400c9000
close(4) = 0
mprotect(0x5c000, 8192, PROT_READ) = 0
brk(0) = 0x93d000
--- SIGILL {si_signo=SIGILL, si_code=ILL_ILLOPC, si_addr=0x4218e} ---
+++ killed by SIGILL +++

Any ideas?
[Android Software/Hacking General [Developers Only]] Surge1223:
Yep probs will recompile in a min, I looked over zipaligns source to make sure that it doesn't do anything (except try to print usage if no args are passed) if no parameters are passed and it indeed doesn't so the error is in the way the binary is compiled and/or your gn device is using a diff cpu architecture or I compiled it with arm cpu architecture type that isn't compatible with your gn device but is with n5 and n7..hmm we could actually test if the cpu architecture theory holds by having you give the output of cat /proc/cpuinfo on all 3 of your devices :)

Either that or we'd have to gdb trace that SIGILL... Which would be painful

But I'll try recompiling in the mean time for more ABIs
[Android Software/Hacking General [Developers Only]] osm0sis:
I always went low SDK version for arm compiles to ensure greatest compatibility. I forget which, but I went through them and one of the early ones is where the toolchain actually starts looking complete, between 9 and like 14 or 15.

N5 and N7 are pretty similar hardware, so no big surprise there. But even with those two being Qualcomm and the GN being TI, all three are still ARMv7 (v71). Features differ though..

GN: swp half thumb fastmult vfp edsp thumbee neon vfpv3

N7: swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4

Thanks again for looking into this. It's funny, but just think, once you work out the kinks, this'll be the new "that zipalign binary" which will float around all the Android community forums probably for the next 5 years or more (until Google makes some new advance we have to keep up with again). :p :)