yashsingh1/xdaforums huggingface dataset:
[Android General] ShaDisNX255:
I would create a backup and delete the folders you are going to replace entirely, instead of overwriting. This ensures you eliminate the oat folders that [MENTION=4315551]Ticklefish[/MENTION] is talking about.

And btw, yes I tried 1024 MB and it seems to be deodexing ATM, I will give it a test to see if it all works fine :fingers-crossed::good:
[Android General] levone1:
I've deleted whole framework folder in recovery, and replaced it with the one in TOAD zip, but it sounds like I need to remove files/folders in the zip, (because it contains odex?).
[Android General] ShaDisNX255:
It contains odex files? That's odd. How many odex files are there? Are they the same you began with?
[Android General] levone1:
Oat folder is there in /framework, the other folders are gone. In /Oat/arm, there are the same 240 files as in original. In /Oat/arm64, there are 4 files, where there were originally 247.
[Android General] ShaDisNX255:
Maybe that's something Ticklefish can answer, I am not that familiar. I finished the deodexing process and there are no other folders in /framework at all so I will begin testing. I guess it wouldn't hurt if you test it out too :confused::fingers-crossed:
[Android General] Ticklefish:
So here's what should be happening if everything works.

TOAD will go through the folders provided and convert the ODEX files into SMALI files using baksmali.jar.

Then it will use smali.jar to turn those files into classes.dex files.

Then 7z is used to put those DEX files into the APK file.

If the above steps work then the /oat folder for that APK is wiped, including any ODEX or VDEX files in the subfolders.

For a JAR file in /framework, the ODEX AND VDEX files are instead wiped from the /oat/arm64 (or /arm if that's your processor) folder.

Once all the JAR files are processed, if they are all deodexed there'll be nothing in the /oat/whatever folder so the whole /oat folder should get wiped as well.

What I didn't mention a few posts ago (because I'm not by a computer, I forgot and I'm low on coffee) is that boot JAR files like framework.jar have their ODEX files inside /framework/whatever, not /framework/oat/whatever.

If those are all correctly deodexed, the /arm and /arm64 folders should be wiped.

So the fully deodexed framework folder should have a collection of JAR and APK files but no other folders except those relating to the APK's (SemcGenericUxpRes for Sony for example).

Thats what should happen and it works on my setup, but only on the system dumps I was able to find. I didn't try it on a Sony device, sadly. Every OEM likes to do their own thing but their roms should be fairly standard by now..

Hope all this makes sense!
[Android General] Ticklefish:
Hmm. In that case, 4 of your JAR files didn't deodex I'm guessing. Have a look in the log file.
[Android General] ShaDisNX255:
Ok, testing on a Samsung J6, Android 8.0 and the zip TOAD creates does not delete anything, in fact, it does not copy anything

Even after deleting the app, framework and priv-app folders, running the TOAD zip does not copy them on to /system

I had to manually delete app, framework and priv-app and copied the folders over.

Been booting for 20 minutes stuck in the Samsung screen, I don't think it will load at this point.

Wiped dalvik, ART and cache

Will have to consider this a loss, I will wait another 10 minutes but I don't think it'll work
[Android General] Ticklefish:
I'll have to take another look at the updater-script and see why it's not working. Maybe I should make multiple scripts - one for each combination of folders (app, framework and app, etc). A bit long-winded but simple enough and it might be more effective..

When you copy the folders over, do you set the permissions as well?
[Android General] levone1:
Well, I got my services.jar anyway. I just copied it manually and deleted odex/vdex, and working fine. I'll look into it more for testing anyway...
[Android General] levone1:
So I tested again, just deodexing the framework folder. The 4 files that are left in arm64 are odex and vdex for 2 files that there's no jar for. I don't know how or why that would be, (my guess is Sony stuff...), but everything went fine, no errors. It would seem like it should still boot fine, if that's the case, unless those odex/vdex files have something to do with a different jar with a different name...(?)
[Android General] Ticklefish:
So it looks like the general process works, which is good. The updater-script may have some issues, and there's this business about the library files, but it's still pretty promising so far. Thanks everybody for all your help!

Does the lib folder exist in the odexed APK's folder?

You have files for JAR files that don't exist? Well that's handy.. Lol
[Android General] ShaDisNX255:
Yes, it exists in the odexed apk folder.
[Android General] Ticklefish:
And in the deodexed folder?
[Android General] ShaDisNX255:
Yes, opening the deodexed .apk file with winrar I also see the lib folder there
[Android General] Ticklefish:
Damn. I admit I was hoping you'd say it wasn't. Then I'd know why it's not working..
[Android General] ShaDisNX255:
It's weird that not all apps FC.

For example, SystemUI seems fine when I delete the oat folder and replace the apk with the deodexed apk

But the keyboard ain't having none of that lol
[Android General] Ticklefish:
Right...I've just uploaded a new version of TOAD to the downloads tab. This has updated Method files, using -Xmx1024m for the lines that call the Java files.

I've also updated the code that constructs the ZIP file. It turns out the updater script was okay, I just wasn't putting the ZIP together properly in the first place.

There's still this issue of some library files not working with deodexed apps in Android 8.0 (26.bat) and there may be a problem with deodexing the framework folder in Oreo as well. But we're getting there!
[Android General] levone1:
[MENTION=4315551]Ticklefish[/MENTION] - could you take a look when you have a chance - https://mega.nz/#!9hcURQaI!teTuR7gyRRfEplLCtqh1aqNNrknrLTKw-uCCEwkaDzQ It's my stock framework folder, (build.prop is in there). Seems like all apks are good, but something's funny with the framework.
[Android General] Ticklefish:
I'm downloading it now. I'm at work (don't tell anybody!) but I'll see what I can find out. :)
[Android General] Ticklefish:
The first thing I've noticed is that your files all have a "read only" attribute, which is stopping TOAD from deleting anything. That's easy enough to change - I've added some lines like these to the batch file:

attrib -r Your_Files\%1\oat\%TOAD_ABI%\%Filename%.* >nul

You're right, there are some files left over that don't seem to relate to any JAR file. It could be that Sony just left those behind and they're not needed. It wouldn't be the first time..

I'm running the tool again now and we'll see what happens.

--a few minutes later--

Your framework\arm64 folder doesn't have any ODEX files in it, only VDEX ones. That's odd. VDEX wasn't supposed to be used until Android 8.1. Trust Sony to be different..

I'll add a section for using vdexextractor on them, hopefully that will work out.
[Android General] levone1:
Thanks - I think that's what was getting me. Based on my experience with previous versions, I typically expect framework.jar and core-oj.jar to be pretty substantial framework files, but they are left unchanged in the deodexing process. There are no corresponding odex/vdex files for them in /oat, but there are boot. files referring to them in /arm64. I don't know what that means, if anything, but it's different...
[Android General] Ticklefish:
Those are boot JAR files, or bootclasspath files, or whatever you want to call them. In 'stock' 8.0, their DEX files are all inside boot.oat.

It looks like Sony have taken something from 8.1 and created "boot-*.vdex" files instead. So you have "boot-framework.odex" inside "framework\arm64" as well as boot.oat. You also have "boot-framework.oat" and "boot-framework.art" but I think we can ignore those.

I've added a few lines to 26.bat that will use vdexextractor on these if there's no ODEX files. This, as far as I understand, creates the "classes.dex" files directly from the VDEX file and we can just put those straight into the JAR file.

It does make 26.bat longer and more complex but that's the price you pay for developers doing their own thing. :eek:
[Android General] Ticklefish:
Okay, I think I've got something working now. I've uploaded a new version to the downloads tab. If you could try it out, I'd be very grateful.

Currently you'll still have the "framework\oat\arm" folder. There will still be a few files left in "framework\oat\arm64". If that folder's empty, both folders are removed. Hopefully keeping the folder won't cause any problems.

But, if the ZIP doesn't work, you could try removing "framework\oat\arm" and seeing what happens.
[Android General] ShaDisNX255:
Saw something very interesting

Tried to decompile services.jar that TOAD deodexed but the smali folder didn't decompile correctly (smali_classes2 decompiled correctly)

Smali folder was empty

Looking through the jar files with winrar, noticed that classes file was named classes.dex and classes2 is classes2.dex.dex

The deodexed services.jar makes it so that it won't boot (without modifications, straight out of TOAD)

Looking in the oat/arm folder I see 3 services files

services.art

services.odex

services.vdex

I deleted these when I tried to boot with the deodexed services.jar

Should I keep the .art file when trying to boot the deodexed services.jar? Or even de vdex file?

CORRECTION: its actually classes2.dex.dex and classes.dex
[Android General] msubhani:
Delete services.vdex and services.odex then try.
[Android General] Ticklefish:
Is this using 26.bat? If so, can you try the latest version please?

If it's still happening, could you possibly share your framework folder?
[Android General] Ticklefish:
You need to delete the entire arm folder. If it's still there, Android will look inside it for the odexed files so it can process them when it rebuilds the cache.
[Android General] ShaDisNX255:
Ok so if I only want to test services.jar, I have to pretty much copy the entire deodexed framework folder and delete the oat and arm folders? Because if I delete the arm folder I am deleting all other odexed files

I will download the newest version and try again
[Android General] Ticklefish:
Ah, I see where you're coming from.

TOAD isn't set up for only deodexing single files. You can do it, but you'll have to manually replace the JAR yourself and delete all the services.* files from the arm folders.

Set the permissions, wipe the cache and reboot.

As long as the deodex process worked you should be good to go.
[Android General] ShaDisNX255:
Yup, that's what I've been doing but let me try with the newest version.

If I can't get services to work or even decompile correctly, I will send you the whole framework folder :)

---------- Post added at 04:28 PM ---------- Previous post was at 04:13 PM ----------

Also just FYI, I let TOAD deodex the whole framework folder. I just decide to delete and replace the services files manually ;)
[Android General] Ticklefish:
I can look at adding an option to deodex individual files but I think we'd better get this working first.. ;)
[Android General] ShaDisNX255:
Ok same situation regarding services.jar, I can't decompile it. However I got a new interesting pop up while it was deodexing

"The program can't start because cygwin1.dll is missing from your computer. Try reinstalling the program to fix this problem."

I don't know if this is a TOAD thing or a my PC thing, but I'm seeing I need to download this file

I am looking for ways to download this file, but can't promise anything because I don't have administrative rights to this PC LOL (work PC, but don't tell :P)

Once I get this file I will try again, but as for now I think (without even trying) it will end up the same because the smali folder didn't decompile correctly
[Android General] levone1:
Ok, so everything good until file 350 of 367, when this happened:

Processing file 350 of 367 - "Your_Files\framework\arm64\boot-android.hidl.base-V1.0-java.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 351 of 367 - "Your_Files\framework\arm64\boot-android.hidl.manager-V1.0-java.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 352 of 367 - "Your_Files\framework\arm64\boot-apache-xml.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 353 of 367 - "Your_Files\framework\arm64\boot-bouncycastle.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 354 of 367 - "Your_Files\framework\arm64\boot-conscrypt.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 355 of 367 - "Your_Files\framework\arm64\boot-core-libart.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 356 of 367 - "Your_Files\framework\arm64\boot-core-oj.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 357 of 367 - "Your_Files\framework\arm64\boot-ext.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 358 of 367 - "Your_Files\framework\arm64\boot-framework.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 359 of 367 - "Your_Files\framework\arm64\boot-ims-common.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 360 of 367 - "Your_Files\framework\arm64\boot-legacy-test.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 361 of 367 - "Your_Files\framework\arm64\boot-okhttp.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 362 of 367 - "Your_Files\framework\arm64\boot-org.apache.http.legacy.boot.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 363 of 367 - "Your_Files\framework\arm64\boot-tcmiface.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 364 of 367 - "Your_Files\framework\arm64\boot-telephony-common.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 365 of 367 - "Your_Files\framework\arm64\boot-telephony-ext.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 366 of 367 - "Your_Files\framework\arm64\boot-voip-common.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

-----------

Processing file 367 of 367 - "Your_Files\framework\arm64\boot-WfdCommon.vdex"

ERROR!!!! EXTRACTION FAILED!!!!

03-10-2018@1245 --- Deodexing Process Finished

And an error popped up saying something about "...unable to extract vdex because cygwin.dll was not found...", (sorry, I did take a screenshot, but then accidentally cropped the important part off. I'm running it again to get the error message back...)
[Android General] Ticklefish:
Cygwin.dll? I don't have that on my PC so that's a bit of a puzzle. It's used to help Linux applications run in Windows and...wait...wait...let me do a quick search of the C:\ drive..

I do have it on my PC! Hidden away in one of the many programs my company's installed. I've attached it in a ZIP for you. Put this into the Tool_Files folder and try it again.
[Android General] levone1:
Here's the error...
[Android General] ShaDisNX255:
Ok, added it to the tool files and trying again (I am assuming that I do not need to edit anything regarding the TOAD exe itself or anything)

Trying again!
[Android General] levone1:
Same error, different file: cyggcc_s-1.dll
[Android General] Ticklefish:
I'll have a look and see if I have a copy of that on my PC at home. In the meantime, it looks like you can get all those DLL files from here: http://cygwin.com/install.html
[Android General] Ticklefish:
Here's all the cygwin files I have. You might not need them all but just in case they're all in this file.
[Android General] ShaDisNX255:
EDIT: Nevermind, I added them to the tools folder and it did its thing

I'm a complete noob, I added these files in C:\cygwin\bin but I am still getting the error because cygwin1.dll is missing from my computer D;

Is there anything else I'm missing? Do these need to be installed some other way?

---------- Post added at 02:43 PM ---------- Previous post was at 01:57 PM ----------

Ok no problems on deodexing, but again, I can't decompile the services.jar correctly, still getting the regular smali folder empty and the smali_classes2 folder is just fine

Here is a link to download the framework folder and build.prop if you want to take a look directly :)

https://drive.google.com/open?id=1a3IzcBLsgc5mjt38YQ14dTHVPzsby8Kg

I don't know if you want me to try to run the deodexed services.jar again to see if it runs, but I am already suspecting that it will be the same result :/
[Android General] levone1:
That did it. Deodex completes normally now, and everything looks good. Zip still doesn't flash, but I'm having a hunch that it might be a TWRP issue. I'm going to try with a different version. I tried flashing the new framework folder with a different script, (from one of my old zips that I've used many times), and the result was the same - no errors, and "ok" message, but nothing happens. Even the stuff in the script that says 'ui print' doesn't print. Anyway, I manually copied folder in recovery, and everything is good. BTW - I deleted the extra folders in /framework, and no problems, but a new arm64 folder was created, that is empty. Thanks
[Android General] Ticklefish:
Thanks for your files. TOAD was creating a classes2.dex.dex file, you're right. I've fixed that, it was just a small typo, but your service.jar file still doesn't deodex properly.

Baksmali.jar isn't able to process the ODEX file fully. So I'm adding a few lines of code that will use vdexextractor instead if baksmali fails. So far this seems to work and the new, deodexed JAR decompiles correctly.

It does make the batch file longer and more complex though, which isn't quite what I had in mind when I started this. I could remove the lines relating to baksmali and smali completely, so we only use vdexextractor but I'm concerned that this could cause issues.

Although the VDEX files apparently contain exactly the same information as the ODEX files, so we could.. And it would certainly make the whole process a lot faster. Hmm..

So it all works and everything boots, after you're wiped the cache? That is good news!

I'm a bit puzzled about this arm64 folder being created but, as long as everything works, I'm happy!
[Android General] ShaDisNX255:
Well if you wantto test TOAD going full vdexextractor, I'm your man ;)

I don't mind going through a couple of bootloops. But that's your decision, because I know you want to keep it as accessible as you can to everyone xD
[Android General] Ticklefish:
I think it's probably going to be the best option. If vdexextractor can cope with 8.0 files then we might as well use it. It's faster and it will stop the batch file from getting far too messy.

I'm on the case right now and should have it done soon.
[Android General] Ticklefish:
Do you mind trying "TOAD - v1.0.1.1.exe" for me? I've rewritten 26.bat to use vdexextractor only. It's simpler and it's a lot quicker..but I do need to test it still works!
[Android General] ShaDisNX255:
Ok it looks like it deodexed correctly, will test out fully (all folders including app and priv-app) and see how it goes

Wish me luck lol
[Android General] Ticklefish:
Good luck! :highfive:

It's quite a bit faster, isn't it?
[Android General] ShaDisNX255:
Yeah it is faster :D you rock!

Well

Good news and bad news

These are the tests I did:

1.- Copied all folders (app, framework and priv-app. Generated ZIP does nothing still lol). Deleted current folders and manually copied the over. Does not boot

2.- Copied app and priv-app. Still same problem that keyboard FCs due to libs

3.- Replaced services.jar and deleted .odex and .vdex files (did not delete .art file). Boots normally without problems (app and priv-app are back to odexed at this point)

So services.jar at least works now :)
[Android General] Ticklefish:
Does the ZIP not do anything at all?

What happens if you delete the boot.art file?
[Android General] ShaDisNX255:
Well, the ui_print works fine on the ZIP :P

Do you want me to delete the boot.art file when trying out the deodexed framework folder?

Because it gets deleted along with everything else anyway.

When I try the deodexed framework, I delete the whole folder and copy the new framework folder over.

I guess I could keep the arm folder (where all the boot oat, vdexes and art files are) and just delete the oat/arm (where all the odex and vdex files are)

Or that's not what you had in mind?
[Android General] Ticklefish:
Not quite. The framework folder shouldn't have any arm, arm64 or oat folders in it if it's fully deodexed. So no odex, vdex, oat or art files.
[Android General] ShaDisNX255:
Yeah TOAD's output has no oat or arm folder in it. But if I replace the whole folder, it doesn't boot. Maybe 1 jar is causing trouble. But it would be tough to find out which one. I'd have to try each jar 1 by 1 to figure out the culprit. I will do exactly that, let me just get out of work lol
[Android General] Ticklefish:
I've been taking a look at the ZIP file today, since everybody says it doesn't work. This is either going to be something wrong with the updater-script, something wrong with the ZIP file itself, or something wrong with the update-binary.

The script seems fine as far as I can tell. I've used this a few times on various phones without any issue. The file is the right format, the syntax seems fine and it's compressed the right way. I suppose it could be a problem with busybox not being installed on the phones. That would explain why the files are not being copied over but not why the ui_print commands are being ignored.

The ZIP seems fine, the compression rates are okay and it's not corrupted in any way.

So I'm wondering if it's the update-binary. Does anybody feel daring and want to try swapping it with one from a ZIP file that works on their phone?

I'm also looking at how to deodex 9.0. Google have introduced 'CDEX' files into the mix. These are compact dex files that sit inside the verified dex files that are being used instead of optimsed dex files which we used to deoptimise to get the ordinary dex files.

(I can feel a migraine coming on just looking at that sentence..)

Vdexextractor can extract the cdex files but another tool is needed to extract the information from those files. This second tool is meant for Linux but we might be able to use it now we have all those cygwin files from a few posts ago.

I don't suppose anybody reading this happens to be running an odexed Pie rom I could try out?
[Android General] Black OPS Acid:
Plz create for htc u play
[Android General] Ticklefish:
Have you tried using this on your phones files?
[Android General] Astrako:
Hi [MENTION=4315551]Ticklefish[/MENTION] and thanks for this new tool! Finally a good deodexer and unique tool for Windows!

As a suggestion, you can update vdex binary from here: https://github.com/fOmey/vdexExtractor/tree/master/bin

Regards ;)
[Android General] Ticklefish:
Thank you! Hopefully it proves useful to people. There are so many excellent deodexing solutions out there that are no longer updated. I made this with the idea that it can always be updated without any trouble. So far, it seems to be working!

And thanks for the link. From what I understand, and I'm not an expert at github at all, this can only be built in Linux?
[Android General] Astrako:
Is more easy than that :) You only need to download the vdexExtractor_x86_64.zip file and extract it in TOAD\Tool_Files folder (replacing existing ones). I've already tried with LG G5 Oreo and worked perfectly :o

That link has 0.5.1 vdex. Hopefully the owner will build newest 0.5.2 too ;)
[Android General] Ticklefish:
Oh. I didn't realise it was that easy. That's a little embarrassing.

I'll update the ZIP later today.
[Android General] Ticklefish:
Thank you for letting me know about this. I've put 0.5.1 into the ZIP and uploaded it to the downloads tab. Exciting stuff!