yashsingh1/xdaforums huggingface dataset:
[Android General] ShaDisNX255:
Hey, ummm I know MM 6.0.1 is pretty much out-dated at this point, but while I was trying to deodex all apps and fw, I got this error:

Processing file 57 of 275 - "Your_Files\app\Photos\oat\arm\Photos.odex"
Processing "/system/app/Photos/Photos.apk"
Exception in thread "main" java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.FileNotFoundException: Tool_Files\out\lha.smali (Access is denied)
at org.jf.smali.Smali.assemble(Smali.java:116)
at org.jf.smali.AssembleCommand.run(AssembleCommand.java:96)
at org.jf.smali.Main.main(Main.java:99)
Caused by: java.util.concurrent.ExecutionException: java.io.FileNotFoundException: Tool_Files\out\lha.smali (Access is denied)
at java.util.concurrent.FutureTask.report(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at org.jf.smali.Smali.assemble(Smali.java:112)
... 2 more
Caused by: java.io.FileNotFoundException: Tool_Files\out\lha.smali (Access is denied)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(Unknown Source)
at org.jf.smali.Smali.assembleSmaliFile(Smali.java:153)
at org.jf.smali.Smali.access$000(Smali.java:56)
at org.jf.smali.Smali$1.call(Smali.java:103)
at org.jf.smali.Smali$1.call(Smali.java:101)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

Any ideas? o:
[Android General] Ticklefish:
I've edited your post slightly to highlight the actual error causing the trouble.

Thank you for your files. I've looked at them and I think I know what the issue is. It's not good news though.

In stock Oreo and Pie rom's, we'd need to process "boot.vdex" to be able to deodex the relevant JAR files whose classes.dex are contained inside it. This is done with vdexExtractor.exe.

But your boot.vdex is tiny. It's 68 bytes only. Your boot.oat is 4,519 kilobytes. Even with amazing compression, there's no way they both contain the same information.

So processing the VDEX file won't work in this instance. We'd need to process the OAT file instead.

That's the good news - we know what the issue is.

The bad news is that vdexExtractor doesn't work on OAT files. We can't use baksmali.jar either as that currently doesn't work on Pie files. So it doesn't look like we can process them at all. Or, at least that's how it looks to me.

I suggest getting in touch with the developer. They may be able to come up with something.

Sorry about this. It's the downside of letting developers make their own rom's - sometimes they construct the things in odd ways!

It looks like smali.jar isn't able to process one of the smali files in that APK. If you can share your files, I can take a look and see if I can come up with something clever.
[Android General] JuniorCaesar:
[MENTION=4315551]Ticklefish[/MENTION] thank you so much for your investigation! i will report this to device's maintainer. good news for me is your tool successfully deodexed my services.jar file, which i want to deodex in the first place (for signature spoofing to use microg). so i deodexed it, put it in my device and it is working flawlessly. thanks again for your help and keep up the good work :)
[Android General] Ticklefish:
And you're on Pie...which means this definitely does work on Pie after all! Yes! That is great news! Thank you for letting me know. :highfive:
[Android General] JuniorCaesar:
no problem, and thanks again for creating this tool :highfive:
[Android General] Beetle84:
Im downloading the Aussie single sim FW now, I was using vietnamese dual sim (had to try pie haha). I will try TOAD and report.
[Android General] Ticklefish:
I'm not a genius at reading logcats so, unfortunately, a lot of that went over my head. But the line I left in your quote looks interesting. Android is saying that it can't recognise (or recognize since it's American..) the version number of the DEX file in core-libart.jar.

This might be an easy fix.

Could you try using the attached 23.bat instead of the one that's currently with TOAD? Just extract it from the ZIP and put it in the Method_Files folder, overwriting the one that's there.

I've changed the lines that build the DEX files from:

java -Xmx1024m -jar Tool_Files\smali.jar assemble -a 26...

to:

java -Xmx1024m -jar Tool_Files\smali.jar assemble -a [color="red"]23[/color]...

It's possible that we've been creating files for a higher version of Android than your rom knows about.
[Android General] Beetle84:
Works like a dream, the installer said it finished with an error, but I couldnt find it in the log. I flashed the zip and it took a while to reboot as expected, no bootloops. Thanks alot buddy, now to decompile and edit a few things. Have to use your other awesome tool, TMA.
[Android General] Ticklefish:
Exactly what I needed to hear during a crappy day at work. Pie has been defeated. Ha!

Hopefully Q won't take so long when that's released.
[Android General] Beetle84:
Well, hopefully. I debloated a lot before I deodexed and now I keep getting 'home network has stopped' message. I think it's related to me removing what's new (Sony entrance), as it seems to only pop up on Sony apps. I'm reflashing and deodexing again, I will post the log.
[Android General] Beetle84:
The error is still there (home network has stopped) so I iust disabled that, no probs so far.

There was a ****load of oat files in priv-app and a couple in app folder, I deleted them all and everything seems ok.

There is arm,arm64 and oat folders in framework still. ARM is empty, arm64 has a few files and oat folder has heaps in it. Do you think its safe to delete those too?

Edit: they're gone :)
[Android General] Beetle84:
Heres a zip of the remaining files in framework.
[Android General] levone1:
[MENTION=4315551]Ticklefish[/MENTION] - I got the script to work for me by changing

run_program("/sbin/busybox", "mount", "/system");

to

mount("ext4", "EMMC", "/dev/block/bootdevice/by-name/system", "/system");

(Not my idea - I just borrowed from someone else's script...)
[Android General] levone1:
BTW - any help with this... https://xdaforums.com/showpost.php?p=78193245&postcount=20

update - solved it by putting /system/framework/arm64 folder back. Not sure if it's a fix or workaround... Is there any issues with that?
[Android General] Ticklefish:
Ah, yes. This was the rom that had the ODEX and VDEX files still with the APK files, despite those APK files being deodexed, wasn't it? I suspect that Sony must have done something odd when they constructed the rom. Maybe those files aren't supposed to be deodexed? Maybe they forgot to remove the ODEX and VDEX files?

Oh well. The main thing is that it worked and you now have a deodexed rom to play with. :good:

I'll be honest - I'm not sure about this. I did some Googling but all I could find suggests that it's something to do with the library. Possibly.

Which could mean that the symlinks are broken. That's an area I don't know much about yet. Yet.

Cool. Busybox is supposed to be able to mount the system partition as that code you used won't work on every rom. But I'm glad you got it working! I'll have to create a FAQ soon of little suggestions like this for people who are having trouble.

So the issue is that the phone is noticeably slower when deodexed. When you put the arm64 folder back, did it have any files in it?
[Android General] levone1:
After deodexing, phone boots fine, but is extremely laggy - pretty much unusable. That error in logcat just repeats over and over again. When I restore /system/framework/arm64, with all of the files in it, (.art and .oat files), it runs fine, but, (and this has nothing to do with TOAD), now phone bootloops when flashing Xposed, which is the only reason I even care about deodexing at this point, (Xposed for Oreo doesn't work on recent Sony firmwares, because of a security patch, [June, I think], but a developer on XDA has made a fix by patching a few framework files). So I can boot and use deodexed fine with the arm64 files, but no Xposed, or I can use Xposed on a buggy / laggy phone...
[Android General] Ticklefish:
There's really not much information that I was able to find about this online. Which is frustrating.

Your logcat looks like this:

I've Googled bits of the error messages to see if i could find a solution.

Somebody encountered a similar "Could not create image space with image file" error when trying to install Xposed on an Xperia Ray. They were able to solve it by changing the watchdog timeout. I'll be honest, I have no idea what that means so I also have no idea if it'll help at all. But it's worth trying. This is what they said:

KazuSoap commented on Oct 4, 2015
I solved the problem by myself. This problem occurs by watchdog timeout as you say. I turn watchdog timeout off completely with the --no-watch-dog option by adding this to /system/build.prop:

dalvik.vm.dex2oat-flags=--no-watch-dog
dalvik.vm.image-dex2oat-filter=--no-watch-dog
Then Xposed framework installation succeeds.
Thank you for your mention.

Or there's this found on the smali github:

JesusFreke commented on Apr 6, 2017
You should rebuild the various classes.dex files and add them back into their respective .jar files, and then either regenerate the art/oat files on the system partition, or delete them and let the system generate them and store them in the /data partition on first boot.

This should be happening anyway but perhaps it's worth checking that the cache is being wiped. Maybe there's an issue with the data partition?

I also found this on stackoverflow:

I am trying to modify Android framework *.dex file. I rebuilt boot.oat and boot.oat in /system/framework/arm64, it is fine. Then I remove all odex file in /data/dalvik-cache/. When I reboot, it is supposed the new boot.oat will generate *.odex in cache. But Android cannot boot, shows cannot relocate boot.oat to /dalvik-cache, Only the zygote can create the global boot image.

How to resolve?

Nobody provided a resolution, unfortunately. But it does show that it's not a unique problem.
[Android General] Beetle84:
I couldnt be more thankful mate, with this program and TMA, im now able to start adding the little tweaks I like to SystemUI.apk and Settings.apk.

(I pushed the clock a little too far to the left, I know)

Cheers again.
[Android General] Ticklefish:
Well, that's the fun part, isn't it? :D
[Android General] levone1:
Interesting - I added the two lines to default.prop in ramdisk, but no change. I saw 'unable to write to...' in log, so I thought of permissions. /data/dalvik-cache/ and /data/dalvik-cache/arm64/ folders were both 771. Changed to 775 and rebooted - still the same. Checked permissions and sure enough, they were back to 771. Tried chattr +i, and bootloop - wouldn't even get past Sony logo. Something force changes the permissions at boot...?
[Android General] Ticklefish:
I don't know, to be honest. Perhaps the folders are deleted when the cache is wiped? So the permissions aren't changing so much as the 775 folders are being removed and new ones with 771 are being made.

I've been trying to find out how to get around this issue of deodexing breaking symlinks. There is pretty much no information about this online but I did discover this interesting post on [MENTION=5132229]xpirt[/MENTION]'s (now discontinued) Fulmic's Deodexer thread:

I've missed out a few lines of the original post but perhaps we need to leave the boot.* files in the rom? That'd be annoying if you want to edit the smali in those files but that's normally only one file in Oreo I've noticed.
[Android General] levone1:
Found it in init.rc, but it seems like it's not unusual, and I compared it to a boot img from another deodexed rom that doesn't have the same problems, and it's there too, so I guess that's not the issue...
[Android General] Ticklefish:
What happens if you put the boot.oat, boot.art, etc back into the folder?
[Android General] levone1:
Tried with the 3 "boot.." files, and no different. Also tried with the 3 "boot.framework.." files. Don't know how much time I want to spend testing individual files, since there's about 60 of them. I figure at some point maybe I'll add the whole folder back and maybe remove some suspects... I wonder about ones like "boot-android.hidl.base", "boot-android.hidl.manager", "boot-org.apache.http.legacy.boot"...
[Android General] Ticklefish:
The error talks about "boot.oat" being the issue but if putting it back didn't work, I'm really not sure.

I'm going to be working on version 3 of TOAD over the next few days. I think I've got a pretty good idea of how to avoid symlinks getting broken, but I don't think that'll help in this instance.

Sorry, I like to give people an answer but I just don't know about this one.
[Android General] AnonVendetta:
[MENTION=4315551]Ticklefish[/MENTION]: The post I made that you quoted isn't in this topic, it's in another (closed) thread at https://xdaforums.com/showpost.php?p=71121224&postcount=52. I got a notification and I knew that I never posted in this topic. So I got curious.

I haven't owned the LG v10 for quite some time now, I have long since moved on to better devices. The v10 truly is the worst phone I have ever owned, nothing but problems. I can't be of any help here, but I never did manage to produce a deodexed stock Marshmallow ROM for the v10 that would boot. I still suspect that the boot.oat/boot.art files were the bigger issue behind everything. I'm not sure how [MENTION=272754]Eliminator[/MENTION]74 managed to produce a ROM that booted while leaving these files intact.

The simple answer to avoiding broken symlinks is.....deodex on Linux, not Windows. Linux fully supports Android's symlinks, since Android uses a filesystem that is based on what you would find on the average Linux system. I've seen been using the Linux version of SuperR's Kitchen for deodexing, which has both a Linux and Windows version, but started out with only Linux support. It's easy enough to install Linux in a virtual machine (which you can boot from Windows), then set up an environment that is ideal for working with Android projects.
[Android General] Ticklefish:
Curiously, my last phone was on Marshmallow and I was able to deodex that completely without any troubles. No issues with the boot.* files at all.

It could be that some roms deliberately look for those files and complain when they're not there. Perhaps. Maybe.

As for symlinks, I have an idea that should solve the issues with those.

Right now TOAD works by creating a deodexed copy of the "app", "priv-app" and "framework" folders in a flashable ZIP. This ZIP deletes the original folders from your phone and copies across the new versions. I think Android is getting upset because the replacement symlink files, while exactly the same as the old ones, are technically different files.

So, as far as the rom is concerned, the symlinks aren't there.

We can put lines in the updater-script to make new symlinks, but I have a plan to create a flashable ZIP that leaves the symlinks alone completely and only copy across the actual APK and JAR files that have been deodexed.

I've got the code planned out in my head, I just need to type it up and test it.

Deodexing does seem to be a lot easier on Linux but I'm a masochist - I'm determined to get it working on Windows as best as it can!
[Android General] UltraM8:
It's not only Sony - Moto One (sprout) also have apps that seem deodexed, while framework contains odexed files + vdex. I've been using batchapktool for a long time, but it seems since Oreo, it can't crack any new framework, so now I'm stuck & have no idea how to solve this issue.
[Android General] Ticklefish:
Have you been able to deodex your framework with TOAD?
[Android General] UltraM8:
Haven't tried yet tbh) But I've noticed another device with vdex & partial deodexed apps.
[Android General] rizky_zulfikar:
the version is not compatible :( why? i use 32-bit pc
[Android General] Zackptg5:
TOAD seems to be able to deodex pie stuff fine with batch script 28 :)

One request though. Can you add the capability to just extract the contents of the odex files rather than merge them into the jars/apks? Thanks!
[Android General] UltraM8:
Tried yesterday - TOAD worked well. Great job, dude!)
[Android General] DaRdamnDoG:
I'm not able to launch it with Windows 7 Ultimate SP1 x86

I have Java V8.191 installed as well. Please help!
[Android General] rizky_zulfikar:
it work only on 64-bit

---------- Post added at 11:27 AM ---------- Previous post was at 11:25 AM ----------

i've tried deodex and the result app and vendor/app is successfully deodex but the priv-app and framework is failure, anything wrong?
[Android General] DaRdamnDoG:
No, look for the download file without .64 in the name (i.e. : TMA_v15.0.0.1.exe). Another problem is I have to deactivate my Microsoft Essential in order to let TMA work as it was said to contain Trojan file.

However, although I managed to launched it, I am not able to pull system files for deodexing purpose. I only managed to get SystemUI.apk, framework-res.apk, tw-framework-res.apk. The program renamed my SystemUI.apk to y.apk and it failed to decompile it. I just put everything in the bin.
[Android General] Ticklefish:
That's not so easy. On earlier versions of Android, we could use baksmali.jar to turn the ODEX files into smali files. Then we could use smali.jar to turn those into classes.dex and put them into the APK or JAR files, if we wanted.

But Marshmallow, Oreo and Pie are different. Baksmali can't process the files.

We use a different program to turn the ODEX/VDEX files into classes.dex, skipping the stage in the middle.

To turn that into smali files, you'll need to decompile. And I already have a decompiling tool..

Good to hear! Thank you for letting me know. :highfive:

I'll make a 32 bit version when I'm next back at a PC. That's likely to be Monday but I might get an opportunity this weekend.

As for the failure, look in the log file and see what it says. This could be for many reasons.

I'll take a look at the code and update it if needed. When you pull the file, are you pressing ENTER when it asks you if you want to rename the file? Or are you entering something else as well?
[Android General] DaRdamnDoG:
Sorry for the late reply, yes I think I am the one who entered the 'y' as to the result of the y.apk, there was no problem pulling files from "Theming" side, when I am in the "Deodex" side & tried to pull it only managed to pull Build.prop and it got stuck there. I can't get the log as I have deleted the whole thing but if you need it I will redownload and try my luck again.
[Android General] Ticklefish:
That's okay. TMA doesn't deodex anymore, I've deactivated the option in the latest version.

All deodexing is done by TOAD instead.
[Android General] DaRdamnDoG:
Yes I've noticed that as I managed to pull the folsers manually however it took afew attempts as the progress got stuck halfway once or twice.

Can't wait for ToAD x86 sir ?

Thanks for your hard work.

P/S: So basically TMA is a full theming/packing service now?

Cheers
[Android General] Ticklefish:
TMA no longer deodexes as it was getting too complicated. The pulling option didn't always work and the code got harder to manage with every new version of Android.

So I disabled it and put together something simpler and easier to maintain. TOAD isn't perfect, yet. But it's getting there.
[Android General] DaRdamnDoG:
Are you able to convert it to x86 tho? It would be very disappointing if not ?
[Android General] Ticklefish:
You want TOAD to be able to process roms from a x86 device?

It should be able to already.. What kind of problems are you having?
[Android General] DaRdamnDoG:
Sorry sir I'm not a tech savvy guy I'm not so clear about x86 vs x64, also English isn't my main language so I am confused of what I read lol.

If it's x86 compatible now I'll give it a go later. Whether the rom is made from x86 or x64 does not bother me as long as it gets things done, just like my Samsung Galaxy J2 for the past 4 years ?

Thanks for your prompt reply & cheers.
[Android General] Ticklefish:
That's my fault for making things more complicated. Android devices usually run on a 'arm" or "arm64" type of processor. But you can get Android to work on "x86" processors as well. So I thought you meant one of those. Sorry about that.

I've just uploaded a version of TOAD 2.0.1.0 that's been made for 32-bit Windows. You'll find it in the Downloads tab.
[Android General] DaRdamnDoG:
Yes I managed to launched it but yet to try as I'm struggling to adb pull /system, kept stuck halfway.

i.e. [69%]Pulling /system/..../.... [100%],

always stuck at this situation & it won't let me enter command. I'm using stock manufacturer drivers & cable. (Newbie question: What goes first? Modding res files [theming] first or deodex? Sorry I just want to make sure I don't do double job ?

Cheers! You are extremely kind & patient fior helping me!

P/S: I spent 25 minutes trying to figure out the first question from TOAD, which is:

I can't find your framework folder in your folder!

<< TRY AGAIN >>

<< EXIT >>

THAT IS A CHEEKY ONE.1
[Android General] DaRdamnDoG:
TOAD said "Everything is Ok".
[Android General] Ticklefish:
I guess I need to rewrite that bit of the code!

Pulling entire folders by adb can prove tricky. I had exactly the same sort of issue as you, yet another reason why TMA no longer deodexes. I couldn't find a solution.

Deodexing means that you can change the smali files in your system apps. You can do it before or after changing your icons or XML. It's entirely up to you.

And I'm always happy to help if I can!
[Android General] DaRdamnDoG:
This is my first time deodexing files, just wondering if the finished folders will have any .odex files remaining inside? As my finished project (zipped) there are .odex files remaining in the app folders.
[Android General] Ticklefish:
If the APK deodexed, there shouldn't be any ODEX files left behind. If it didn't, those files will still be there.

There have been some Sony roms mentioned in here recently which have deodexed APK files that still have the ODEX files in the folders as stock. TOAD doesn't detect for that yet so will still try to deodex them.
[Android General] Ticklefish:
Processing file 5 of 21 - "Your_Files\framework\oat\arm64\services.vdex"
[INFO] Processing 1 file(s) from Your_Files/framework/oat/arm64/services.vdex
[ERROR] [12240] 2018/12/01 13:00:09 (vdex/vdex_backend_010.c:467 vdex_backend_010_process) Unexpected checksum (c83a2c09 vs fc2ef625) - failed to unquicken Dex file
[ERROR] [12240] 2018/12/01 13:00:09 (vdexExtractor.c:251 main) Failed to process Dex files - skipping 'Your_Files/framework/oat/arm64/services.vdex'
[INFO] 0 out of 1 Vdex files have been processed
[INFO] 0 Dex files have been extracted in total
[INFO] Extracted Dex files are available in 'Tool_Files'

ERROR EXTRACTION FAILED

So this looks like vdexExtractor is doing a check on the VDEX file, creating a checksum based on the contents of that file, and that checksum isn't matching the one in the file itself.

There's not much we can do with it. Either the VDEX file was changed after it was constructed, without the checksum being changed, or something went wrong while it was being constructed. But vdexExtractor doesn't see it as a valid file so won't process it.

I suggest getting in touch with the developer for help with this.
[Android General] napolionnardo:
From where I downloaded TOAD, I can not find the link to download it. I also want to ask you a question, is this guide useful for Note 8?
[Android General] Ticklefish:
You'll find the links to download TOAD in the DOWNLOADS tab towards the top of the thread page. The latest version is 2.1.0.0 and there is a version for 32bit and a version for 64bit.

I'm hoping to get version 3.0.0.0 out later this week. I'm doing something different with the code and I'm having a few issues but I'm nearly there.

TOAD should be able to process your rom, just make sure you do a backup first. Any problems, let me know.