yashsingh1/xdaforums huggingface dataset:
[8125, K-JAM, P4300, MDA Vario ROM Development] tadzio:
Hi guys,

let me briefly explain what my version of mamaich's tools do and don't do, and where they differ from mamaich's original.

- the unpacking part is identical to mamaich's original, I just added a bit of error processing.

- when packing, I do cut off the .nb file after the last used block. mamaich's version always used the original file size. So, if you remove files, your resulting .nb will get smaller.

- the other major difference, and the one which has the biggest impact, is that my tools reclaim the space Microsoft reserved for its future online Windows Update system. This gives you back a few megabytes of ROM.

- on a Hermes, the unused ROM goes to Storage. This is not (yet) possible on the Wizard, as the boundaries between DOC and Storage are fixed. But of course the reclaimed memory can be used to add more files into a ROM while you're cooking it. In fact, you can add files until the resulting .nb is back at the size of the original .nb.

- there is no file compression going on while packing.

- when you run a ROM repeatedly through unpack - pack cycles with my tools, the end result is always the same. So it's not true that every time you do that some space is wasted.

- a ROM treated with my tools may theoretically boot marginally faster. This is due to the fact that the OS partition header in an original MS ROM is not at the very beginning of the partition. MS left some space for more partition headers which will be written by the online Windows Update procedure (that procedure does not overwrite the existing header, but just prepends it with a new one - for data integrity reasons, I guess). So, at boot time, the OS loader has to search for the first partition header. In a "tadzioised" ROM, it'll find it immediately, as it is right at the beginning of the partition. In an MS ROM, the loader needs to scan a few hundred kilobytes. But this is a very theoretical speed advantage!

Hope I could help clarify things a bit.

Cheers

tadzio
[8125, K-JAM, P4300, MDA Vario ROM Development] notyourdaddy:
weird. previously in my rom i was unable to add any more files as i had it filled to the max. Using these tools i was able to add Acrobat LE and Streaming Media player. it did also seem a little snappier, but it may have been due to some reg tweaks i made i don't know for sure.

if i understand correctly, having windows update in a rom made using your tools is pointless as the reserved space is removed?
[8125, K-JAM, P4300, MDA Vario ROM Development] mfrazzz:
Yeah I can confirm that my Crossbow Reloaded v1.5 could not fit Transcriber or Voice Command in it how I was building it before. Now after running the rom packer piece, I can now put Transcriber back in for sure (haven't tried to fit anything else back in) so you do gain back space that you can reuse since the rom is built more efficiently. Not sure I'm going to put those back in, was more just playing around to see what I could do.

So thanks for the method and explaination tadzio, and thanks to edhaas for making this easy to use (I had read the original thread but backed away from trying to implement it until I absorbed it more).
[8125, K-JAM, P4300, MDA Vario ROM Development] tadzio:
So, what's weird about that? :-)

It is pointless, as there's no space Windows Update could write files to. And since it doesn't work, the Windows Update executables are also a waste of space.

It may even be dangerous, since I do not know how well Windows Update handles "out of flash memory" situations. It should treat "tadzioised" ROMs the same as normal ROMs where previous (hypothetical - seems to not exist yet) Windows Updates filled all remaining free space, but that's only the theory...

Cheers

tadzio
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
Well, I have to say I'm relieved that people are finding its working. And thanks tadzio for sharing your knowledge (and the base tools)!

After discussing it with tadzio, it doesn't appear there's any benefit to artificially inflating the size of the original base OS.nb, so don't worry about that.

There may also be no benefit to doing the "pre-packing" that is currently the first step. On the other hand it doesn't hurt. If someone wants to try it both ways and give some feedback that would be cool.

Thanks for the support.
[8125, K-JAM, P4300, MDA Vario ROM Development] mfrazzz:
The prepacking has DEFINITE benefits. It pulled almost 1.5meg out of the size and gives me room to add more stuff to my rom. So that is a huge plus IMO!

Basically I used your kitchen to do the pre-packing step. Then took the result and put it back over into the Faria kitchen and built from there (since both are basicaly the same after that step and I was going to have to replace BOS.exe with Faria's BuildOS.exe I figured I'd just move the base rom image after doing the pre-pack.

Let me know if by doing this I may miss something else, but everything else appears to be pretty much the same between the two kitchens.
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
Ah, I see.. Well, the prepacking in your case is helpful because you're pulling it out of kitchen before the final build. If you were to leave it in the kitchen for the final build it likely wouldn't matter. Each build essentially repeats the process.

From what I gather, you are inefficiently packing only those couple of packages you are adding to your final build, a difference which may be negligible for a large package or two, but might be significant for several smaller packages.

Glad you're finding it helpful.
[8125, K-JAM, P4300, MDA Vario ROM Development] vivi:
Works well.

GREAT.

I can add 8 MB of mms package into my modified ROM without deleting any files.

Thanksss,

vivi
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
Version 1.3

Thanks everyone for the nice feedback.

Today I am putting out version 1.3. In this version I have added the option of eliminating the .rgu files from the final build, and thus from the /windows directory in the Wizard. Early reports are that this works well, and eliminates about a meg or more from the final build. Please report how this works for you. If you do have problems with it you can use the Start-V1.2.bat file to do it the old way.

If you are building "base roms" this will make it difficult to "reverse build" the rom, so please don't do it in that case.

Let me know if you have further ideas or suggestions!
[8125, K-JAM, P4300, MDA Vario ROM Development] mfrazzz:
Is it only the the start.bat that is different from v1.2 to v1.3? If so, can you just provide the new version of start.bat? I'm now completely using your kitchen since I didn't realize it repacked it again after bos.exe and I'm amazed at how much more I can add. If I can now gain some more by having the .rgu files go away, that would be great too.
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
Yes, it is only the start.bat and the readme that have changed. You can catch up on the readme by re-reading the first post. Here is the new start.bat...

Best wishes,
[8125, K-JAM, P4300, MDA Vario ROM Development] mfrazzz:
Cool thanks! I'll give this a whirl a little later and let you know how it goes.
[8125, K-JAM, P4300, MDA Vario ROM Development] mfrazzz:
Not sure why, but I've run this twice now, and both times my rom size is .4MB larger than it was when I run it with V1.2... Still looking, but seems very odd...

[...Edit...]

Ok, not sure whats going on now, but last night my nk.nbf file was 56.7MB. Now this morning its 57.1MB if I run V1.2 and 57.0MB if I run V1.3... I was up late so maybe I changed something since my last build. Sheesh, I can't remember now ;) But yes, I gained a small amount with the .rgu files being removed.
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
Doesn't sound like much though. I'm at my day job now and can't check it out myself at the moment. I want to actually reflash and make sure its working correctly. I'll work with it tonight and see what's up.

Thanks for the feedback.
[8125, K-JAM, P4300, MDA Vario ROM Development] mfrazzz:
Last night I built a rom using the V1.3 script with no problems. Today, I've tried building the rom 3 times with V1.3 and while the build works and the rom flashes to the phone, the entire OEMAPPS package gets left out (so in the Faria ROM, this is things like Backlight, Power, and other utilities and settings). I then built the exact same ROM (same choices) but used V1.2 script and there was no problem with this build (OEMAPPS is all there). So something with the V1.3 where it deletes the .rgu file seems to cause that problems.

The difference between my build last night (that worked fine) and the one I'm doing this morning is I'm adding more items into the rom build (no other changes or anything with OEMAPPS package).

Since I was seeing very little size difference between V1.2 and V1.3 start scripts, I'm going to just stay with V1.2 for now.
[8125, K-JAM, P4300, MDA Vario ROM Development] ADB100:
I have been attempting to build a ROM and I think I am right at the limit with regards to the size. I eventually got my ROM to build by removing some files (.bmp & other less important files). However when the Wizard booted up I was missing some stuff - Office Mobile folder didn't get built, security wasn't disabled, plus some other minor stuff but all my apps were physically there. I think I am so near to the limit that what I thought was the limit isn't......... I think that even though the ROM builds and installs OK it still needs some more space - I think with the latest base ROM the nk.nbk file needs to be less than 58MB.

This might not be your issue but certainly with me it was....

Andy
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
I have to say I have also had some problems with the new version after removal of .rgu files. There were links which weren't moved and registry entries that apparently weren't made. I've also started having active sync problems which I am still struggling with. Thank goodness for the Tmobile rom, because at this point nothing else is loading. It's probably unrelated, but its troubling for me.

Anyway, I guess I'm recommending people use the 1.2 Start version unless they're feeling experimental, until we can figure out which rgu's are the really important ones.

I have a new technology I'm working on, so I'll wait until thats ready to put out a new version. I'll probably leave out the rgu deletions in that version. Having said that, I think it was a good idea, and worth a try.
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
New Version!

Ok, so here we go! In the latest version, I've removed the .rgu deletions, and introduced file compression! Remarkably, this actually seems to improve the performance of many programs, while increasing the space available for more packages. I have automated the process for "squeezing" the main Office programs, which automatically gains an extra 2+ meg. For example, powerpoint goes from 2.6 meg to <1 meg. In addition, I've included a module, the ZSqueezer, which makes it very easy to compress other large .exe files, both those in packages and in the storage card. I would suggest running the ZSqueezer on camera.exe, which will also decrease the load time noticeably. See the readmes for more information.

At this point, I'm actually running out of useful things to put in my rom! :D

Enjoy! And let me know how things are going!

Regards,
[8125, K-JAM, P4300, MDA Vario ROM Development] RevAtB:
Just to let you all know, the link shows

http://rapidshare.com/files/33616241/Wizard_ROM-Packer_Kitchen_V2.0.rar

but actually if you click on it it goes to

http://rapidshare.com/files/32964857/Wizard_ROM-Packer_Kitchen_V1.3.rar.html

I'm sure that Ed will fix this once he reads this post. so copy and past and make sure you are actually downloading the V2.0!
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
Oops!

Thanks for the post! I've now (hopefully) fixed the link.

Best wishes,
[8125, K-JAM, P4300, MDA Vario ROM Development] ADB100:
Absolutely fantastic :) :)

I have been trying for a while to get all my stuff into a ROM and kept running out of space. I now have the ROM I want with all the packages integrated - sweet.

A big thumbs up to you for finding the UPX utility, I would have never have thought of that....

Does anyone know how to get TomTom 6 into a package that I can integrate into my ROM :rolleyes:

Andy
[8125, K-JAM, P4300, MDA Vario ROM Development] freeyayo50:
I have a question....Can you copy .exe files from your PPC(of any program) and put them into the SqueezeRoom dir, compress them and copy them back to your PPC?? Will this work.
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
Exactly. Pretty powerful stuff. TomTom compresses well BTW!

I think I'm going to put out the ZSqueezer module as a standalone for non-cookers to use for just this purpose.

Best wishes,:)
[8125, K-JAM, P4300, MDA Vario ROM Development] freeyayo50:
Nice!! I think that will be a great idea.

IM going to be testing this out rite now...and see how much files get squeezed :)
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
Standalone Squeezer Module Thread

Link for standalone Squeezer module thread:

http://xdaforums.com/showthread.php?p=1283338#post1283338

Regards,
[8125, K-JAM, P4300, MDA Vario ROM Development] masr1979:
Using the same base rom, I got a 56,129KB nk.nbf file with one kitchen, and 49,473KB nk.nbf with your kitchen edhass...... impressive :D.

Plus I have never seen the screen calibration cross move so DAMN FAST. Opening up the \Windows\ directory with Total Commander also seems to be pretty fast. I can only hope the rest of the stuff works likewise.
[8125, K-JAM, P4300, MDA Vario ROM Development] RevAtB:
Sqweezed ROM, Totatlity

Well I am working on diong a full .exe and .dll UPX on the entire OEM dir from the latest pandora, as well as what I can recomboulate in the sys folder. Not sure how to go about reassembling the .dll listed as folders to compress them and then make them folders again so the kitchen will work. Any help would be greatly appreciated.

I'm sure that with this complete Compacting, Stuffing, Cramming we can get a rom thats BLAZING even when the cpu is underclocked, and loads more room.

Will keep you all posted on my findings.
[8125, K-JAM, P4300, MDA Vario ROM Development] masr1979:
I can also compress dlls??????
[8125, K-JAM, P4300, MDA Vario ROM Development] RevAtB:
UPX and what it can do for you!

http://mobile-sg.com/software/?p=UPX4PPC&platform=ppc

this refers to the UPX4PPC version that actually runs on the PPC

I include this in my own roms to compress extras I install after.

There are also links to the desktop version which is currently UPX 3.0.0

Gotta love compression.
[8125, K-JAM, P4300, MDA Vario ROM Development] freeyayo50:
Thanx for this. Does either the PC or PPC app compress the EXE/DLL files anymore than the one edhaas posted??

I ask this because from the screenshots you can choose the amount of compression.
[8125, K-JAM, P4300, MDA Vario ROM Development] freeyayo50:
Does anyone know how I can compress ppt.exe, pword.exe & pxl.exe files??

My PC and Total Commander wont let me copy any of those files to any other folder. I can't even change the permissions on the files.
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
UPX

Hi Guys,

Thanks for all the comments. The UPX I'm using is the latest version. I just downloaded it a few days ago.

Be careful about doing wholesale compaction. If something doesn't work with a system program you may not be able to troubleshoot it very well. One at a time trials is likely best. You don't always need to rebuild each time to test. You can copy the packed version into the ppc on top of the one in rom to test it. It's not sure-fire, but its a good initial test.

Also, background processes I would expect may not always work, as sometimes the files are self-referencing.

But please report what works or not. I like the idea of the "blazing fast" compact rom!:)

Best wishes,
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
Once these are flashed into the rom, they cannot be changed or removed. All of the roms I've seen have them flashed into the rom. You can overwrite them (temporarily) to test a compacted version, but on a hard reset they will reappear, and the compacted versions will also use up your user storage memory. Time to start "cooking" your own!
[8125, K-JAM, P4300, MDA Vario ROM Development] vivi:
upx them and replace in ROM by RK works fine.
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
You can play with compression in the kitchen as well. I chose a mid-high level which is touted to be a good trade-off between loading speed and compression, and indeed it seems to be. I tried the "best" compression level, and it only gave me perhaps a 5% decrease in size. I went for the speed. Higher compression also adds to the probability that the file won't run correctly.
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
DLLs compress also, but from my experience and other's reports they are less likely to work.

Have fun!

EDIT: Late reports from the Standalone thread are that .dll compression is working well.
[8125, K-JAM, P4300, MDA Vario ROM Development] computhug:
This registry key:

​

HKLM\System\ObjectStore\RegistryUpdate

contains binary values for every .rgu included in a rom. Might not be worth investigating, as folks seem to be freeing up 8-9 megs with your latest release, but I noticed it and thought I'd pass it along.

Thanks for the great kitchen, I loved it so much I decided to come out of hiding and actually post something :D
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
Welcome to the forum! :)

Thanks for the comment and thanks for the tip. I'll cogitate on it awhile and see if I can do something with it.

Best wishes,
[8125, K-JAM, P4300, MDA Vario ROM Development] mfrazzz:
Ok, from the cooking side... I want to compress some of these DLLs. I've compressed them by hand and dropped them in the rom and verified they work that way. Now I want to cook them in compressed... But I'm apparently missing something. Here's an example:

Windows Media Player has a DLL called wma9prodecoder.dll. This guy is 875.6K in size by default. I ran the squeeze on it and its now only 281K. I can drop that directly on the phone, and I see no ill effect to WMP so now I want this in my rom this way. So I go to the SYS\MediaOS and I find there is a Directory called wma9prodecoder.dll. Inside that directory is this dll, so I change that. But when I cook the rom, the one on the phone in the rom is the original at 875.6K. I'm assuming then that this .dll file is really not being used but instead its coming out of the S000 file or something in that directory? This appears to be like a CAB. Do I have to make this into a cab, and then modify the file that way?

I'm finding a lot of dlls that are very large that compress down nicely and things seem to work just fine afterwards (also finding some that don't). If I can get this figured out, I'll be happy to share the list of things I'm trying to compress.
[8125, K-JAM, P4300, MDA Vario ROM Development] masr1979:
I experienced some hang ups, when I compressed every single dll I could, so there's definitely some that don't agree with the procedure. Also voice command just stopped working for no apparent reason, so I'm blaming compression again on this one, that is until I find a more reasonable explanation. It was working fine, but suddenly it crashed and never worked again, I'll update yous guys if I find out what the reason is.
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
Hi mfrazzz, Nice to see you. I'm afraid my technical knowledge of .dll structure is about where yours is (or less). I do know that some programs do not compress well, because they contain self-referencing code, with relative offsets, which may not be preserved well in the compression. Having said that I'm not sure that's what you're running into. It sounds as though the .dll is being built during the buildos phase, using instructions built into the structures you are seeing in the package. Faria has commented in the Standalone thread that some things are not meant to be compressed, but he did not give a reason, other than its been tried in the past.

You could try replacing the .dll in the dump. This did not work for .exe's, but for .dll's who knows? Wait until buildos has finished, then before exiting look in the temp directory for the dump. Replace your file, then exit buildos. A bit of a long shot.

Maybe someone with more technical knowledge of .dll structure will help us out.

Best wishes,
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
Yes, I have been cautious not to compress "resident" or "today screen" type programs, as they are more likely to have self-referencing code with relative offsets as I described above. Some may work though, but I'll let brave souls like you figure out which ones. :)
[8125, K-JAM, P4300, MDA Vario ROM Development] mfrazzz:
Yeah, while the actual .dll file is in this directory, from what I can see that .dll is never moved into the dump area, just the directory itself and the files in it (minus the .dll) is in the temp/dump area. I'd love to be able to do this, as it appears a lot of space can be conserved, but its figuring this piece out...

I may try removing the directory itself and just leaving the dll and see if that works. Worst case, I'll just have to reflash my phone ;)
[8125, K-JAM, P4300, MDA Vario ROM Development] mfrazzz:
Nope, didn't work. WMP complains there are either missing pieces or parts are unsigned. Transcriber just flat doesn't work now. Oh well, it was worth a shot :)
[8125, K-JAM, P4300, MDA Vario ROM Development] masr1979:
Brave souls.... funny :p anywho, it would be good is we post results of which packages function properly when compressed; individual file results would make for a somewhat lengthy list.
[8125, K-JAM, P4300, MDA Vario ROM Development] mfrazzz:
Here's some commands I put into a command file to compress some stuff in my kitchen. Should be self explanatory what each of them are. I mainly went after larger .exe files so others I'm sure will compress too and this was just a start:

tools\upx -6 oem\1-Calc\1-Calc.exe

tools\upx -6 "oem\camera+streamer\camera.exe"

tools\upx -6 "oem\camera+streamer\StreamingPlayer.exe"

tools\upx -6 "oem\COnTACTS MANAGER\PPCPimBackup.exe"

tools\upx -6 "oem\COnTACTS MANAGER\TaskMgr.exe"

tools\upx -6 oem\HTC_Audio_Manager\AudioManager.exe

tools\upx -6 "OEM\pocket_rar\Pocket Rar.exe"

tools\upx -6 oem\Smartskey\Smartskey.exe

tools\upx -6 "oem\Total Commander\totalcmd.exe"
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
Nice, mfrazzz

You can add voicecmd.exe (not the .dll's:rolleyes:) gsfinder, and phm registry editor, besides those already reported. Commercial apps we're not including here. (Check the standalone thread for listings of those.)

Regards,
[8125, K-JAM, P4300, MDA Vario ROM Development] mattk_r:
I went crazy with mine, compressed all of the exe's and dll's in the OEM packages (except Camera) as well as a few in SYS. Amazed at how many fit, and that the performance is great. There were a few packages that didn't work, but overall I think this is the next best thing since KY! :eek:
[8125, K-JAM, P4300, MDA Vario ROM Development] edhaas:
Well, yeah, its hard to beat KY. :cool::o

BTW, is this a PG rated thread?

It's neat you're having good luck with the .dll's I'll have to get more daring and give those a go too. I was happy with the .exe's but now folks are taking it to the next level!
[8125, K-JAM, P4300, MDA Vario ROM Development] mattk_r:
I can easily take an inch and drag it way out (you should hear my fish story...), or crash in flames trying to do so.