yashsingh1/xdaforums huggingface dataset:
[Windows Mobile Software Development] maja:
:(Same with a virtual machine with XP sp 3:(
[Windows Mobile Software Development] anryl:
LZX

try this one from china lzx package :cool:

echo.

..\TOOLS\IMGFS\ImgfsFromNb OS.nb.payload imgfs.bin

if not "%errorlevel%"=="0" goto error

xvi32 imgfs.bin

..\TOOLS\IMGFS\ImgfsFromDump imgfs.bin imgfs-new.bin

if not "%errorlevel%"=="0" goto error
[Windows Mobile Software Development] spocky12:
Hey, I didn't have time to test lzx compression, but I guess it's time to shared a cecompr_nt.dll file I found some time ago (I can't remember where) and which is supposed to fix a memory leak when using lzx compression.
[Windows Mobile Software Development] ivanmmj:
I love you.

This fixes the memory leak AND fixes the issue with it crashing with certain ROMs.
[Windows Mobile Software Development] xda2_haseeb:
Well if it does what you say than THNX ALOT MAN!!
[Windows Mobile Software Development] Ameet:
Well spocky, how do we use it though? The actual module that we have in OEMXipKernel is "cecompr.dll" and the one posted by you is "cecompr_nt.dll" Does the imgfs needs any more hexing to include this module? Or should I rename this to "cecompr.dll"?
[Windows Mobile Software Development] !Aman!:
this is not to be cooked. u have to replace this in ur kitchen (tools\imgfs\).
[Windows Mobile Software Development] xda2_haseeb:
Actualy ur kitchen will have a module with tht name :D I guess u have to replace tht old module with this one :D i tried doing tht and cooked the ROMs f9 now just testing if memory leaks still there :D
[Windows Mobile Software Development] Ameet:
Well the RAM usage looks stable during cooking now :D Lets have a look at the flash if it boots well :D Will know in a few minutes ;) RAM usage stays at about 37% on my machine :)

EDIT: After cooking I gained about 10MB of size. Earlier .nbh was 67.9MB with this LZX compression its now 57.7MB :) Flashing to see what happens

EDIT 2: Booted well without any problems. Storage memory after boot is 150.35MB from 138.27MB :D Quite good :) Time to make the posts into tutorials, with credits to all contributors ;) Thanks
[Windows Mobile Software Development] xda2_haseeb:
By your post looks like if the DLL was fixing the PC's Memory leakage not PPCs Memory Leakage :confused: which memory leakage's fix is it?? PC or PPC's?? THNX!!
[Windows Mobile Software Development] !Aman!:
it is related to PC's memory leakage
[Windows Mobile Software Development] xda2_haseeb:
well that's SAD :( i thght it was relating to PPCs Memory leakage and was happy tht atlast my problem might have been solved :(
[Windows Mobile Software Development] maja:
Thanks now work great:D:)
[Windows Mobile Software Development] d_train:
So I replace my cecompr_nt.dll file in the kitchen and cecompr.dll in the XIP

I have editted the kitchen as follows...

..\TOOLS\ImgfsFromNb OS.nb.payload imgfs.bin

echo Edit imgfs.bin, if needed for LZX compression
pause

..\TOOLS\ImgfsFromDump imgfs.bin imgfs-new.bin

When it pauses I open up imgfs.bin with XVI32 and search for the string "XPR". Replace the FIRST one with "LZX" and continue the cooking process?

Do I need the ImgfsFromDump.exe that anryl posted, or is the one in my kitchen okay?

Thanks for all your efforts and assistance in the forums, helping us less experienced :)

DT
[Windows Mobile Software Development] CRACING:
Oh thanks a lot. This made my PC to be coooooooooool while building LZX compressed ROMs :cool:

Thanks again.
[Windows Mobile Software Development] utak3r:
it's me again... :D

the mythic cecompr.dll, supporting both LZX and XPR :eek:
[Windows Mobile Software Development] CRACING:
Awesome :)

Thanks for info, utak3r. The osnbtool is very usefull.
[Windows Mobile Software Development] xda2_haseeb:
Is this for XIP or the ktichen?? because the one in the XIP is cecompr.dll(same name) and the one in the kitchen is cecompr_nt.dll(different name) :confused:
[Windows Mobile Software Development] utak3r:
it's for the XIP :)
[Windows Mobile Software Development] xda2_haseeb:
Wht effects will it have?? does it make any difference?? my old cecompr.dll was running f9 on XPR and is running f9 on LZX :D so do i need to replace it with this one??
[Windows Mobile Software Development] utak3r:
If dll of yours works ok, you don't have to replace it :) It's for those who don't have such dual libraries.
[Windows Mobile Software Development] udK:
Thank you, now works fine ;)
[Windows Mobile Software Development] xda2_haseeb:
ok :D still THNX :D
[Windows Mobile Software Development] seatone:
Thanks for all of you, it's great now :)

More than 10 mb gain...
[Windows Mobile Software Development] ivanmmj:
You both ROCK!
[Windows Mobile Software Development] !Aman!:
the LZX version of cecompr.dll posted by u also worked for XPR compression on my Elfin.
[Windows Mobile Software Development] xda2_haseeb:
I guess it was for both XPR and LZX :) if im not mistaken ;)
[Windows Mobile Software Development] !Aman!:
yeah, so we don't need to use the other one posted by utak3r coz that will take a few extra KBs :D
[Windows Mobile Software Development] utak3r:
you mean - me and me? ;)

true - that mine is bigger as it's older one. I took it from one of my old ROMs for Wizard. So... which one is this smaller one? I checked from the tut and there wasn't any dual one. In which post is it?
[Windows Mobile Software Development] !Aman!:
the one posted by ivan is smaller than urs.
[Windows Mobile Software Development] utak3r:
ok, I can see it: the one titled as LZX one supports both algorithms:

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17

00003198 06 00 02 00 01 00 00 00 43 45 43 4F 4D 50 52 2E 64 6C 6C 00 4C 5A 58 5F ........CECOMPR.dll.LZX_
000031B0 44 65 63 6F 6D 70 72 65 73 73 43 6C 6F 73 65 00 4C 5A 58 5F 44 65 63 6F DecompressClose.LZX_Deco
000031C8 6D 70 72 65 73 73 44 65 63 6F 64 65 00 4C 5A 58 5F 44 65 63 6F 6D 70 72 mpressDecode.LZX_Decompr
000031E0 65 73 73 4F 70 65 6E 00 58 50 52 5F 44 65 63 6F 6D 70 72 65 73 73 43 6C essOpen.XPR_DecompressCl
000031F8 6F 73 65 00 58 50 52 5F 44 65 63 6F 6D 70 72 65 73 73 44 65 63 6F 64 65 ose.XPR_DecompressDecode
00003210 00 58 50 52 5F 44 65 63 6F 6D 70 72 65 73 73 4F 70 65 6E 00 00 00 00 00 .XPR_DecompressOpen.....
[Windows Mobile Software Development] ivanmmj:
LOL. That's what I get for not paying attention. I didn't realize that mine supported both, either. lol
[Windows Mobile Software Development] spocky12:
May anyone be kind enough to post the small xpr+lzx version with imageinfo.txt, please ?
[Windows Mobile Software Development] evertec:
Ok, I'm a bit of a newb on this whole compression thing. I've edited the LZX string and the rom cooked fine without any errors, but I'm not sure where to put the cecompr.dll file. I put it in the rom/XIP folder but it's not booting? Is this wrong? Where do I need to put it?
[Windows Mobile Software Development] utak3r:
you have to overwrite your original one... it can be various locations, depends on your kitchen. It could be ROM\XIP - but only if you have it there already...
[Windows Mobile Software Development] evertec:
I've searched throughout the whole kitchen and there is no existing cecompr.dll file. All I have are the cecompr_net.dll and CECompressv4.dll in the tools folder. Any ideas?
[Windows Mobile Software Development] seatone:
Did u try in the oemxipkernel?
[Windows Mobile Software Development] evertec:
I don't have oemxipkernel either
[Windows Mobile Software Development] seatone:
what kinda kitchen do u use?
[Windows Mobile Software Development] evertec:
Calkulin's diamond kitchen
[Windows Mobile Software Development] seatone:
cdma, sorry I'm out...
[Windows Mobile Software Development] utak3r:
well... so you have to manually get your xip.bin out of your os.nb, dump it, replace cecompr.dll and put it back.

Find some tutorial about porting XIP - it will contain all the required info.
[Windows Mobile Software Development] evertec:
Ah, thanks... that must be the step I was missing. I'll look into that.
[Windows Mobile Software Development] seatone:
Answered at ppcgeeks...
[Windows Mobile Software Development] Ameet:
And this is suggested in a thread that starts with the XIP porting tutorial :rolleyes: :D
[Windows Mobile Software Development] utak3r:
yeah, I spot it later :D
[Windows Mobile Software Development] evertec:
Ok, I read the tutorial at the beginning of this thread and tried to go through the process. I extracted the xip.bin from the os.nb.payload, dumped it using xipport, clicked make pkgs. Then, I replaced the cecompr.dll and tried to write maps after realloc P. Xipport then gives an error of "An entry with the same key already exists" Any idea what's going wrong? Thanks again
[Windows Mobile Software Development] Ameet:
That means there is an overlap in the addresses. Check each module's imageinfo.txt to know which has an overlapping address. There's actually many ways to do it easily but you need to check it out with the way you feel comfortable
[Windows Mobile Software Development] evertec:
What is one of those easy ways? :-)
[Windows Mobile Software Development] Ameet:
My way normally is,

dump a working XIP.bin

run XIPPort tool, do P aloc, write MAPS.txt

copy the new module to the folder

change the value in new imageinfo.txt as per working XIP

then check the MAPS.txt again with XIPPort tool and use M'reloc

;)
[Windows Mobile Software Development] CRACING:
Has any faced problems with graphic in LZX compressed ROM?
[Windows Mobile Software Development] xda2_haseeb:
No, they are running f9 n gr8 :D Let me guess?? ur facing it right??
[Windows Mobile Software Development] spocky12:
Thanks in advance.
[Windows Mobile Software Development] mbarvian:
do we need to do this step on the "mythical" cecompr.dll?

thanks
[Windows Mobile Software Development] ivanmmj:
Yes.

The cecompr.dll is what tells the XIP how to read the LZX compressed imgfs.bin. The hex editing is what tells the kitchen tools to write the imgfs.bin in LZX compression.

I worked with evirus and now his kitchen has an option for LZX compression. It will do everything automatically. ^_^
[Windows Mobile Software Development] CRACING:
yeah something like that, I'm facing this problem rarely. Might be bad flash, I should reflashing and see.
[Windows Mobile Software Development] utak3r:
in other words - it defines, which of the algorithms will be used to prepare an imgfs.
[Windows Mobile Software Development] béotien:
Hello , I' m a newbie and an Omnia owner . I have read the whole thread ( several times ) ; I managed , using Ameet 's tutorial , to do my very first XIP porting from stock ( 20270 ) to 21198 build ; thanks to all who provide us all these infos and tweakings ( sorry for my poor english )

I would like to have a bit more answers before flashing it : Stevenh says when extracting OS.nb , first section is IPL , second is XIP.bin and the third imgfs.bin ; and as far as I understood , there are in IPL , somes hardcoded values for Ram allocation

1) In my map.txt , I have some " wasted " place in initialized data xxxxxxxx and in Virtual Base Adress xxxxxxxxx ; so , sorry for my noob question , but is it possible to use this " wasted " space to increase RAM ? I saw a FarmerPalmOs 's answer about it , but I am a little confused with it

2) I had to remove from XIP LsOmaClient.dll : I read that I have to put it in OEM Drivers , but I was told that I may get this module into file first : how to do it ?

3) There is an overlap in the stock XIP , whatever extracting tool you use , and this is true for three different Roms ( one WWE and two french Roms ) : someone could explain me how the device is therefore able to boot ?

Regards

Philippe

Edit : here is my map.txt
[Windows Mobile Software Development] xda2_haseeb:
Yep :D try reflashing :p
[Windows Mobile Software Development] Ameet:
There's also a section before XIP, that is the ULDR which takes up about 3 MB space, depending on every device. I have a tutorial in the 1st post to remove this from Diamond. Aman has 1 for removing it from Elf(In) :)

This wasted space can surely be used (although will not make huge difference to available space) You will need to calculate the address of all the modules to use this space, the way you would do manual porting ;)

Use a tool called "recmod" I found it in Tadzio's IMGFS tools thread to convert the module back to file

Never experienced overlaps in the stock XIPs so wont be much in a position to comment on this
[Windows Mobile Software Development] utak3r:
Probably you don't have to do it, try first without this step. Just copy a whole module to OEM\OEMDrivers.

And as a converter from module to a file I'd recommend nueRecMod, which is better from the old recmod (it produces better headers).

I have seen on some devices such overlaps.... the funny thing is that after fixing them the ROM couldn't boot ;) so it seems it has to be in such situation :eek:
[Windows Mobile Software Development] béotien:
Thanks for your quick replies ; I will try to begin the reallocation from stock to increase available space and do a search for nueRecMode :)
[Windows Mobile Software Development] utak3r:
http://www.nuerom.com/BlogEngine/page/nueRecMod.aspx
[Windows Mobile Software Development] spocky12:
Hi guys, could anybody here answer my request please ? (If you managed to get 12kb version to work, just dump your resulting xip with xipport and that'll be done).
[Windows Mobile Software Development] Ameet:
Hi spocky. So sorry, missed out your earlier post :o

Here's what I'm using, maybe this is the one
[Windows Mobile Software Development] spocky12:
Exactly what I needed. Thank you Ameet.
[Windows Mobile Software Development] Ameet:
Someone donated $1 to me after a long long time :rolleyes: :D

Thanks Nazaliyah, whoever it is :)

LZX Tutorial post added here: http://xdaforums.com/showpost.php?p=3720172&postcount=9 :) Thanks a lot guys for all the support and knowledge sharing :)
[Windows Mobile Software Development] ivanmmj:
I'm still a little confused about removing the ULDR.

I'm gonna give it one more try and then come here and ask for help if I can't figure it out. (Working on a Herald.)
[Windows Mobile Software Development] Ameet:
No worries brother :)
[Windows Mobile Software Development] ivanmmj:
OK, I don't get it... It might be a little over my head at the moment (or it might be that I'm at work while doing this...) but I still don't understand where I'm getting the values for removing the ULDR from.

Here's my payload, stripped of everything. (Even most of the XIP which is why it's compressed so small.)
[Windows Mobile Software Development] utak3r:
you just have to take it easy, no rush... I succeeded at the second try, after taking a break from the first time ;)
[Windows Mobile Software Development] Ameet:
True it needs a lot of patience :D

Will be able to check and reply on this tomorrow since I'm travelling currently :)
[Windows Mobile Software Development] tonio94:
Hi everybody,

I finally decided to try this full xip porting and I would need some help :

I think I got the full process (dumping xip, make packages, replace MSXIPKernel with new xip's packages, undo, reallocating overlapped modules/files, rebuild, insert into payload), the problem is that the last point seems to fail and lead to a "IMGFS Signature not found" message when rebuilding the ROM.

I did some tests to try to check where it comes from and I noticed some strange stuff :

1) I took, my original payload, extract xip, reinsert it (with xipport or xipporterex, the result is the same), try to cook... Same error (No IMGFS Signature)

2) If I dump the xip, it only contains 3700Ko of files even if it's origina size is 4608Ko.

3) If I extract the xip again from the upped payload, it's size is not the same as the first time (the first one extracted/reinserted was 4608Ko and the second one extracted is only 2908Ko).

Any idea what I'm doing wrong ?
[Windows Mobile Software Development] utak3r:
the XIP's size may differ (to be exact it can be SMALLER) - as it doesn't always occupy the whole space (the rest are just 0xFFs). And in such situation the problem is inserting it back: it depends on your device... in older devices the rest of space should be filled with those 0xFFs, you can't move the IMGFS back, as most of nowadays tools do.

So - at what exactly command you get this error message? We'll find another way of reinserting it back then :)
[Windows Mobile Software Development] tonio94:
I've got the error message when dumping the imgfs from the modified payload with

..\TOOLS\ImgfsFromNb OS.nb.payload imgfs.bin

I'm working on a Diamond btw.
[Windows Mobile Software Development] utak3r:
ok... try this procedure:

1. dump xip.bin:

osnbtool -d os.nb.payload 1 xip.bin

2. work with XIPPort or whatever else

3. insert it back:

osnbtool -c os.nb.payload 1 xip.bin

it *should* work... right now it seems you are destryoing your partitions layout while reinserting it back into payload.
[Windows Mobile Software Development] tonio94:
Unfortunatly it seems to fail too

I tried it with a Stock Rom (RUU_Diamond_HTC_Europe_2.03.401.2_Radio_Signed_Diamond_52.51.25.26_1.09.25.23_Ship.exe), just dumping and extracting back the xip.
[Windows Mobile Software Development] utak3r:
1. The first red message is NOT an error, just an information that XIP is not compressed ;)

2. and more - you did type it wrong ;) not the pipe sign "|" but the one digit "1" :D it means: "extract/insert SECOND (it's zero-based) partition from the image".
[Windows Mobile Software Development] itje:
+1 on that
[Windows Mobile Software Development] Noonski:
Out of My League , This XIP porting.

You must have red eyes and all your nerves shot.

Good job Pit bull.

And you know how i appreciate, when Stuff is open and documented.

Oh yeah i forgot that's what we're supposed to do.
[Windows Mobile Software Development] hellonasty:
:D
[Windows Mobile Software Development] Ameet:
Next stop - more free RAM station :D Classes commence immediately ;)
[Windows Mobile Software Development] !Aman!:
+ 1 :) ...
[Windows Mobile Software Development] cruzzmz:
+1 on that too :D
[Windows Mobile Software Development] cruzzmz:
AB

i am still confused wz all the free ram thingy ... so if i have this

what actually i can do? & how?

regards
[Windows Mobile Software Development] Ameet:
80413000 - 80413000 L00000000 Start: start of RAM
80413000 - 80414000 L00001000 initialized data of region_1 giisr.dll
80414000 - 80420000 L0000c000 NUL
80420000 - 80426000 L00006000 uninitialized data of region_2 nk.exe
80426000 - 80449000 L00023000 initialized data of region_3 nk.exe
[COLOR=red]80449000[/COLOR] - 80449000 L00000000 ------ [COLOR=blue]start of RAM free space[/COLOR]
[COLOR=blue]80449000 - 83c00000 L037b7000 NUL[/COLOR]
83c00000 - 83c00000 L00000000 End: end of RAM

The blue space is what I mean by the free RAM space. More the better ;) You may start increasing that by reallocing the nk.exe to go up (keeping in mind the addresses to be divisible by 1000 :)) and then using the value of your new address (red) for ulRAMFree in ROMHDR.txt

In my previous post I have posted my new MAP.txt & dump_memoryMaps.txt to show the modded dllfirst, dlllast & rest all the addresses co-relating to the imgfs VM values. But thats a deeper study yet to be actually tested ;) So for now, you may start playing around with your free RAM space
[Windows Mobile Software Development] DupinBJK:
Where do you modify those values ? In ROMHDR.txt ? And then using realloc on XipPort ?

You can get those values from XIP.bin, the ROMHDR struct :

typedef struct ROMHDR {
ULONG dllfirst;
ULONG dlllast;
ULONG physfirst;
ULONG physlast;
ULONG nummods;
ULONG ulRAMStart;
ULONG ulRAMFree;
ULONG ulRAMEnd;
ULONG ulCopyEntries;
ULONG ulCopyOffset;
ULONG ulProfileLen;
ULONG ulProfileOffset;
ULONG numfiles;
ULONG ulKernelFlags;
ULONG ulFSRamPercent;
ULONG ulDrivglobStart;
ULONG ulDrivglobLen;
USHORT usCPUType;
USHORT usMiscFlags;
PVOID pExtensions;
ULONG ulTrackingStart;
ULONG ulTrackingLen;
} ROMHDR;

starts at offset 0x1AB158 in my .bin. If you wish, you can send me your xip_out to check...
[Windows Mobile Software Development] DupinBJK:
The pointer to the structure is at 0x00000048, it should be struct ROMChain_t

which points to ROMHDR. That way, it should be possible to find it no matter where you've extracted xip.bin from. The only thing is that I'm not sure if xip.bin has to pass a checksum verification of sort or not. I should disassemble xipport, to be sure of that - with cmonex permission of course ;)
[Windows Mobile Software Development] evalu8t0r:
I got tried of writing down numbers so I wrote a little app to do it for me, maybe it's usefull for someone else...

EDIT: App taken down for maintenance... well... it didn't work as is should :o)
[Windows Mobile Software Development] DupinBJK:
Basic FREE Ram Increase

Well, let's see an example. Here is my map:

80306000 - 80306000 L00000000 Start: start of RAM
[COLOR="SeaGreen"][B]80306000 - 80310000 L0000a000 NUL[/B][/COLOR]
80310000 - 80316000 L0000[COLOR="Red"][B]6000[/B][/COLOR] uninitialized data of region_1 nk.exe
80316000 - 8033b000 L00025000 initialized data of region_2 nk.exe
8033b000 - 8033b000 L00000000 ------ start of RAM free space
8033b000 - 83400000 L030c5000 NUL
83400000 - 83400000 L00000000 End: end of RAM

I have to get rid of the green NUL range, by stacking up the data of region_1 and region_2 of nk.exe.

1] To stack them up, I have to use m'reloc (m'reloc_nk in this particular case -

take note

) and then edit the corresponding imageinfo.txt

In m'reloc_nk the value to be changed is o32_realaddr. It will show 80310000 and it will need to be changed to 80306000.

the entries in imageinfo.txt that need to be edited are these:

o32[1].o32_vsize: 00006000
o32[1].o32_rva: 0006E000
o32[1].o32_psize: 00000000
o32[1].o32_dataptr: 00000000
o32[1].o32_realaddr: R=[COLOR="Blue"][B]80310000[/B][/COLOR]
o32[1].o32_flags: C0000080

o32[2].o32_vsize: 00024A0C
o32[2].o32_rva: 00074000
o32[2].o32_psize: 00000A00
o32[2].o32_dataptr: P+00074208
o32[2].o32_realaddr: R=[COLOR="blue"][B]80316000[/B][/COLOR]
o32[2].o32_flags: C0000040

They have to be changed to 80306000 and 8030c000, respectively.

(0x80306000 + 0x6000 = 0x8030c000 : New address of "uninitialized data of region_1 nk.exe" + Size of "uninitialized data of region_1 nk.exe" = New Address of "initialized data of region_2 nk.exe").

2] XipPort must be used to realloc, ending with this resulting map:

80306000 - 80306000 L00000000 Start: start of RAM
80306000 - 8030c000 L00006000 uninitialized data of region_1 nk.exe
8030c000 - 80331000 L00025000 initialized data of region_2 nk.exe
[COLOR="Blue"][B]80331000[/B][/COLOR] - 8033b000 L0000a000 NUL
[COLOR="Red"][B]8033b000[/B][/COLOR] - 8033b000 L00000000 [B]------ start of RAM free space[/B]
8033b000 - 83400000 L030c5000 NUL
83400000 - 83400000 L00000000 End: end of RAM

the final thing to reloc will be the start of RAM free space, in red.

3] To reloc the start of RAM free space, I have to modify the ROMHDR.txt file:

[COLOR="Silver"]ulRAMStart: R=80306000[/COLOR]
ulRAMFree: [COLOR="red"][B]8033B000[/B][/COLOR]
[COLOR="silver"]ulRAMEnd: 83400000[/COLOR]

the value to be changed has to be that of ulRAMFree, from 8033B000 to 80331000

4] XipPort must be used to realloc, ending with this resulting map:

80306000 - 80306000 L00000000 Start: start of RAM
80306000 - 8030c000 L00006000 uninitialized data of region_1 nk.exe
8030c000 - 80331000 L00025000 initialized data of region_2 nk.exe
[COLOR="Blue"][B]80331000[/B][/COLOR] - 80331000 L00000000 [B]------ start of RAM free space[/B]
80331000 - 83400000 L030cf000 NUL
83400000 - 83400000 L00000000 End: end of RAM

and 40kb of additional memory :D

Not too much, but it's just to explain how to do it.

BJK
[Windows Mobile Software Development] vivi:
ababrekar,

THANKSSS.

This is one of the greatest posts in xda-dev I've found.

vivi