yashsingh1/xdaforums huggingface dataset:
[Windows Mobile] CRACING:
I don't know about smartphones coz I never played with it. :p

Well, how big that single DSM and RGU be? It is also very necessary for the system to work. Expertise here will tell you deeply the need of this single DSM and RGU. What I have seen is, all the files and modules contained in ROM are written in this single DSM include there flags, when packages are merged. No idea about that RGU but its also contains a necessary thing.

BTW; if you are looking to save space, I have a suggestion. Use png crush tool to compress png files. It will save 15 - 20%, with lose in graphics and works great. I'm using it since beginning. Not sure you know that already. Just a tip. :)

Thanks...

Best Regards
[Windows Mobile] -=Barin=-:
Forget it. Shadows can be used while building packages.sof but not in OSBuilder. OSBuilder uses packages.sof (you can edit it manually) and the ONLY packages.sof defines registry processing order. Provxml execution order is also defined by packages.sof
[Windows Mobile] tobbbie:
Ah! - more insight - and finally a reason to potentially drop the DSM and RGU completely (or emtpy them potentially) in ROM. If it is just the packages.sof that is needed later for defining the provxml sequence then the rest are not needed on the device to run the ROM.

I know it is tweaking at an edge where benefit is small, but still it is good to understand how things are connected and what is needed where (during build or later on the device). Reverse process of ROM dump is also a purpose that I support, so I may neither use the "strip relocations" nor the "merge packages" but getting rid of the FWUpdate is really useful if you also drop the USPL (which most are doing anyway).

---------- Post added at 08:05 PM ---------- Previous post was at 07:42 PM ----------

just get it from your folders - 2 MB (uncompressed) are easily piled up for dedicated DSM + RGU. This shrinks compressed but it may still leave a 200-300k footprint. Not a big deal if your Flash has 256 MB but for 64 MB shared between ROM and Onboard storage it may leave you 1 MB less of the latter if you cross the 1 MB with your setup. And as usual - all the small steps count to a bigger one. For the devices I use, the ROM size is allocated in 1 MB steps to either ROM or storage, so the size above could give you additional applications cooked in if desired.

RGU are the registry updates to the standard hives for that dedicated package, useful as backup like FarmerTed mentioned for later reference. This is however done during ROM build ("compiling registry..." or similar) and is not needed later anymore for the device to operate. DSM are more tricky, but also not needed later on the device as far I understand. Different OS versions may treat that differently (basically depending on the FWUpdate package mandatory dll linked from coldinit.exe) - so removal is potentially dangerous.

Both would definitely be needed if the whole workflow of Package Update that the referenced Da_G post explained would work - but it does not. So the only (and important!) purpose is to allow a ROM dump.

Well I did a lot of tweaking in my initial ROM for the Tornado, mainly UPX-ing exe and DLL where possible - also some PNG and JPG was worked on (but there are hardly any) and even some WAV. However if that was really useful is another story. I just wanted to learn how small I could get that ROM :-)

---------- Post added at 08:25 PM ---------- Previous post was at 08:05 PM ----------

I noticed that my gwes.exe already has the section flag "08000000 - IMAGE_SCN_MEM_NOT_PAGED" set for all sections - so nothing to gain from that. When checking the modules properties for others (I did for GPSID.DLL) then the OSB option is exactly setting this flag "08000000 - IMAGE_SCN_MEM_NOT_PAGED" for all sections of the module in the list. So you may want to check the modules properties first before adding to that list. On the other hand it does not do any harm either.

One thing should be noted in that context I think: If you add all modules to the "exclude from demand paging" list - then it is useless to have a fixed paging pool left, or not?
[Windows Mobile] -=Barin=-:
packages.sof contains GUIDs of packages only, but dsms contain modules/files list. Thats the reason why we cannot remove dsms completely. If we do some programs would not start.

And one more thing... This topic IS NOT DEDICATED TO THE COMMON THEORY OF ROM COOKING

If we start to discuss common questions or internal OSBuilder methods the topic will grow up to tons of posts.
[Windows Mobile] ultrashot:
well, some modules may have this flag but it doesn't mean all builds have this module unpaged.

I usually set it myself, so most of my x1 drivers have this flag set on all sections and have "M" in dsm.

page pool is used not only by modules.
[Windows Mobile] tobbbie:
...oh well - yes. The following will get off topic for OSB and I moved my post to here:

http://xdaforums.com/showthread.php?p=17694590

---------- Post added at 11:09 PM ---------- Previous post was at 10:34 PM ----------

Oh well - I have gone too far again :-)

Where else could I ask such things?! Should there be a dedicated OSB internals thread or better as you suggest "Common Theory of ROM Cooking"? I am not sure how many would be able to follow the discussion (I would probably not be) and also how many are still interested in doing any cooking. The community caring about old WinMo stuff seems to shrink day by day and all wisdom maybe buried some time - at least for things before WM7.
[Windows Mobile] tobbbie:
Merge packages does more than merge...

As I found out just now, the "merge packages" option does more than just what it says. It will merge the packages (as announced) but will also delete all the RGU and create new packages to hold only the Provxml for later execution on the device. This can easily be seen from the MergeDSMLog.txt and the OSB userguide (WWE) where I could have looked earlier. Sorry for entertaining you with my speculations.

This may be in-line with the platformbuilder behavior before but it is new to me as a smartphone user. So OSB does the maximum saving regarding this option. It seems such a prominent option that it even deserves an own toolbar button which depicts the relevant settings entry.
[Windows Mobile] tobbbie:
XIP optimization - removing modules?

Tempted to get more RAM I wonder about the removal of modules from XIP. There are posts regarding this, e.g. the full manual XIP porting thread here and also on mobileunderground here but it seems that some are safe to remove and others are not.

I wonder especially about the cachefilt and the diskcache parts. While others may serve dedicated functions you may accept to drop (e.g. encryption or debugging) I am reluctant to remove basic things like above. I could not find documentation on these OSB options, probably depicting the pros and cons for the modules listed with an own checkbox.
[Windows Mobile] sh4d0w86:
if i use OS Builder v1.3.163 to dump an HD7 WP7 ROM, how would i rebuild it?

and yes, i do understand that it can't be rebuilt using OS Builder, right now.

thanks,

sh4d0w86.
[Windows Mobile] HD2Owner:
well... for now, you can use imgfdfromdump.exe and imgfs2part to rebuild.

just stay tuned.....
[Windows Mobile] sh4d0w86:
well, i have been using ImgfsFromDump to build my intrinsicROM...

but can't figure out how to use it with a Dump from OS Builder.

it fails because it lacks imageinfo.bin.

is this where Imgfs2Part comes in? haven't heard of this .EXE.

edit. well, i feel dumb...

just released there are two separate ways to dump.

sh4d0w86.
[Windows Mobile] Farmer Ted:
There are some useless modules in there (can't think of them off the top of my head, as I don't have my kitchen with me), as well as the splash screen/warning, that is pretty big (I think it's an nb0 file). But I would recommend leaving things like cachefilt in there. You will get much faster benchmarking with out cachefilt, but in my experience my device is slower at routine tasks, and also buggy with cachefilt removed. I've tested w/o it for several months, and I remember the last time that I switched back to including it, it was really, really obvious how much snappier my device was, despite the benchmarking (for file system read/write). I'm pretty sure that when you remove it, it just jacks around with how the benchmark tests are done and makes them faster without helping the real device performance.
[Windows Mobile] AndrewSh:
Farmer Ted, absolutely right conclusion - concerning cachefilt. Benchmarks (read, write) are incredibly high without it. But it absolutely does not affect on the real device productivity. Besides - I managed to get the same read/write scores with cachefilt included.

Also with that lib in xip device deals better with free program memory amount.
[Windows Mobile] ultrashot:
Andrew is talking about this tweak:

[HKEY_LOCAL_MACHINE\System\StorageManager\CacheFilt]
"CacheVMSize"=dword:00400000

(that's almost the best value for HD2).

CacheFilt also makes a hashmap of filenames in \Windows\ folder, so enumeration might be better.
[Windows Mobile] Stephen:
I currently us oskitchen to build roms. I would be very keen to move accross to osbuilder. How difficult is it to transfer my packages from oskitchen to osbuilder?
[Windows Mobile] Farmer Ted:
Jeez, I've been looking for tweaks for cachevmsize for a long time. I never even knew what the bloody range was for it. I'm going to have to play around with it. So, what is that? Around 4 MB? I thought the stock setting was 32 MB.

It's really not that bad. OSKitchen has dsm's included, right? You basically need to move all files out of the sub-directories of each package, then rename the app.reg files to {guid}.rgu, using the {guid} of the dsm. The packages are all flat (i.e., you don't have sub-folders for Files and lang crap). The structure is the same as the OEM packages. And you take your EXT's and just dump them in the OEM directory. You also need to edit the packages.cif/.sof, to set up the order of module processing and registry building. Then you need to bang your head against the wall for 2-3 days to work out all the bugs, like converting pretty much all the files used in the rom build to unicode.
[Windows Mobile] CRACING:
Here you go -

Tools you need:

Ext to OEM by JooJooBee666

Unifier 5 (or any ANSI to Unicode converter)

Producer:

As the name says, Use Ext to OEM to translate oskitchen ext packages to oem and place in osbuilder oem folder. Later you can categorize into ext folder (In OEM format), if you like (*see below for more info about ext folder in osbuilder).

Secondly, use Unifier 5 to convert *.rgu and initflashfiles.txt files to Unicode format (UTF-16) since oskitchen dumps rgu's and initflashfiles.txt files in ANSI and its not supported by osbuilder. Atleast when I first entered to osbuilder. Newer version may support but not sure.

FYI; Don't convert Unicode rgu and/or initflashfiles.txt files Unicode itself in Unifier 5. If you do, you will get bad strings in start of the file (i.e %$& ).

Thirdly in osbuilder, go to packages.cif/.sof and sort the packages. 1st XIP, 2nd SYS, 3rd OEM, 4th ZZZ (Your customizations).

Fourthly, auto fix DSMs in ext and oem folders using DSM autocorrection in osbuilder.

* Ext folder in osbuilder is same as oskitchen but the packages should be in OEM format (*.*, *.rgu, *.DSM). Categorized sub folders can be possible here.

Hope this helps. I can provide you detailed guide via pm if you need. :)

Thanks...

Best Regards
[Windows Mobile] AndrewSh:
Yes it's 4*194*304. And yes - you're right about defaults :)

Here is it http://msdn.microsoft.com/en-us/library/ee490232.aspx

It's possible to play from 1 to 128 but ultrashot offers 4. I don't have enough knowledge to argue with him

:)
[Windows Mobile] ultrashot:
AndrewSh, Farmer Ted,

You both aren't right: default cache size is 512 kb, minimum is 256 kb, maximum is 10 MB.
[Windows Mobile] ai6908:
That is the key! Trying to figure out what works and what doesnt takes days! But got to love the new kitchen though...;)
[Windows Mobile] tobbbie:
Merge Packages does not honor the value "delete" operation

Just found that the merge packages to one is not properly taking the deletion of values into account when merging the rgu and later applying it. I have checked the content of the created default.hv regarding one special value that is deleted in the last package that I have sorted in the packages.sof+cif. It is still present when the "merge packages" is used and correctly deleted if this is not used. Registry preview is correctly showing that the value is not there as expected.

I had to check the created hive as the merged rgu is deleted before imgfs creation. I checked the HKLM\ObjectStore\RegistryUpdate and found that there is just one package instead of the many in case of the non-merged handling.

Operation in question is this here:

[HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Power\State\UserIdle]
"bkl1:"=-

I only found this because the backlight handling was suddenly different after using the option "merge packages". Took some time to exclude other options that may also make this appear on the device (e.g. .provxml) - but the proof is certain.
[Windows Mobile] -=Barin=-:
This can happen only if you delete parameter which is defined in boot.rgu.

I'll correct it in next version.

Difference in HKLM\ObjectStore\RegistryUpdate key is normal.
[Windows Mobile] Stephen:
Hey Cracing yes that would be really good if you could. My oskitchen has evolved a lot over the last year and is really well structured now. I would be keen to transfer accross to osbuilder but having only used the software once to dump a rom and then build as a test I am a complete noob.

If you could PM me a really really details transfer process that would be good. I would need to start from scratch, install software, easy enough, but then porting is were I am really lost!

Any help would be appreciated :)
[Windows Mobile] tobbbie:
Indeed that was set in the boot.rgu initially. I have not found a log of the registry activities and no option either to turn that on. The documentation tells:

Looking at the MergeDSM.log I see only the DSM merge, but not anything related to registry merge activities, I suspect that there also the boot.rgu has to be included at first.

Which is the recommended order in the packages.sof/cif?

Shall the XIP packages be included(yes)?

I would go for: XIP, SYS, OEM, private, each group then sorted by title to keep the MUI packages close to the basic ones for a proper list. I will also cross check this with HD2Owner's recommendation in this thread.

Regarding sof/cif I found this thread - worth reading for those that don't dare to post here. Go to post 28 and later for some more insight.

It would be good to have a tool that identifies registry branches that are touched by more than one rgu files. This would much ease to identify potential sequence problems. So many utilities are there already - did I just not find this in OSB?

Yes, I thought so - I just only noticed this way that you first merge the RGU to one new guid.rgu and then insert it - you could also have done the insertion one-by-one and then just delete all the rgu. This had probably given the same trace for both cases then.
[Windows Mobile] -=Barin=-:
:eek:

Did you try registry preview in advanced tools? It will show you everything you wish....
[Windows Mobile] tobbbie:
Of course I did - this shows the result after all merge is done. It does not show WHICH RGU finally did the change in case more than one has touched a value.

Sequence is only important if the same value is touched by more than one rgu. So I was asking for a tool that tracks/lists only these values and identifies the rgu that touches them. So you can take care about the right order.

Somewhat usable is the registry compare if you already suspect two rgu to touch one value, but the delta view also lists touched branches even if the values are not conflicting but just complement.
[Windows Mobile] -=Barin=-:
Just click on parameter - and the change sequence would be shown in the lower right window (table). Doubleclick on the row of this table and rgu file would be opened.
[Windows Mobile] tobbbie:
THANK YOU - I knew it must be there! I had clicked the right pane, but on the value and NOT the parameter - it is even in the documentation - shame on me (again) - just give me a RTFM in such cases please :-(

Testing the entries, it seems that the "Modified only" applies to my wish - is it? So the "modified" means not "just added", but only added+changed and changed+changed?

The sheer amount of such multiple changes to same parameters is shocking me, Even within the same SYS .rgu you see that happen or across different SYS packages. So knowing the shocking truth now - back to my initial question:

What is the recommended sequence of packages? I doubt that alphabetic package sorting is a wise decision, while I still believe that (boot.rgu)->XIP->SYS->OEM->private is the right one. I think I may have to check an original WM-x dump to get it right.

Sure I am not first one to ask this - is there any wisdom documented somewhere to read?

Edit: I checked the WM6 shipped Excalibur ROM and have not seen a logic in the sequence of packages. SYS, XIP and OEM go wild in sequence there - now I am as clueless as before. Need to check more shipped ROMs...

Edit2: Can I get information if a certain package will influence registry parameters that are also set by other packages? So if moving a certain package will create impact? Could I somehow identify a "whitelist" of packages that are free to move anywhere (i.e. there is no need to put them in the packages.sof)? Thinking further you could create a package matrix where cross-points are listing the parameters that both packages impact - or first just tell the number of such parameters and hovering over it show the parameters. Aside each package you could also tell how many other packages/parameters are impacted. The data should be there in your Registry preview.

Please don't tell me that THIS is also there already (I checked the manual)...
[Windows Mobile] HD2Owner:
it's simply said preceeded according to packages.sof/cif.

in registry preview, if you click on a key, on the bottom, all packages that touch that specific key are listed (afaik in correct order of building hives).

i would advise to use packages.sof/cif of original rom and add new stuff you add to the kitchen to the bottom of the lists.

if you are porting a new sys, check a stock roms packages.sof/cif order to place new sys packages in the right order.

if there is no stock rom for your device with your desired sys build,

you can't talk your way out of manually comparing and adjusting/sorting. :p

you could open two instances of osb and two instances of packages.xxx editor to compare order of different roms.
[Windows Mobile] caliban2:
don't make it too complicated, you already got it. :)

attached see my order of packages.sof: ROM - SYS - OEM - EXT

never had a problem with it like this.

as you can see it's alphabetically ordered. i put my own stuff at the very end of EXT, so that reg-keys don't get overwritten. (keep in mind that provxml's are processed during first boot, use the reg-preview to check)
[Windows Mobile] Farmer Ted:
Stick with the stock order for sys; if you mess around, you can end up with tiny system fonts. I dunno why microsoft does it, but they put some of the font registry keys in two different packages, with different values. One set of values is awful (it's in base/lang, or somewhere like that).
[Windows Mobile] -=Barin=-:
@tobbie:

One more tip for registry preview...

Osbuilder supports line numbers for registry editor. Editor must also support line number as command line argument.

For example if you use notepad2 as rgu editor set the command for it as

"C:\Program Files (x86)\Notepad2\Notepad2.exe" /g %2 "%1"

If you use notepad++ set the command as

"C:\Program Files (x86)\Notepad++\notepad++.exe" -n%2 "%1"

%1 will be replaced automatically with rgu file name

%2 will be replaced automatically with line number

If you get an error while registry preview you can open rgu file and cursor will be placed directly to line which caused an error.

It also works for table of changes in ComplexRGU
[Windows Mobile] tobbbie:
After seeking for some external tools to deal with sorting/cleaning of the RGU I found that OSB has that also built in. Looking at OndraSters tool and description it seems you are doing just the same. It may be handy some time to get the old RGU back for reference (due to comments getting lost in the registry import-export cycle). I wonder if it could be useful to have the rgu be backed up (to *.rgu.bak) and have the *.bak added to the exclusion list.

I helped myself by doing a backup with a zip recursing to all directories and storing the original rgu along with their path - which may be a toolable alternative if you want to keep the package folders clean and tidy.

The result was not so overwhelming - still a lot of redefinitions visible in the Registry preview from several packages.

I wonder why there is nothing visible of the rgu merge to registry in the build log or extra log file - just like you list actions when doing the registry preview.

Another thing handy for the build log would be to see the activities of the steps

- imgfsbuilder

- ROMBuilder

in either separate logs (like the MergeDSMLog) or the BuildLog.

Especially when tweaking builds the ROMBuilder output could be handy to compare results. Or did I miss again to read something in the docs or the settings?!?
[Windows Mobile] tobbbie:
I noticed that the initflashfiles merge is not giving a warning when invalid data formats have been used to store the initflashfiles.txt. The initflashfiles.dat preview gives a good idea where to fix, but you need to get the idea first to actually look there. I missed one .lnk copied to the startmenu and found 3 others not stored properly as well.

As a side note - and good night joke:

I am about to finalize my Typhoon ROM cooking with OSB (yes 64MB ROM and 32(!) MB RAM) - it is possible with some tricks in preparation of the kitchen (5 modules cannot be relocated). Additional options of OSB have given me another ~1MB of ROM space (and 256k RAM). The ROM image is limited to 27MB but now there is even JAVA inside (along with .NetCF 3.5 and other smaller tools). Will publish ROM and Kitchen soon in the Typhoon section - not sure who cares, but I made it finally :-)

Tornado already works well (as expected) and Hurricane will follow (with 18 modules that cannot be relocated).
[Windows Mobile] Farmer Ted:
I'm not sure what the deal is, but I can't compile my registry with this entry. The bugger gives me an error at that line in the boot.rgu every time. Anyone got a clue what the deal is? If I comment out the line, it builds fine.
[Windows Mobile] -=Barin=-:
May be it's just a syntax error.

Is it the last line in boot.rgu? If it is add empty line after it...

P.S. This parameter works fine for me
[Windows Mobile] Stephen:
Ok,

I have managed to get my base rom setup in osBuilder and compiling and working ok!! I use base rom Leo 1.66 WWE as I dont like the 576mb hack and I use all the ext packages from leo 3.14 WWE. Two things I want to do now.

Fist I want to update the os from 21869 to the latest 6.5.0 OS - could somebody tell me which is the latest and how to import it to osBuilder?

Second. I want to start using XPH compression. I am not sure how to check that I have it!

See screen shot below. XPH not supported? Im using the latest version of osBuilder.

View attachment XPH.bmp
[Windows Mobile] HD2Owner:
dude... stop it right now...! :mad:

don't start to flood this thread specifically with your personal cooking history... :eek:

only ask technical questions about osbuilder here! no general cooking talk!

if you have questions about xph, ask in xph compression lib thread by ultrashot.

also it's not even a question at all to ask. download my kitchen and have a look.

it's all written down in tutorial, initial posts and whole thread..

read, try, read, try... how we all learned to cook..
[Windows Mobile] + Que PPC:
mmmmm interesting :cool: