yashsingh1/xdaforums huggingface dataset:
[Windows Mobile Apps and Games] ShaDrac:
:eek: :eek: :eek: :confused: :( ...

...

...

Thanks for testing.

btw. if you replace the app.dat with an mxipupdate_etc_etc.provxml, do the shortcuts get where they should be?

Thanks,

ShaDrac
[Windows Mobile Apps and Games] twopumpchump:
i havent tried making a new provxml for those two shortcuts themselves, but file explorer is installed by provxml and that is in kitchen. i will try making a new provxml just for those two shortcuts and see if that will work.
[Windows Mobile Apps and Games] RoryB:
My app.dat files for copying shortcuts work. It is the files in the zip. So we are still having different problems. I wish I could test latest isolate for you, but I cannot swing it until tomorrow evening. Meetings all day and just enough time this evening for a quick check.
[Windows Mobile Apps and Games] zenkinz:
I resolve it by changing the order of the extendir to be after cachefilt. Although this works, but it creates a performance hit as a consequence, so I clean up the boot.rgu and XIP further (while keeping the order of cachefilt & extendir), removing away some files in XIP, remove unnecessary filters, and keep cachefilt only to sdmemory.

eventually I got a benchmark that's close to where I was before (before resolving the bug). The only bugging performance hit is directory list (28 ms instead of 18ms previously) and internal database read (300-400ms instead of 195ms previously)

I wonder if that's the expected result of performance for changing the order of cachefilt over extendir?
[Windows Mobile Apps and Games] bgcngm:
ShaDrac, I was following your tricks and trying to tweak my boot.rgu.

Normally my boot.rgu looks like this:

[HKEY_LOCAL_MACHINE\System\StorageManager\AutoLoad\NANDFLASH]
"DriverPath"="Drivers\\BuiltIn\\NANDFLASH"
"LoadFlags"=dword:1
"Order"=dword:0
"MountAsRoot"=dword:1
"MountAsBootable"=dword:1
"BootPhase"=dword:1
[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\NANDFLASH]
"DefaultFileSystem"="IMGFS"
"PartitionDriver"="mspart.dll"
"Name"="Microsoft Flash Disk"
"Folder"="Storage"
[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\NANDFLASH\IMGFS]
"MountHidden"=dword:1
"MountAsROM"=dword:1
"XIP"=dword:0
[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\NANDFLASH\FATFS]
"Flags"=dword:14
"FormatTfat"=dword:1
"CheckForFormat"=dword:1
"EnableWriteBack"=dword:1
"MountAsBootable"=dword:1
"DataCacheSize"=dword:00000400
"FatCacheSize"=dword:00000100
"MountAsRoot"=dword:1
[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\NANDFLASH\FATFS\Filters\fsreplxfilt]
"Dll"="fsreplxfilt.dll"
"Order"=dword:1 ; must be loaded before other filters

I tried to remove Autoload, so it was changed over to this:

[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\NANDFLASH]
"DriverPath"="Drivers\\BuiltIn\\NANDFLASH"
"LoadFlags"=dword:1
"Order"=dword:0
"BootPhase"=dword:1
[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\NANDFLASH\IMGFS]
"MountHidden"=dword:1
"MountAsROM"=dword:1
"XIP"=dword:0
[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\NANDFLASH\FATFS]
"Flags"=dword:14
"FormatTfat"=dword:1
"CheckForFormat"=dword:1
"EnableWriteBack"=dword:1
"MountAsBootable"=dword:1
"DataCacheSize"=dword:00000400
"FatCacheSize"=dword:00000100
"MountAsRoot"=dword:1
[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\NANDFLASH\FATFS\Filters\fsreplxfilt]
"Dll"="fsreplxfilt.dll"
"Order"=dword:1 ; must be loaded before other filters

Device boots fine (even faster than before!). The only problem that I've noticed is that for some reason Phone 2 radio firmware (which resides in ExtROM - hidden to the user, although) can't be found. I have an ETEN DX900 dual SIM, by the way. Do you know if any additional tweak is needed? Don't I have any other option but to have this driver autoloaded?

I'm only trying this because I wanted to be able to set bootphase to 0 (and place WinCE.nls under SYS).
[Windows Mobile Apps and Games] l_evo:
Yes,after i change the value of "BootPhase" form 0 to 1,everything seems be OK now,both 1st boot and reboot with no problem,and i can use Preset Installer.

btw:i'm use X1 with wm6.5

This is my boot.rgu
[Windows Mobile Apps and Games] stepw:
Very nice, great job ShaDrac!

Out of curiosity, what kind of optimizations did you implement in extendir driver? I'm taking it's doing more than simply handling symlinks to files within virtual FS, correct?

My dream is to see applications portable across devices/ROMs through the use of virtual file system and virtual registry. Hard resetting a device will not require reinstallation of software then. There's no such thing as registry filter driver, but there are other ways to hook registry access.

Is there any particular reason underlying file system is used as a container for extendir's virtual FS? Wouldn't memory-mapped FS image containers (like FreeOTFE is using for instance) be even faster? Single file containers may even be portable across devices, registry changes should only be required to deploy an application stored in extendir container.
[Windows Mobile Apps and Games] ShaDrac:
Extendir:

I've managed to get my device (Diamond) also hanging after soft reset!!! :cool:

It was after I added the packages ConnectionSetup & ConnectionSetupDownload!

After I removed those two everything was working fine again! :)

Can you guys check if this is also the problem on the Blackstone?
[Windows Mobile Apps and Games] bobsbbq:
Oh heck, I missed that I thought it said to change the order to 1, I will try it now with bootphase changed to 1.
[Windows Mobile Apps and Games] bobsbbq:
Ok with changing the bootphase to 1 it is now booting fine. Oh thank you.
[Windows Mobile Apps and Games] ShaDrac:
Hi stepw,

You've got really great ideas and not impossible to!

Maybe nice in the near future to brainstorm about this.

But, with memory-mapped FS image containers being faster, well not in my experience.

The whole idea behind Extendir was to create performance that is easy to implement. And to be honest, probably 70% of the performance is just good architecture.

If we want a way bigger performance increase, the memory architecture and storage architecture of windows mobile should be completely rewritten.

(but that would be the end of notebooks ;) )

ShaDrac
[Windows Mobile Apps and Games] stepw:
Ok, thanks for your follow up... Feel free to PM me if you'd like to brainstorm together :)

So in your research TFAT FS driver is faster than memory-mapped file reads/writes on WM? I guess you know better... This must be an architectural issue indeed, as it's meant to be the opposite.
[Windows Mobile Apps and Games] bobsbbq:
Great News this is now working on my Full Premium WM 6.5 ROM. The only side affect I can see at the moment is my AutoRun for GarminXT on the storage card runs after every soft reset where before it would do it only once.

Now to start working with packages
[Windows Mobile Apps and Games] ShaDrac:
This is great news indeed!

Enjoy!

btw What was the package that was a problem?
[Windows Mobile Apps and Games] Kraize:
Okay I got it to boot, but apparently extendir isn't in my rom? I placed the extendir.dll in my OEMXIPKernel and I took the provxml and dsm, put it into a folder, and converted it to EXT and put it in with the rest of my packages. What did I do wrong? :S
[Windows Mobile Apps and Games] turbotechnology:
Would be nice to see this in an installer that will work on an already flashed ROM without Extendir, kind of like no2chems battery drivers. Are the extra steps that are required preventing it?
[Windows Mobile Apps and Games] mbarvian:
you need to put the .dll in the EXT folder, not in the XIP ;)
[Windows Mobile Apps and Games] ShaDrac:
And if that doesn't work,

I've tested it with just placing the complete Extendir package in the OEM folder and modified the boot.rgu and boot.hv in the OEMXIPKernel

Good luck!
[Windows Mobile Apps and Games] ShaDrac:
Sorry that's not possible.

Extendir needs to be loaded before your boot partition is loaded.

Your boot partition is where you install programs and has also your registry on it.
[Windows Mobile Apps and Games] Kraize:
Okay. But since I have the visual kitchen, do i place it in \EXT\BlackStone\COMMON\My Packages\Extendir\files or just \Extendir? I don't know how dll's work in EXT's.
[Windows Mobile Apps and Games] ShaDrac:
Well, I'm not that familiar with this kitchen, so I also don't know how it works with EXT's but place the Extendir package you've downloaded with everything in it in the OEM folder. It should work.

It would be like this:

\OEM\Extendir
[Windows Mobile Apps and Games] biglol:
TUTO

you can made a tutorial because I don't understand how we use your programs

thank you
[Windows Mobile Apps and Games] Kraize:
Btw, Can you make your preset installer WVGA compatible? (480x800) because the background get's screwed up on my HD.
[Windows Mobile Apps and Games] SmS4SYR:
very good program

many thanks
[Windows Mobile Apps and Games] Kraize:
That worked! Thanks! Except I put it in \OEM\BlackStone\COMMON\Extendir
[Windows Mobile Apps and Games] ShaDrac:
@ RoryB & twopumpchump,

Do you cook with the cert patch?

If not, can you try it?

And can you try to cook a rom with attached package with security settings?

Thanks! :)

ShaDrac
[Windows Mobile Apps and Games] twopumpchump:
i already cook with cert patch, and i have sys security off in rom as well...but i will try yours and see what happens after work today :) also i tried last night using provxml for shortcuts and it didnt work. this morning i cooked rom without extendir and shortcuts worked fine as normal :confused:
[Windows Mobile Apps and Games] ShaDrac:
Thanks!

Same results with provxml and if your security off is about the same as mine, then I think I've got an idea. :)
[Windows Mobile Apps and Games] twopumpchump:
ok i went ahead and flashed at work lol...same results with your security off in kitchen. i think later i will unzip one of my old kitchens and see if i have the same problem adding extendir to it.
[Windows Mobile Apps and Games] zenkinz:
Shadrac,

any comment on why I must order cachefilt to be loaded, before extendir, in order for the photo tab to show properly refreshed thumbnails?

I also noticed, that if I disable cachefilt for main storage, files cut and paste from the source folder to target folder, will not be refreshed properly. the target folder will not show the pasted file unless I get out of the folder and come in again.
[Windows Mobile Apps and Games] jagan2:
Thank bro for your interest in my question.

I checked with fdc task manager and found that imgfs.dll was there when i clicked filesys.exe for details but ONDisk.dll was not there.
[Windows Mobile Apps and Games] ShaDrac:
@ RoryB & twopumpchump,

Attached another test build.

Thanks for all the testing and feedback! :)

ShaDrac
[Windows Mobile Apps and Games] ShaDrac:
Hi zenkinz,

I don't have those problems, while we've both got an HD2. :(

Could you post me your current boot.rgu and boot_ms.rgu?

Thanks!

ShaDrac
[Windows Mobile Apps and Games] ShaDrac:
Hi jagan2,

Do you also got an boot.rgu from a shipped rom?

Thanks!

ShaDrac
[Windows Mobile Apps and Games] ShaDrac:
Most of the time an driver is autoloaded for good reasons... (but not always :D)

Could you post your boot.rgu and boot_ms.rgu?

Thanks.

ShaDrac
[Windows Mobile Apps and Games] bgcngm:
Here is my boot.rgu. I usually remove boot_ms.rgu.
[Windows Mobile Apps and Games] twopumpchump:
test 16----shortcuts worked :D

so does this mean you finally found it?

edit. i noticed that my stuff in sdcard\autoinstall didnt install at first boot like normal. i wonder if this(autoinstall) could have something to do with it?
[Windows Mobile Apps and Games] ShaDrac:
Mystery solved

:) YES! :)

Thanks for all the testing!!! :)

I think your phone could use some rest now :D :D :D

@RoryB:

I really love to hear if this version also solves all your problems (since you've got more variants).

Thanks! :)

ShaDrac
[Windows Mobile Apps and Games] twopumpchump:
yeah i think me and rory should get beta tester of the week awards lol. im glad you found it, is this version(16) the complete pkg? or is something disabled in it?
[Windows Mobile Apps and Games] BlackFriar:
Haha! Definitely, I just wanted to say thanks and congrats to you guys and ShaDrac!

So, what was the problem?!?
[Windows Mobile Apps and Games] ShaDrac:
:D Yeah, I think we should open up a thread special for that "Beta tester of the week awards" :D

The next 3 years I would vote every week for the two of you! :D

Test version 16 is not complete, I intentionally left a small bug in it which maybe could fail one of the zip files of RoryB.

If it fails, or not, I know completely sure what was causing this.

Thanks! :)

The problem was again some very special (read: sneaky) undocumentated security features of Window mobile / CE.

Maybe that's why it's not documentated. :D

ShaDrac
[Windows Mobile Apps and Games] twopumpchump:
well whatever you do, dont document finding this security feature or they will add another one and im tired of flashing so much for a while :)
[Windows Mobile Apps and Games] RoryB:
Stealing a peak at the progress. I will play catch up tonight.
[Windows Mobile Apps and Games] RoryB:
Isolate 15

: No good

Isolate 16

: All previous problems gone without adding your SYS_SecurityOFF Ext package.

YAHOO!!!:D

I removed my start_menu_certificates and SYS_SecurityOFF packages because I read somewhere you did not need them if you run implantxip.exe with nocert. I did keep mxip_SecurityOff_1.provxml file.
[Windows Mobile Apps and Games] selyb:
How do you know all this stuff? You work(ed) at MS?

I mean, I'm not the dullest tool in the shed, but in your presence, I feel about this big
[Windows Mobile Apps and Games] bobsbbq:
Congratulations Shadrac, you are the man! Nice to hear that you have finally found something. I know TPC & RoryB were feeling like they were on a winding road to nowhere. Now we all feel like it is our birthday. Well actually tomorrow is my birthday. :D
[Windows Mobile Apps and Games] bobsbbq:
After having the beta out for testing it does seem to have a refresh issue. When copy and paste it will not update until you go out of the folder then go back in to it. I think this is also the reason for the shortcuts not showing after install, and you have to do a soft reset then they show.

Zenkinz has already had these problems as well. He said they were resolved I think by cachefilt.dll active on all storage. I currently have it only on Storage Card.

How about you? have you noticed this? It will be good if we can get around the cachefilt on the main storage as it is too slow.
[Windows Mobile Apps and Games] seatone:
Hi ShaDrac, :)

Have you seen this: http://xdaforums.com/showthread.php?t=647764

What do you think, can we have some advantage implementing it with your masterpiece?

Cheers,
[Windows Mobile Apps and Games] ShaDrac:
Thanks for all the testing :), you've made my day! :cool:

Hopefully later today I will have the latest Preview/Beta without cache uploaded. :)

Thanks again! :)

ShaDrac
[Windows Mobile Apps and Games] ShaDrac:
I think the problem is the combo Extendir & cachefilt, since cachefilt starts writing back after 5 seconds (default) and on the HD2 TMOUS shipped roms I even saw settings with 20 seconds (how to lose your data with a powerfailure :rolleyes:).

Easiest way to get your folder updated is to add "EnableFileCacheWriteBack"=dword:0 to the cachefilt reg entry.

Or you can change the order of Extendir so it has an higher value then the order of cachefilt.

If you only need cachefilt for the recent documents tab on your SD card, keep the normal Extendir register enties with order 0

and create a profile for Extendir, only for your SD card, with an higher order value then the order of cachefilt.

But I've got good hopes that with Extendir V2 including cache, we don't need cachefilt anymore for the recent documents tab in Sense. :)

Enjoy!

ShaDrac
[Windows Mobile Apps and Games] ShaDrac:
Hi seatone,

If the RAM disk is just a normal RAM disk, I think it should work without any problems. (Atleast I did it once with the old Extendir V1.06) :)

But after Extendir V2 get its cache.., a RAM disk would be obsolete. :D

Enjoy!

ShaDrac
[Windows Mobile Apps and Games] seatone:
This is what I thought as well ;)

Cheers :)
[Windows Mobile Apps and Games] RoryB:
BTW currently I have no cachfilt and no encfilt in my ROM. I am using old M2D ver 1917 and do not have problems.

Now that I have SPB benchmark, I am running tests on identical ROM verions of no Extendir, ver 1.06 with 2MB cache and may do ver 2 isolate 16 unless your next version comes out before I get there. I am keeping log files so I can compare to future versions too.
[Windows Mobile Apps and Games] bobsbbq:
I tried this for cachefilt and did not work:

[HKEY_LOCAL_MACHINE\System\StorageManager\FATFS\Filters\CacheFilt]

"Dll"="CacheFilt.dll"

"Order"=dword:2

"EnableFileCacheWriteBack"=dword:0

[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\SDMemory\FATFS\Filters\CacheFilt]

"Dll"="CacheFilt.dll"

"Order"=dword:2

"FileCacheWriteBackPriority256"=dword:FE

"FileCacheWriteBackTimeout"=dword:A ; 20000ms , defaule is 5000ms

I also tried with cachefilt only on Storage card and a profile for Extendir with order set to higher than Cachefilt, but it does not fix the refresh problem either.
[Windows Mobile Apps and Games] RoryB:
@bobsbbq please give me some samples of copy-paste from which folder to which folder. I will try to duplicate on my phone. I assume it can be whatever file I can find. In other words your situation is consistant no matter the file type.

I am almost finished benchmarking version 1.06 with 2MB cache and will reflash version 2 isolate 16 after a brief breakfast meeting.

I had a similar problem a long time ago when I tried to copy files to windows, or was it to a new folder in the Extendir series. I think the Extendir folder. I think the error was file already exists even though I was just then copying it new.
[Windows Mobile Apps and Games] bobsbbq:
It seems any file. I for testing have been trying to copy a mortscript file from Storage card\My Documents to Device\My Documents. But I think it is the same for any of the copy and paste functions.

With the new HTC Sense with the Document tab we have to have CacheFilt.dll at least on the Storage card
[Windows Mobile Apps and Games] RoryB:
Ok, I'll need to to cook SDCard cachfilt back in to try to match your set up. Probably will be later today. I'm waiting for benchmark to finish ver 1.06 and then I'll have to go to a meeting. I'll try your set up before I bother to benchmark ver 2.
[Windows Mobile Apps and Games] twopumpchump:
the refresh problem that you guys are having with the cachefilt reg doesnt just happen with extendir, i told a friend who doesnt use extendir last week to try the reg and he had the same thing happen to him.
[Windows Mobile Apps and Games] zenkinz:
so you know why the document tab requires cachefilt,. and you are enabling this capability in your extendir?

If so, that'll be great.

From what you have described earlier, cachefilt acts like a cache to the filesystem and applications are accessing cachefilt instead of the filesys directly?
[Windows Mobile Apps and Games] RoryB:
@bobsbbq I was setting up my kitchen for later. I noticed you have

[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\SDMemory\FATFS\Filters\CacheFilt]

is SDMemory FATFS or should it be

[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\SDMemory\Filters\CacheFilt]

similar to encfilt settings we delete to not have encription? Gotta run.
[Windows Mobile Apps and Games] bobsbbq:
Well I was trying both way with it on all storage and with it on just Storage card, this is reason for both of them.
[Windows Mobile Apps and Games] RoryB:
Okay, test results for copy file from \storage card\My Documents to \My Documents:

My current batch - file shows up immediately

Add

[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\SDMemory\FATFS\Filters\CacheFilt]
"Dll"="CacheFilt.dll"
"Order"=dword:12
"FileCacheWriteBackPriority256"=dword:FE
"FileCacheWriteBackTimeout"=dword:A ; 20000ms , default is 5000ms

- copied the file and did not see it. Tried paste again and it asked if I want to overwrite the file. So it is there, but I cannot see it either. I quit file explorer and reopened it and the file showed up.

Change to

[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\SDMemory\Filters\CacheFilt]
"Dll"="CacheFilt.dll"
"Order"=dword:12
"FileCacheWriteBackPriority256"=dword:FE
"FileCacheWriteBackTimeout"=dword:A ; 20000ms , default is 5000ms

- file shows up immediately. A subtle difference, but it works. Please try it and see if it works for you guys.

left Extendir at 0
[Windows Mobile Apps and Games] zenkinz:
Hi Rory,

thanks and you are spot on here. I was troubleshooting all the possible combination but totally missed out this particular option. not only it resolve the photo bug & refresh problem which I have reported previously, it improves the performance since I no longer have to apply cachefilt to all the storage.

One more thing I observed, if I take away HTCFSFilter, the file system performance will take a (small) hit. So for now, I've included it to the list of filter to be loaded.
[Windows Mobile Apps and Games] bobsbbq:
Wow this is great news, Thanks RoryB. I'm going to try it now.
[Windows Mobile Apps and Games] RoryB:
I hope it works for you bobsbbq

Here are my results
[Windows Mobile Apps and Games] zenkinz:
sorry to throw wet blanket here. But I realised that it will break the document tab problem ("Missing sd card"). Rory probably won't see it because I read somewhere that you are not using Leo device. But for Leo, this will not work.

so we are back to square one (for Leo chef)
[Windows Mobile Apps and Games] bobsbbq:
I have tried it, Copy and paste works instantly. I did take a hit in write and copy performance. My speed was 1597-1610, now is 1758-1800, my copy speed went from 965 to 1100. But as Zenkinz said the document tab will not work. So I guess for now we have to go back to the way we had it. And hope that with the V2 with cache will fix it.
[Windows Mobile Apps and Games] RoryB:
Have either of you tried

[HKEY_LOCAL_MACHINE\System\StorageManager\Filters\CacheFilt]
"Dll"="CacheFilt.dll"
"Order"=dword:12
"FileCacheWriteBackPriority256"=dword:FE
"FileCacheWriteBackTimeout"=dword:A ; 20000ms , default is 5000ms

Another arrangement I have thought of

[HKEY_LOCAL_MACHINE\System\StorageManager\Filters\CacheFilt]
"FileCacheWriteBackPriority256"=dword:FE
"FileCacheWriteBackTimeout"=dword:A ; 20000ms , default is 5000ms
[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\SDMemory\FATFS\Filters\CacheFilt]
"Dll"="CacheFilt.dll"
"Order"=dword:12

This is similar to concept of fsreplxfilt which gets its parameters set under filter and its order set under profile.
[Windows Mobile Apps and Games] bobsbbq:
No but it's worth a try. But I think this would apply to all storage, not SDMemory
[Windows Mobile Apps and Games] RoryB:
see my edit above your response
[Windows Mobile Apps and Games] zenkinz:
this will certainly work, even though I've not tried. When I applied it at StorageManager\FATFS\filters, it works, so I can imagine it'll work if the filter is apply to all the storage systems.
[Windows Mobile Apps and Games] RoryB:
That is why I went to the second one. There we are defining parameters under filters and applying the filter under profile. I see this as how fsreplxfilt.dll is done.
[Windows Mobile Apps and Games] bobsbbq:
I tried the bottom one and the document tab works, but the copy and paste refresh bug is back. So it's really the same as having the profile only on storage card.
[Windows Mobile Apps and Games] RoryB:
Sorry, I was hoping it was some subtle tweak like that.

I was reading your thread about deleting cachefilt etc. so I tried deleting fsreplxfilt.dll and it gave the notes not selectable in syncing error for me too. Which M2D are you guys using? Can I get it working on an old Wizard.
[Windows Mobile Apps and Games] twopumpchump:
shadrak, is it possible you could write a small app to help with making zip files? where we could point it at a bulk folder and it would divide the files up to a specified count and make zips automatically :)
[Windows Mobile Apps and Games] sunsetdk:
+1 on this one