[MDA III, XDA III, PDA2k, 9090 Software Upgrading] Tuatara:
Well I aim to please ... or was that ... I'm pleased to aim ... *fire* ... eh ... whatever. ;-)

In the simplest terms for our devices it's both. Disk defragmentation is just the example everyone understands ... so I'm using this 'understanding' to explain it.

A flash disk doesn't need ANY defragmentation - it's indexed like RAM, and we can get at any data directly. No need to make any data contiguous like on a hard disk (which spins). So, that's why they use the word "compaction" instead of "defragmentation" ... but the principle is the same.

So ... putting it simply:

It really doesn't matter where it's stored to ACCESS it. But it does matter where it's stored if we need to ERASE it.

Remember we have 64 Storage Units per Erase Unit. This means that if we are actively using even one of those Storage Units, but need to erase the rest of the block to get more "free" space, we need to move the used Storage Unit somewhere else first, before we can erase that block, to use it again.

If you have an application writing to storage during this time ... well ... then the 'free' Storage Units in those nice-new-empty Erase Units can be used up, and the compaction thread has to start over again - trying to pack things in to make more available free space.

Yes ... that would be true - although I have no idea what running the compaction thread at time critical would do to your device!!! I would probably set it to a NORMAL (0xfb / 251) priority ... maybe something at most "THREAD_PRIORITY_ABOVE_NORMAL" (or 0xfa / 250) at a highest value. It may be worthwhile to experiment and see what the result is. Backup your device before doing this though!

I can't answer this one unfortunately ... there are so many intricate details in the OS which could have an effect. Experiment with the priority settings if you wish to ... might be an option to see what the real cause is. If the compaction happens quicker, then we know it's due to flash usage collisions.

Thinking about it - change only the CritPrio256 values ... i.e. when the "s*it really has hit the fan" do it or else ... and see if it completes in less time. If you don't hit critical compaction - then try setting the standard priority as well.

It would be possible - I think that mamaich may have written such a tool for himself, but it is just that - a tool, and not something which could (or should) be actively utilised.

I'm not sure if this is possible ... part of the problem is - where would you put the result data? It would need to be a memory mapped file or something, which is locked exclusively in RAM, so as to not touch the flash, and then your reporting app would need to operate in a similar manner. Not sure ... maybe there is a way to get some info (from the TrueFFS driver) but I wouldn't know how to.

Along the "bad block" concept - the TrueFFS driver has management within itself to control bad blocks and to transparently reassign blocks which have been deemed 'bad'. It is possible that this logic is somehow failing - i.e. not working as advertised.

You could try changing the Read/Write and Verify methodology which the TrueFFS driver uses. Here are some registry entries which might help you track down the problem. You should have registry entries similar to:

[HKEY_LOCAL_MACHINE\Drivers\BuiltIn\TrueFFS???]

“VerifyWrite”: When using this parameter, TrueFFS performs an immediate read operation
from the flash after every write cycle.
The value of this entry can be as follows:
dword:0 - Sets TrueFFS to Optimized Verify Write mode. This is the default TrueFFS mode for
devices supported by INFTL, such as DiskOnChip Millennium Plus and Mobile DiskOnChip. In
this mode, a sector is assumed to remain valid after it has been verified as long as no sudden
power-down event occurs; therefore, no additional read operations (verifications) are performed.
When recovering from a power failure, TrueFFS gradually verifies the validity of additional
sectors as data is written to the disk partition. All the sectors on the disk are eventually verified,
at which point no further verification is done and peak performance is achieved (default).
dword:1 - Sets TrueFFS to Read Before and After Write mode. In this mode, each write
operation is preceded by a read operation verifying the flash area is indeed erased, and followed
by a read operation verifying the data was written properly. Only after both conditions are met is
the sector considered to be written properly. Selecting this option results in lower write
performance, but insures absolute data protection against power failures or any other event (such
as errors caused from noisy buses, faulty flash, or flash close to its end of life).
dword:2 - Sets TrueFFS to Standard mode. In this mode, a sector is considered written
successfully when the flash media reports that the write operation ended successfully. Additional
verification (using a read operation) is not performed, so TrueFFS works at peak performance
regardless of past power failure events. This is the default mode for devices supported by NFTL,
as these devices do not require additional verification for most applications.

“UseCache”: Enabling this option improves performance, but requires additional RAM
resources. The NAND Flash Translation Layer (NFTL, INFTL or SAFTL) uses a small part of
each flash unit and sector for control information that allows accessing the data stored on
DiskOnChip as a virtual block device. Setting this option to 1 enables caching this information in
order to improve performance.
dword:0
dword:1 (default)

“RemoveFatFilter”: The Windows CE TrueFFS driver contains a FAT filter that monitors file
system write operations to the FAT area. The TrueFFS driver uses this information to improve
performance by deleting sectors from the media when the corresponding file is deleted. Use this
entry to disable FAT filter operation.
dword:1

So I'd try “VerifyWrite” = 1, “UseCache” = 0, and leave the FatFilter enabled ... see if this helps 'eliminate' that possible bad block issue.

The fsdspy filter is used to monitor the filesystem - incl OS level and TrueFFS driver performance checking (and for hooking). For the fsreplxfilt filter I believe this is required for data replication ... but the inner details of what it does, I'm not sure either. You do need to specify WHICH directories and files are NOT included - so therefore I believe it to control which files should NOT be replicated to the Flash as soon as possible (i.e. they are maintained in RAM cache, and only flushed at a later date, or by a special seperate thread - things like the registry files, temp files, object store, etc.).

Heh ... I'll just need to find more time to answer then! :D