[Touch Pro, Fuze ROM Development] ktemkin:
I believe most of the problem here is that you don't precisely understand what's going on behind-the-scenes. Your CPU/processor is only capable of running a single instruction at once- and thus only process a finite amount of data per unit time.

When designing an algorithm (in this case, the algorithm that reads SMS/E-mail database entries and displays them onscreen), tasks which apply to more than one or two items are usually completed by either iteration or recursion. For a task such as enumerating SMS messages onscreen, the process would be iterative; for example, if you were to break it down into simple steps, those steps might be:

1. Read the current SMS message from the database.

2. Display it on screen.

3. If there's another text message after this, repeat this process for it.

Step 1 itself actually is composed of several detailed steps, which take the time to perform a second algorithm, which tries to locate the current SMS message in the database, usually based on a unique identifier and a hashing algorithm. This takes time, but is faster than the alternative- which is to check each SMS and ask "is this the one I want?". Instead of having to check each message, the device usually only has to check a few until it finds the right location.

You can think of this as the device automatically categorizing messages into virtual 'boxes'- you have to spend a hell of a lot less time digging through a box to find 1 document out of 10 then by searching through a much larger heap of 4000 documents.

Step 3 is also composed of several steps, as the device has to figure out which SMS message is really next. As the user is capable of configuring the way in which the messages are sorted (by date, sender, etc.), the message that's next in the database is probably not the next message that's stored in the database. Time has to be spend finding the message, usually by yet another algorithm.

The problem is, each of these steps, and each of their sub-steps, and each of the sub-steps required to run those (all the way down to the machine code level) take time, and each have to be executed for every message the application would like to display.

Thread-view further complicates the matter by creating additional relations between the messages, and requiring the OS to look for even more posts during each step.

And this process isn't all that's going on at once- the operating system is trying to do something called TDM, or Time-Division Multiplexing (Multitasking), which allow you to run more than one program at once on a single CPU (which in turn can only do one thing at once). This basically lets the individual processes take individual turns so quickly that they seem simultaneous.

Because of all this, computer scientists rate their algorithms by degradation. We actually have a measure called Big-O (asymptotic) notation that tells you how well an algorithm handles load.

A good algorithm generally has performance of O(N) or better. This means that for every N elements (in your case, e-mail), it takes approximately N iterations to complete. Note the assumption that each "base iteration" executes in a (roughly) fixed time.

Assuming the WM message app uses a sensible algorithm (and it would be difficult for it not to), we can assume its efficiency is approximately O(N)- as it is simply O(N) for non-threaded, and it is O((N/S) * S) for threaded, which simplifies to O(N). This means that for every 4000 messages, it will need to iterate 4000 times.

Considering the application can't store 4000-messages worth of data, it spends a lot of time during each iteration moving messages in and out of memory.

Given all of this, the Windows Mobile message parsing algorithm is the cause of your 'slowness' and 'hangups'. These periods of non-responsiveness are simply WM trying to run through the algorithm for all of the huge quantities of messages.

Hence, the problem is that Windows Mobile simply wasn't designed to bear the load you were forcing it to bear. This means the problem isn't related to the implementation in any given ROM.

A ROM isn't going to be your solution- every ROM uses the same core Windows Mobile messaging applications.

It is possible that a combination of a data structure and message parsing algorithm (with the addition of hash and cache optimized for the way you specifically use messaging) could handle all of these messages without any considerable degradation; but it would run far less efficiently on small amount of messages than the current WM scheme. (And think: how many users have as many messages as you?)

Since no one really needs as many messages as you seem to, consider your options:

1) Implement a database optimized for large amount of messages. Implement a program that hooks all messages and places them in this database instead of the WM one. Write your own message reading/writing application and use in place of the WM one. Note that no one will do this for you. It's not a public interest: if you want it, you'll have to write it. We can help you- but we're not here to do the work for you.

2) Try a third party SMS reader/writer like Vito SMS-Chat. I don't think these will fare much better (in fact, they may fare worse), but if they implement any database of their own and any type of localized caching (especially of recent messages), they may work a bit better.

3) Remove your older SMS messages. This is probably the best option. If you don't need those messages anymore, a Microsoft applet called InboxExtender adds buttons to delete all messages (and to mark them all as read.)

4) Don't use threaded mode. This will decrease the time each parsing iteration takes.

---

This isn't a bug; nor is it a glitch, nor is it WM being crappy. It's much the same as if you tell photoshop to open an 8GB file on your desktop. Your computer may slow to a halt and take forever doing it- but the cause isn't that your computer is crappy, but rather that you're trying to open such a huge file.