yashsingh1/xdaforums huggingface dataset:
[Touch HD Windows Mobile ROM Development] Traasje:
My Results

on my fully installed origional rom (latest rom downloaded from the htc site)

mobile shell 3 is running

connected to activesync

programs running on the background

i scored an overal performance off 87,40%

Not bad actually

Write 1 MB file (LESS IS BETTER)

i came FIRST with 546

Read 10 KB x 100 files (LESS IS BETTER)

last with 470

Directory list of 2000 files (LESS IS BETTER)

last with 1669

Internal database read (LESS IS BETTER)

last 920

Graphics test: GAPI Bitblt (LESS IS BETTER)

last 7

Arkaball frames per second (MORE IS BETTER)

FIRST with 9

CPU test: Whetstones MFLOPS (LESS IS BETTER)

Last with 3647

CPU test: Whetstones MWIPS (LESS IS BETTER)

last with 7564

Copy 1 MB using memcpy (LESS IS BETTER)

Last with 9

although i mainly came last, 87,40 is not bad

i good here :-)
[Touch HD Windows Mobile ROM Development] s1rl4ncel0t:
Hi daeron.be,

I would like to know which radio version you had installed on each rom you tested.

It would be good to clarify this, as radio can influence the results.

HastaSSSS
[Touch HD Windows Mobile ROM Development] daeron.be:
I've got not idea which radio was in every rom tested, because I don't flash rom at all, unless it's included in rom I test. Are you really sure it can influence the results?
[Touch HD Windows Mobile ROM Development] s1rl4ncel0t:
I'm positive about it, as the phone uses cpu resources. For example, don't you feel the device slower when the phone is searching for network?

If radio 1.13 searches more times per second for network than 1.14, then scores with 1.14 should be lower than with 1.13...or vice-versa.

I would suggest one thing: put the phone in airplane mode before testing. In this case, it would eliminate the eefect of the phone on the device.

But, on the other hand, it wouldn't be a real-life test, as nobody uses the device in airplane mode...unless their in an airplane :D

HastaSSSS:)
[Touch HD Windows Mobile ROM Development] fwt:
Hi,

great Idea in this Thread!

Just one thing about your Spreadsheet:

Maybe you should think about highlighting the best / worst value with different colours (green / gray).

Thank you for the time you put in benchmarking all this great ROMs!!

regards

fwt
[Touch HD Windows Mobile ROM Development] bouzoubaa:
Guys !!!! i don't really get it ... why are we comparing theses roms in term of performance !?? they are all different with different components !! we are creating some kinda performance competition that will kill the reliability and usability of these roms !!! i don't really think this is healthy ...

anyway .. what is known for a fact is that all the cooks are great in this forum and they have all our respect .. Dutty , Bepe, Mirri, Topix and all the rest .. they are all really great people that are spending all there free time working on this stuff ....

I would suggest that the comparison be made upon stuff like buggs free, stability, language supports, innovations and such things that we really use and need daily ... but tweaking performance mmmmmm, i 'm personally not so into this ... they all score almost the same anyway !!!

hats down for all the cooks out there :cool:
[Touch HD Windows Mobile ROM Development] s1rl4ncel0t:
Why always this discussion when someone decides to benchmark roms? Same discussion had place on Diamond thread, and also in Trinity.

Honestly, I don't see the harm. Competition is good and healthy, if the rules apply equally to all roms. That's why I asked about the radio version installed.

It's a basic concept really, that all roms have to be tested under the same conditions. If so, then it's honest and will bring added value to this forum.

HastaSSSS:)
[Touch HD Windows Mobile ROM Development] sonydesouza:
Thanks for precisions but i always think that spreadsheet could be better organized, in particular for dummies...

Moreover, i agree with Bob ! If Daeron was less arrogant and more comprehensive, this thread could be interesting...

Anyway...

See ya !
[Touch HD Windows Mobile ROM Development] daeron.be:
No problem.

Thanks for your advice. I really appreciate your comments on my person. And I'm watching this thread on a daily basis, read all comments and I respond only to those I like.

pzucchel -> I'm really delighted to see somebody who can use this spreadsheet in a proper way - by customizing it to see only things that matters to him. Keep going ;)

About battery testing - It'd be cool, but that'd be the most time consuming test, especially if you'd have to leave blackstone unused before it discharges. Even if we use piece of software that'd only measure battery usage for a short time, that'd be very inaccurate, unless we do it for a longer time, like half an hour or longer. Personally, I don't have time for something like that. And that'd mess up my ideal overall score system, (it's an irony!) because I'd have test all roms yet again. Don't you think it's a little time consuming thing to do? But if YOU want to do this, you're welcome.

And don't tell me I'm not cooperating with users and I don't accept their tests, because I do it. I've added all of them (all PMed to me) all to the spreadsheet, for what I was often criticized by BobsBBQ. Some of them were removed then they were proven inaccurate by other tests.

And again, don't tell me I don't follow user's suggestions - you asked for OS column and I've added that. Even through in my opinion it's pretty useless feature, but once there was enough voices for it, I've added that. And asked you if you could specify me what some OS roms are, because I don't remember every rom I and Houms tested. And NOBODY answered this call.
[Touch HD Windows Mobile ROM Development] bobsbbq:
Dareon the battery test would take less time than any of the other test you due. It's a current draw test. The more current that draws on the battery the faster it will go down and vise versa. You don't have to wait for a battery to go dead to test the time.

What I was referring to is you have your own way of looking at this benchmark and that is your right. But you have been close minded to suggestions. Yes you changed the OS with a new column and I thank You but it was only after many people had to ride you about it. You can be assured that if a chef is making suggestions on a way to make it better then he may know something as a cook you do not, and it doesn't hurt to be more open minded and receptive when dealing with these kind of things.
[Touch HD Windows Mobile ROM Development] daeron.be:
Bobsbbq, please run this test on 5-6 recent roms and send me your results via PM. If they make any sense I'll consider adding them to the "test method".

Houms tested following roms for you:

Black Pearl v3.3 WWE

Moon v1-2

Miri 6.5 v23.0 Premium

Miri 6.5 Premium Manila v2.5 REV2

Dutty's 6.5 RC4
[Touch HD Windows Mobile ROM Development] bobsbbq:
Wow, did I detect a nice tone? We might be getting somewhere. I will test them when I have the time. Right now working on new build with Extendir. It is very very fast many of the spb results are the fastest even compared to 6.1. It would definitely be at the top of all 6.5 ROMs. See for yourself this ran just 10 min ago.

<benchmark-results>

<info>

<platform-name>

Touch HD T8282

</platform-name>

<device-name>

touch hd

</device-name>

<start-date>

07.10.2009

</start-date>

</info>

<main-tests>

<fs-largefilewrite>

537.800000

</fs-largefilewrite>

<fs-largefileread>

23.800000

</fs-largefileread>

<fs-largefilecopy>

625.700000

</fs-largefilecopy>

<fs-manyfileswrite>

4359.100000

</fs-manyfileswrite>

<fs-manyfilesread>

263.300000

</fs-manyfilesread>

<fs-manyfilescopy>

3171.900000

</fs-manyfilescopy>

<fs-dirlisting>

125.200000

</fs-dirlisting>

<fs-db>

743.000000

</fs-db>

<grf-ddb>

11.100000

</grf-ddb>

<grf-dib>

34.435000

</grf-dib>

<grf-gapi>

1.260000

</grf-gapi>

<misc-zip>

2568.900000

</misc-zip>

<misc-jpeg>

397.900000

</misc-jpeg>

<misc-arkaball>

3.885500

</misc-arkaball>

<misc-mflops>

3434.300000

</misc-mflops>

<misc-mops>

1004.200000

</misc-mops>

<misc-mwips>

7144.900000

</misc-mwips>

<misc-memcpy>

8.508000

</misc-memcpy>

</main-tests>

</benchmark-results>
[Touch HD Windows Mobile ROM Development] Ascenca83:
Hi

Perhaps Laurentis26 used THIS to get a faster windows mobile under the hood.

I think only he can deny or confirm this.

Ps: I didnt use the ROM til now, as I am just fine with Duttys last installement V3.9 (which is awesome imo).

Ascenca
[Touch HD Windows Mobile ROM Development] Laurentius26:
Nope I didn't use that.

And about 'the lag' in Manila story have a look:

http://www.youtube.com/watch?v=FnGTh5VMszo
[Touch HD Windows Mobile ROM Development] kwbr:
You get these benchmark results when you remove fsreplxfilt.dll from the xip. It gives me a SPB dirlist score of 200. There is a but ... in the end it will give you problems with activesync. fsrplxfilt.dll is needed by some applications. So it is not smart to remove this :)
[Touch HD Windows Mobile ROM Development] Ascenca83:
Thanks for the explatation!

I am really looking forward to see the first ROMs to use the solution linked be me here.

I think could be a nice thing if used correctly by chiefs (only my opinion, I'm no chief).

Ascenca
[Touch HD Windows Mobile ROM Development] pzucchel:
Hi Leo,

i can immediately tell you it is not what i had, otherwise i would have kept using it! and i believe your movie, of course.

Probably i didn't install it correctly - do you hard-reset after installation? I do it systematically (by buttons). any other trick i could have missed - e.g. do you understand what could be the problem?
[Touch HD Windows Mobile ROM Development] KukurikU:
Share, share...said somebody once upon a time...:p
[Touch HD Windows Mobile ROM Development] kwbr:
See my post which explains this.
[Touch HD Windows Mobile ROM Development] daeron.be:
2 roms tested by Houms, online! :)

Topix 2.6.1 (best results of Kwbr's rom so far)

M-Amine v3 up3

Enjoy. I'm glad results are improving step by step ;)
[Touch HD Windows Mobile ROM Development] pzucchel:
....is this what you also did in 2.6.1? this analysis of speed shows, without ambiguity, the absolute value of a ROM benchmark. the fastest is not the better....;)

Admittedly, yours and Dutty's ROM are, in my hands and on my touch HD, those that feel faster. The fact that both are highly ranked by the test is a confirmation that - without being perfect - this benchmark is indicative....if we don't try to mislead it!
[Touch HD Windows Mobile ROM Development] vorenius:
kwbr, u r 1 of my favorite cooks but please do have a look at L26 rom. This dll is in there cooked and part of xip.;) So... the key from the palace is somewhere other place,

Salute
[Touch HD Windows Mobile ROM Development] vorenius:
I apologize in advance of what i would say but...

A d*ck in some hands is hard and in others not.:D So who is to blame? The d:Dck or the hand? :D

And now to stick to the rules i say - your mentioning any names of cooks is not at all OBJECTIVE.

Anyway... who gives a SH(EE)T :D

Apologies again
[Touch HD Windows Mobile ROM Development] kwbr:
Hi vorenius, thanks for your feedback on this. This is really interesting. Can you (or another user) please tell us what value is used for this registry keys:

[HKEY_LOCAL_MACHINE\System\StorageManager\Filters\fsreplxfilt]
"ReplStoreCacheSize"
[Touch HD Windows Mobile ROM Development] KukurikU:
apologies accepted and also yr sense of humor:D you made me laugh by concentrating lots of useless posts from this thread in one, funny proposition:D stay with us, pls!
[Touch HD Windows Mobile ROM Development] kwbr:
Filesystems

Since we can all benefit from this I created 2 (beta)roms. One with a XPR and one with a LZX compressed filesystem. I am curious if these will differ in the benchmarks.

- LZX compressed rom

- XPR rom

Someone willing to test these?

P.s. next thing is to add extdir to the test and see what effect this will have.
[Touch HD Windows Mobile ROM Development] Ascenca83:
Nice to hear this.

Hope there are noticeable improvements so perhaps also other chiefs will try them out (especially curious about the "extdir" thing).

Lets wait for the tests....

Ascenca
[Touch HD Windows Mobile ROM Development] pzucchel:
...Apologies accepted! Following your advice, I am checking my hands, indeed pushing the three buttons alltogether for HARD reboot often doesn't generate the right effect...should I exercise more? :eek:

I have to say that , after all, being a cook shouldn't give immunity to criticisms and/or questions. Yes, they are doing it for all of us - but we should not exhagerate with this holy protection from being cited through an anonymous nickname or benchmarked. Aren't the roms public domain, after all? do the cooks claim more copyright than microsoft itself?

Vorenius, FYI i have also been a cook, so I have immunity. I feel very much offended from your words and therefore I cast back 10 years of battery drain and a broken LCD....not to speak of HARD reboot failures....:o
[Touch HD Windows Mobile ROM Development] KukurikU:
LMAO! and everybody says that swiss ppl are lacking the sense of humor:p doohhhh...
[Touch HD Windows Mobile ROM Development] Prerna:
That statement is basically right, and Pzucchel seems to actually be a French living in Switzerland. But then let's not start talking about French' humor

:D

My first post here, although I've been following it since start. So let me take this chance to congratulate on the benchmark' works.
[Touch HD Windows Mobile ROM Development] houms:
not found

Hello, I look at this registery place but did not found it:

in HKLM\System\StorageManager\Filters there is only two sub dir:

ENCFilt and LsFilter

Let me know if I should look anything else
[Touch HD Windows Mobile ROM Development] houms:
I'll test them

Actually loading them, I'll test them asap and send results to Daeron.5Update: downloading them once and was corrupted, unzipable, I'm redownloading).

Hope it will help to enhance things.

Ay other test you need, let me know.
[Touch HD Windows Mobile ROM Development] houms:
not testable: files are corrupted even after two download.
[Touch HD Windows Mobile ROM Development] pzucchel:
I live in France :o, I work in Switzerland :confused:...but I'm Italian :eek:
[Touch HD Windows Mobile ROM Development] bobsbbq:
To Daeron, just FYI I would suggest that you add a addition step to the method. It seems the results after the first run of the test are slower than after a soft reset. I think it would be best the install the software, soft reset, run 1 test, then soft reset again, then start collecting results. I have noticed a wide fluctuation during this first run. Maybe some others can share their experiences as well.
[Touch HD Windows Mobile ROM Development] houms:
On your request, I had stoped to bench your rom, do you agree that we include again your roms in our bench ?
[Touch HD Windows Mobile ROM Development] Sokota:
Strange values!

Hi all.

I was happy with my HD when someday I played a little bit with the HTC Touch 3G and found out that it was quicker than my device. I thought that maybe this had to do with the bigger screen/resolution of the HD and to the fact that it runs TouchFLO 3D instead of TouchFLO 2D.

Some days later a colleague of mine bought a Diamond 2 and, once again, his device was quicker... and I mean really quicker! Starting Opera in his device is almost instantaneous while in my HD Opera gets about 10 seconds to open. Having the devices almost the same specs I thought this to be very strange.

I finally got some time and run the spb benchmark on my HD. When I compared my results with the results found here I was blown away.

Before I write the results I have 3 things to say:

- My device never gave any problems/errors at all.

- I have a fresh install of Dutty's v3.9 XT Turbo ROM and only installed the spb benchmark software after that.

- Before making the benchmark I disable the today plugins, enabled the airplane mode and soft-reseted the device

Now the results:

(Cyan - Dutty's v3.9 XT Turbo)

(White - Mine)

What the hell is with this Read/Write/Copy/Compress values? They are many times higher than the ones found here.

Do you think this is an hardware problem? Maybe the memory chips?

What do you think about this?

Thank you.
[Touch HD Windows Mobile ROM Development] houms:
your micro sd card

I think it means that you have a slow micro sd card. Or perhaps it's very fragmentated? Or full? Try to empty your card and format it and retest. If you still have the same results, it's probably your card which is very slow.

Houms
[Touch HD Windows Mobile ROM Development] Sokota:
Are you saying that the tests are made on the SD Card?

But I selected the "Main" group and not the "Storage Card" group in the wizard.

I thought the tests were made on the internal memory of the device... Am I wrong?
[Touch HD Windows Mobile ROM Development] houms:
I've tested your propoasal

I've tested what you suggest: but the second measure after soft reset was 1.5% worse than the first measure. I measure it on Topix 2.7.

So as this methode will double my measurmeent time, and do not show any benefits, I do not push for changing the bench measurement methodology.
[Touch HD Windows Mobile ROM Development] Sokota:
You are right. I reinstalled the app in the main memory and now the results are normal. Thank you for your help.

Just one question: Is it normal for the Opera browser to take about 10 seconds to start? Because as I said in my first post in the Diamond2 is very quick to start.

Thank you once again.
[Touch HD Windows Mobile ROM Development] daeron.be:
For everyone waiting, new results by Houms, with test Topix roms provided by Kwbr! Enjoy

List:

Topix v2.7

Dutty's 6.5 RC5 MSVC Turbo

PhamQuang v44 b23004 2009.10.07

PhamQuang v44 b21920 2009.10.07

Maryone b23004 2009.07.10

Topix test rom with XPR

Topix test rom wtih LZX
[Touch HD Windows Mobile ROM Development] MAMeingast:
If you have push-internet activated (as I guess is the case with your friends device), Opera doesn't really exit - it still runs in the background after closing it. Therefore it's back almost immediately whereas it is completely reloaded with no push-internet.
[Touch HD Windows Mobile ROM Development] Ultimaex:
Duttys R5 is first competitive WM6.5 ROM in comparison to 6.1.

Nice to see that the new WM seems to be at least as same fast as 6.1.

gj Dutty
[Touch HD Windows Mobile ROM Development] Sokota:
Thank you for you answer.

I'll find out that information.

Regards.
[Touch HD Windows Mobile ROM Development] daeron.be:
The catch is that chefs to create 6.5 rom have to use some parts of 6.1 anyway (because some files have to be taken from original blackstone roms), but once they use more of 6.1 rom's parts, such roms could be actually hybrides with 6.1 engine and 6.5 look. But that's a theory. I use 6.1 because there isn't ANY exclusive feature in 6.5, that I use.

That's not like say that cooking 6.5 is pointless. Any new developments that are usable for somebody is a good thing. Keep going! :)

And I hope WM7 will be a greater upgrade... and somebody will port it! Because porting Android/Linux on Blackstone is many times harder and time consuming than tinkering with WMs and I don't think there will be a usable version of Android on Blackstone anytime soon. And 3d drivers are even less likely to appear. Unless a bunch of really experienced linux/android programmers will start working on it.
[Touch HD Windows Mobile ROM Development] badaas:
hmmm, but even after all these tests it's down to 'in use' performance.

and having used laurentius 3.2 wm6.1, and now back to onkolog lite wm6.5, onkolog wins if only for the fact screen rotation is 'instant', laurentius 3.2 wm6.1 is about 3 seconds! not good imho. and overall using the wm6.5 is a lot sleeker i feel.

just my 2 cents!
[Touch HD Windows Mobile ROM Development] daeron.be:
Screen rotation time hugely depends on type of 'rotation software' and its settings. Or it can be set in registry too. For example I really don't like when screen is rotating every time I move my phone, so it rotates after such '3 seconds' after my rotation. Ya see?
[Touch HD Windows Mobile ROM Development] duttythroy:
just to clarify what you said about roms been hybrid, the only part of my rom which use any part of a wm6.1 rom is the OEM drivers which is htc.

thanks for ur benchmarks and continue your hard work
[Touch HD Windows Mobile ROM Development] badaas:
hi, yeh. I used same rotation software on both (600GOL) with same hdtweeks settings and couldn't duplicate speed on l26 3.2, same was noted in l26 thread. but anyway, back on topic, thank you for your testing, seems most roms hit 82% score apart from top 2.

:)
[Touch HD Windows Mobile ROM Development] daeron.be:
I see. Well, then good job on your results ;P

Well, it can happen. There must be some reason, through. But that's question to Laurentius26. :)
[Touch HD Windows Mobile ROM Development] ehsanij:
hi,

is it possible for you to include the latest official stock ROM in your list as well?
[Touch HD Windows Mobile ROM Development] houms:
it's already in.
[Touch HD Windows Mobile ROM Development] hurzwa:
Why is "less is better" at "Arkaball frames per second" ? More frames per second should be better.
[Touch HD Windows Mobile ROM Development] daeron.be:
well, that description sucks, because results are actually in ms. Gotta correct this...
[Touch HD Windows Mobile ROM Development] IAmSmoke:
How is Dutty's R6 doing?
[Touch HD Windows Mobile ROM Development] Matthes42:
it would be great to see whether Laurentius L26 V 3.5 is as fast as his 3.2

Please benchmark it as well. Thanks in advance.
[Touch HD Windows Mobile ROM Development] houms:
it's even better than 3.2

I just sent new mesures to Daeron.

Thanks a Lot for this nice rom.
[Touch HD Windows Mobile ROM Development] daeron.be:
First, sorry for a short delay, but I've visited Cracow (beautiful city, btw)

I've added roms benchmarked by Houms:

Laurentius26 6.1 v3.5 (PWNAGE!)

RUU_BlackStone_hTC_Asia_WWE_1.59.707.0_Radio_52.62.25.34_1.13.25.24_Ship

Dutty's 6.5 R6 Turbo

Dutty's 6.5 R6 MSVC Turbo

PhamQuang v46 b21921 ATT 2009.15.07
[Touch HD Windows Mobile ROM Development] EpiphanyNL:
My gawd! Seems Leo has outdone himself this time.
[Touch HD Windows Mobile ROM Development] c4Lvin:
HOLY COW! Has any other ROMS scored that high overall? I just noticed that 3.55 is out and was wondering if it's worth upgrading from 3.5? Also glad to see you here EpiphanyNL!
[Touch HD Windows Mobile ROM Development] xmoo:
Video playback test on Leo's 3.55: http://www.youtube.com/watch?v=NHRiRdOFNuQ
[Touch HD Windows Mobile ROM Development] manowar1978@gmail.com:
I would see a comparison between Laurentius L26 V 3.55 and 3.5.

Please benchmark 3.55. Thanks and compliments for your work!
[Touch HD Windows Mobile ROM Development] daeron.be:
I'm glad you posted that video. But it has no meaning, because we know nothing about file you played - which codecs (audio and video) you used, what coreplayer version you used, how you configured your coreplayer, what's the bitrates (audio and video) of the movie...

About Dutty's and Leo's

Here they come, from Houms! :P

Tested:

Dutty's 6.5 R7 Thriller (Houms said that rom was pretty unstable for him)

Laurentius26 6.1 v3.55
[Touch HD Windows Mobile ROM Development] khldon2000:
from where can I donlowd the file ?

thanksssssssssssss
[Touch HD Windows Mobile ROM Development] MAMeingast:
You can't - there is a link to a google spreadsheet in the first post.
[Touch HD Windows Mobile ROM Development] daeron.be:
In a google spreadsheet you have export option (in file menu - left top corner). There you can choose format of output file. Or just copy it and paste to any spreadsheet file/template.

Added (by Houms):

Miri 6.5 v25.0 Premium

Dutty's 6.1 v4 Turbo X