yashsingh1/xdaforums huggingface dataset:
[Touch HD Windows Mobile ROM Development] daeron.be:
Well, it'll be even more helpful if we checked coreplayer results, because it's usually what matters for users. :)

Spreadsheet is back, but now with Google, but with EditGrid. http://www.editgrid.com/user/daeron.be/Daeron's_Blackstone_Rom_Benchmark

Well, sorting finally works properly, but it's a little more time consuming to sort things and for some reason freezing cells doesn't work properly. I'll try to do something about that later.

Please, if you're really curious about coreplayer results and you have the coreplayer, test it using the method and send us results!

And well, I think it's about time somebody checked in a scientific way if hard reseting device after flashing does affect performance - I need somebody to first flash for example some Miri rom, hard reset it, flash some 6.1 rom, benchmark it with 'a method', hard reset and see if there is any difference.
[Touch HD Windows Mobile ROM Development] Niksie:
Here are the results of my favorite ROM: Miri's 26.1 Premium:

After a hard reset and disabling Manilla:

Coreplayer 1.3.2-a

Low Quality: 144.7%

High Quality: 109.7%

Xtra Hight Quality: 91.54%

Every test is done twice.
[Touch HD Windows Mobile ROM Development] MAMeingast:
Thx! I've added the results to the table!
[Touch HD Windows Mobile ROM Development] MAMeingast:
Added results for Miri WM 6.5 23017 MANILA 2.5 REV7.0, including coreplayer benchmarks.
[Touch HD Windows Mobile ROM Development] MAMeingast:
Added results for Topix 2.8.7, including coreplayer benchmarks.
[Touch HD Windows Mobile ROM Development] Niksie:
Just flashed topix 2.9.

After hard reset and disabeling Manilla, these are the results with CorePlayer 1.3.2-a.

Low quality: 145,12%

High quality: 106,18%

Extra high quality: 88,34%

I've done every test twice.
[Touch HD Windows Mobile ROM Development] MAMeingast:
Thank you very much! :D

I'll include the results as soon as I get to flashing 2.9 myself and doing the other benchmarks this evening! ;)
[Touch HD Windows Mobile ROM Development] MAMeingast:
Added results for Topix 2.9, including coreplayer benchmarks (by Niksie - thank you!).
[Touch HD Windows Mobile ROM Development] Lilith_v2:
Hi guys, excellent benchmarks!

But i have a question: in the red coloumn of 'Overall performances index' a low result is best right?
[Touch HD Windows Mobile ROM Development] MAMeingast:
No: The opposite is true! The higher the percentage, the better the ROM fared!
[Touch HD Windows Mobile ROM Development] Lilith_v2:
Uhm, so for more performances i must choose that with the high number, and of course for the fast with the low number right?

I'm not stupid, but for me is not clear u.u
[Touch HD Windows Mobile ROM Development] MAMeingast:
Now you're confusing me... isn't "more performance" and "fast" the same?!? :confused:
[Touch HD Windows Mobile ROM Development] Matthes42:
the chart is like a ranking. Those on top are those ROMs with the best performance, which is in this case synonym to fast
[Touch HD Windows Mobile ROM Development] daeron.be:
It's a ranking as long as the roms are sorted with an overall performance index...
[Touch HD Windows Mobile ROM Development] warbot:
Maybe someone knows what's the problem...

Everytime I try to run a benchmark test on my HD I get poor results no matter what ROM I use (and I tried them all, including stock ROM). These poor results are mostly when testing main memory and never in CPU tests. spb benchmark installed in main memory (went worse when installed on card).

any idea what's wrong?

Thanks.
[Touch HD Windows Mobile ROM Development] EpiphanyNL:
Poor results as in? Did you follow the procedures posted in the OP?
[Touch HD Windows Mobile ROM Development] daeron.be:
Well, it probably means that there's something wrong with you phone. I suggest to flash default SPL and send it for repair if you can.

Do you have your phone unplugged from PC or AC when you run those tests?

How bad are your results?
[Touch HD Windows Mobile ROM Development] warbot:
Benchmark for Touch HD

I did everything as posted.

Here are my results:

write 1mb 594ms

read 1mb 45ms

copy 1mb 668ms

write 10*100 4410ms

read 10*100 299ms

copy 10*100 4213ms

dir. list 1376ms

internal db read 821ms

graphics: ddb 13ms

graphics: dib 36ms

graphics: gapi 6ms

compress 1ms 2924ms

decompress 415ms

arkaball 9ms

cpu mflops 3463ms

cpu mops 1003ms

cpu mwips 7110ms

copy 1mb memcpy 8ms

so a you can see my cpu works fine, the major problem is in the write/read/copy tests.

btw i am using NRG rom with WM 6.5 manila 2.1

Thanks for your help.
[Touch HD Windows Mobile ROM Development] MAMeingast:
Hö? I don't really see the problem!? Your results are comparable to the ones we have in our table!

You:

write 1mb 594ms

read 1mb 45ms

copy 1mb 668ms

write 10*100 4410ms

read 10*100 299ms

copy 10*100 4213ms

Laurentius' ROM (the fastest one):

510.20

32.00

605.20

3,016.30

158.50

2,629.20

PhamQuang 6.5 v34 (one of the slowest ones):

609.50

42.40

679.80

4,704.00

277.70

4,266.70

Your results are somewhere in between on average!
[Touch HD Windows Mobile ROM Development] MAMeingast:
Added results for Topix 2.9.2 including Coreplayer benchmarks!
[Touch HD Windows Mobile ROM Development] daeron.be:
...

It happened again - our spreadsheet grew too big for another online spreadsheet hoster. Therefore, we have to swich different way to distribute our work - via attachments in this topic. To get newest results, just look for a latest post from me, Houms or MAMeingast. Sorry for the inconvenience, but I'm sick and tired of all problems with those services.

BTW we have over 135 roms tested. We missed our first 100 :/
[Touch HD Windows Mobile ROM Development] houms:
same than posted by Daeron but only corrected formulas

same than posted by Daeron but only corrected formulas
[Touch HD Windows Mobile ROM Development] leobox1:
sorry... where r the results?
[Touch HD Windows Mobile ROM Development] Fallen Spartan:
If you had read first post you would have seen that they ared having difficulties with spreadsheet and they have posted latest results in their last post which is currently this one.

In future try reading (especially the first post of each thread!!!! :mad:) & searching before posting

Ta

Rick
[Touch HD Windows Mobile ROM Development] leobox1:
sorry.. i did :( jsut that the earlier com i was using .. blocked the attachment from showing..
[Touch HD Windows Mobile ROM Development] Blessed HD:
Excellent job you're donig.. Just wonderful! :)

BTW

I do check your benchmark almost daily looking for new results :D
[Touch HD Windows Mobile ROM Development] MAMeingast:
lol - i guess you see them more often than we do! ;)
[Touch HD Windows Mobile ROM Development] D.O.C:
Did anyone benchmark duttys 4.3 yet?
[Touch HD Windows Mobile ROM Development] Ultimaex:
Should be as same as 4.1 because dutty only did some minor bugfixes
[Touch HD Windows Mobile ROM Development] leobox1:
am i right to say 6.5 roms are not fast
[Touch HD Windows Mobile ROM Development] MAMeingast:
Not necessarily... Although many 6.5 ROMs are further down towards the end of the ranking, Dutty's 6.5 ROMs are far above avarage. Atleast when judging by the benchmarking results. But in every day use, to me 6.5 ROMs always (including Dutty's) seemed A LOT slower than 6.1 ROMs. This is a subjective impression though and not backed up by our data! So just try it yourself! ;)
[Touch HD Windows Mobile ROM Development] leobox1:
but i am right to say that the top rom .. Laut3.5 is the fastest even when using righT???

where can i get it? i only find 3.55
[Touch HD Windows Mobile ROM Development] MAMeingast:
Well - as I already said: Personal, subjective opinion can be quite different from the benchmark results posted here. You have to try it yourself to decide if it's fast or not.

You should ask for the download link in Laurentius' thread!
[Touch HD Windows Mobile ROM Development] Fallen Spartan:
MAMeingast is correct. These benchmarks are only benchmarks and your own opinion may differ. Personally I may take a glance at these benchmarks from time to time, but IMHO, nothing compares to trying these out. When rom is put to everyday use with tasks, internet, music etc, you may find your own results differ from these benchmarks and from my experience they do!
[Touch HD Windows Mobile ROM Development] Turbii:
Hi guys,

i was checking for a rom for a friend of mine. He has a Topaz and I went searching for a benchmark-list like ours. I found it, but i saw that they have a very big and more detailed list. Can't we have one? :o

I really love your work for this list and i'm checking it almost daily :D

Byebye,

Turb
[Touch HD Windows Mobile ROM Development] MAMeingast:
Could you post a link to the Topaz thread? I couldn't find it! ;)
[Touch HD Windows Mobile ROM Development] Blessed HD:
:D

& after I tried it my self, I totally agree with that ;)

BTW, the L26 3.55 & 3.5 downlad links are available in his site..

I've just flashed to kwbr's new TOPIX 2.9.3 ROM, & after 2 hours of REALLY using it, it's super fast & very smooth.. I wouldn't be amazed if it topped the daeron benchmark charts :D

Care to benchmark...?
[Touch HD Windows Mobile ROM Development] MAMeingast:
The last few months I have been using Topix' ROMs and I'm personally very happy with them as well!

And I'm going to flash 2.9.3 as soon as I have the time - and then I'll definitely benchmark that ROM! ;)
[Touch HD Windows Mobile ROM Development] MAMeingast:
Topix 2.9.3 benchmarked and added to the table! ;)
[Touch HD Windows Mobile ROM Development] Blessed HD:
Well, it's still the fastest 6.1 ROM+manila 2.1 I used so far :rolleyes:

Thanks for your wonderful contribution, and enjoy the ROM :)
[Touch HD Windows Mobile ROM Development] MAMeingast:
Your welcome!

And yes: It's definitely one of the fastest and smoothest I've used so far! :D
[Touch HD Windows Mobile ROM Development] Turbii:
http://xdaforums.com/showthread.php?t=510385

Here you go ;)
[Touch HD Windows Mobile ROM Development] MAMeingast:
Thanks!

I took a look at the table - what information would you specifically like to have included in our table? Besides the "power drain"-category the other important stuff is already there, isn't it?

Also, if we start to add more info we're not going to be able to complete that new category for the ROMs already tested - it would be far too much work! :eek: ;)
[Touch HD Windows Mobile ROM Development] Fallen Spartan:
Ah, but you have to remember that new roms are created all the time and eventually the older ones would become obsolete. I wouldn't worry about testing existing roms. If you add these new categories into the benchmarks, and test each new rom with these, eventually after a small period of time all the latest roms would have the full benchmarking and it wouldn't matter about the older obsolete ones....
[Touch HD Windows Mobile ROM Development] Turbii:
Yeah you're right, would be to much work to test every rom agian.

Is there already a new fastest rom? :D
[Touch HD Windows Mobile ROM Development] daeron.be:
Hi!

There aren't many new roms around recently, but I've decided to test one of them - PhamQuang's 6.5 v59 from PDAviet with 23047 build. And I'm really disappointed with results and the functionality of the rom. The only 'pro' is better working dictionary (performance). There's nothing left but to come back to good old Laurentius'es 3.5 rom.

updated spreadsheet attached.

daeron

edit: is it a time to change to iphone? :)
[Touch HD Windows Mobile ROM Development] daeron.be:
There's another test for you! Original one!

I've tested 2 roms for speed of their flash files playback in IE using http://www.newgrounds.com/portal/view/465908 (direct link to the swf file: www.daeron.be/benchmark.swf ). It's nice piece of flash movie that score your flash performance. You just have to download it to your blackstone and run it with IE (just put it in a root dictionary (/) and then input /benchmark.swf address in IE. On some roms like topix 2.9.3 you'll have to wait a while before the flash starts - there won't be any signs of loading). Run the benchmark (if you want your result to be compatible with mine you MUST run it with panorama screen and with flash file in a full screen), listen to the music, count bubbles...

This benchmark won't be compatible with flash uncompatible roms. Obviously.

daeron
[Touch HD Windows Mobile ROM Development] MAMeingast:
I'm afraid your sentence is incomplete! What is on some ROMs like Topix? :confused: ;)
[Touch HD Windows Mobile ROM Development] daeron.be:
Sorry, everything is fine now. :) Thanks for pointing this out ;]
[Touch HD Windows Mobile ROM Development] MAMeingast:
Your welcome! :D

Topix 2.9.4 is out and I'll flash and benchmark it today! ;)
[Touch HD Windows Mobile ROM Development] MAMeingast:
I've added the results for Topix 2.9.4, including the flash benchmark!

But you have another mistake in your instructions on how to do the flash benchmark: You write "and then input /blackstone.swf address in IE" but it should be "and then input /benchmark.swf address in IE"! ;) Also, you should specify that not IE needs to be fullscreen, but the flash animation (by long pressing the animation and selecting "full screen" - I did it the other way around first and had a much higher result of 1075! :eek:

EDIT: I forgot to sort the table after adding the new results... I've updated the file to correct that!
[Touch HD Windows Mobile ROM Development] daeron.be:
I'm ashamed.

but thx :)
[Touch HD Windows Mobile ROM Development] LoopyLoo:
Results Table?

Guys,

(I have a horrible feeling I'm gonna regret asking this, but...)

Where's the results table gone?

I cant see a link for it any more!
[Touch HD Windows Mobile ROM Development] MAMeingast:
LOL

Usually I would flame you for your question (not really), but since you already fear my wrath I will spare you and quote from the first post of this thread: ;)

"It happened again - our spreadsheet grew too big for another online spreadsheet hoster. Therefore, we have to swich different way to distribute our work - via attachments in this topic. To get newest results, just look for a latest post from me, Houms or MAMeingast. Sorry for the inconvenience, but I'm sick and tired of all problems with those services."

So just head back a page and download the attachment from the last of my posts on that page! :D
[Touch HD Windows Mobile ROM Development] Blessed HD:
No shame, you're both doin' a supberb job ;)

New kwbr vanila is out !
[Touch HD Windows Mobile ROM Development] jrmccann:
Hi Daeron,

Thanks so much for offering this service!

Just an idea...how about making 2 separate spreadsheets...1 for the 6.1 ROMs and one for the 6.5 ROMS??

Personally I get so much help out of your benchmarks as your scores are vital in making my decision to flash. As flashing takes a load of time to re-set up, if you can show in your bench marks that there is an improvement in flashing then I do...you can't make that decision accurately on the comments of others!

I am particularly looking forward to your analysis of Dutty's 6.5 R4 V L26's new 6.5 LEO.

Keep up the good work.

Regards

J
[Touch HD Windows Mobile ROM Development] Blessed HD:
+1

I think this would make the comparison much easier & more fair :)
[Touch HD Windows Mobile ROM Development] MAMeingast:
Since the results are available in an excel table, you can manage this on your own... Just open the file and sort the results by ROM-version first and overall results second! ;)

EDIT: In the attached file you can see what I mean: The 6.5 ROMs come first, then the 6.1 ROMs! To do this, do the following: Select all ROMs and values, hit "Sort & Filter" and choose "Custom Sort". Then make the dialog look like it does in the screenshot!
[Touch HD Windows Mobile ROM Development] MAMeingast:
Update: Added the results for Topix 3.0!
[Touch HD Windows Mobile ROM Development] MAMeingast:
Update: Added (because of popular demands... ;) ) the results for Dutty's 6.5 Leo R4!
[Touch HD Windows Mobile ROM Development] Turbii:
Hey,

I can't unzip the file :S gives an error that the achive is either in unknown format or damaged...

Is it because of the file, or my pc? :p

Waiting for your anwser, i would love to see the new results :)
[Touch HD Windows Mobile ROM Development] hrcro:
Zip file works fine with me.
[Touch HD Windows Mobile ROM Development] MAMeingast:
It seems to be a problem with your PC! What software are you using to unzip the file?

If you want, I can send you the file unzipped via e-mail - just send me your adress via PM!

I just can't upload the xls-file here because the extension is not accepted by the forum!
[Touch HD Windows Mobile ROM Development] jrmccann:
Hey, thanks heaps for that...its great just to compare the 6.5 ROMS.

P.S. Am i just missing it on the Benchmark spread sheet, or has the EnergyROM 'Genesis' (23016) ★ |Sep 18| ★ Manila 2.5 not been tested?

I would love to see just the best of Dutty, Miri, NRGZ28 and the up coming L26 LEO 6.5 V1 compared side by side!!
[Touch HD Windows Mobile ROM Development] MAMeingast:
Sorry, we (Daeron, Houms and me) are quite busy at the moment - therefore the ROMs haven't been tested!

But maybe we're lucky and some other users will send us their benchmarking results (hint, hint)! ;)
[Touch HD Windows Mobile ROM Development] MAMeingast:
Update: Added results for PhamQuang 6.5 v64 (terrible!) and for RTT Leet R5 (quite ok).
[Touch HD Windows Mobile ROM Development] MAMeingast:
Update: Added results for Dutty's 6.5 R14 Turbo.
[Touch HD Windows Mobile ROM Development] MAMeingast:
Update: Added results for EnergyROM "Leo" 2009.09.23 (as always with EnergyROMs quite slow benchmarking results but very nice looking).
[Touch HD Windows Mobile ROM Development] houms:
some new mesurement: definitly love Topix 3.0

I lot about 7 or 9 new rom: Miri's Topix 3.0, ...
[Touch HD Windows Mobile ROM Development] MAMeingast:
I think you used an old excel-file to post your results into... In the file you posted some ROMs are missing which I've already tested (e.g. Topix 3.0, Dutty's Leo R4, RTT Leet R5 ...). Could you post your results into the last file I uploaded to merge our data?

EDIT: Never mind! I already merged both tables! ;) The one attached to this post contains all results!
[Touch HD Windows Mobile ROM Development] jrmccann:
Hay, the new L26 6.5 ROM is out!!!

Lets get the show down of the century benchmarked!

L26's V1 6.5 v Dutty's R4 v Miri's v Energy Rom 6.5 Leo

I flash the winner!!

J
[Touch HD Windows Mobile ROM Development] houms:
sorry, and thanks

Thanks a lot.
[Touch HD Windows Mobile ROM Development] Pretenderr:
Yes, I would love to see a benchmark of the new L26 ROM

Regards
[Touch HD Windows Mobile ROM Development] MAMeingast:
I'm on it! ;)

EDIT:

Update: Added results for L26 Leo V1 (nothing special... comparable to Dutty's 6.5 ROMs).
[Touch HD Windows Mobile ROM Development] SatScan:
Manila...

Good to see this LEO is again the fastest!
[Touch HD Windows Mobile ROM Development] daeron.be:
Well, sure it's not the best result in a whole table, but it's one of the best result for the HTC Leo based rom in the table. And it's one of the fastest 6.5 roms, because less than 0,2% difference is no really a difference. :)

Great job on both creating and testing this rom, guys ;)
[Touch HD Windows Mobile ROM Development] alper1:
BAD to see it has the most incredible BUGS!
[Touch HD Windows Mobile ROM Development] MAMeingast:
LOL

Yeah, I have to agree: This ROM has some serious problems (it took me several hours to benchmark it because it froze almost everytime when I disabled TouchFlo and on some hard-resets I weren't even able to complete the setup).
[Touch HD Windows Mobile ROM Development] galthop:
Great thread, thanks for your work.

Would you mind adding this stock ROM to the list: http://xdaforums.com/showthread.php?t=552366

I went to it from Duttys 2.8 and found it to be faster. I would be great to see how it compares with the newer cooked ROMs.
[Touch HD Windows Mobile ROM Development] Laurentius26:
Thank you. :)

What kind of bugs, can I have your test reports please?

You seem to know better then the 20 people in my beta team who tested the rom for two months? :)
[Touch HD Windows Mobile ROM Development] SatScan:
WiKi we have one!

LOL

Topic mode: off

Tell me more??

What bugs??

Read WiKi how to flash... and enjoy!

Best WM6.5 (buttons up version), MAN 2.5 rom till now!

Topic mode: on
[Touch HD Windows Mobile ROM Development] MAMeingast:
I've already reported my problems in your ROMs thread. When I finally got the ROM to work I was very pleased with its performance - but I don't feel comfortable running a ROM that freezes whenever I disable TouchFlo. ;)
[Touch HD Windows Mobile ROM Development] daeron.be:
Lol. Sorry, but then I read your first message about radios, I was feeling kinda sleepy. I'm enabling the airplane mode each time I benchmark any of these roms, so nobody will call me. It was worth mentioning in a first post, isn't it? :)
[Touch HD Windows Mobile ROM Development] Eraser85:
thanks a lot daeron!

@ Mods: Sticky the thread pls :P !
[Touch HD Windows Mobile ROM Development] sweet_eye_h:
AT FIRST THANKS alot daeron for the great subject&work coz god knows ive been waiting4 these benchmarks ;

anyway i think we must seperate the benchmarks of 6.5 rom's from 6.1 rom's coz as i see 6.5 is still new and a whole differnet than 6.1

Thnks Anyway man
[Touch HD Windows Mobile ROM Development] axl30:
Another rom

test the nextgen 1.6 LucaP!!!

is great :)
[Touch HD Windows Mobile ROM Development] BaranE:
THX but why is the data for Energy ROM empty?
[Touch HD Windows Mobile ROM Development] houms:
can I help?

Very great job: it help us to make the right choice.

I'm proposing my help. If there is a way to do measurements, let us know:

Perhaps by explaining your methode and used applications to do this measurements and I suppose, at least, one mesure on one the OS you alredy measured to insure that we are on the same page and the measure I'll provide are aligned with yours?
[Touch HD Windows Mobile ROM Development] axl30:
my test oh nextgen 1.6 LucaP

550,3-35,0-634,2-4587,5-182,0-3119,7-1277,9-730,0-7,29-30,95-5,65-2698,8-418,6-8,836-3627,1-1017,1-7281,4-8,517

37649-6002-38208-10219
[Touch HD Windows Mobile ROM Development] Hardqs:
battery test see which rom is most effecient at delivering its high results... or is it just burning more battery.

just a suggestion.:D
[Touch HD Windows Mobile ROM Development] daeron.be:
This test is extremally time consuming, because you have to charge your battery, wait till you run out of your battery. And I don't think results would be trusted, because battery sometimes lasts longer or shorter...
[Touch HD Windows Mobile ROM Development] daeron.be:
I've added latest Energyrom benchmarks!

Go and see then to find out if this battery-eating rom is worth it :)
[Touch HD Windows Mobile ROM Development] rdgut:
I thought you were going to do Topix 1.6.1 and Vanilla Topaz?
[Touch HD Windows Mobile ROM Development] alper1:
I am also waiting for the Topix 1.6.1 test.
[Touch HD Windows Mobile ROM Development] KukurikU:
in my humble opinion is even faster than the champion, Damn Small Topix 2009.05.18. yep, faster!
[Touch HD Windows Mobile ROM Development] nicelad_uk:
Thats a great way to piss the Chefs off isn't it? Everyone has their own perception on what is a good rom and what isn't, its all based on choice. Benchmarks are good, but some people prefer reliablility and stability and battery performance, not how fast they can make their games work or watch movies, I'd get a PSP or an iPhone for this.

I'm with Onkologs rom and have been with Energyrom and both are brilliant roms, very stable and very fast. Wow, benchmarks prove they aren't great in some respects, but if you piss the chefs off you won't get any cooked roms then you'll be down to stock roms, then we'll all come after you ;)

Lets keep to facts about benchmark and just keep the scores on, don't slate the roms because thats not fair when so much hard work is put in
[Touch HD Windows Mobile ROM Development] alper1:
Yes, I agree totally on that point. I think that Topix 1.6.1 is the fastest en most bug free ROM on XDA. No freezes or hangs... Just fast and stable! Thanks 100000x times for cooking such a good rom Kwbr.