yashsingh1/xdaforums huggingface dataset:
[HD2 Windows Mobile 6.5 ROM Development] freyberry:
Don't agree. I know ROMs with high benchmark scores are OFTEN slower than those with lower scores.

That's why I apply only some tweaks. Doesn't give me the highest scores, but also no lag and more stability. Cause benchmarks can't measure lag that occurs in daily use.

By the way, from your benchmarks, I saw that I forgot to apply some tweaks in the 28014 builds. New versions are already on the way, I hope you can benchmark them immediately when they are up, as they will have much higher scores!

(the 23518 versions are already tweaked, see my thread for the scores)
[HD2 Windows Mobile 6.5 ROM Development] pietrucci:
Ok, if you say so. Can you give an example on Leo so I can confirm or try that out? Till now I didn't had that situation.

EDIT: Sorry, you don't need to give me an example. I think that could be seen as an attack to other cooks.
[HD2 Windows Mobile 6.5 ROM Development] duttythroy:
Just want to saw something before this thread start heating up, I use to think benchmarks reflects the true performance of a Rom but realise this isnt alaways the caase.

A xip can be hacked and registry changes made so it can show a highler number but this can also reflects the Rom, This can cause outlook notes not to sync and some programs to crash etc

I'm also and so other chefs out there perform xip hack , it does help sometimes speed up the roms but using HD2 I didnt , or dont think the HD2 needed all that xip hacking stuff as the stock rom itself was fast and doesnt need much chef input to make it better.

I think we should avoid start complaining about which score his higher and just have a feel of the rom , the responsive of the rom and any lagness of the rom in which you can decide to yourself which rom feels faster to you and with performance and stability.

my 2cents

Dutty.

@pietrucci nice job as always and I know yit must take times in doing all these benchmarks so fast.
[HD2 Windows Mobile 6.5 ROM Development] tom_codon:
@Laurentius26 : You did very good research in Xip place , long times i didn't cook Rom . And now back cook for funs only , so not need to angry me :)

@pietrucci : i know bro , manila disable but some dll of manila still run as Service , and one of them make chefs can't hack xip like L26 did .
[HD2 Windows Mobile 6.5 ROM Development] xmoo:
Don't think it would make such a big diffrence, Laurentius roms have always been fast....
[HD2 Windows Mobile 6.5 ROM Development] pietrucci:
Your absolutely right dutty. This has been discusses soooooo many times before.

Benchmarks are just an indication, play with the ROM an choose whichever you like most.

And I'm just a poor lonesome tester...:o
[HD2 Windows Mobile 6.5 ROM Development] freyberry:
You know, from making ROMs myself, I know that it often happens that they give higher benchmarks but lower real life performance.

I'm not saying you should not benchmarks ROMs, but a benchmark can definitely not replace a real life test.

Another thing: As I said before, new versions of my ROMs are on their way and the benchmarks you have from my 28014 ROMs are already outdated.

Could you please take those down and benchmark either the 23518 versions (because the upcoming versions will be like that) or wait for the upcoming versions?

Sorry for that, but I really would be glad if you had benchmarks of the actual versions of my ROMs in your list, cause they are a lot better that what's now in you list.
[HD2 Windows Mobile 6.5 ROM Development] Laurentius26:
I will fix the sync notes in next upcoming V1.1 and show you it doesn't mather.

You know that just as me. ;)

My 2 cents...

Leo :)
[HD2 Windows Mobile 6.5 ROM Development] Hellas4:
We are keep talking about how fast is a Rom from one chef or another.. i use Tom's ROM right now,previus was itje's,dutty's,xanny's,miri, and all the good stuff out there...Leo is by itself fast,so there is no need to argue about how fast is roms cause benchmarking is just numbers.I think the "fast" and "slow" rom on Leo it's just a difference of a blink of an eye!
[HD2 Windows Mobile 6.5 ROM Development] gianton:
+1 on that

It's really a matter of taste which ROM you like more. Later in say 6 months time when wm7 may arrise then chefs will have more work to do!
[HD2 Windows Mobile 6.5 ROM Development] pietrucci:
I will test a new version soon, but I hope you understand I won't take down any results. In my benchmark sheet there are several ROMs from a while ago or that are replaced bij newer ROMs and I don't take them down either.

As for the discussions here I don't really like them. This kind of discussions are going on for years now since I started my benchmarks. I just want to test and compare ROMs looking for the perfect ROM for me (I still haven't found it). And I still think SPB Benchmark is a good indicator for that.

And I want to share my results and experience with people who like it. No more.
[HD2 Windows Mobile 6.5 ROM Development] freyberry:
Well, you have to accept that those discussions will continue. Cause actually, things like what's included in the ROM, how stable is it, does it lag while using it etc. are so much more important than benchmark scores, that can hardly say anything about the real life experience.

Anyway, I hope you can test the V5SE_23518 version or the new V5.3 version soon, cause the current scores in your list give a false impression of my ROMs and to be honest, I don't like that much.

Anyway, I do benchmarks of my ROMs myself (that's why I know high scores don't mean a fluid real life experience) and you can see the scores of my most recent ROMs in my thread.
[HD2 Windows Mobile 6.5 ROM Development] pietrucci:
New ROM

Benchmarked new ROM: XannyTech Evolution 1.0 RC1 (23518/28205, Sense 2.5.1922)

See post #1.
[HD2 Windows Mobile 6.5 ROM Development] NIKOSXRI:
21887 and other builds

It appears that there is a huge difference between 21887 and other builds in scores.

Since I have not yet a device to test, is this great difference seen while using these builds?

I mean does the user feel this great difference between 21887 and other builds?
[HD2 Windows Mobile 6.5 ROM Development] c4Lvin:
Battery Drain reading/clarify

Can someone explain how to read those numbers? I'm just abit confused of which is better power consumption or the more hungrier. Can't tell from those numbers. Sorry for a stupid Q'

Regards.
[HD2 Windows Mobile 6.5 ROM Development] band27:
I was just going to comment on that. Xanny's latest one (23518/28205) stands alone at the 90% level.

Wonder what build 21889 will bring as the 2188x branch has been doing well.

Keep up the great work Pietrucci!
[HD2 Windows Mobile 6.5 ROM Development] faraz.riaz:
I always did tell a person being ‘popular’ has its price. I might be having an illusion but I feel people are jealous of someone’s ROM performing better than others. I think I was the only ‘juvenile’ but people come and surprise me. “Tom_Condon” shouldn’t you be one supporting the efforts of people who spent their time to make something good for others. I don’t think jealousy is something you should be concerned with. To be honest you sound a little desperate and a lot like an idiot. You should never say I never gave you some good advice.
[HD2 Windows Mobile 6.5 ROM Development] thomas_pieps:
any chance off benchmarking the new Miri V2.0?

because I'm this one currently and it is a really fast one. So I think that the score would be higher than the V1.0
[HD2 Windows Mobile 6.5 ROM Development] appelflap:
I have a little problem with interpreting the benchmarking results as it comes to mapping them to my experiences with using the device. For example, for me it is important that menus will open without much delay, to which variables should I look then? Or when I find it important that lists (not directory lists :) ) scroll without any hampering, where should I look to? Or, when I find it extremely important that the HTC Music library is build up really quick after a softreset, to which chef should I turn to?

Maybe one could define (in addition to those figures) a series of tasks and benchmark those tasks (the way they do it for example with cross-platform browser testing, and pietrucci has done for movies). At least it would be nice to map the benchmarking variables to higher level OS tasks.
[HD2 Windows Mobile 6.5 ROM Development] xmoo:
Jealousy is one of the things.

But also not sharing sources with some chefs is the same thing.

This way they stay ahead of competition in having new stuff.

XDA has changed a lot :(
[HD2 Windows Mobile 6.5 ROM Development] Audio Oblivion:
Please chefs... dont argue!

You are all doing great work and us users appreciate EVERY single rom that gets released.

Thre is no need to bicker about whose rom has this or that feature and whose rom is fastest etc... because in the real world it doesnt actually matter!

People have their own preferences and this will always be the case.

Me for example... i am sticking to my stock rom at the moment as i dont feel there is any significant improvement available in ANY custom rom that is out there.

You are all doing brilliant stuff so share the wealth and take pride in yours AND other peoples work.

This will only lead to better firmware for all of us!
[HD2 Windows Mobile 6.5 ROM Development] bobsbbq:
Well I'm the quiet one. I agree with some of the other comments made here about using this only as a guide. You really need to flash and test to get what you want.

My first ROM had almost no speed tweaks as I wanted to get used to the HD2 as I flashed it on day 3 after purchase. But hold on to your hats as this might be my next canidate V0.2
[HD2 Windows Mobile 6.5 ROM Development] kwbr:
Agree throy. What I don't understand is why people use some mods although it is known these will cause all kinds of problems. Only to get a high synthetic benchmark which says nothing about hickups, crashes, reboots and hangs ;)
[HD2 Windows Mobile 6.5 ROM Development] bobsbbq:
I agree with this as well.
[HD2 Windows Mobile 6.5 ROM Development] freyberry:
That's exactly what I'm saying. A few more benchmark points are worth nothing when the ROM crashes, lags etc.

That's why a real test is so much more important than benchmarks and benchmarks can really give a very false impression of ROMs - those with the lowest scores might actually be the most fluid and stable ones in real life.
[HD2 Windows Mobile 6.5 ROM Development] rm001:
I`ll never believe that. The most fluid will be the most stable of the fastest ROMs, but the slow ROM can`t be the #1 while exists a stable but faster ROM.

As soon as we can`t yet do any conclusions about stability of any ROM, naturally we try the fastest ROMs at first )

Besides, let`s be honest: those who has bought HD2 after HD, will trust Laurentius26, kwbr, LIA, Dutty and others...and first of all will try their ROMs, especially if results of testing are not disappointing. All as in a real life )
[HD2 Windows Mobile 6.5 ROM Development] freyberry:
Read that again.
[HD2 Windows Mobile 6.5 ROM Development] chris_ah1:
agreed, especially when SPB is so biased towards the file system figure.

What if you disable flash memory safeguards to verify writes, the device wrongly thinks a sector is written, and then on read it crashes.

It might be fluid and quick 90% of that time, but if you cannot rely on it, what is the point?

For a business device, it must be near 100% stable without glitches, or at least as good as the stock rom for stability.

remember, fluid + stable is key.
[HD2 Windows Mobile 6.5 ROM Development] Laurentius26:
OMG OMG don't make me laugh :D:D:D

You guys are all patching XIP, applying spinlocks, changing dlls, changing regs to get more speed and now your complaining here that these patches make no sense? :p

Oh my.. what a bunch of hypocrits :eek:
[HD2 Windows Mobile 6.5 ROM Development] freyberry:
Laurentius,

As much as I appreciate the work you're doing, but this post is just ridiculous.

We never said that those tweaks make no sense. And yes, we do use them ourselves.

We only said that a few more benchmark points are worth nothing when the ROM isn't stable and fluid. This means that sometimes it's better not to apply certain speed tweaks, which may lead to lower benchmark scores but to a much more fluid and stable real life experience.
[HD2 Windows Mobile 6.5 ROM Development] mightymn:
guys, since we all agree that these benchmarks do not necessarily reflect real life performance/stability wouldn't it be best to just state this in big bold red in the first post so that newer users know they shouldn't judge the ROMs only by these?
[HD2 Windows Mobile 6.5 ROM Development] Laurentius26:
Haha +1 thank you. :D
[HD2 Windows Mobile 6.5 ROM Development] freyberry:
Makes sense and would probably put an end to those discussions.
[HD2 Windows Mobile 6.5 ROM Development] bobsbbq:
Well I have done some testing. I can 100 % say that on my ROM's flashing without Storage card does make a difference. The benchmarks are faster, but more importantly it is more smooth and stable. I will be instructing all my users to flash without storage card, not for the benchmarks but for better stability.

There is some strange things going on with the storage card ever since the Documents tab was introduced. This has also led to problems with Extendir.

I suspect that they could be related.
[HD2 Windows Mobile 6.5 ROM Development] pietrucci:
Yes I could do that. A big "READ AT YOUR OWN RISK!".

But that would be the 10.000.000.000th time I did so. See this and all of my previous benchmark threads, it's all over the place that this is "just" a benchmark.

Guys what are we all talking about, a phone... A very very nice phone but still a phone.

And a benchmark which even might not be very reliable nor objective...
[HD2 Windows Mobile 6.5 ROM Development] freyberry:
It's up to you, but if you add this every time you make a benchmark thread, you probably won't see this discussion again, as everything would be said with the firt post already.

@bobsbbq

Interesting indeed... will try that.
[HD2 Windows Mobile 6.5 ROM Development] mightymn:
Well it's not like every one reads all the thread to know these are just benchmarks. And with the HD2, there've been lots and lots of new users around that may not know how these benchmarks are supposed to be interpreted or what they actually reflect.

So I'm just saying it will save everyone a lot of redundant explanation. It's up to you ;)
[HD2 Windows Mobile 6.5 ROM Development] pietrucci:
Ok, added it to first post.
[HD2 Windows Mobile 6.5 ROM Development] Mikulec:
and sterile.

lol :rolleyes:
[HD2 Windows Mobile 6.5 ROM Development] bobsbbq:
If half the effort was made on the ROM's as has been made in this thread we should have ROM's that will melt our HD2 down. :D
[HD2 Windows Mobile 6.5 ROM Development] Laurentius26:
Nah I don't think it's that necassary as every human with comon sence would know this to my opinion.

I was only joking with my +1 as some guys are so anoying here and they just can't stop. :rolleyes:
[HD2 Windows Mobile 6.5 ROM Development] freyberry:
Haha, you're an optimistic guy...
[HD2 Windows Mobile 6.5 ROM Development] mightymn:
was thinking the same :D
[HD2 Windows Mobile 6.5 ROM Development] Laurentius26:
Yup that's me. :D

I'm not walking on clouds like some people do here. ;)
[HD2 Windows Mobile 6.5 ROM Development] tnyynt:
Could we get back to the topic please?
[HD2 Windows Mobile 6.5 ROM Development] freyberry:
Isn't the topic benchmarks and isn't that what we're talking about?

Just sayin'...
[HD2 Windows Mobile 6.5 ROM Development] pietrucci:
New ROM

In the meantime...

While you where speaking I was testing: freyberry ROM.V5.3 23518 (23518, Sense 2.5.1922).

Fast, almost as fast as XannyTech's 23518/28205. But a few graphic glitches.

See post #1.
[HD2 Windows Mobile 6.5 ROM Development] gianton:
That's the feeling it gives in real life as well a fast ROM. Will keep testing it for any stability issues.
[HD2 Windows Mobile 6.5 ROM Development] freyberry:
Thanks, what graphics glitches do you mean? EDIT: Nevermind, saw you posted it on my thread. Thanks for benchmarking again.
[HD2 Windows Mobile 6.5 ROM Development] pietrucci:
See my post in your thread. Also noticed it in a previous ROM.
[HD2 Windows Mobile 6.5 ROM Development] freyberry:
Haha, just when I saw it and edited my post, you posted here :D

Haven't you seen this on other build 23518 ROMs? Cause I thought it was related to the WM build. (You can answer in the other thread, as it's a bit OT here)

Interestingly, your benchmark results are quite a bit better than mine, especially the file system index. Do you benchmark without the storage card inserted?
[HD2 Windows Mobile 6.5 ROM Development] 4tune:
I loved reading the last few pages of this thread. All the big guns blazing… at each other.

Freyberry arguing that benchmarks don’t mean **** and in the next sentence almost begging pietrucci to test his new ROM, you gotta love that :D

You know what, I think it’s a good thing. It means you all give a damn. A little banter like this keeps you all as sharp as a knifes edge. And that’s good news for users like us. It means fast and frantic development. Go for the top.
[HD2 Windows Mobile 6.5 ROM Development] freyberry:
I'm not saying that benchmarks 'don't mean ****'. What's wrong with you guys?! Why are you always misinterpreting what others say? First Laurentius, then you...

I even said that I benchmark my ROMs myself (as you can see in my thread). Would I do that if I thought benchmarks 'don't mean ****'?

It's just that some people here think that high scores always mean that the ROM is fast in real life - which is just not the case. And this should be clarified in the first place, so that discussions like this would not be necessary.

I really hate when people are twisting my words, deliberately misinterpreting what I say just to push their opinions. Anyway, all has been said, so why did you have to start this again?
[HD2 Windows Mobile 6.5 ROM Development] Bugblatter:
I remember Anand of Anandtech many years ago saying that benchmarks are actually a pretty poor way to compare gfx cards.

Mfrs optimise their drivers for the games and benchmarks that are commonly used, make use of lower-quality AA and lower image quality just to pump up the numbers.

In the end what matters is that the card is fast enough for your needs and gives great image quality, but because it's subjective, and qualitative rather than quantitative, it's hard to compare different cards.

His conclusion was that they'd continue to use benchmarks because they couldn't think of anything better, but to always bear in mind your own requirements and priorities.

Same goes here. Even the stock ROM seems 'fast enough' to me. Extra performance is always nice, but more important to me is stability and leaving me some room to install apps!
[HD2 Windows Mobile 6.5 ROM Development] Xeon:
Thanks for confirming...I have reported here http://xdaforums.com/showthread.php?t=615398

Some results would be appreciated :)
[HD2 Windows Mobile 6.5 ROM Development] band27:
Very interesting and yet bizarre observation regarding the SD card removal and better performance results....:confused:

AND THIS....

READ THIS THREAD AT YOUR OWN RISK!

MISINTERPRETATION COULD MAKE YOU BLIND!

Hilarious!!
[HD2 Windows Mobile 6.5 ROM Development] x3dfx:
This thread should be a sticky ^^
[HD2 Windows Mobile 6.5 ROM Development] pietrucci:
As I can't believe that I will do my benchmark on the XannyTech Evolution_RC1 again. Now without my sd-card (and part of it without the buggy AmeBa).

I predict no speed gain. We'll see...
[HD2 Windows Mobile 6.5 ROM Development] manuelcalavera:
Pietrucci,

tnx for the great job, please update every rom updare, most of them updated new base rom and packages
[HD2 Windows Mobile 6.5 ROM Development] pietrucci:
If you want the updates to be updated you must ask the cooks...

But seriously I'm not gonna bench every new ROM because that's just not possible. Every ROM takes 2 hours, one day has 24 hours, so theorethically I could bench about 12 ROMs (yes I still can count). I guess I could do that... NOT!

Unless you send me about 12 HD2's? I'll PM you my address if you want.
[HD2 Windows Mobile 6.5 ROM Development] pietrucci:
Just as I expected: no difference whatsoever. Look in first post, results without sd-card are about the same as with sd-card. So another mythe busted I'm afraid...
[HD2 Windows Mobile 6.5 ROM Development] Laurentius26:
Hi pietrucci,

Some cookers are using Chainfire's drivers I think to have higher graphics index.

If you look here you see that Chainfire expleciet asked not to cook in these drivers.

I would say that isn't a fair comparison isn't. ;)

http://xdaforums.com/showthread.php?t=592663

Maybe it's good if these chefs mention if they used them or not.

Grtz Leo :)
[HD2 Windows Mobile 6.5 ROM Development] bobsbbq:
Well I don't know about yours, but with my device and my SD card it is always higher without SD Card when flashing. I also find that manila is smoother, more noticeable on the 2.02 Test than on 2.01.

So in my case there is no myth. But we are not here to talk about who's device works and who's does not.

Maybe when I release my next ROM we will see if it is only me. Maybe you will get the same results.
[HD2 Windows Mobile 6.5 ROM Development] manuelcalavera:
agree about the timing problem, but u can do it like tournoment :D

like u can test new releases of best scores in your benchmark table , semi final thing i guess, lol
[HD2 Windows Mobile 6.5 ROM Development] wizard6601:
XannyTech Evolution 1.0 RC1 battery drain (min) is 2ma !!!! :eek::eek::eek:

is it real ?????????

are u sure ?????

any body sense it in practice ?
[HD2 Windows Mobile 6.5 ROM Development] capital:
Dear pietrucci's !

Can u test benchmark of Tom v1.1 ?

Look like it much faster than 1.0

Thanks
[HD2 Windows Mobile 6.5 ROM Development] Xeon:
I wouldn't have opened a thread about it if it was only you. I almost start believing it's true. I don't know for others but I am sure there's a difference....
[HD2 Windows Mobile 6.5 ROM Development] pathologo:
The difference in performance is noticeable when you *flash* the ROM with *card inserted or not*,

but I think pietrucci did not test that but instead he *benchmarked* with *card inserted or not*.

But as we all know there is no performance difference after flashing, the difference is made *while* flashing.

Oh well.
[HD2 Windows Mobile 6.5 ROM Development] bobsbbq:
This was the exact same ROM I just hit the cook button twice one with storage card and once without.

With:

Without:
[HD2 Windows Mobile 6.5 ROM Development] Laurentius26:
Hi Pietrucci,

I'm having a weird issue and I taught you are the benchmark expert so lets ask you. ;)

My CPU Index is doing very strange.

I mean I almost returned my device for repair as my device was not scoring a CPU Index higher as 4000.

I tried to benchmark with stock and cooked roms but it doesn't mather.

My device itself isn't slow and performing as it should be.

Then suddenly this week I did the benchmark using the USB cable to PC and my device scored a CPU Index of 7000. :confused:

So I was very happy and also the benchmarks after (stock or cooked rom) scored the regular index of 7000.

But yesterday I benchmarked my device and suddenly it's 4000 again. :(

Atm I'm flashing stock roms or cooked roms, using adapter and using USB to pc but it doesn't mather I'm back to a CPU Index of 4000.

Any idea, did you experienced something like this?

Another friend was having the same issue with his device so I taught maybe more people experiencing this.

Thanks,

Leo
[HD2 Windows Mobile 6.5 ROM Development] Mikulec:
Leo, u could try the latest sktools and their benchmark tool. Its updated to show the benchmark scores of stock HD2. So u can benchmark the cpu there and see if there is some problem.