yashsingh1/xdaforums huggingface dataset:
[Touch Diamond, MDA Compact IV ROM Development] mondilv:
Not, of course.

But always 1.00.25.05 was the fastest by spbbench. And the "trend" was very evident. At every rom in any time.

Even by subjective look to your's last table. I didn't summ index - just compare by strings.

Other (jounger) radios are very similar each to over in this test. Tried original Diamond & "alien" radios.

Possible index calculating have any summary coefficients?

P.S. I didn't compare battery consumption .05 to others.

Situation can be not so good as at later radios. By straight logic.:)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
mondilv and other interested,

I think I know where the "pain" lies...

It's the fact that the indices are calculated with a weight factor. This means that all individual testresults have their own weight.

As you can see on the technical page from SPB, http://www.spbsoftwarehouse.com/products/benchmark/indices_descriptions.html , the formula for calculating the indices is:

where C1, ...,Cn are the weight factors.

When you look closely at this formula you'll see a obvious contradiction: the HIGHER the weight of a test, the LOWER the influence of that test on the index! So a testresult with a weight factor of 5 has

more

influence than a testresult with a weight factor of 20!

Now, look at the File System results of both of the radio's .05 and .20 in your Nataly ROM. You'll see that almost all results from radio .05 are BETTER than .20. BUT... the results from test "Directory list from 2000 files" from radio .20 is faster than .05 (1185 vs. 1211). And this individual test has a weight factor of 5, while the rest have a factor 20, 15 or 10!

Conclusion: because of this individual test with a weight of 5, radio .20 in the File System index get's faster than .05!!!

I do think it's a bit strange that a high factor has the least influence, but that's the way SPB calculates it's results. I guess they have a meaning doing this because of the sysntax of the formula... (they give the most important tests the lowest factor).

I hope you get my point.
[Touch Diamond, MDA Compact IV ROM Development] alphazero:
welcome back pietrucci! i didnt know your a math genius. :D

thanks for the formula explanation.
[Touch Diamond, MDA Compact IV ROM Development] mondilv:
Thanks, clear now.

Did think so.

Now I'm relax.:D
[Touch Diamond, MDA Compact IV ROM Development] mondilv:
May be is a reason to test radios too?

:D

Take one rom & check performance at diff radios.

What do you think, Pietrucci?
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Maybe I could do that.

But first of all it's a lot of work...

As you know I do all my tests 5 times. And I experienced that some hardresets give a slow result (don't know how that's possible, but it is!), so I have to hardreset and test again...

So it takes very very much time.

Secondly I think it's very strange that different radio's gives different testresults. My simcard is removed, my radio is off, so what's the influence of the radio??? (none I would say.). But radio DOES matter, I already did a simple test comparing radio's .05 and .23. You can see that in my Excel sheet on the tab Radio. Radio 0.5 is faster...
[Touch Diamond, MDA Compact IV ROM Development] nicholas99:
Anyone can help or tell me am i done something wrong with my diamond phone coz i have try few of the ROMs that the program memory should be around 60mb to 70mb like nataly v2.0 final rom and CTs v6.4 Org TF3D rom, But after try flash few time and the rest of the ROMs got big size for the program memory still i got only 50mb. Why was that....
[Touch Diamond, MDA Compact IV ROM Development] mondilv:
Look at the first post: Manila & PhoneCanvas are disabled before tests.
[Touch Diamond, MDA Compact IV ROM Development] nicholas99:
lol i got it now....sorry didnt look on it.....cheers.
[Touch Diamond, MDA Compact IV ROM Development] Dr.Tenma:
Hello, i got question, in first post, you give link to your opinion, i read it. But noticed that this opinion is old(2.03 is out now), is your opinion updated though ? I mean is still cooked rom's based on 1.93 still better ?
[Touch Diamond, MDA Compact IV ROM Development] losdrivare:
I miss your benchmarks, Pietrucci.

There are new and interesting ROMs out there, like Nataly 2.1, udK Syrius Topaz R1 and PDAcornerIllusion V3, just to mention a few. But how do they compare in the challanging Pietrucci benchmarks? :D
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Ok, because you miss me that much, losdrivare, here some new benchmarks:

swtos V2.6 Light (20764)

udK Syrius Topaz R2 (21032)

Nataly V2.1 Final (20771)

Cloudy WM 6.1 V1.0 (21018)

PDACornerIllusion V3 (21039)

With it's first Diamond WM6.1 ROM, Cloudifa's goes straight to the top: fastest ROM I benchmarked (except for the exotic one's)!

See post #1
[Touch Diamond, MDA Compact IV ROM Development] mondilv:
Thanks for test!

Just a little comment: in Nataly V2.1 Final Opera is present.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Ok, I didn't notice, I thought it was in sdconfig like previous versions. I'll correct my comment for the next version of my spreadsheet.
[Touch Diamond, MDA Compact IV ROM Development] dysje:
You may want to update the your advice post you made, it tells me to go for a 1.93 rom but it seems old since the 2.03 finals are out for a while now?
[Touch Diamond, MDA Compact IV ROM Development] cloudedhopes:
welcome back pietrucci :) was missing ur benchmarks :) btw, you forgot to update the date in the title :D
[Touch Diamond, MDA Compact IV ROM Development] losdrivare:
Awesome, thanks! You're the man! =)

It's tight on top these days, a lot of ROM's are getting top scores +/- a few points.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Ok guys, thanx for your reactions.

I changed the date in post 1 and my advise in post 199.

EDIT 16-03: Corrected some small errors in Excel en recent ROM file. No errors in figures, just in texts.
[Touch Diamond, MDA Compact IV ROM Development] gregmits:
According to the new file, Cloudy is not the fastest. Is that right?
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
No that is NOT right! Cloudy IS the fastest!

Sorry guys I really ****ed up this time, I think I'm getting a bit rusty...

All measurements are correct, only the calculated corrected index was wrong, because I forgot to exclude a ROM with "strange" results.

Now everything corrected.

Sorry again...
[Touch Diamond, MDA Compact IV ROM Development] losdrivare:
Never you mind. No one could do what you do better! :D
[Touch Diamond, MDA Compact IV ROM Development] drgupta1981:
hi there,

you mentioned that you do automated installs for programs after downloading naked versions of ROMs.

Can you please tell me the detailed procedure how to do that??

Regards.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
I already PM'd you...
[Touch Diamond, MDA Compact IV ROM Development] DefJamz:
hi.. is possible to have these 2 roms benchmarked?

aFast v2.03 (b35)

vs

[url="http://xdaforums.com/showthread.php?t=422417] A.Z.T.O.R. X2 Ultimate[/url]?

i need to know the faster of the 2 that's as close as possible to the stock 2.03 ROM.

Thanks in advance.
[Touch Diamond, MDA Compact IV ROM Development] laser21:
Thanks for your time pietrucci! I think a lot of us would be lost in the sea of new roms floating around without your xls sheet :-)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
News ROMs: udK and Cloudy

2 more new ROMs:

udK Syrius Topaz R4 (21040, 2.04)

Cloudy WM 6.1 V1.1 (21040, 2.04)

Both ROMs are 21040, 2.04 version.

And both give very bad SPB-results especially in CPU and File system!

Don't know why, I hardresttet many times, just to make sure I didn't test wrong. But always these bad figures...

There are two possible reasons for this: 1) SPB cant handle this 2.04 ROMs or 2) there ROM version stink on our Diamonds...

See post #1.
[Touch Diamond, MDA Compact IV ROM Development] cloudedhopes:
its good to see you are active once again! :)

just wondering - any chances of benchmarking the aztor's X2?
[Touch Diamond, MDA Compact IV ROM Development] mondilv:
...or didn't change pagepul from 8PP. As I see, udK use 2.04 drivers. If payload from "Orange" rom, PP is 8M as default.
[Touch Diamond, MDA Compact IV ROM Development] Dr.Tenma:
Yes please the Aztor's x2, the aFast and Nataly v2.1.1 if possible...Thank's !

Also, i saw in your excess file, that the wildberry has best sound score, does it mean, sound is more powerful or quality is better ?
[Touch Diamond, MDA Compact IV ROM Development] aussie1234:
Thanks for your stats

Hi Pietrucci

Thanks for your ROM Statistics.

I have found them very useful in choosing my ROMS.

And I do agree that the new 2.04 ROMS are noticeably slower than previous ROMS. I have seen this in UDK's R4 and Cloudy's 1.1 ROM.

Keep up the good work.

Aussie :)
[Touch Diamond, MDA Compact IV ROM Development] wencyv:
May I know which ROM has at least all the new setup for HTC Diamond2? I would really love to install one.

please advise.

thanks
[Touch Diamond, MDA Compact IV ROM Development] cloudedhopes:
since XDA got restored - it lost the last update

attached is his last benchmarking done on 29th March 2009

until pietrucci gets chance to update his posts :)

EDIT: pietrucci already updated
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
New ROM

Tested new ROM: swtos V2.71 Full (21042)
[Touch Diamond, MDA Compact IV ROM Development] shaks1979:
Hi

Any chance of testing my latest: Shaks.P v4.00 (21042/2.03MRII) 12Mb PP?

Thanks for taking the time to do all these benchmarks. Much appreciated.

Regards
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
New ROM: Shaks.P 4.00

Tested it, see post #1.

Hate to be a pain in the ass, but at the moment I'm disappointed about the speed of the 21040 and 21042 builds. They just don't seem to fit that well on our Diamonds. Not only benchmarks but also in normal use I just dont' feel them so snappy.
[Touch Diamond, MDA Compact IV ROM Development] cloudedhopes:
pietrucci - i've noticed you're now testing the ROMs with radio 1.09, could that be?
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Yes, you are right that radio can make a difference. But I compared radio 1.00.25.05 and 1.09.25.23 with two recent ROMs (Nataly 2.2 Test and udk R4): results were about the same!

Just to do another test: right now I'm using Cloudy 1.0 with radio 1.09.25.23, I will test this one again and compare it with the results of radio 1.00.25.05 (already in my benchmark).
[Touch Diamond, MDA Compact IV ROM Development] losdrivare:
Judging from your scores 21040 really seems slow, both udK and Couldyfa usually makes really fast ROMs. But 21042 seems as fast as the fastest builds to me, at least Mondilv got a great score with it. If you compare the 21042 ROMs, the only significant difference is that Nataly has a higher File System score.
[Touch Diamond, MDA Compact IV ROM Development] mispook01:
CDMA ROMs

Which of these ROMs can be flashed to the Sprint Diamond.

Thanks
[Touch Diamond, MDA Compact IV ROM Development] shaks1979:
Thanks for testing. It's slower than my last roms according to the benches however feels very quick here. Feedback I'm getting is it feels very quick and I don't get slowdowns which I did on the previous builds.

P.S was this run with opera still installed? I don't think it would make much difference however I remember you uninstalled opera on my 3.10 build. It's not important anyway.

Regards & thanks for your hard work & testing.
[Touch Diamond, MDA Compact IV ROM Development] band27:
Interesting results consider the snappy overall feel of Shaks latest installment. Quite surprised!
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Large cache = slower testresults!

Ok here my little theory, try to explain in English...

When you set the system cache and glyphcache, the system loads everthing in cache first and that takes a little time. The larger the cache the more time you need to put everything in it.

For normal use that's no problem because you re-use the caches so everything works faster.

BUT... in these benchmarks everything is tested only once so the caches aren't really re-used. But still the tests do need some time to put it in cache. And therefore the testresults are SLOWER when caches are larger...

How about this theory (hope you understand what I mean?)

I tested it with Dutty 2: the results are much faster with small caches!
[Touch Diamond, MDA Compact IV ROM Development] duttythroy:
That makes lots of sense and I think ur right probably, as if u notice in my roms if u open a foler e.g multimedia it takesaround 2 seconds but next time its like instant
[Touch Diamond, MDA Compact IV ROM Development] hallatore:
Run the test 3 times and select the results from the last run.

Will give a much better result for "overall usage" :)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
well, I could do two things: testing the ROMs as they are. That means that cache settings are different because cooks often use their own settings...

Or I could set cache values to a default value before testing. But then I don't test the ROM as it is.

So what's the best strategy?

Why I changed Dutty's settings? Well I got very bad testresults and couldn't understand this because it feels so fast. After many setup changes I finally decided to lower the caches. And then Dutty was very fast, both in SPB and VSBenchmark!

So I figured out the cache theory a few posts back...

Don't know how or what exactly, fact is I get very different (= better) results with small caches!
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Yes I guess it will...

But maybe you understand flashing and testing a ROM with both SPB Benchmark and VSBenchmark takes about an hour.

Doing this 3 times for each ROM...

Last week I tested about 20 ROMs (my poor device...).
[Touch Diamond, MDA Compact IV ROM Development] hallatore:
Ah, 2 times and take the last result then :p

3 times is only to get a "perfect" result.
[Touch Diamond, MDA Compact IV ROM Development] bepe:
I'm voting for the first strategy, bc most of the users do not know about this settings or are not changing them... and I think it is part of making a rom to set this to the best value.

One interesting thing to add that would be the bootup memory, but just a suggestion.

I know how much work you are doing there with flashing and testing every rom, thank you very much for that!
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Ok, I can live with that strategy. Only problem is that sometimes tests will be bad (with large caches) while in daily use these ROMs are fast (also because of the large caches).

I already have the bootup memory, but left it out in the published file. Here a version WITH the bootup memory (on the bottom).
[Touch Diamond, MDA Compact IV ROM Development] panosha:
I think Diamond works better (the first PPC) and with faster feeling with standard caches. ;)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Second Dutty added

Ok, second ROM from Dutty added.

Tested with default cache settings, make testresults better (writing to cache slows down)

From now on I will test every ROM with default/disabled cache settings so I compare the ROMs with same cachesettings.
[Touch Diamond, MDA Compact IV ROM Development] kurgandk:
could you guys include standard roms in your tests? It would be nice ti see how much reason there is to venture into the undiscovered land of cooked roms... ;)
[Touch Diamond, MDA Compact IV ROM Development] murks_muell:
bepe has a new rom, v0.93...
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Bepe 0.93

Yes I know, I tested it already yesterday.

But to be honest: I was so disappointed with the graphic testresults I hadn't the courage to publish it.

I tried several cache settings to make it perform better in the tests but nothing really worked, it stayed slow.

I'd really wanted it to be fast(er) because Bepe's 0.54 also was very fast (in fact: overall the fastest around)

Although it feels pretty fast during normal use (not really superfast but "pretty" fast) I just can't make more of the tests...

Post #1 updated...
[Touch Diamond, MDA Compact IV ROM Development] saveferris:
Wow, bepe has gone from the fastest Rom to the slowest in his next ROM, thats quite interesting. Looks like his exams are getting ther better of him. Keep plugging away bepe. :)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
I'm sure it's NOT because of Bepe, I think its the new ROM itself.

Also the tests don't fully represent how we users experience the ROMs in daily use.
[Touch Diamond, MDA Compact IV ROM Development] walshieau:
1.93 has MAJOR issues.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
perhaps time to test Elite 2.5 beta...

:D:cool:
[Touch Diamond, MDA Compact IV ROM Development] walshieau:
fingers crossed tonight :D

two more flashes left and i reckon im done
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
pssssst... (don't wake up anybody...)
[Touch Diamond, MDA Compact IV ROM Development] saveferris:
I know that, I was just doing a bit of **** stirring, I 'm a big fan of bepe to the point i'm still using his 0.54 Rom. :)
[Touch Diamond, MDA Compact IV ROM Development] mgs333:
none (10 chars)
[Touch Diamond, MDA Compact IV ROM Development] cloudedhopes:
did u get a chance? any interesting findings?
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Well yes. I did several tests with radio's 1.00.25.05 and 1/09.25.23.

I took de average differences in radio results and recalculated the benchmark results from the recent .23 ROMs I tested, back to radio .05.

The total results in short (will publish my sheet with detailled results soon):

.....................................Benchmarked.......Recalculated

Radio...............................1.09.25.23...........1.00.25.05

udk Syrius Topaz R4..................906......................915

Cloudy WM6.1 V1.1...................909......................919

swtos v2.71 Full.......................938......................948

Shaks.P v4.00..........................926......................936

If you would take a look in my benchmark spreadsheet, you'd see these results wouldn't change the overall ranking, the best ROMs stay the best.
[Touch Diamond, MDA Compact IV ROM Development] cloudedhopes:
wow - thats very interesting, however i still can't understand why the change when the radio is turned off... but anyway

thanks for the info! :)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Do not try to understand this, radio off and still of influence in the tests. It just is that way. Has been discussed many times before, I didn't believe it either at first.

But already tested this about a year ago, it really is true.
[Touch Diamond, MDA Compact IV ROM Development] laser21:
Is Nataly 2.2 coming up? Thanks
[Touch Diamond, MDA Compact IV ROM Development] boscaiolo:
why you dont bech rom with wm6.5 without manila?
[Touch Diamond, MDA Compact IV ROM Development] boscaiolo:
last manila rom's VS wm 6.5?
[Touch Diamond, MDA Compact IV ROM Development] megahenebi:
Macadam 1.2 rom please
[Touch Diamond, MDA Compact IV ROM Development] devilsneck:
hmmm...ive tried macadam and its full of bugs....wont use until thers is a more stable version
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Some more ROMs

Tested 2 more ROMs:

AZTOR X2 WWE (20764)

Nataly 2.5 beta test1 (21043) (private)

Got myself the new Diamond 2, so I won't be around so much anymore at the Diamond part.

Thnx for your positive responses, hope to see you soon at the TD2's!
[Touch Diamond, MDA Compact IV ROM Development] ssj5trunks:
It's normal that when i soft reset my Diamond with last Official HTC Rom the speed of programs is more quickly?

But after 3/4 restart of programs it half the speed!!!!

Why????
[Touch Diamond, MDA Compact IV ROM Development] losdrivare:
Thanks for those last 2 benchmarks, it seems Mondilv keeps pushing the speed forward. :)

Congrats on your new purchase, hope you'll be happy with your new device! I'm a little torn between the HD and TD2 myself, if I'm going for a bigger device than my Diamond, then the HD is not much bigger than TD2 but with considerably more screen estate + 3,5 mm jack. Decisions, decisions... :D

Anyway, your benchmarks will be missed here in the Diamond section. Thanks for an awesome achievement on testing all these ROMs.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
First benchmark diamond 2!!!

Check this out: first benchmark of the new Diamond 2 (Topaz). See first post.

It's faster and it has much more memory!

(by the way: and it's more beautiful...)

Had to test VS Benchmark in VGA mode, VS Benchmark doesn't support WVGA.
[Touch Diamond, MDA Compact IV ROM Development] cyron_at:
...great but we don't have a diamond 2 :-(
[Touch Diamond, MDA Compact IV ROM Development] mondilv:
Great!

Just one thing - hardspl.... and test results will be much better.:)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Yes, anxiously waiting...
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Pubished in the Diamond 2 section a sheet with benchmark results of several HTC devices from Hermes to Kaiser to Diamond to Diamond 2.

You can see the evolution of speed...

See http://xdaforums.com/showthread.php?t=510385
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
NEW ROM: Nathaly 2.5 Final: Fastttttttt...

hi,

mondilv asked me to benchmark his latest Nathaly. Well, it fast, in fact the fastest Diamond ROM I ever benchmarked!

See first post.
[Touch Diamond, MDA Compact IV ROM Development] losdrivare:
Thanks for the test, it confirmed my suspicions about Nataly 2.5! =)

If I may make one last fan request, before you are forever lost to the Diamond 2 section, it would be to benchmark udK Topaz R7 (preferably with 12mb PP). I'm really curious to get an indication on how much LZX compression affects performance. 5 mb more RAM is worth alot... :)

Only if you have the time and willingness. Thanks again for all your work, it's invaluable.
[Touch Diamond, MDA Compact IV ROM Development] 3dward87:
Update this very good topic !
[Touch Diamond, MDA Compact IV ROM Development] norgan:
sorry if this is a repeat but the 50+ threads are hard to look through. have you tested the energyrom? i really liek this rom and am interested to see if it's worth trying others as far as speed.

i think i am in love with this rom, using the 6.5.3 build 28xxx build. stable very minor bugs and very pleasing to use visually.
[Touch Diamond, MDA Compact IV ROM Development] ceevee369:
Don't understand with all respect to the chef above.

1.93.4656.2 WWE cooked by Diamond Project team is running stable without any significant bugs and hits the highest graph scoring...?

Are we talking about the same 1.93 Core?

I am confused ...
[Touch Diamond, MDA Compact IV ROM Development] tom_codon:
Which one ?

Funny post :)

Remember it's shiped version by htc
[Touch Diamond, MDA Compact IV ROM Development] ceevee369:
Hoped you did not found that remark - statement :D

We don't want a "Battle of the Chefs" :rolleyes:
[Touch Diamond, MDA Compact IV ROM Development] walshieau:
activesync issues
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
REMOVED...
[Touch Diamond, MDA Compact IV ROM Development] saveferris:
That really isn't major issues is it? I was under the impression that major issues meant that the ROM all round had some serious flaws.
[Touch Diamond, MDA Compact IV ROM Development] KukurikU:
you must be joking or you are using yr diamond for mp3 only....

without sync you can not sync yr contacts, calendar, tasks. there are programs that can be installed only when in sync mode. you can use yr diamond as modem for yr pc\laptop.

if this is not a serious flaw than what is? today screen animation not snappy enough? bad sound quality of sound from the speaker when listening to music?:D
[Touch Diamond, MDA Compact IV ROM Development] SecureGSM:
Could you please be a little bit more specific? Thank you very much!