yashsingh1/xdaforums huggingface dataset:
[Touch Diamond, MDA Compact IV ROM Development] naser404:
Hi,

It might be, but again there might "also" be a chance that its actually a better test, let me explain.

lets take a windows based machine again: If you disable 2 services for example:

1-File indexing

2-Super fetch

Things might "seem" snappier, & "some" benchmarks might give better performances, cause it cuts on using cpu cycles & obviously valuable ram but once you start using your machine on a day to day basis then these two really start coming into effect & actually help performance, File indexing for example will make your search a LOT faster, instant search & SF (depending on the settings 1,2 or 3) will PRE-fetch your appz, boot into ram to give a faster bootime and/or applications startup.

Again, different configuration will give different results, but with quad core cpus & 2GB ram i would say these are two services that should be enabled & you will defiantly benefit from them.

PDA are made equally in configuration so that should help to determine a setting that should in general work best for all Diamonds.

Again im not in the know how on pda but i think it makes the same sense & again i might be wrong & id be glad to hear anyone else opinion.

once again thank you for doing a great job testing different roms with different settings.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Like every ROM: right after install I remove the startup folder in windows, disable TF3D, install SPB Benchmark and VSBenchmark, remove sim-card, connect to AC-adapter, restart twice (because sometimes it does some post-installing and post-settings things), remove running programs from memory (like SAPSettings, BioTouch, Opera) , then run SPB Benchmark, then restart and then run the VSBenchmark.

Takes about about 1.5 hour.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Thnx, primary I'd like to test everything for myself, but I'm happy to share it with others.

After some testing with the startup programs and TF3D loaded I think I'll stick to my original teststrategy: remove all startup programs and disable TF3D.

Why? Because that's the most objective test I think. With everything loaded the ROMs are often very different, some are light some are heavy loaded. When I test without extra loadings the configurations are more (or less) the same.

When people want to know TF3D or other extras they know the ROM will be slower. I assume when using TF3D the decrease in performance will be the same for every ROM because there are not much differences between the TF3D versions (yet).

Another reason to stick to my original plan is that I can continue comparing with the already tested ROMs.
[Touch Diamond, MDA Compact IV ROM Development] sev7en:
Thanks for the sheet. Benchmarks are with the first release of TLR? It would be good to test the TLR2: I just flashed the device with this and it seems faster than original Diamond ROM.

I haven't tested other ROMs but the Elite v2 seems a good alternative.

I was expecting better performaces for the Surface roms...
[Touch Diamond, MDA Compact IV ROM Development] JanDaMan:
no w8 i think you misunderstood me, i meant how did the TLR2 ROM do compared to the Elite 2.0 ROM, thats what i ment by 'how did the TLR2 perform' lol soz for the trouble
[Touch Diamond, MDA Compact IV ROM Development] athanaso77:
New dutty's and Tom's ROMs must be added in this benchmark! ;)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
TLR2 is in the list...
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Must???

I'll think about it after getting some sleep...
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
First Dutty's and Tom's

Added the first Dutty's and Tom's.

Another good example of the difference between cold testing and the feel of a ROM during use: both ROM's are very responsive and feel very fast.

But testresults show these ROMs perform only mediate...

Let's test the new one from Dutty (downloading now...)!
[Touch Diamond, MDA Compact IV ROM Development] duttythroy:
Your right but all roms I tested before including my last this new one feels the fastest but I guess we cant exactly know the truthabout benchmarking with wm6.1 roms with spb
[Touch Diamond, MDA Compact IV ROM Development] bepe:
It is strange; because of the new ROM base I had much better graphic results with your rom

Something must be wrong... I'll do the benchmarks again.
[Touch Diamond, MDA Compact IV ROM Development] duttythroy:
very true I tested my roms on 4 different occassion with same atmosphere and get 4 different results the highest I got was 475 but still the rom felt faster:confused:
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
dutty and bepe, I also had only 450 or so on the new dutty. then I set the system caches back to disable and auto, glyphcache to 8k. now it flies through the tests: 475!

also VSBenchmark much beter results.

but I fear in daily use it will be slower with these low cache values...
[Touch Diamond, MDA Compact IV ROM Development] bepe:
I'm getting very stable results like this:

But have to say that I had problems with "duttythroy test 2"... forgot to disable the auto display off. Anyway, I've done the tests many times and got almost the same results.

... And never got graphic results with my rom that were better than yours.

Somehow that is very strange, could you redo the benchmarks for this roms?

And could some ppl please do the same with different devices, would be really interesting...
[Touch Diamond, MDA Compact IV ROM Development] duttythroy:
Its all strange bepe, not too confident in spb right, :confused: seems the new build has very good graphics
[Touch Diamond, MDA Compact IV ROM Development] bepe:
Yes, it has... but I completely disagree on the confidence of spb.

As I said, I get stable results with the same test conditions... also with your rom.

I'm making many benchmarks like this, for example to boost the file system performance I'm changing hardcoded values in dll's. To find a good value I sometimes need to make more than 5 benchmarks, so I know what I'm talking about.

I know about the fluctuations of some results, but not like this.

For me there is no way to declare this results implausible, bc I know better.

Anyway, let’s do other ppl some tests too, to see the credibility of the device used for these benchmarks.

... or would you like to end this benchmarks???
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Bepe my testresults with Dutty's 2 are almost the same as yours, but only after disabling/lowering the system and graphic caches! (see next post)
[Touch Diamond, MDA Compact IV ROM Development] duttythroy:
iT WOULD BE GOOD to see how others devices do on the same roms with benchmark scoring this would give us a rough idea on how we all doing on roms, but like i said sometimes devices performs different wether the benchmarks score are high or low. so we still continue
[Touch Diamond, MDA Compact IV ROM Development] bepe:
Great that means the results are accurate :)

Anyway, don’t really understand why you changed back the caches, because I think benchmarks are about testing exactly things like cache settings.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Dutty's latest...

Dutty v3.6 Full CVSD (20764), see first post.
[Touch Diamond, MDA Compact IV ROM Development] alphazero:
hello pietrucci my friend,

would you suggest to revert on the 1.93 roms with old drivers for future roms ? your tests show that all the new builds and drivers are slower than the older ones. what is your suggestion on this for developers ?

as promised end of this week you get a very experimental rom to play and test with. should outscore the results of v7 but your tests are more precize than i could do on my own.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Well maybe part of them???

I mean, the "old" ROMs seem very fast but they also have "older" drivers and application versions. Are the drivers and applications of the newer versions slower but better???

I think the trick is to combine the fast parts of the old ROMs with new drivers and applications which are better than the old ones... But that would be very difficult I fear.

At the moment I'm back to Aztor v7 and installed the new Opera 9.5 beta and some other stuff. I'm quite happy with it...
[Touch Diamond, MDA Compact IV ROM Development] alphazero:
well tell me why you are quite happy and not full happy with it ? maybe i can take it as a suggestion for next ver
[Touch Diamond, MDA Compact IV ROM Development] suiller:
sorry friends... but I have to disagree... even if benchmarks show slower performance for 2.00 test rom, I must admit that it's really smoother in every kind of operation & it stays smoother also after 1 whole day of heavy usage... so in conclusion there must something deeply changed inside the rom, not just drivers... benchmarks just take note of "pure" performance without considering caching techniques & "long" run terms

I started from 1.93, then switched to 1.96, then 1.97 but didn't like it, then made a combination between 1.96 + 1.97, but didn't like too, so restarted from 1.96 & included only updated apps/drivers (very few), liked a bit but still missing something, so tried 2.00... & I think I'll never revert back to 1.9x series

When I'll have time (perhaps during Xmas holiday) I'll prepare a "simple" script to simulate the real use of phone (open menu, activate & close progs, etc...) taking notes of time to do all these operations... this should give a real meter performance as human eyes reception :rolleyes:

or am I totally wrong??

bye,

davide
[Touch Diamond, MDA Compact IV ROM Development] christonge:
I have to agree with you Suiller, the overall "feel" for me is better.... believe me I´ve flashed and cooked them all... plus various combinations but this latest 2.00 test ROM cooks up the best for me. I never updated from my last 1.93 release because firstly lack of time and secondly there wasn´t that much diffrence.... but I feel this latest build is definately worth an upgrade.
[Touch Diamond, MDA Compact IV ROM Development] duttythroy:
I also agree the v2 series are better I'm using a slightly newer drivers but benchmarks are still slower than 1.93 but the rom feels better and stable with v2 drivers
[Touch Diamond, MDA Compact IV ROM Development] alphazero:
to really test if the System is better on 1.93 or 2.00 one of the cooks with spare time should do this:

take both roms:

1. OEM 1.93

delete all folders except oemxipkernel, oemlang, oemdrivers, recook upload here and test

2. Same procedure with 2.00 rom

3. ask pietrucci kindly to test the roms. its important that no other additional tweak was done.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Ok I'm in! Awaiting ROMs to test...
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
I'm happy that you cooks all are happy with 2.00. I also agree that benchmarks are only benchmarks (although they DO say sómething about speed...).

I also tried them all and I'm not yet convinced that 2.00 is faster. I went back to AZTOR and a previous swtos ROM yesterday and they both react very very responsive and speedy.

The problem is that there's no benchmark (yet) to measure the normal use of a ROM.

So suiller or anyone else: if you could make a kind of test I'm glad to use it!
[Touch Diamond, MDA Compact IV ROM Development] alphazero:
should be very very interesting!

this way of test ignores all the provider specific packages and oem parts. its this way the "PURE" system wich can be tested. i use this system to test roms.

all credits go to pistonripper he did the roms on request
[Touch Diamond, MDA Compact IV ROM Development] pistonripper:
Lets see what happens :)

I personally think (have not tested the above stripped versions) ver 2.00 is smooth and stable.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
I did my usual tests on both of them:

1.93

SPB Benchmark 469,50

VS Benchmark 2832

Storage total 146

Storage free 116

Program total 111

Program free 77

2.00

SPB Benchmark 487,23

VS Benchmark 2843

Storage total 146

Storage free 116

Program total 107

Program free 72

Will publish the total results later...

Indeed seems that the 2.00 version is faster. Like most of you cooks already stated.

Only strange thing is that the current cooked 2.00 based ROMs mostly aren't that fast ín my benchmarks than the ROMs from some time ago.

I retested some "old' ROMs yesterday - just to make sure my device hasn't gone slower - but I got the same results then before. So that's not the issue.

mondilv's latest ROM is an exception, his recent ROM is very fast. But I tested it without Opera and QWERTY keyboard.
[Touch Diamond, MDA Compact IV ROM Development] christonge:
Nice work my friend... look forward to your full results ;)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
New test ROMs

As promised: three more new test ROMs:

Pistonripper 1.93 bare (16-12-2008)

Pistonripper 2.00 bare (16-12-2008)

christonge non-tweaked v6

See post #1.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
One more thing: graphic results in SPB Benchmark of christonge's ROM are very very fast! How come christonge?

Strange that VS Benchmark doesn't show quite the same...
[Touch Diamond, MDA Compact IV ROM Development] christonge:
Strange indeed :p
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Speed: radio matters!

As mondilv suspected that different radio's might influence benchmark results I did a test: I took HTC stock ROM 1.93.456 and tested two popular radio's .05 and .23.

I tested both radio's twice: once WITH SIM-card and radio ON and once WITHOUT SIM-card and radio OFF. And indeed IT MAKES A DIFFERENCE!

I repeated the tests but always the same results: in SPB and VS Benchmarks radio .05 is faster than radio .23. Especially the file system speeds.

I don't understand why (yet), certainly when SIM-card is removed and radio is off, there shouldn't be any difference, should it?
[Touch Diamond, MDA Compact IV ROM Development] christonge:
Yeah I completely agree... in fact on one of the Spanish forums... http://www.htcmania.com/showthread.php?t=10763 .... they did the same test a few months ago with the older radios and the 05 made the ROM faster. Here are those results with one of my early Spanish ROMs
[Touch Diamond, MDA Compact IV ROM Development] Blazeitup123:
so the next question in my mind is how can we benchmark only the radio?
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Well, I fact I DID benchmark the radio, because I used the same ROM for all these tests. Doing so you can test ALL radio's and compare them with each other. Like christonge's link already shows...

Or do you mean a special radio test without file system, cpu and graphics tests? Well I personally don't think that's really neccessary, because with the above test we can compare radio's with each other, can we?
[Touch Diamond, MDA Compact IV ROM Development] bryangopan:
Did you check any of the V2.00. 421.1 ones ..... please do if you havent
[Touch Diamond, MDA Compact IV ROM Development] bryangopan:
I completely agree with you brother .. even the test Rom v 2.0 is much much faster than any other 1.9 series ...
[Touch Diamond, MDA Compact IV ROM Development] bryangopan:
Piston brother .. i have been using your version of the new stock Rom ... have to tell you .. one of the best ...please do learn more and cook more
[Touch Diamond, MDA Compact IV ROM Development] Blazeitup123:
if that is the case, how much work would it be to post test results of each of the current radios available in sblop's radio thread?
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
much work...
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
How do you know? How did you test?

People often say: this ROM feels verry fast, that ROM is much faster. But are they really??? I mean, I've had them all, also the 2.00 ROMs and really can't tell which RM is the fastest. Yes, they FEEL fast... But much ROMs feel fast.

My benchmarks say that some 2.00 versions are significantly slower than many 1.9x ROMs and in my opinion also FEEL slower then the 1.9x.

But there are also soms 2.00 which are very fast, maybe fastest of them all!

I did a test with ROMs from christonge: I compared the same ROM with 1.93 and 2.00. The result: on that ROM 2.00 is slightly faster than 1.93.

My opinion is that maybe the 2.00 versions are faster but not that much...

So much fo my opinion!
[Touch Diamond, MDA Compact IV ROM Development] alphazero:
For Your Interest:

the recent benchmark between the 1.93 and 2.00 roms "diffrence" in benchmark scores had this reason: "PagePool".

The one was set to 16 mb like we know from 2.00 roms and the 1.93 rom had 8 mb. I have set the 1.93 rom to 16 mb pp and rerun the test. 1.93 Roms was even faster than the 2.00 with 16 MB Pagepool.

Please stop the 2.00 HYPE. They arent faster at least not for me.

Thx for all people who contributed to this 2.00 HYPE and contributed with faulty informations about it :D

Pietrucci was right from the beginning ! New roms are slower than the old drivers. I hardly advice not only to go after builtnumbers. Newer isnt better on the windows mobile market. All experienced users from other wm devices will confirm this.

cheers alphazero

EDit: For those people who are not familiar with rom cooking who think still: " but the manila touchflo speed is faster"

you people are right. the increase in felt speed is result of the manila version. If you want the same felt speed on any other rom simply use the package_manila from the 2.00 rom.
[Touch Diamond, MDA Compact IV ROM Development] domenic79:
Hi to all, in a file bechmark i see that are the most fast rom, where i can download it?I whanna try but don't found this rom, thx

mondilv KitchenTree 1.96/20755 Grey
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
It's (almost) the same ROM as his Nataly ROM, you'll find it in the ROM developement section.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
New ROMs...

Two more new ROMs:

CT's V6 HD TFlo Full

swtos WWE v2.0 Almost Full
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
New (review) ROM

Benchmark of a preview ROM: AZTOR V9preview (version 22-12-2008).

See post #1.
[Touch Diamond, MDA Compact IV ROM Development] christonge:
Thanks as always for a fine service!! Merry Christmas mate ;)

Uploaded new v6.1 with original manila... give them a try when you get chance (but after Christmas please!! Take a break!!!)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
New ROM

Benchmarked swtos V2.2 Naked (20931), see post 1
[Touch Diamond, MDA Compact IV ROM Development] gunar100:
CUPRA 2.3 ???

No "NEW ROM" ?

21.12 2008
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Yes there are some ROMs on my list: christonge's, Cupra. Coming soon...
[Touch Diamond, MDA Compact IV ROM Development] gunar100:
Thanks! :)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
New ROMs again...

Tested three more babys:

Swampy395 2.00.421 v2 (19965)

CT's V6.1 Full classic TF3D 16PP (19965)

CUPRA v2.3 Full (19965)

See post #1
[Touch Diamond, MDA Compact IV ROM Development] spippo:
new rom wwe dib!

http://xdaforums.com/showthread.php?t=423418
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
New winner!

Ladies and gents,

We got ourselves a fat new winner!

I benchmarked a recent ROM and it's a winner: fastest benchmark I've ever tested (except for the special editions from swtos and alphazero...)!

Will publish the name and results on the first day of the new year, a new start with a new reference ROM...

Happy new year everybody!
[Touch Diamond, MDA Compact IV ROM Development] akil3x:
Must be the new Rom from Seatone , CUPRA FINAL....

This is my bet :)
[Touch Diamond, MDA Compact IV ROM Development] band27:
Hey pietrucci,

I would like to thank you for testing all these Roms extensively. I have to say, checking out your charts is one of the first things I do before picking out a new Rom.

Bravo once again to all your hard work. Awaiting the results....

Happy New Year !!
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Thnx, happy new year!
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
New winner!

AND THE WINNER IS..................

mondilv's NATALY v1.1 LIGHT!!!

His SPB Benchmark is 531, very high on my device.

His overall score is the highest ever (except for the extreme swtos and alphazero's).

On January 1th 2009 this is the new benchmark reference.

Congratz with your work mondilv!

And no, I didn't test the recent dib's, vic's, suilers's, christonge's, seatone's, ect.ect yet.

BUT I WILL SOON!
[Touch Diamond, MDA Compact IV ROM Development] xenon_art:
very useful benchmark.

Thanks a lot
[Touch Diamond, MDA Compact IV ROM Development] carburano:
I don't see DIB's roms in your Benchmark... why? for me are great... and now are WWE too...
[Touch Diamond, MDA Compact IV ROM Development] spippo:
dib is the great;)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
First of all: because it never was a WWE version and I usually don't test other languages.

Second: I can't every ROM, I have to do my test in my spare free time and therefore I primarely test ROMs I can use myself.

Thirdl if you read two post back you'll see that I have the intention to test it, now that there's also a WWE version.

So be patient please...
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
4 new benchmarks:

udK Syrius HD RO beta6 (20764)

TestROM seatone

Anja ROM GER HD Manila

Anja ROM GER Diamond Manila

Seatone's ROM is for testpurposes only (request from seatone).

Anja's ROMs are also a request, both are German versions

See post #1
[Touch Diamond, MDA Compact IV ROM Development] band27:
Your work here is invaluable to the xda community.

Keep up the great work !! ;)
[Touch Diamond, MDA Compact IV ROM Development] hc_blacki:
When you put Anja´s Rom on the list then you have to give some minus points because that rom´s are buggy like hell!!!
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Sorry I won't do that... I only test the benchmark speed not the stability. Besides, what criteria should I use: one freeze a day = minus 10 points?

When I personally experience that a ROM is unstabile I will mention that, but no more than that.
[Touch Diamond, MDA Compact IV ROM Development] anja31:
Hi pietrucci,

Thanks a lot for benchmarking... very kind of you to test a wwe outside rom!

Now I know where I stand between the other master cooks. Your test will bring me forward to change some things...

with best regards to the netherlands

Anja31
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
New ROM

Added DIB V1.28 Full WWE (20940).

See post #1
[Touch Diamond, MDA Compact IV ROM Development] spippo:
thanks pietrucci, good!:)
[Touch Diamond, MDA Compact IV ROM Development] king_dani:
what about Shaks ROM ? it's currently on v2.40
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Three more ROMs: udK's and AZTOR's (twice), the first ROMs based on HTC 2.03 ROM:

- udK Syrius HD RO beta7 based on 2.03 (20764)

- AZTOR X based on 2.03 (20764), radio .23

- AZTOR X based on 2.03 (20764), radio .05

As you can see I tested AZTOR once with radio .23 and once with my favorite radio .05.

Some interesting details:

- udK's results are certainly faster than AZTOR's (sorry alphazero)

- as confirmed before: radio .05 gets better results than radio .23!

See first post...
[Touch Diamond, MDA Compact IV ROM Development] losdrivare:
Good work mate, I was really curious about udk beta 7! Thanks! :)
[Touch Diamond, MDA Compact IV ROM Development] Magicacaca:
Battery life benchmarking

Hi,

Have you done any or do you know of any battery life benchmarking for this array of ROMs?

Cheers,

Magic
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
read posts 398 and 399
[Touch Diamond, MDA Compact IV ROM Development] Magicacaca:
Battery life benchmarking

Nothing about batteries in 398 or 399...?
[Touch Diamond, MDA Compact IV ROM Development] mchus:
Hi, the first thing i do every time i open this page is to see ur thread as a reference.

thanks