yashsingh1/xdaforums huggingface dataset:
[Touch Diamond, MDA Compact IV ROM Development] duttythroy:
k just to add I sync with vista no issues for 3 days, battery is good with .05 radio, so not sure really if its a major bug, But this build significant faster and much stable than all builds so far:)
[Touch Diamond, MDA Compact IV ROM Development] pizzachips:
elite 3.0 beta is out....

hopefully scores will be much better than the amazing ones for 2.0 =)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
see post #1

pretty fast, benchmarks like Dutty's latest!

even a little bittie better!
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Will test to new (pre-)ship 1.93 now...
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Wwe 1.93.456.2

Added off. WWE 1.93.456.2
[Touch Diamond, MDA Compact IV ROM Development] ceevee369:
I do have to agree with this despite my postings on DPT V2 thread.

Active sync is indeed for some a hastle and has a bug vs 1.37 builds.

Continously dropping connection with and without "enable advanced network" checked -unchecked.

If a Sync connection is lost while flashing a ROM, we might have some bricked Diamonds..
[Touch Diamond, MDA Compact IV ROM Development] SecureGSM:
ActiveSync is not involved in firmware update (fortunatelly). I can confirm succesfull sync with Win XP and Vista over last 3 days. NOTE: sync between 2 computers still does not work and causes sync errors. That is not something specific to this firware though.
[Touch Diamond, MDA Compact IV ROM Development] saveferris:
So you're calling that major issues with the1.93 ROM, well it doesn't seem to be a major issue considering that the ROM is now used in all of the current cook releases. if they had an issue with it it didn't take them long to figure it out. Personally i'd call it a bug.

Major issues I would classify as multiple issues within a ROM to the point its unsuable, whether it be speed stability etc........ Time to get over yourself bud.
[Touch Diamond, MDA Compact IV ROM Development] ceevee369:
I guess this responds indeed to my findings though partely.

Yes, I believe that most problems are reported when using 2 PC's (more specific one XP and the other VISTA.

No, the same "bug" was not found in any 1.37 or 1.39 build.

Maybe we better make a new Active Sync thread as it does not belong here :o
[Touch Diamond, MDA Compact IV ROM Development] Ce:
Is it possible to test the new Panosha clean 2 because the versionnumber didn't change....the core ROM did...it's now 1.39xxxxxx. TIA!!
[Touch Diamond, MDA Compact IV ROM Development] bepe:
I've found now what the problem is related to!

Test 1: Disconnect your device from the PC, make a soft reset and run spb graphic test

Result will be very good

e.g.: 6 - 25 - 6

Test 2: Connect the device to the PC, run spb graphic test again

Result will be bad!!

e.g.: 7 - 30 - 26

Test 2: Disconnect your device from the PC, run spb graphic test again

Result will stay bad!!

e.g.: 7 - 30 - 26

So this is why the results were that different.

btw, you don’t need to disable any apps like manila for this

This is really bad; I think it might be drivers related

Perhaps some ppl would have time to try this on different roms and post a result like this

bElite 0.94 - 1.93 based (not public)

6 - 23 - 6

7 - 30 - 26 (after connecting to PC)

Heard that would also happen on 1.37

18 - 37 - 6

18 - 40 - 14 (after connecting to PC)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Don't know...

After flashing I never connected to my PC and still got bad results.

And good... and bad...

Really don't know. Someone suggested to disable the Diamond dialer since it uses a lot of memory. But then he tested it and the results stayed the same.

My only positive find till now is testing on battery: my results on battery are better than on AC-power.
[Touch Diamond, MDA Compact IV ROM Development] KukurikU:
sync with 2 pc is a wm 6.1 related problem. look here:

http://xdaforums.com/showthread.php?t=377823

i can confirm that i am able to sync with 2 pc my athena with wm 6 but not my trinity or diamond with wm 6.1
[Touch Diamond, MDA Compact IV ROM Development] bepe:
But soft reset your device and just do the benchmark, so you will get good ones for sure.
[Touch Diamond, MDA Compact IV ROM Development] sandro.bertini:
I tried with my device (Using bElite 0.93) and got very very close results to yours, Bepe.

It seems to be repeatable.

Regards,
[Touch Diamond, MDA Compact IV ROM Development] bepe:
All Diamond roms have that bug 1.93, 1.39, 1.37, ... and all cooked roms

But I think I've found a way to solve the problem
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
I softresetted many times, but still bad.

I just tried Dutty's new hybrid. First test it tested not so good (but VSBenchmark did, fastest ever!).

Then tried very different settings, including softresets. The test keep getting worse and worse, I never reached the results I got the first time.

However, we keep on trying...
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Dutty's Hybrid added

I tested Dutty's new Hybrid, see post #1.

Very strange testresults: SPB Benchmark very bad (graphics and CPU fast, but very bad file system index).

BUT: VSBenchmark superfast! (this doesn't really test the file system).

Did several tests in SPB, couldn't get is better. Now I'll do a hardreset and test again once more. Still don't know exactly what the best testprocedure is.

EDIT: Did a hardreset, disabled TF3D, installed SPB Benchmark, set saving settings off on battery power, no SIM-card, softresetted and ran Dutty's Hybrid: exactly the same bad results I got the first time.
[Touch Diamond, MDA Compact IV ROM Development] ceevee369:
TLR V3

Hi Pietrucci,

yes, I do know it is rather impossible to test all ROM releases as there are now more ROM's than Diamond owners..

Though, TLR's Roms seen having some fans and he uses some tweaks since his first release, so any chance to benchmark his?

Thanks
[Touch Diamond, MDA Compact IV ROM Development] saveferris:
Thats his next ROM he will be testing, he confirmed it in a private message.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Added TLR 3 (twice)

Tested the new TLR 3 twice: once with TF3d installed (but not active) and once with TF3D uninstalled (took ages to uninstall...).

The version without TF3D is pretty fast, as expected.

See post #1
[Touch Diamond, MDA Compact IV ROM Development] flashmp3:
Hi pietrucci,

I love your thread ! Very useful! I'm sure everyone had been waiting for this kind of thread for ages! CONGRATULATION

Could you add last swiftblade's please?

Do you know why there is such a gap between sound performance in stock and cooked roms?

Thanks and thanks again for this thread and your initiave
[Touch Diamond, MDA Compact IV ROM Development] walkerwalk:
can add panosha light 5? this ROM seems interesting... thanks. :)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Thnx!

Today I'll test Panosha Light 5

And Swiftblade's

And Cyphol's

I hope...
[Touch Diamond, MDA Compact IV ROM Development] panosha:
Certain spb benchmark measurements in this Device are perfectly irregular. Other you realise at Diamond using and other show in measurements. This is for all ROMs, maybe Qualcomm, maybe something else, I don't know... :(
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Yes you're absolutely right. SPB Benchmark isn't fully reliable for WM6.1.

But haven't a better alternative yet.
[Touch Diamond, MDA Compact IV ROM Development] milleniumbug89:
i've edited it for excel 2007, easier consultation...

i did it for me, but i think is right share my work with the comunity... :P

thanks again, pietrucci :D
[Touch Diamond, MDA Compact IV ROM Development] gurkburk:
Very exciting and see what results you get :D
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
GM360D.3

Tested Cyphol's GM360D.3, see post #1

Installed Opera and TF3D like all the ROMs I test, but disabled TF3D (like all the ROMs I test).
[Touch Diamond, MDA Compact IV ROM Development] cyphol:
As I predicted it would be since I did not tweak any cache parts in device registry. As I said in the ROM thread, I leave that parts to users liking and efforts... for now.:D

Right now I'm playing with multiple conditions for preparing a good ROM for fans.

Very nice benchmark pietrucci. The value is what I need to accomplished my future missions since I don't like to do the benchmarking myself. You know why Pietrucci.:) I hope you will always do the benchmarking for all my future ROMs.

Cyphol.
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Ok, thnx. I'll see what I can do.

As long as the wife and kids aren't complaining and at night I need some sleep sometimes).
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Panosha Light 5

New test: Panosha Light 5
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
SwiftBL-WWE 2.0

Last one for today: SwiftBL-WWE 2.0
[Touch Diamond, MDA Compact IV ROM Development] VisitorSam:
Hello,

i want to flash rom on my O2 Diamond, but I don't know what ROM is better in what point. So I decided to make a Benchmark of my orginal ROM and I am surprise.

My Orginal Rom is in some points faster than other ROMs? Why it is so? I don't understood why the copy of 1MB file and the dir list is very faster but the read and write time is many slower.

Sorry for my bad english I am german.

PS: in the Excel table is one point wrong: Arkaball frames per seconds (higher values are better)

PPS: why is the attachment limit four PDF only 20kb?
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
With the benchmarks we use nowadays you can't compare our devices that good. It seems that every device gets (some) differente results.

The only thing you can do if you want to know which ROM is fast is test different ROMs on the same device and under the same conditions. And these conditions change very often en very fast: if you do a test, then do a simple softreset and then test again you get different results already. Not too much most of the time but sometimes I wonder why results are suddenly that different...

Hope you understand what I try to say: just test different ROMs on your device immediately after flashing.

My tests are all on the same device more or less under the same conditions, so they do say something about speed I think...
[Touch Diamond, MDA Compact IV ROM Development] VisitorSam:
I have flashed bebe 9.4 premium in german,

and it is very faster. Show the Bench :)
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Yes you're right, Bepe's new ROM is fast!

I got about the same figures you got, see post #1.
[Touch Diamond, MDA Compact IV ROM Development] Huey85:
I wonder how Bepe's latest 0.95 will stack up against the pack ;)
[Touch Diamond, MDA Compact IV ROM Development] ceevee369:
i bet slower than 0.94. Opera is set as default browser and takes more background CPU power , en if idle.

But, yes, I am curious.

0.95 is a for me a candidate release to test further on stepping (temporarely) away from DP V2 which was/is perfect still for me.

Piet kerel, you took a hell of a job on your shoulders.

Maybe you should take a rest, and analyze the collected data a bit.

Like pointing out to flashers which score's they should loook at if they want speed - quality- functionalit - super graphics - best audio etc.

Thanks for your valuable contribution to XDA and the Diamond Forum!
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Thnx for the advice to rest. But I believe my Diamond is one of my resting points... (sad? o no, not for me). I wonder of my device ever wants a break! Always flashing to a new identity, heavy attacks on his battery...

I don't think it makes much a difference between Bepe's 0.94 and 0.95 because of the Opera browser. I mean in 0.94 links to Opera also were loaded in memory (Opera9.exe and OperaPreL.exe) and as long as you don't use Opera there won't be much difference. Ok, there are some extra registry settings.

Best way yo find out is to test it!

About your remark to start some analyzing: I was just about to do that!

I was thinking to do some other ranking: now I rank the graphics twice: with SPB and with VS. Furthermore VS ranks the sound, what's the use? And why is that ranking so low in the latest ROMs? Will play around in Excel with weights and stuff.

My first obvious conclusion is that the 1.93.xxx ROMs have the future (till there's a better one), the "older" ROMs seem dead (hey what's old in this world, we're talking about 1-2 month old ROMs).

Althought: why score the first ROMs so high on CPU? Why scored Bepe's 0.54 so high on the file system? And why scored his 0.93 so low on graphics in SBP Benchmark and rather well in VS Benchmark?

Questions questions...

In fact why don't YOU all people do some analysing too???

One final thought, alrady mentioned many times: don't take this **** too serious. These benchmark do say something about (relative) speed but certainly not all of it! And are certainly not 100% reliable in WM6.1.

The responsiveness in daily use, the stability, the look, those factors are much more important.

That's my word.
[Touch Diamond, MDA Compact IV ROM Development] meow0623:
OMGOSH

Dude you're the man!!!!! you completed the mission impossible (for most ppl) LOL
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Added Bepe 0.95 WWE Premium

hi,

added Bepe's bElite 0.95 WWE Premium: fast and much free memory...

see post #1
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
How About Some Analyzing???

Now we've collected such an enormous amount of figures it's time to do something with it...

First my question: which ROM overall performs best according to these two benchmarks (SPB en VS)???

This is difficult to answer: do you like file system speed, do you want CPU speed or speedy graphics?

Well probably all of them I guess...

So this is what I did:

> I considered all individual tests from SPB Benchmark and VS Benchmarktool. VS only tests graphics and sound, so I divided VS in two parts: one part was the sound test, the other all other (graphics) tests. Doing this I came to these individual indices:

1. SPB File system

2. SPB CPU

3. SPB Graphics

4. VS Graphics (so all trest excl. the sound test)

5. VS Sound

> Now I normalized all indices to the same score level. This means I made the scores for the CPU index as high as the index for the graphics test ect.

To achieve this I took the average of every index and calculated the weightfacter for that index. Then I multiplied every individual score with that indexfactor. Doing so every score got about the same height (for example: the File system index got a factor of 14,82 because the average value of the file system index was 183,56 and the average value of the highest index (VS graphics) was 2720,46.

> Then I added all results with factor 1 except for the sound index, this got only factor 0.5. Because I don't think the sound is that important. (by the way: the speed of sound, what is that...?).

> Results are scores from about 10380 till 12844.

AND THE RESULTS ARE............................ (20th July)

1. Walshieau Elite 2.0 12844,60

2. Bepe Elite 0.95 Premium 12807,22

3. Bepe Elite 0.94 Premium 12719,19

4. Walshieau Elite 3.0 12692,22

5. Diamond Project 1.5.2.19965 12653,24

6. Panosha Light 5 12602,59

7. Org WWE 1.93.456.2 12600,91

8. Dutty's Hybrid Publ. Beta 12583,62

9. Panosha Light 4 12578,48

10.GM360D.3 incl. Opera and TF3D installed 12548,34

11. Diamond Surface 6.1.1.16 NLD 12513,51

12. GM360D.2 12481,73

13. SwiftBL WWE 2.0 12460,26

14. TLRDiamond 3.08.07.16 TF3D 12433,78

15. PDAVIET 1.0 12382,50

16. Bepe Elite 0.54 12187,26

17. Org Dutch 1.35.4.. 12109,03

18. Diamond Project 1.5.2.19593 12083,93

19. TLR Diamond 2.08.07.01 11995,58

20. Org WWE 1.37.401.3 11963,52

21. Panosha light 3 1.37.401.3 11895,40

22. Diamond Project 1.5.2.19960 11873,79

23. Panosha light II 1.37.401.3 11848,97

24. Panosha light 1.37.401.3 11839,43

25. Diamond Project Lite 1.5.2.19593 10604,90

26. Bepe Elite 0.93 10381,30

IS THIS SCIENTIFIC??? NO

IS THIS OBJECTIVE? NO

IS THIS OF ANY PRACTICAL USE? HARDLY

(it's just fun...)

Note: If anyone is interested in the individual results of the above ranking I could post a detailed sheet.
[Touch Diamond, MDA Compact IV ROM Development] panosha:
pietrucci

The better friend or our better enemy ? :p
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Could you explain, I'm not that bright...
[Touch Diamond, MDA Compact IV ROM Development] mgear356:
one question..how do u benchmark sound?

as in the loading of the song OR the sound quality?

weird to see all cooked roms getting lousy score for sound from Stock..
[Touch Diamond, MDA Compact IV ROM Development] Swiftblade:
Hi pietrucci.... what a nice benchmarking works, bro and you throw in your analysis too.:) Thanks for the hard works and now I can save doing the benchmarking myself for comparison purposes.:D

Well Done and Keep Up the Good Works!
[Touch Diamond, MDA Compact IV ROM Development] zjsteven:
thanks! Great job!
[Touch Diamond, MDA Compact IV ROM Development] :
Wow, really tons of work! Thanks, I hope this help who want install new rom and also who want cook :)

Bai bai!
[Touch Diamond, MDA Compact IV ROM Development] ceevee369:
Hoi Piet,

Seems you had the same idea about some analyzes - paralyzes.:D

Yes, you're right about the scoring, but me , not a Tech guy understand vagely what the scoring per criteria mean.

I do not know if you have somewhere a " guide" from within the 2 tools You used to point out what the scoring does.

CPU I-II-III for example, what resources are used. etc

I would like to share some time to analyze further on, but without more indepth meaning of score it is diffcult.

FE, Sound , where you gave a lower score might maybe important for some drivers which take a lot of resources (High sound score -lower graph index maybe?)

The good thing is -there is no right or wrong and you used your own device, with same settings so we got ourselves a very very good benchmark.

I am also convinced that the Chefs themselves are looking into your scoring and adapt their tweaks as such :)

One thing I am a bit afraid for is that some people would look at the scorings as "best ROM" meaning overseeing that some of thsoe ROM's might still in BETA mode where bug's might appear. mmmhh. maybe I think too much.

Glad your post become a sticky as it is more than worth!

Kei-goed man! - You Rock pall!

Followin further your thread in my subscribed list, and yes, I might give BePe 0.95 a try , but it hard to step away from DP V2 now that it is being tweaked to MY almost perfection standard :)
[Touch Diamond, MDA Compact IV ROM Development] egal02:
Hi pietrucci,

can you bench this rom to?

It's great I think, maybe it has good results..
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
I would be glad to test it, but... in a few hours I'm going on a vacation for a 2 weeks.

Perhaps in Sharm EL Sheikh there's internet somewhere... and hope I can find taht.

But no more testing for 2 weeks...

Bye bye
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Thnx for you reaction ceevee (is that your name?),

Yes, you pointed out to do some analysis, I was just planning to do so.

I do think these test say something about speed. When I feel a ROM is fast and snappy, it always gives me a good benchmark. But you're right that doesn''t always mean it's a good ROM. It can have freezes, bugs, not the programs we want or programs we dón't want...

I was in doubt publishing a ranking because people might conclude it's an absolute truth. Well it isn't and I think people know by now.

Well that's it folks. Hope to hear from you in a few weeks after my vacation. Ready to test...!!!
[Touch Diamond, MDA Compact IV ROM Development] mgear356:
:) bump bump :)
[Touch Diamond, MDA Compact IV ROM Development] suiller:
hi guys,

just installed bElite.95, after having reinstalled all the software I need I tried to run vbench... and results are 2875 :eek: with Vbench?? tried 3 times, waited 1min after soft reset... why a value so high??

2nd set test I've enabled all common tweaks and result are about 2600 :confused:

so all those tweaks are unuseful?? :cool:

bye,

davide
[Touch Diamond, MDA Compact IV ROM Development] Gissan:
So ... which is the fastest ROM of them all?
[Touch Diamond, MDA Compact IV ROM Development] hitman220881:
Almost all of the cooked ROMs are fast after applying tweaks.Maybe it more useful what ROM is more energy efficient. Can anybody do some tests with batterystatus to find out? Maybe with and without TouchFlo3D activated?
[Touch Diamond, MDA Compact IV ROM Development] suiller:
it depends also by energy saving parameters, 2g or 3g, gprs auto attach, and so on... we have all different setups that just fit our needs, surely radio rom can make the difference but I don't think a so BIG difference.

for example I made this test: I can reach 2 "working" days without recharging by disabling wifi, gprs auto attach/search, g3 and bluetooth.

if I enable all stuffs my battery dies within 12 hours :D

bye,

davide
[Touch Diamond, MDA Compact IV ROM Development] wonderiuy:
this thread got sticky but don't get further updates
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
That's partly because I'm on vacation in Egypt. I do all the testing in my free time, it's hard to find the means to do this in Egypt.

In a week I will test all the newer ROMs' I hope...
[Touch Diamond, MDA Compact IV ROM Development] flashmp3:
Enjoy your trip ;) this thread has to remain sticky altough it isn't updated evrey day !
[Touch Diamond, MDA Compact IV ROM Development] zyborg:
Please add a date in front of the thread headline, so that we can see when it's updated with new benchmark results. Thank you.
[Touch Diamond, MDA Compact IV ROM Development] DarkRay_:
what are conditions used for the tests?

with SIM or whitout SIM?

I read that the test is launch after the flashing a rom. But can you more specifics in conditions?

Where install VS and SPB Benchmarks, in main memory or internal?

I will test all Spanish roms, for the moment: Original 1.37, Original 1.93 , Vodafone 1.37, Movistar ??, Diamond Project Team Rom V2 Lite ESN , Vauss 1.2 (from HTCmania forum).
[Touch Diamond, MDA Compact IV ROM Development] pokevitek:
Hi, very good work, but I noticed one "bug" here. From your test results, it seems that graphical performance of orig 1.93 ROM is terrible, but it is not. I noticed days ago that device is very slow after active sync connection, if you not restart device after that. So, I tried orig rom 1.93 before restart - get same results as you, and after restart, and I get 2536 points. I dont know how, it is a mystery for me, but active sync terribly slows device.
[Touch Diamond, MDA Compact IV ROM Development] iliescu902:
well great job on the testing. however, like suiller said, because we have different needs the ROMs , radios and battery life will perform differently each time being tested.

all in all these benchmark results are good for giving you roughly an ideea of what ROM to try out.

anyway, keep up the good work
[Touch Diamond, MDA Compact IV ROM Development] Knoore:
Could you maybe update this with the newest roms?
[Touch Diamond, MDA Compact IV ROM Development] Udi:
Nice work.

It would be great if you could benchmark latest official ROM. I think that would be the best comparison over all.

Edit: Sorry, I see it was already made. :)
[Touch Diamond, MDA Compact IV ROM Development] creatix5:
Benchmark Mossis Rom 2.0

Can you test Mossis Rom 2.0? Its my favorite, very stable, fast, but I think only in German

Link: http://download.stanigel.de/

greetings

CreaTiX
[Touch Diamond, MDA Compact IV ROM Development] pietrucci:
Ok, I usually don't test other language ROMs, only WWE and (sometimes) Dutch. But hey, it's thursday...

See post #1 for some recent ROMs:

Dutty's 20273 V1.5

Dutty's priv beta V1.6

Dutty's 20273 V1.6

Panosha Shine On 1.1

Tom Diam Prj v3.3 WWE

Mossis ROM 2.0 GER

HTC WWE_EE 1.93.479.3

SpeedUP TOR_ROM 1.93.456.2
[Touch Diamond, MDA Compact IV ROM Development] yiannosk2:
Why you didn't test any ROMs from SwiflBL?