yashsingh1/xdaforums huggingface dataset:
[Hero, G2 Touch Android Development] rakig:
One last request.

Can you make the ROM name Font size a bit smaller so its not so hard to read?
[Hero, G2 Touch Android Development] shokkapic:
No. But I tried it. I'm doing the graphs on Google Docs. It lacks a lot of funcionalities. I'm looking for another alternative. :)
[Hero, G2 Touch Android Development] Arkymedes:
Good job man :)
[Hero, G2 Touch Android Development] CountC00l:
test request

Could U test

SenseHERO-2.1-090610-B5-1

Plz
[Hero, G2 Touch Android Development] shokkapic:
http://xdaforums.com/showpost.php?p=7124839&postcount=277

I tried. I even sent him a PM. He didn't answer. I don't care. :)
[Hero, G2 Touch Android Development] rakig:
As some people can't or don't want to run OC kernels, maybe the next tests can include roms with stock kernels (sense/non-sense) and OC kernels (sense/non-sense).

I know, its a lot of work :p. But where possible it would be cool to test as many at stock frequency and then OC'ed as well.

hmm. that would bring you up to how many roms that need to be tested? :p
[Hero, G2 Touch Android Development] sajseven:
I thought Super Hero was a non-Sense ROM? :confused:
[Hero, G2 Touch Android Development] dalet11:
can you test HebVillain 1.1.2?

its based on VR12 but it using other kernel..
[Hero, G2 Touch Android Development] shokkapic:
See this:

http://xdaforums.com/showpost.php?p=7168836&postcount=58

Affirmative. :)
[Hero, G2 Touch Android Development] sajseven:
Ahh okay. Sorry 'bout that! ;)
[Hero, G2 Touch Android Development] shokkapic:
I want everyone here, right now. :D

Check the 1st post and compare the Linpack and the BenchmarkPI graphs.

In the BenchmarkP graph, the names are cleared, but since the x-axis start at 0, it's harder to compare the values, because the bars are smaller and more similar.

In the Linpack graph, names are all weird and sh*t (and it will become worst everytime I add a new ROM), but it is easier to compare the values, because the graph starts at 2200 and not 0. (compare both Linpack graphs).

So... Blue graphs or green graphs (Google Spreadsheets or Zoho) ?
[Hero, G2 Touch Android Development] dkelley:
well... the green ones are easier to read and better lined up imho.

the results appear different to me? maybe I'm wrong, but I'm sure the results are different and that makes zero sense to me (I'm sick though today so might be seeing this in a cold medication haze...)

for example my VE4.0 rom seems better now in V2 chart in speed tests than it used to. why? am I just wrong (I can accept that LoL) :-)

anyway, I'm preferring green. much cleaner charts.
[Hero, G2 Touch Android Development] embrion:
Please update to RoboMIX_v0.5.5 :)

http://www.multiupload.com/Q0WP2XUT0F
[Hero, G2 Touch Android Development] shokkapic:
I've already benchmark it. Now I'll benchmark HebVillain and I'll post both at the same time.
[Hero, G2 Touch Android Development] embrion:
Thx.

And about your question about charts look.

Old fonts were too big, new might be too small to many people :P Hard to decide.
[Hero, G2 Touch Android Development] shokkapic:
What do you like the most? Visually and to values comparison.
[Hero, G2 Touch Android Development] embrion:
Green ones, especially considering more ROMs to come.
[Hero, G2 Touch Android Development] shokkapic:
HebVillain 1.1.2 and RoboMIX 0.5.5 added.
[Hero, G2 Touch Android Development] SpooXter:
How about using Microsoft Office to make the tables. There you can change fonts, types of charts etc. The only thing that you must be careful about is for the whole chart to fit on the screen. Then you can capture it with the print screen button and then save it using paint (I think everybody knows how to do that).
[Hero, G2 Touch Android Development] embrion:
Not on charts ;)
[Hero, G2 Touch Android Development] shokkapic:
Take 4 printscreens everytime I add a new ROM, plus saving them as .jpg. Host them somewhere on the Internet, and then edit the 1st post to add the new links. :D

I prefer Zoho. :)

Thanks. :P
[Hero, G2 Touch Android Development] embrion:
I see RoboMIX got worse. Which kernel did you use?
[Hero, G2 Touch Android Development] shokkapic:
Default. But I think I screwed the results up. I don't remember if I installed just the 4 benchmarking apps or all the apps and games I have to install (23).

I'm flashing it again. And I will re-run quickly, just to check if the values are very different. If yes, I will benchmark it fully, again. :)

Edit: I always use the default kernel, except if I can't overclock it to 691Mhz. When that happens, I look for a non-BFS and non-JIT kernel to flash. :)
[Hero, G2 Touch Android Development] shokkapic:
Hmm...

Point of situation: The 3rd test (Benchmark 1.03) needs to be re-run to every ROM.

The reason is each result is a 3-digit integer number, plus decimal places.

The application has the functionality of exporting each test as a .csv (some kind of Excel Document) stored in the memory card. But, when opening each file in Excel the results appear with the hundreds and the thousands separator.

ie. in the device, the CPU score 163.28296 appears as 16.328.296 in the .csv.

Now, implying each value may have more or less decimal places.

ie. the value 163.00 and 163.00001 are pretty much the same, but they will appear as 16300 and 1630001, in the tables. :)

Averaging those two values, we obtain 823150.5, but due to the bug with the decimals and the hundreds separators, we get 8231505. :D

So... I'll delete the 3rd graph, and I fix it on Tuesday or Wednesday.

Sorry for the inconvinience.
[Hero, G2 Touch Android Development] rakig:
Not to say you're wrong, but...

Excel tries to guess the data type in CSV fields, then formats it using the display settings for that data type. Its strange what it ends up giving you sometimes (integers as dates, dates as floats etc...) I usually confirm what is in the actual file with a txt editor and don't go by what excel shows.
[Hero, G2 Touch Android Development] lynx2k50:
Whats going on with ChocolateEclair 2.0 ? :)
[Hero, G2 Touch Android Development] RaduG:
I will post here the benchmark results... and if shokkapic wants, he may use them... Give me a few minutes.
[Hero, G2 Touch Android Development] lynx2k50:
I love you and ur Rom RuduG :)
[Hero, G2 Touch Android Development] RaduG:
The procedure was the same as Shokkapic described in the first post... so here you are:

Boot time: 48.2s (less is better)

Linpack(avg out of 5): 3.0312 (more is better)

Linpack(best out of 5): 3.045 (more is better)

BenchmarkPi(avg out of 5): 9236.4 (less is better)

BenchmarkPi(best out of 5): 9144 (less is better)

Softweg(avg out of 5): cpu ->225.464846 ; memory->157.032886 ; filesystem->33.9670722 (more is better)

Softweg(best out of 5): cpu-> 232.4373; memory->164.49925; filesystem->35.173603; (more is better)
[Hero, G2 Touch Android Development] rover.prince:
Plz test Chocolate Eclair 2.0
[Hero, G2 Touch Android Development] lynx2k50:
Thank you.
[Hero, G2 Touch Android Development] shokkapic:
I think I forgot to mention... xP

I'll just be back for benchmarking on Tuesday / Wednesday.

Until then, I'll not update none of the ROMs.

Studies first, then girlfriend, then Hero. :D

I'll be navigating the forum though, just to see the latest news.
[Hero, G2 Touch Android Development] scouzer:
Can't wait to see the VanillaEclair 5.0 results. :)
[Hero, G2 Touch Android Development] shokkapic:
Feel free to compare. :D

Edit: Failure. It is for ChocolateEclair... Nevermind, then. :P
[Hero, G2 Touch Android Development] djdavis:
could you benchmark mocado r5 with the teknologist kernel please?
[Hero, G2 Touch Android Development] embrion:
Doesn't it does not have OC ?
[Hero, G2 Touch Android Development] shokkapic:
I tried. The ROM froze, because teknologist kernel isn't for r5. There is no custom kernel to r5 yet, afaik.
[Hero, G2 Touch Android Development] sajseven:
The boot time for the original ROM seems to have disappeared? :p

Good work. :cool:
[Hero, G2 Touch Android Development] MacaronyMax:
Nice pleas add the fusion rom (2.2 vanilla)
[Hero, G2 Touch Android Development] djdavis:
http://android.modaco.com/content/h...netfilter-and-compache-ramzswap-for-gsm-hero/

the kernel in this link is meant to work with r5, and theres also one with 710Mhz enabled
[Hero, G2 Touch Android Development] shokkapic:
Regarding the 2.2 ROMs... Read the latest pages.

Regarding the other ROMs... Read the latest pages.

>> No more benchmarking until Tuesday. <<

As I said... There is no ROM to flash. :D

Next time, read the thread you linked to me, before linking it. :)

I try so hard to get my work done but I end look like stupid... I know what I'm doing and I know why I haven't benchmarked some ROMs (SenseHero p.e.)

And sorry for screwing with the last graph...

Now, let me study for my exam and stop requesting what has already been requested. I read every comment, to your information.

Thank you, and see you on Tuesday.
[Hero, G2 Touch Android Development] djdavis:
alright, sorry for the inconvenience lol, sorry if I seemed ignorant, i know u know what your doin, infact i just flashed the rom and i couldnt overclock! so yes your right i should have read the whole thread first.

Good luck for your exam!
[Hero, G2 Touch Android Development] lsxrx7:
ChocolateEclair 2.0
[Hero, G2 Touch Android Development] shokkapic:
Can you please read the thread before posting?

Jeez... -.-
[Hero, G2 Touch Android Development] shokkapic:
The first to request FroydVillain 1.0.0 will die in a painful way. Just kidding. :D

Just to let you guys know.
[Hero, G2 Touch Android Development] rakig:
Good luck with your exams.
[Hero, G2 Touch Android Development] dkelley:
ya know guys, someone else with time on his hands coudl always start another thread with their own test results for just the froyo roms. there are 3 main froyo roms now although comparing back to test0 (the original for hero, super quick linpack score, bugs and all) would be interesting too which would make it 4.

someone with a few hours to kill (sorry, I don't have the time right now) could consider that.

this list is amazing for 2.1 roms - very informative. and imho the new charts are useful to show more the difference between vanilla and sense than the difference between different roms.

But overall I'm really happy that I've been running VE4.0 for the past week :-). worked amazingly well on my family trip this weekend for photo and video taking and gaming for my kids (and using it as a phone etc LoL)!
[Hero, G2 Touch Android Development] shokkapic:
I don't mind of doing it. Let me just have my exam: tomorrow @ 9:00 GMT+0 (Portugal).

I said I'll test that all. You hust have to wait 24/48 hours. :)

(BTW, my vacations will start on August, 2nd and I gues I'll not have internet there, so... You'll have to wait 15 days. :D)
[Hero, G2 Touch Android Development] rakig:
Someone else could do the benchmarks, but different phones and processes wouldn't necessarily be comparable to shokkapics numbers. Maybe if a few people worked together you could standardize a process and people could pitch in with all the results averaged or something of that nature? community benchmarking?
[Hero, G2 Touch Android Development] shokkapic:
Hello there. :D

Benchmarks delayed until tomorrow, because I borrowed my Hero to my gf. :)

Regarding this:

If someone else shows interest, I'd be very glad.
[Hero, G2 Touch Android Development] seshmaru:
Froydvillain v1.0.0 (froyo build)

Linpack: 5.11 MFLOPS

CPU freq: 710 MHz

Didn't time boot time but boot time is faster than anything I have seen as of yet.
[Hero, G2 Touch Android Development] dkelley:
you should note that your'e using the alternative kernel (that's the only way you could have gotten thos specs).

it will likely become default soon, but afaik the default kernel in the downloaded rom is still the buggy one that only gives 3.5 MFLOPS.

agreed, boot time is shockingly fast with froyd.
[Hero, G2 Touch Android Development] seshmaru:
If you reboot enough times you can get 4.1 on the stock kernel. Also testing kernel is free to be picked up by everyone.
[Hero, G2 Touch Android Development] lynx2k50:
Please don't forget the new cronos :-)
[Hero, G2 Touch Android Development] shokkapic:
Hi all...

My father decided to get me a job... Not everyday, but some days per week.

So, since I don't have my Hero yet (my GF really likes it :(), today will not be possible for me to benchmark any ROMs.

For the next 4 days (Thursday - Sunday), I'll be working all day.

And I'm pretty sure I'll have to work next Monday and Tuesday, too...

It's basically to say I don't know when I'll be able to test the ROMs again... Maybe next Wednesday.

Sorry guys.
[Hero, G2 Touch Android Development] btdag:
How do you create the graphs?

How do you time the boot up?

Can you send me or upload the most recent data?

I don't have a job at the moment. I might be able to take over this project.

I might integrate the GPU benchmarks and CPU benchmarks into one topic if I do.
[Hero, G2 Touch Android Development] shokkapic:
There is no need to join everything in one topic.

Read the PM I sent you. :)
[Hero, G2 Touch Android Development] btdag:
Just to let everyone know - I now have access to add new details to the images in the first post. I'll be updating things over the next few days (when i get some spare time) with new results from the following:

Chocolate Eclair 2.0

Vanilla Eclair 5.0

FroydVillain 1.0.0 (stock and possibly with better kernel)

Cronos 2.1

Cronos Froyo

EzHero 5.1

Fusion!ROM 2.2 (update4) - maybe

Latest CyanogenMod6 build - maybe

Please also bear in mind that i'm also working on the backend spreadsheet that this data is coming from, as well as trying to fix the broken benchmark results from the benchmark app (which is dooable but will take time) so please don't think nothing is happening if there are no new posts. I am sure i'll be working behind the scenes on something :)
[Hero, G2 Touch Android Development] Wagster:
I've just flashed the Fusion Froyo ROM and did a quick benchmark for my own curiosity but thought I'd share here.

With no overclocking (so at 518MHz)

Linpack 3.565

BenchmarkPi 10084

So that's better than the overclocked 2.1 ROMs at Linpack but worse at BenchmarkPi.:confused:

Overclocked to 691MHz

Linpack 4.584

Benchmark Pi 7726

Which is much better than the 2.1 ROMs :D
[Hero, G2 Touch Android Development] btdag:
I'll be updating the graphs shortly - unfortunately I have to enter the data manually, instead of copy paste from my personal spreadsheet - which i use because I find the online one to lag my computer. This topic is likely to be updated less frequently than the GPU one but I am collecting data for both at the same time. So the same ROM's which are tested for the GPU charts will be included in these charts in the future.
[Hero, G2 Touch Android Development] embrion:
Great, I can't wait :)
[Hero, G2 Touch Android Development] btdag:
I'm finding it difficult to edit Shokkapic's spreadsheet so for the time being (until he has a chance to copy my data into his spreadsheet) here are the more up to date sheets:

In the works (bit of a sneak preview):
[Hero, G2 Touch Android Development] dkelley:
so wait a minute...

I take it VE5 has JIT enabled?

and it's performing better than froyo...

I wonder how stable it's jit mode is? all of the other jit 2.1 roms I've tried became really unstable after a day or two of use (might just be me but it ALWAYS happened...)

froyo's stable over long periods. if VE5's jit is as stable as froyo's somehow then with those performance gains I think I've found my new favorite rom (and I haven't even tried it yet LoL, but I'm a HUGE ve4.0 fan already).
[Hero, G2 Touch Android Development] btdag:
Do bear in mind that VE5 was clocked at 768 MHz in comparison to Froyd's 748 MHz.

I'm guessing it did have JIT enabled, though I only flashed the extreme kernel to get it, nothing else.

I had no stability issues when testing it.

Both VE5 and Froyd were a dream to use, even at stock speeds. Though bear in mind I don't really do much on them.
[Hero, G2 Touch Android Development] seshmaru:
So you know btdag, FroydVillain performs better 1 step below your max possible, so for you at 729.
[Hero, G2 Touch Android Development] btdag:
My max possible is 768 MHz - but FroydVillain doesn't allow me to choose that.
[Hero, G2 Touch Android Development] robocik:
So now you also have to test roms with JIT enabled ;)

You find JIT for robomix here :) (page google translated but i bet you get it, but HE JIT is JIT ON lol)
[Hero, G2 Touch Android Development] btdag:
I'll give it a go - but it probably isn't top on the agenda at the moment. Trouble with JIT is that it looks great on benchmarks but it doesn't actually do that much for general use. How is the stability on RoboMix? What is RoboMix based upon - the translation was flaky at times.
[Hero, G2 Touch Android Development] robocik:
Based on WWE EU 3.32.something ;)

You have to change wallpaper to whatever before flashing JIT package, you can change back the wallapaper later.

Stability is good, dont have any FC or other instabilty issues ;)

Overclockable @748.
[Hero, G2 Touch Android Development] dkelley:
tell you what though - the combination of great test results in this thread and great test results in the graphic benchmarking thread means absolutely incredible gameplay!!!

I'm surprised to say this but playing Raging Thunder 2 with VanillaEclair5.0 with it's JIT enabled (but still using stock kernel @ 691 top speed) gives WAAAAAY better fps than any of the froyo roms have given me until now, and I can tell you that before froyo, even with jit roms, I couldn't play that game at all due to crappy fps.

somehow the latest 2.1 JIT roms (at least VE5.0 but likely some of the others too) are actually the fastest gaming roms I've tried. this is a big change from the past.

and they're faster than the current (As of yesterday) flashable froyo roms on the hero, although those are certainly fast too and one can play RT2 on them ok.

so it seems the benchmarks are actually quite helpful in this case.
[Hero, G2 Touch Android Development] skydg:
i am using this rom too with the same settings, tried RT2 lite today and it was relatively smooth ie. playable with the occasionally frameskip/slowdown.

Had fc, hanging at boot when using JIT with Ve 4.0. All buttery smooth and no crashes so far with VE 5.0.
[Hero, G2 Touch Android Development] shokkapic:
Hi there. Finally I've some time to update the thread (though I don't have the Hero yet.)

Performance order and the GPU ones, though I prefer my CPU ones.

When you insert 10+ ROMs Google Spreadsheet graphs will become all ugly. :)
[Hero, G2 Touch Android Development] Feeyo:
Cronos Droid 2.1.1

Cronos Froyo 0.1 :)
[Hero, G2 Touch Android Development] shokkapic:
The graphs will disappear for the next minutes, because I'm updating the data (thanks to btdag).
[Hero, G2 Touch Android Development] shokkapic:
Forget what I wrote here.

Charts added.

Changelog will be added later at night. :)

See you in 13 hours.
[Hero, G2 Touch Android Development] dkelley:
deleted - redundant
[Hero, G2 Touch Android Development] ddotpatel:
WOW! This is some comprehensive stuff man, how have I not looked at this?!

Great work for the community, BRAVO!!!
[Hero, G2 Touch Android Development] Sow:
you're doing a good job guy, but please can you test the froyo of cronos, because it's a very good version of froyo, thanks (sorry for my bad english, i'm french)
[Hero, G2 Touch Android Development] dkelley:
so is Hik@Ri_2.2.0_5_os [Froyo vanilla]. both it and cronos are marching to their own drummer. nice to see :-)

but the problem is that froyo is changing so fast on the hero it's already impossible to keep up with even villainfroyd (notice he only tested 1.1.0, we're at 1.1.1 PLUS 2 OTA updates since then and it's made big differences in performance in real use, although I suspect the test results would be similar under good conditions).

I'd love to see the others tested too, but it might not be practical until everything has stabilized a bit more in froyo build land.
[Hero, G2 Touch Android Development] Sticky-Micky:
Currently getting reasonable results from enabling jit in sensehero 2.1, would be interesting to start comparing jit stuff as well.

Current rom is around 3.6mflops at stock speeds, which seams ok, wondering if overclocking would help?
[Hero, G2 Touch Android Development] adwinp:
Why don't you use Quadrant?

You can run customised tests if you acquire the full ver.
[Hero, G2 Touch Android Development] btdag:
I might start testing with that in the future - what makes it any better than the other tests? Problem is that it isn't going to have many results initially.
[Hero, G2 Touch Android Development] adwinp:
It has a plethora of tests to select from:

CPU integer, floating point

I/O: file + database

Graphics: 2D + 3D...

Why use 3 apps when you can use 1, standardised?

Although, for it to have any real meaning, the full version would be required. The free version doesn't let you run customised tests.
[Hero, G2 Touch Android Development] btdag:
Ok - Quadrant is unlikely to be tested as you are unable to run the tests in a clean environment. It refuses to give you a final score unless you have internet access. Which for testing purposes is disabled. It appears to do a good variety of tests but I will not be adding it to my test rota any time soon.

I also don't think having to pay for a benchmark tool is good. Everyone should be able to run the same tests we do for these results if the program has to be bought to do this it will ruin the whole point IMO.

FYI Cronos Froyo 0.1.33 will be added shortly. Overclocked Cronos to come soon after.
[Hero, G2 Touch Android Development] MentalDeath:
::Edit:

saw your post above
[Hero, G2 Touch Android Development] btdag:
Both of those have been done. I'm just waiting on Shokkapic to update the spreadsheet with the details. I gather he is quite busy at the moment so if you bear with it the details should go up shortly. If needs be i'll post up some temporary graphs in the near future.
[Hero, G2 Touch Android Development] dkelley:
how bout quick text-only numbers? :p

not anxious or anything...

really...

LoL
[Hero, G2 Touch Android Development] kindastuff:
Id like to see benchmark for SenseHero 5.1
[Hero, G2 Touch Android Development] btdag:
They don't look perfect and some of the older builds have been removed for simplicity but here are the most recent graphs:
[Hero, G2 Touch Android Development] dkelley:
cool info! is cronos froyo the (formerly default) experimental .32 kernel? and how fast is it's cpu running in these tests? also how fast is froyd's kernel speed in the unmarked tests (it's something like 578 in froyd's default settings, right?)
[Hero, G2 Touch Android Development] btdag:
They are all tested at 691 MHz unless stated otherwise. Cronos Froyo is in the default state with the .32 kernel.
[Hero, G2 Touch Android Development] dkelley:
thanks btdag, that value isn't what I expected somehow (you probably have it written in big letters somewhere, sorry if I'm blind :-)

good to know.
[Hero, G2 Touch Android Development] pulser_g2:
That's some huge set of results. As someone not too familiar with some of the benchmarks used (linpack and neocore are all I use :P), would annotating the graph with a little "PC Pro-style" lower is better or higher is better.

It's obvious for most of them, but some made me think a few times, when stock ROM massively outranks the custom ROMs :)

Or maybe I should just spend less time skimming through graphs... BTW, see for benchmark pi, what are the units? Is it time-based or something? Just interested :)
[Hero, G2 Touch Android Development] dkelley:
+1 (10 characters)
[Hero, G2 Touch Android Development] btdag:
Hehe - the ones at the top are always better. These graphs are only temporary and have been made very quickly. The stock rom is slower in every benchmark (another good indicator). All the details are in the first post of this topic.