yashsingh1/xdaforums huggingface dataset:
[Nexus S General] :
Ummmm..... try a different kernel
[Nexus S General] Oodie:
:P really .. don't you have any other work in your house :rolleyes: Have you brushed your teeth today :confused: :p
[Nexus S General] :
Wait til I publish the UX benchmark, it'll be worth losing a few of my teeth.
[Nexus S General] :
UPDATE: Pixel ROM 1.72 benched!

Now leads the pack, we have a new #1
[Nexus S General] tchaari:
I am seeing an update in the spreadsheet. Pixel ROM is taking the lead again.

Congrats again WP ;)

I am also curious about the new SLIM ICS rom. Users report that there is a considerable performance boost...
[Nexus S General] :
Cool. It'll have to wait until morning when I have unrestricted downloads (very close to my peak-time limit at the moment.)
[Nexus S General] :
Here's the words I'll be using as the base for my UX benchmark:

kernel	5988

thanks	4463

problem	2859

wipe	2536

how	2284

issue	2080

fix	1998

great	1907

works	1897

good	1849

reboot	1768

fine	1286

issues	1274

help	1137

bug	1090

fixed	1051

problems	1009

tweaks	998

sorry	946

stable	939

matr	938

love	857

fast	848

nice	836

reboots	811

question	799

hours	762

smooth	728

lol	684

wrong	666

restore	658

awesome	616

kernels	615

confused	584

matrix	578

error	575

debug	482

worked	469

missing	454

permissions	423

quick	415

feature	409

force	399

perfect	366
[Nexus S General] spupuz:
Cybergr rom is very good so i'm not surprised, don't you like his rom or there something else?
[Nexus S General] krarvind:
I got a project for you if you want to develop an app. Pm me if interested. I sold my nexus s but slim should continue though. 3.2 benched?
[Nexus S General] rasengan82:
Yes I flashed the Feb 08 nightly and it was faster. Found a bug though. Min Freq is set to 200 But clock still goes down to 100.
[Nexus S General] Ibn Saeed:
After checking the bechmarks, ill give Slim ROM a try. Right now im on Bugless Beast, which is quite good.

Lets see how Slim behaves.
[Nexus S General] :
Too bad for pete. I thought it was meant to be bugless. I'm compiling a list of the state of play as it was on the 5th of February.

I'm not going to keep retesting ROMs until they do better. It's extremely time consuming and boring.

What I'll do is, every time I think there has been a

major

release for any ROM, I'll re-test, and update the sheet.
[Nexus S General] :
FINALLY finished. Phew, that was the biggest project yet. Not as scary as the battery testing though. Perhaps a little more contentious.

Result: Looks like we'll all be enjoying some boosts when 4.0.4 drops, as evidenced in nuhetri-reloaded v2.0.0

Oodie pRiMe is notable, in that it outclasses everything when equipped with T132

Slim ICS v2.4 is probably the one to go for if pure performance is your desire. However, cyanogenmod is of course close behind, and offers a completely different experience.
[Nexus S General] Ibn Saeed:
I just flashed Slim 2.4 over Bugless beast

I thought bugless beast was fast, but Slim is the fastest for me so far. Really like it.

Its very smooth, so ill test Slim for a week or so.

I just tried Slim when i saw the benchmark results, otherwise,I would not have tried it since its not that famous.

Thanks for the hard work beladus, it really paid off.
[Nexus S General] :
Thanks for following my work Ibn Saeed.
[Nexus S General] :
I'm only doing one's that have XDA's blessing!
[Nexus S General] :
Yeah, I've been using Slim ICS for ages in my kernel work. I've never had any issues with it. Miss having status bar widgets, that's all. And krarvind is a nice and helpful dev.
[Nexus S General] krarvind:
Great work bedalus. We need ppl like you for the community. regarding 4.0.4, that was sprint nexus s drop of the zip file, and status bar missing has something to do with that. I will release 3.0 when the next google source drops, could be 4.0.5 going by the history of ICS drops. Until then it will be 2.x
[Nexus S General] kuzn.dima:
Running apps (long press home button) seems very snow for me.

So slow that I prefer the old 2.3 one...
[Nexus S General] tchaari:
Really, really very good work bedalus. As krarvind said, your efforts are so precious to the community. Your benchmarks will give some lights to help users in choosing the ROM and the Kernel to flash on their Nexus S. With the big amount of ROMs and Kernels available on XDA, many of us get (positively) confused :D

@ bedalus, if you still have the benchmark addiction, I have some interesting ideas for you :D

EDIT1: A little surprise for me : Mandaman2k_v6 ROM is a very light cosmetic mod of Petter Alfonso Bugless Beast but it has a higher score :eek:

EDIT2: @bedalus, you didn't included the ICS 4.0.4 IMM26 for GSM Nexus S (port from NS4G) in the benchmark. I think it doesn't have the Status Bar bug and it is a little bit different from the nuhetri-reloaded.
[Nexus S General] Ibn Saeed:
So far, Slim has been performing quite good. I am thinking that I might replace my regular Bugless Beast with Slim for good.
[Nexus S General] :
Cool, let's hear the ideas!

The worst case scenario is a 1% difference in the score. So if Mandaman's performed at it's best, it's score will be 1% above it's true mean score. Conversely, if peter alfonso's ROM did it's worst, it's score would be 1% below it's true mean. This may account for the differences (although it is highly unlikely for normally distributed data, hence, worst case scenario). More likely is that peter alfonso introduced some changes in his build that reduced performance. Mandaman2k's is a 31 Jan build, and peter alfonso's was 6th Feb.

I only downloaded ROMs that were already available on the 5th of Feb. I'm not adding or updating any more ROMs for a while because it's far too time consuming.

EDIT: Sorry tchaari, you were right again! This was available on the first of feb. I just missed it somehow! :eek: Tiredness probably.
[Nexus S General] :
Remember, when the 4.0.4 or 4.0.5 source code drops, every ROM will benefit from improved RAM and javascript, but it should be an equal gain for every ROM. It shouldn't affect the overall rankings.
[Nexus S General] evilisto:
weird... because that rom is first 4.0.4 rom for gsm nexus s and initially uploaded on feb.01 (before 'nuhetri-reloaded' rom updated to 4.0.4).

anyway, i don't think that rom will perfom well on benchmark because it is really stock rom...(doesn't include any tweaks)
[Nexus S General] Jamalsid:
Awesome, simply awesome!!

Your benchmarks make it so much easier to help everyone pick a ROM, KERNEL, SCHEDULER & GOVERNOR to suit their needs.

Excellent work really appreciated.
[Nexus S General] :
Oh Jesus, another oversight. I apologise. I did in fact leave numerous posts and a message on my spreadsheet that said "Have I missed any?" and nobody pointed out yours until after I was done.

I shall download and include it. Hopefully I won't have to re-colour every row with pastel green and grey all over again. I hate that tedious "make it presentable" ****.
[Nexus S General] thegtfusion:
i knew the 4.0.4 build would be higher than SLIM ICS
[Nexus S General] :
I think Slim ICS will gain a little when the next google version drops. Same for all ROMs though.
[Nexus S General] Oodie:
where's the proper link for trinity kernel you used to benchmark ?

edit : never mind :) found it here
[Nexus S General] tchaari:
I think that it should be better to look for expressions such as "no problem", "doesn't work", "running smooth"... instead of words. Even short expressions with just 2 words can considerably improve the precision of the results ;)
[Nexus S General] :
3.2 soon in a few hours

kernels ; battery ; ROM ; gov/sched

Now with summaries in the first posts. Convenient for XDA app users!
[Nexus S General] :
Too late tchaari I've already modified my program to only include those words! Bedtime now. Hopefully have ux benchmark complete early next morning.

kernels ; battery ; ROM ; gov/sched

Now with summaries in the first posts. Convenient for XDA app users!
[Nexus S General] tjackbjorn:
would you like to bench the new play ROM (rc4)? :-)

Sent from my sandwich using a Nexus S
[Nexus S General] :
Sure, what's the link?

UPDATE: Column F now makes more sense (I hope... feedback please!)

UPDATE: Slim ICS 3.2 benched!

Also noticed update to AOSP+ this morning, so will bench that next.
[Nexus S General] tchaari:
thanks bedalus, column F is now easier to read but why do you have some negative values in green (-1%)?
[Nexus S General] krarvind:
you know Java? Pm me for an interesting project.
[Nexus S General] :
Yeah, it's a bit arbitrary, anything lower than -3% is highlighted as being a bit poor.

krarvind needs java skills tchaari!

UPDATE: UX complete! Just need to add it to the sheet...
[Nexus S General] tchaari:
Nice, so let's say the T132 score is 6% and the default score in column F is -1%. Then, can we conclude that flashing T132 will boost the performance by 7%?

Oh you take a big risk by showing the results to me. I'll bother you by my continuous suggestions to improve them as always :D

Yes thanks bedalus. I PMed him and I hope I can help with the small free time I have :)

By the way, if one day you want to learn Java, I can point you to some interesting docs ;)
[Nexus S General] Oodie:
One thing i can get from colomn f is , that no kernel can match T132 ?

( it's obvious that you try the stock kernel with 1000 & T132 on 1320 . It's the reason right ? So does the colomn f reasonable regurding other kernels which tested with 1000mhz ?)
[Nexus S General] :
UPDATE: User Experience benchmark is up.

The words shown occur more than average (where the baseline is generated from words extracted from ALL threads).

If a word is occurring more than average, I see this as significant. The words are ranked in significance from #1 to #10 according to how far above the baseline they are.

I'm not going to try and interpret whether each word has a positive or negative implication, I leave that up to you.

Attached is the complete data. Also see here for how I developed the software to help me do all this.
[Nexus S General] :
Okay, column F now shows the boost gained from flashing T132.

The average boost is 18%
[Nexus S General] tchaari:
With colomn F, we can have an idea about how much the stock kernel is far from T132 even if they are not equally clocked ;)
[Nexus S General] Oodie:
That's better :D

Now i know you hired ninja assassins to kill me . One just tried to kill me , got a slap from my wife and went away :D he dropped a cheque given from a man named " bedalus " .
[Nexus S General] :
Tear that cheque up. He can't get paid if he doesn't do the job.

Now for the correct smiley. Umm... :mad: also :eek:

Mainly :D
[Nexus S General] glennkaonang:
I still don't get the idea of column F.

Statistics has always been a subject I couldn't catch up with.

So, in Column F, is it the higher the better or otherwise?
[Nexus S General] Oodie:
lol. just read the user experience column . & it's just ... "lol" :D
[Nexus S General] :
It shows how much of a boost it got from flashing the T132 kernel.

Compare Column F for AOSP+(n.5) and Peter Alfonso's Bugless Beast Nightly.

AOSP+ 26%

Bugless Beast 13%

AOSP+

When I flashed T132 on AOSP+ the result went up 26%

This is more than the average of 18%

You get a bigger gain than expected by flashing T132

So... maybe it is a good idea to replace the kernel on this ROM.

Bugless Beast

When I flashed T132 on BB the result went up 13%

This is less than the average of 18%

You get a smaller gain than expected by flashing T132

So... maybe it is a good idea to keep the default kernel.
[Nexus S General] glennkaonang:
That's it, I got it now.

Those you bolded really explains much.

I'm that kind of guy who understands words better than numbers :D :D

Thanks a lot, bedalus.
[Nexus S General] AeroEchelon:
Sweet so what I gather from this is Pete's ROM has the best scores on benchmarking if all the ROMs were pitted against each other using the supplied kernel?

(Also, the User Experience column says that "kernel" is the most said word in his thread. And probably for good reason.)
[Nexus S General] kong:
Just saw the UX results.

Thinking about changing the ROM name to LOL MIUI :D
[Nexus S General] :
I didn't actually lol, but that remark made me very happy!
[Nexus S General] :
Hi Aero, yeah, looks like Pete's kernel is doing the business, and also Cyanogenmod. Looking forward to getting my hands on the latest cyanogenmod kernel with its 4.0.4 upgrade. I know brainmaster is on it, and there may be a few others up to the task. Anyone seen mobile_pc lately?
[Nexus S General] simms22:
you want to try another rom? heres rascream by @rascarlo(known for his awesome boot animations). its an aosp build with some tweaks and mods. im running it right now w/trinity t11. its a nice rom for the nexus s http://fitsnugly.euroskank.com/rascream/maguro/.upload/rascarlo_crespo_ota_04-07-12.zip
[Nexus S General] :
Thanks simms... I'm trying to get on top of a backlog of stuff at the moment. If rascarlo (nice guy, helped me make a boot animation once) releases here on XDA I'll add it to the table.

I'm already accumulating UX data though...
[Nexus S General] tchaari:
Lol, I am posting just to increase your "lol" occurrences :p

Is it too difficult to change the words with expressions bedalus?

By the way, are you exploding your data limit by downloading all the content of the dev section for your UX data? :p
[Nexus S General] :
No, check this out: http://xdaforums.com/printthread.php?t=1481856&pp=50
[Nexus S General] :
CM9 v5.0 kernel courtesy of androidphone2012

Be sure to thank him!
[Nexus S General] rrohanjs:
miui comes out with official 4.0.4 and so does kong with catalyst miui :)

http://xdaforums.com/showthread.php?t=1483033
[Nexus S General] kong:
@bedalus

I made some final changes to the released ROM. Dunno if it will make any difference though.

Just wanna let you know that I don't mind if you'd just use the prerelease scores, but if you have some free time.... :D
[Nexus S General] Harbb:
Excellent work as usual going on around here. Also, the perfection of your spreadsheets amaze me every time.

I've sussed out all of this new mathgic. I actually really like the way you've done it. You've managed to compare the supplied and T132 kernels perfectly with a percentage, you're testing and methods are spot on and it is unbelievably easy for anyone to have a look and see just how well each ROM scales with the competition. A current average for the T132 increase might be something to add here though so i don't have to average it myself later ;)

The most common words is a good start. I had a quick browse through a couple of the threads to see how they portray against the single-word UX list and it did give a decent look into many posts and often the atmosphere of the thread. As something that is likely simpler than the expression suggestion that i'm sure you're working on, maybe try and seperate words and phrases by date. Many of the threads have changed a fair bit over the months and you might find large changes for UX. Overall thread top words can be shown similar to the raw data, last couple of months or so can be shown as the current top words. Just a suggestion to try and keep everything current, critiquing very welcome (i'll sharpen my pitchfork in preperation :)).

I like the UX idea though. Great way to incorporate C++ and get a brief view of where the users place each ROM for those who don't have the time to scour through each thread like your modem and CPU. For a device that is often used several hours a day that we keep with us a slight emotional attachment is a good thing (i hear it increases screen on time too).
[Nexus S General] urko95:
For Bedalus:

Hi, I'm urko95, and I'll be pleased if you bench my Rom, PlayRom!

Now it's stable and you can bench it!

Regards,

Urko
[Nexus S General] :
Cool. I'll check it out asap.
[Nexus S General] :
Also cool, will check this out too. I'll drop a note into your thread when it's done.
[Nexus S General] tchaari:
Brainmaster cooked a new ROM based on CM9 too :)

Nexus MV is updated too ;)

Too much work for you with all these updates and these new ROMs. It is your fault in some part :p

We are the happy users of the new ROMs and the lucky readers of your Benchmarks ;)

Cheers.
[Nexus S General] :
Thanks for the feedback Harbb. I went through about five iterations of the presentation of the 'Supplied Kernel' score. I settled on the one that is the most basic. But the average percentage is important too. At the moment it is 18%, but obviously this will shift around as more ROMs update to 4.0.4 and get the full treatment.

I'm glad you went to the effort of comparing the UX to the threads themselves. It's something I neglected to do :eek: But you've reassured me. Your idea about the date is good. I know you like background reading, so here's something relevant: http://www.bores.com/courses/advanced/windows/files/windows.pdf

The issue of windowing applies here also. How to obtain a specific figure for fluctuating data > use an average. The longer the window, the better the average, but the lower the time specificity. Conversely, the shorter the window, the better the time specificity, but the worse the average.

To be honest, I don't intend to resolve these issues. The UX benchmark was just a bit of fun, and I may remove it unless I (or anyone else) can think up some way to make it more useful.

Emotional attachment is bad for the battery. Perhaps I should add that to the list.
[Nexus S General] :
Thanks tchaari!
[Nexus S General] brainmaster:
Hi Bedalus, would you please bench my new CM9 ROM?
[Nexus S General] Oodie:
My hired assassin just confirmed the death of bedalus . Been planning this for severel months and at last he got him while bedalus is sitting on a bench doing marking :D

I'm so happy that my money didn't get wasted .

..Generated from Tapatalk..
[Nexus S General] :
zombie bedalus is here... I now benchmark and eat people.

Harbb - I've added the average gain to the very bottom row.
[Nexus S General] Oodie:
Holy sh*t . That one failed also ?

Now i have to hire another assassin to kill the previous assassin :| useless buggers .

Btw , how's benching going . Gonna release another rom ( not miui ), hopefully tonight , maybe tomorrow . My only intention is being a nuisance to you :D

..Generated from Tapatalk..
[Nexus S General] :
Grrr... :mad:

:D
[Nexus S General] original21:
Bedalus am using android000001 naepolitan N8 Rom with matr1x V18,5. On antutu it scores 3600 and on quadrant only 1300. Which is the correct or better which is more reliable?
[Nexus S General] :
I haven't evaluated the new quadrant, because I lost faith in the old one. In the pro version of old quadrant you can view the scores of CPU RAM IO 2D and 3D graphics individually. The CPU RAM and IO scores varied horribly from run to run. The 2D and 3D scores reached a limit due to the frame rate maxing out. So the sum total of the result was meaningless.

For testing graphics I recommend GL Benchmark.

For ram and cpu I recommend antutu and linpack

For SD Card I recommend SD Card Tester (but you need to set at least a 2GB run for an accurate result)

For javascript I recommend Vellamo's sun spider test.

For database performance I recommend SQLite
[Nexus S General] original21:
So the score on antutu (3600), is it considered to be good for my nexus s
[Nexus S General] :
I can only tell you if your ram and cpu scores are good. What were they?
[Nexus S General] :
UPDATE: Brainmaster's Black CM9 benched!
[Nexus S General] original21:
Setup:

Matrix V18,5

min20 max 1300

SmartassV2 (deadline):)

results:

screenshot
[Nexus S General] :
Can you make sure you select the performance governor before you run the test?
[Nexus S General] original21:
Done
[Nexus S General] rrohanjs:
woah shocked to see kongs miui so low, im using it currently and its the same if not faster than the previous release ..
[Nexus S General] :
Okay, I think the scores are about right. I think matr1x is a good kernel, but I disagree with UV completely, and would recommend raising the voltages, both ARM and INT to the nominal levels. My own study showed smartassV2 to combine best with noop (although deadline is a good choice too).
[Nexus S General] :
It's the SD Card speeds. The previous release had a high SD Card Score, but it was because my testing method was flawed at the time.
[Nexus S General] glennkaonang:
I've seen changes in the spreadsheet.

It has another new column, isn't it?

Looking at, for example, Brainmaster's Stock 4.0.4, am I wrong to say that the supplied kernel performs better than T132?

Or there's another way to interpret the numbers?
[Nexus S General] :
Hang on Glenn, I'm going to make another change...
[Nexus S General] migueldbr:
bedalus why not insert an extra column with gl benchmark tests for each rom?would like to see if there is any significant change between each rom
[Nexus S General] :
UPDATE: [MIUI][2.4.6] Oodie pRiMe rOcket fUeled benched!

Moves up to #4!
[Nexus S General] :
Check out the raw data here: https://docs.google.com/spreadsheet/ccc?key=0AntDDKv-lS6IdFZ6VDRQV0cxRXFVT2dKYm94bWJyQ0E

...graphics scores are proportional to bus and cpu speeds alone.
[Nexus S General] Oodie:
:mad: :mad: Name of the rom is incorrect , calling my lawyer to sue you bedalus :cool::D

Should be :

[MIUI][2.4.6] Oodie pRiMe rOcket fUeled
[Nexus S General] :
Sorry! Sorted now.
[Nexus S General] migueldbr:
hey bedalus to benchmark pixelrom did you use 1.72 with modpack?or just plain 1.72?
[Nexus S General] :
Hey miguel, I used plain!
[Nexus S General] tchaari:
With his benchmarks, bedalus is explaining where each ROM is having a low score (RAM, SD card writing...). Consequently, developers easily know what they should improve ;)
[Nexus S General] :
UPDATE: Brainmaster's Black CM9 9APR ; Nexus MV 2.4.8 9APR ; [MIUI][2.4.6] Oodie pRiMe rOcket fUeled 9APR ; Crossbones 0.3.0 9APR

I've also come up with a way of testing ROM/Kernel compatibility by examining the performance gain of flashing T132. The spreadsheet looks a lot different now!
[Nexus S General] glennkaonang:
The spreadsheet looks a lot more interesting now.

Many thanks, bedalus.

I now see your point of locking your other benchmark threads.

Focus on this one and yeah it's now getting better than ever.
[Nexus S General] Oodie:
yeah !!! Colors are great
[Nexus S General] :
UPDATE: PlayROM v2 Benched!

UPDATE: AOKP Build 31 Benched!
[Nexus S General] glennkaonang:
I see that AOKP b.31 has Pete's 11 Jan kernel as the supplied one.

Does AOKP team always include Pete's kernel as the default?
[Nexus S General] :
I don't know. They have the same issue as Slim ICS, where the 4.0.4 update hasn't improved their javascript. I wonder why.
[Nexus S General] cowst:
Bedalus, could you check latest cybergr please?

He fixed at least an installation bug and changed to air 3.9.5.
[Nexus S General] tchaari:
bedalus, what a nice idea to add colomn H ;)
[Nexus S General] maxone001:
+1
[Nexus S General] ehiggins:
I'm trying the t132 kernel since you test with it. Wow is it fast....and then I saw I shouldn't need one on peter Alfonso's rom oh well. Seems to play nice.
[Nexus S General] :
That's not a sufficient update for me to spend an hour retesting it!
[Nexus S General] :
I have mixed feelings about telling people what to do... But I think if people are visiting my spreadsheet, they probably care about performance! So I just wanted to make it clear which ROMs/Kernels may have compatibility issues. People need to use their own judgement of course!

No no... feel free to try any kernel you want! But if you don't like to overclock, then Bugless Beast offers great performance out of the box.
[Nexus S General] cowst:
Damn, you are lazy...

Just kidding, but I think he mentioned other tweaks and I feel it visibly better.

Thanks for the efforts anyway :)
[Nexus S General] :
Okay, I'll take a look at the changelog.
[Nexus S General] RicardoOneri:
Hey bedabus can you bench CyberGr v10 from 9 April? Cyber updated it yesterday: D

Thank's!!
[Nexus S General] :
I'm waiting for him to post his changes before I decide it's worth the time. I want to do some programming today. I'm also running a kernel comparison at stock speeds, which is taking up a little time.
[Nexus S General] thegtfusion:
New rom here :D

http://xdaforums.com/showthread.php?p=24653651