yashsingh1/xdaforums huggingface dataset:
[Nexus S General] :
Found this: http://wiki.metalayer.com/index.php?title=API_Sentiment

I can do this in windows:

curl http://api.metalayer.com/s/datalayer/1/sentiment -d "text=In 1972, a crack commando unit was sent to prison by ... A-Team."

Returns a slightly negative sentiment. (-0.5 on a scale from -5 to 5)

...using this version of curl: http://www.paehl.com/open_source/?CURL_7.24.0

(top most download)

I also found this list of sites: https://sites.google.com/site/twittersentimenthelp/other-resources
[Nexus S General] :
These people seem to have a nice service (including a free option): http://www.alchemyapi.com/api/gstarted.html

...and there is an SDK so I may try to integrate this directly into my C++ program rather than relying on curl.
[Nexus S General] tchaari:
Yes bedalus. It can be automated. A HTTP request (call of a web page from a program with strings at the place of input box) can be done using any language curl, C++, Java...

This is discussion about how to do it in C++

http://stackoverflow.com/questions/1011339/how-do-you-make-a-http-request-with-c
[Nexus S General] :
Wow, programming is amazing. I'll try to build that example into a c++ program tomorrow. Then I'll see if I can do a full page of text to the api (there is a character limit). I'll aim to attach dates too.

http://www.alchemyapi.com/api/sentiment/textc.html

The above link gave me a slightly positive rating for my rank of word-triples for Slim ICS, whereas other sites had given negative. After looking at the word triples manually, it seems the sentiment is definitely positive. I think this is a good sign that the api will prove to be reliable... perhaps it will be more reliable with virgin text. Exciting!
[Nexus S General] :
UPDATE: Checked my original program for tallying threads worked when threads got longer. Expected behaviour was to only read the necessary pages and update the .dat file. Unfortunately, it reads the new pages, and replaces the .dat file. Fixed now (I think... I have to wait until a thread gets longer to test).

I cannot understand how to integrate curl into c++ so I will just use system calls and place a prebuilt curl program in system32. It's an easy workaround.
[Nexus S General] :
UPDATE: Nearly have it working.

I have threads sending their entire text (but no quotes) to the sentiment api in 400 word chunks, with scores returned.

I plan to share the results as a graph.
[Nexus S General] tchaari:
wow, this is very very interesting bedalus. Are you interested to mount a software engineering company with me? lol

EDIT: I am thinking about the next challenge: smoothness benchmark :D
[Nexus S General] :
Don't know if I'll have the time to go global... :p

Program now works completely. I can't share it yet, I just need to write a method to import my private key from a file. Then other people can use it (but would need to get their own key). It wouldn't be much work, but unless I get demand for that, I actually think will just keep it private for now.

... because now it's working well, I will start working on a public chart!

EDIT: Okay, here's the new graph format, shows the accumulation of sentiment. So far only NexusMV (CM9 base) visible.

Link: https://docs.google.com/spreadsheet/ccc?key=0AntDDKv-lS6IdG9YU2h0LWVKT0RrZzVlMmZzNlhwSWc
[Nexus S General] glennkaonang:
Bedalus = A math teacher and a developing C++ programmer having great analytical instincts. Damn you've just blown me off!
[Nexus S General] :
Does this work?

<script type="text/javascript" src="//ajax.googleapis.com/ajax/static/modules/gviz/1.0/chart.js"> {"dataSourceUrl":"//docs.google.com/spreadsheet/tq?key=0AntDDKv-lS6IdG9YU2h0LWVKT0RrZzVlMmZzNlhwSWc&transpose=0&headers=1&merge=COLS&range=B29%3AB140%2CF29%3AF100%2CJ29%3AJ40&gid=0&pub=1","options":{"vAxes":[{"viewWindowMode":"pretty","viewWindow":{}},{"viewWindowMode":"pretty","viewWindow":{}}],"curveType":"","booleanRole":"certainty","useFormatFromData":true,"animation":{"duration":500},"lineWidth":2,"width":790,"height":371},"state":{},"view":{"columns":[{"calc":"emptyString","type":"string","sourceColumn":0},0,1]},"chartType":"LineChart","chartName":"Chart1"} </script>

EDIT: Try legend: {position: 'none'}

Need to host this on my own separate page I think.
[Nexus S General] :
It's here at last:

http://goo.gl/IOpOE (link to graph)

The UX graph you've all been waiting for... Works great on computer and phone too! On a computer hover over a graph-line to see which thread it represents, and on a phone, tap it.

I just need to wait for more allowance from the sentiment API and then I can add the rest of the threads. Any improvement ideas anyone?

Also, does anyone know how to get recognised contributor status? I pm'ed six senior mods altogether and so far got no reply. I think I deserve it after all the benchmarks! :D
[Nexus S General] glennkaonang:
That's because the mods are discussing about promoting a new title "Recognized Contributor & Determined Analyst" :D :D

Anyway, regarding the chart, what does steeper mean? I see that Slim ICS is currently the steepest one.
[Nexus S General] :
:)

Steeper means the same as higher gradient, or the one that rises at the fastest rate.

The one rising at the fastest rate is the one getting the most positive feedback. So, although I haven't added all the ROMs yet, currently Slim ICS is winning.
[Nexus S General] AeroEchelon:
Wow, this is great work bedalus. It was amazing that you were able to parse through the threads but then you went out of your way to graph it from the results of an API. Impressive!

Also, you may have answered this in a previous post but what does the quantitative number on the Y axis represent? I understand that a steeper line means a better UX experience.
[Nexus S General] :
It's not a standard scale. The y axis (y to the sky kids...) is accumulated sentiment.

For example, the cyanogenmod thread has about 150,000 words in it, and I sent them to the API in 400 word blocks, that's 375 calls to the API, each one returns a score between -100% and 100%.

100% would be the most positive thing it is possible to say (amazing/excellent/outstanding)

-100% is the most negative (rubbish/useless/awful/disastrous)

So cyanogenmod is at 15, which = 1500%

However this is over 375 API calls,

so that's an average of 4% positive.
[Nexus S General] tchaari:
Wow, what a wonderful way to present the results bedalus. Very very good work man.

I have one question, when you grab the words from the threads, are the given thanks included? did you also exclude html tags?
[Nexus S General] :
Like "58 people thank you for this useful post"? I didn't include that. Only what was said. What is a html tag?

kernels ; battery ; ROM ; gov/sched

Now with summaries in the first posts. Convenient for XDA app users!
[Nexus S General] tchaari:
A HTML tag is what is used in the web pages source code...

I've just built a Java program that automatically:

1) grabs the text from the threads that are on the Nexus S dev forum's first page

2) analyzes all the posts (by chunks of 50) of each thread using the wingify API

3) computes and prints the average opinion score of each thread

I can generate these scores in a file.

Bedalus, can you please tell me which format (for this file) is better for you to facilitate presenting the data using graphs?

I have also some improvements for my program in mind but I have to sleep now ;)
[Nexus S General] :
Nice, a different approach.

Yeah, my program throws away all html tags. It differs to yours in that you first have to specify which threads you want. Also it breaks threads up into equal amounts of text, not equal amounts of posts. (I'm not sure there for whether one way is better?)

I use all the scores to produce a graph, but i can easily add an average to the page too.

I was very confident in the alchemyAPI scores. Do you think wingify is better? I thought it was focused on twitter? Perhaps both can be amalgamated?

kernels ; battery ; ROM ; gov/sched

Now with summaries in the first posts. Convenient for XDA app users!
[Nexus S General] :
UPDATE: I now include a bar graph at the top of the site, which shows the average sentiment score. This makes it simpler to compare threads overall UX.

UPDATE: Received my free daily API allowance (1000 calls) so I've added nuhetri-reloaded, CyberGR-MOD, crossbones, and I've re-done Pixel (it didn't complete properly the last time - I ran out of allowance about two thirds of the way through). I'll see if I can get another thread too.

UPDATE: I haven't run out of allowance, so I've added Oodie pRiMe rOcket fUeled, CodeName Android, and AOKP
[Nexus S General] tchaari:
Wingify is not specifically designed for twitter. Can you please make a little comparison between them for a same text?

My program seems to take too much time to analyze the long threads. I am thinking about reducing the amount of text to analyze. I'll try to look back into this tonight after work :)
[Nexus S General] :
Wingify>wings>birds>twitter... must be why I thought that :p

I compared some text about the American September 11th attacks that I copied from wikipedia. The scoring systems are different. If they were on the same scale, AlchemyAPI would give it (0.44) and Wingify scored it as (0.34)

Not miles apart... both slightly negative.
[Nexus S General] :
If anyone else thinks I'm just doing a word count, read through my methodology in the second post!

This person thought i was counting posts: http://xdaforums.com/showthread.php?p=25295292
[Nexus S General] tchaari:
Yes I saw the discussion and I think that we can have an idea about the time window of the posts. I can extract it from the threads but it will take more time to analyze the posts :)
[Nexus S General] :
I've just added a new graph for the most recent 10% of all posts. I think that should cover it?

(Thanks to original21 for the inspiration: http://xdaforums.com/showpost.php?p=25302700&postcount=5088)
[Nexus S General] tchaari:
Nice work bedalus.

This is what my Java program produces (using wingify API):

[ROM][DEV][PRE-ALPHA] HTC Sense 4.0 | ICS 4.0.3 | 2/17/12 - 0,579 - 0,571 - 0,659 - 0,596 - 0,621 --> {avg=0,605}

[ROM][MIUI 4][2.4.20] Catalyst MIUI with OTA - 0,839 - 0,502 - 0,603 - 0,763 - 0,322 --> {avg=0,606}

[ROM][PORT] Nexus Sense 3G v1.4 | Full HTC Sense Experience | 03/04/12 - 0,811 - 0,75 - 0,239 - 0,819 - 0,819 --> {avg=0,688}

[ROM][AOSP][4.0.4][Crespo] Gummy 102 - 21 APR 2012 - 0,668 - 0,605 - 0,741 - 0,673 - 0,824 --> {avg=0,702}

[ROM] Ultimate ICS v4.0 - 0,857 - 0,537 - 0,776 - 0,596 - 0,797 --> {avg=0,713}

[ROM][IMM76D][19/04/12] Next ICS |STOCK|ELEGANT|INVERTED| CM9 Kernel | - 0,849 - 0,849 - 0,793 - 0,677 - 0,459 --> {avg=0,725}

[ROM] [i9020/23] CyanogenMod 9 v4.0.4 v7.0 + CM Night builds - 0,727 - 0,76 - 0,77 - 0,7 - 0,763 --> {avg=0,744}

[ROM] Androxide 2.0.0 | AOSP | 4.0.4 | 24-04-2012 - 0,866 - 0,881 - 0,637 - 0,783 - 0,648 --> {avg=0,763}

[ROM][INFO][AOSP][ICS] Android Open Kang Project-build 22 - 0,61 - 0,794 - 0,795 - 0,855 - 0,807 --> {avg=0,772}

[ACS][ROM][4.0.4 NS 3G][GSM] Pool Party V .5 ICS - 0,737 - 0,737 - 0,88 --> {avg=0,785}

[ROM][4.0.4]Slim ICS 68MB- Clean, Simple & Fast - The Google Way - 0,857 - 0,846 - 0,857 - 0,681 - 0,838 --> {avg=0,816}

[ROM][ICS][AOSP] Bugless Beast (Nightlies) - 0,891 - 0,748 - 0,857 - 0,773 --> {avg=0,817}

The scores before each average represent the opinion analysis of the latest 300 posts. It gives an idea about how the opinions have evolved until the last post.
[Nexus S General] :
I think crossbones got a good score with mine because it is a very short thread. Usually a new thread begins with a lot of well-wishers posting encouragement. It's strange how you got a low score for it. Anyway, I compared some of our mutual scores:

--- ROM Title = Tchaari's method / my method

CyanogenMod 9 v4.0.4 v7.0 + CM Night builds=0% / 0%

Android Open Kang Project-build 22=+19.4% / +44.5%

Slim ICS 68MB- Clean, - The Google Way=+54.8% / +67.1%

I compared your most recent to my most recent.

Assuming that wingify has 50% as neutral, I've calculated the proportional improvement of AOKP and Slim ICS compared to Cyanogenmod using your method and my method. (There were only three ROMs that we both did).

I looks like our methods produce comparable results.

EDIT: I'm see if I can get a key for wingify and see if I can use curl for their api like I do with alchemyapi.

EDIT2: yeah... curl works fine. I may need a key from them first though.

EDIT3: Okay, I tried a brief test with wingify (I had previously tried it before with AlchemyAPI)

With AlchemyAPI

"really good" - 60.3% (positive)

"really not good" - 42% (negative) (50% is neutral)

With Wingify

"really good" - 91% (highly positive)

"really not good" - 91% (highly positive)

So it looks like Wingify may have some issues with negation.
[Nexus S General] :
UPDATE: [ROM | 4.0.4][MIUIV4] MIUIXPERIENCE #5 | Smooth! Duration! Stock!

[ROM][CM9] Brainmaster's CM9 & Black CM9 v3.0 OTA

[ROM][MIUI 4][2.4.20] Catalyst MIUI with OTA

[ROM] Next ICS |STOCK|ELEGANT|INVERTED| CM9 Kernel

all added to the UX site.

I found some of the threads too short to get a reliable recent UX, so I skipped them (e.g. Pool Party).

I will have a look through the development section to see if any new 4.0.4 ROMs have arrived, and if they have had sufficient enough posts to get a usable UX benchmark.
[Nexus S General] :
I think if it works better on a small string, it works better on all text!

I will include 3G Sense and Androxide in the next update (also gummy). But from now on, only threads being actively developed and with at least 400 replies will be included. In fact I may take crossbones out because it has very few posts, and I think the developer has had a new baby (=0 time for dev work!)

My mutual ranking is

Next ICS

Catalyst MIUI

AOKP

BB

Cyanogenmod

So only AOKP out of place (higher up in mine... but I changed the way I rank now, a simple addition of entire thread UX + recent UX)

I found this list: https://docs.google.com/spreadsheet/ccc?key=0Ajqw0BPB53J7dFZmTGhCeGFvNzBGazliQXRlYnBfeFE

Some of them might have API/Curl for free. I'm happy with AlchemyAPI for now.
[Nexus S General] tchaari:
Cool, let's keep the two rankings then. Two different approaches :)

I'll, try to generate the graph script when i have some free time. Can you please pm me your graph script bedalus?
[Nexus S General] :
I'll just attach a spreadsheet here. It's easier. Is this what you need?
[Nexus S General] tchaari:
Thanks buddy. Can you please give me an idea about how did you integrate the graph in your webpage?

EDIT: Btw, I am loving your webpage bedalus. Really nice work ;)
[Nexus S General] :
It's my first ever web page!

I made one with Google sites. All you have to do is import the chart Not sure, but probably it has to be a chart in a Google doc, so email that excel doc to yourself first, then open it as a Google doc. Paste in your own data, then create a chart. You'll be able to import it from your Google site.

kernels ; battery ; ROM ; gov/sched

Now with summaries in the first posts. Convenient for XDA app users!
[Nexus S General] :
Some kernels now up: http://goo.gl/i466T

Trinity will have to wait until tomorrow, and there is also GlaDOS (just ran out of API allowance part-way into the trinity thread).
[Nexus S General] stewarta:
Good stuff. I'd still like to put the posts through Leximancer and see the output. I can explain via PM if you are interested. ;)
[Nexus S General] :
Pm sent!
[Nexus S General] :
UPDATE: Trinity and GlaDOS added.

Trinity ate three quarters of my API allowance today, GlaDOS the other quarter. But I don't think there is much else I can add at the moment anyway.

I removed the average and recent bar charts for the ROMs, leaving only the combined bar chart to make things simpler (and I kept the line-graph because it looks cool...)

There are now links to the data in its original numerical form (on google docs spreadsheets) at the bottom of the sidebar.

I plan to update all the ROMs and Kernels weekly.

EDIT: I see I have recognised contributor status! Thanks community! :D
[Nexus S General] Jamalsid:
Congrats!!

I think it is safe to say you earned it!!
[Nexus S General] :
Thanks Jamalsid!
[Nexus S General] tchaari:
+1,

Congrats buddy, you really deserves it after this quantity and quality of work that you have done ;)
[Nexus S General] :
UPDATE: Apex added to the NSUX chart. (bigxie has pushed out a 4.0.4 test version)

Also, put all the ROMs in rank order.
[Nexus S General] :
UPDATE: Performed my first attempt at a global update (all ROM and kernel threads)

...discovered a bug in my program, corrected it (thank goodness for having the sense to backup all of my working folder!)

...after that, the update went smoothly!

Currently the bars show the global average plus the most recent 10%. I think I will change this to add half the most recent score, a quarter of the one before that, an eighth of the one before that... etc. I think this is a better way of ensuring the results reflect current sentiment.
[Nexus S General] tchaari:
bedalus, there is a problem at the ROM graph display. The graph isn't shown and an error message is displayed (All series on a given axis must be of the same data type).
[Nexus S General] rrohanjs:
congrats bedalus .. :)

i hope you bring back rom benchmarks :D
[Nexus S General] kong:
I hope not, lol.

It turns my ROM-making hobby into a serious part-time job :D
[Nexus S General] :
Oops technical difficulties...

...and we're back!
[Nexus S General] :
I don't think there's any need to bring back the performance benchmarks. I think all the 4.0.4 ROMs perform very close to each other:

41.86%

41.08%

40.53%

40.34%

40.09%

39.77%

39.72%

39.20%

39.09%

38.19%

In the top ten, if we take the average for the top 2 as 41.5% and for #9 and #10 as 38.5% that's only a difference of 3%

Let me roughly organise that list from the most modded to the least:

MIUIXPERIENCE #1 | StuffedCrust

[MIUI][2.4.6] Oodie pRiMe rOcket fUeled

MIUI Catalyst 2.4.6

CyberGR-MOD|NS.NGN ICS v10 HYBRID

Pixel Rom 1.72 ICS

Pool Party V .5 ICS

Crossbones v0.3.0

Brainmaster's Stock ICS 4.0.4

AOSP+ N.5

Stock ICS 4.0.4

...that's quite a wide range, from stock, to MIUI with various mods, all performing at near enough the same level.

My point is that what seems to affect performance most with 4.0.4 isn't the individual mods, but perhaps conflicting mods. I think some new developers have thrown everything possible into their ROMs when it would have been better to exercise restraint.

So I think what we have now is the need for developers to provide the right balance of eye candy and extra utility in the ROMs.

So how do we find out if they have got the balance right? Feedback from the users! I'm not saying that this UX benchmark is perfect for assessing that, but I think what it does do is roughly highlight which ROMs are drawing a crowd, and for the new root users and crackflashers alike, the UX provides a handy snapshot of where the action is.

I think if you take a relatively new ROM like Next ICS for example, where the developer has a distinctive creative vision, the skills to execute it, and is responsive in his thread, that will attract attention, and I think that's very clearly showing up in a good UX score. And if there were performance issues with the ROM, people would complain about it!

To summarise, I personally believe that when performance is not an issue, what matters most about a ROM is its unity, character, seamlessness, and these things can best be provided by developers with not only a creative vision, but also restraint and focus.
[Nexus S General] glennkaonang:
A very nice argument you got there, bedalus.
[Nexus S General] tchaari:
lol, yes true words. Since bedalus started his several benchmarks, ROM developers have produced more in a better quality ;)

by the way Kong, there isn't better than a good update to have a boost in bedalus graph with many positive feedbacks. So, I think it's time to update your MIUI ROM and cook its language pack again :D
[Nexus S General] :
I'm going on holiday for a week!

I'll refresh the results when I get back, and continue refreshing on a weekly basis.
[Nexus S General] tchaari:
enjoy your week buddy and don't think a lot about benchmarks in your holidays :D
[Nexus S General] glennkaonang:
Oh I thought he would do "Holiday Family Members Experience Benchmarks" :D :D
[Nexus S General] Oodie:
He gets sweet dreams & also nightmares of " BENCHMARKS " .

As i heard from one of bedalus neighbor , Sometimes when bedalus sleep he says ,

1. " Oh ! my dear benchmark . my love , ***** ( adult only words occurs after that )

2. " Ohh ! what the hell benchmark , you go to hell , you f**** ( adult only words occurs after that )

Then he wakes up & runs to his PC & so on , so on

Edit : Enjoy your holiday buddy ! :D
[Nexus S General] :
Benchmark the holiday, good idea!
[Nexus S General] Oodie:
ho hoo .. calculate the given food proportional to the given Liquor :D
[Nexus S General] viralbug:
Wow! You sir are amazing! First it was those benchmarks and now this user experience.

Your time and efforts are very much appreciated. Thanks a lot for providing us with all these comparisons!

Enjoy your holidays and have a great time! :)
[Nexus S General] tchaari:
If we calculate OodieSkank updates proportional to euroskank's. The result is not outstanding : 1/2 or 1/3 if I am not wrong :p

Just joking buddy. I like the idea of your modded skankwich ;)
[Nexus S General] Oodie:
with my Data limit for a month ( only 3Gb ) i really can't release any roms frequently . That's the reason :( I'm thinking of quiting from rom modding & keep on testing other roms & kernels :rolleyes: :D
[Nexus S General] tchaari:
Oh no, please don't. Don't worry, I can wait an entire month for the next Oodie Skank :D

Really Oodie, if you have enough free time for developing and you really like it, I highly encourage you to keep modding. I would like to do it but not enough free time (teaching, scientific research, student projects supervising, administrative tasks, wife, family...) and I really feel happy when young devs like you do things that I wanted to do ;)
[Nexus S General] tchaari:
This is the new ranking of the ROMs according to my sentiment analysis program. I removed the ROMs that were not updated since one month and I also removed the threads that have less than 200 posts.

1) [ROM] Androxide 2.0.0 | AOSP | 4.0.4 | 24-04-2012 [26th March 2012 09:52 AM : 0,866] [29th March 2012 03:52 PM : 0,881] [10th April 2012 06:41 AM : 0,637] [25th April 2012 05:26 PM : 0,783] [30th April 2012 11:00 PM : 1,008] --> {

avg=0,835

}

2) [ROM | 4.0.4][MIUIV4] MIUIXPERIENCE #5 | Smooth! Duration! Stock! [20th April 2012 05:57 PM : 0,821] [23rd April 2012 12:44 PM : 0,803] [25th April 2012 04:57 AM : 0,822] [28th April 2012 07:39 PM : 0,765] --> {

avg=0,803

}

3) [ROM][ICS][AOSP] Bugless Beast (Nightlies) [13th April 2012 05:18 PM : 0,748] [16th April 2012 11:52 PM : 0,857] [23rd April 2012 07:57 PM : 0,773] [1st May 2012 12:33 AM : 0,785] --> {

avg=0,791

}

4) [ROM][4.0.4]Slim ICS 68MB- Clean, Simple & Fast - The Google Way [5th April 2012 12:28 AM : 0,846] [14th April 2012 10:21 AM : 0,857] [23rd April 2012 09:04 AM : 0,681] [30th April 2012 03:43 PM : 0,652] --> {

avg=0,759

}

5) [ROM][MIUI 4][2.4.27] Catalyst MIUI with Language Packs and OTA Update [20th April 2012 04:59 PM : 0,502] [24th April 2012 03:30 PM : 0,603] [25th April 2012 04:56 PM : 0,763] [29th April 2012 12:02 PM : 0,758] [1st May 2012 01:09 AM : 0,878] --> {

avg=0,701

}

6) [ROM][INFO][AOSP][ICS] Android Open Kang Project-build 22 [28th March 2012 07:54 PM : 0,794] [16th April 2012 04:32 AM : 0,795] [24th April 2012 12:44 AM : 0,855] [29th April 2012 12:34 PM : 0,807] [30th April 2012 01:20 PM : 0,185] --> {

avg=0,687

}

7) [ROM][AOSP][4.0.4][Crespo] Gummy 102 - 21 APR 2012 [14th April 2012 10:24 AM : 0,668] [17th April 2012 07:46 PM : 0,605] [21st April 2012 04:17 PM : 0,741] [25th April 2012 08:31 AM : 0,673] [29th April 2012 03:40 PM : 0,687] --> {

avg=0,675

}

8) [ROM] [i9020/23] CyanogenMod 9 v4.0.4 v7.0 + CM Night builds [24th April 2012 02:33 PM : 0,7] [25th April 2012 09:49 PM : 0,763] [27th April 2012 01:55 PM : 0,536] [29th April 2012 10:14 PM : 0,721] [30th April 2012 10:47 PM : 0,644] --> {

avg=0,673

}