yashsingh1/xdaforums huggingface dataset:
[Android General] TrevE:
Hey guys, an EFF volunteer - Jered Wierzbicki - has created an application called IQIQ to decode Carrier IQ profiles back to XML. Their public git repo is in the above linked article. In a nutshell, Profiles contain what data is sent back at what interval and to where.

We are now able to see sets of metrics called from phones. The problem is right now we only have default profiles. The EFF is looking to collect as many profiles from as many different devices as possible trying to find real metrics.

Github Proof of concept code for a "profile scraper": https://github.com/TrevE/IQTool

(right now it scans, but only works for automatic sending of archive.img from tmobile, but it should be for root or non root)

If you can contribute go for it! It would help to have a "one click profile sender" out there for everyone.

Possible Methods

FINDING .PRO FILES

We are not too sure yet where updated profiles get stored, so for the time being root users can run the following to search out all .pro files on your disk.

adb shell busybox find / -iname "*.pro"

It will likely be some sort of IQProfile.pro, CIQProfile.pro, defaultprofile.pro. Once you locate it just:

adb pull /full/path/to/profile.pro .

Other strategies, such as grepping for a characteristic string like " CONSTANT " across the whole filesystem, might also turn out to be useful. Use this thread to experiment with and improve such techniques!

Waiters suggests a command reference:

Getting archive.img from non-embedded CIQ

We don't yet know if all profiles will be .pro files, or if they'll sometimes be embedded inside of other things. If you are on tmobile the profile information is potentially contained in an archive.img file. The file could be world readable, so you might not even need to be rooted.

WARNING ABOUT ARCHIVE.IMG FORMATS:

Please be warned that sensitive data could be in this archive.img file such as URLs, IMEI, SMS metadata, etc.. EFF will always do its best to keep archive.img files confidential, but please DO NOT send them if there may be any private information on the handset you are working with

Example of where Tmobiles CIQ archive is:

adb pull /data/data/com.carrieriq.tmobile/app_iq_archive/archive.img .

KNOWN STOCK MD5SUMS

Were really trying to find some new profiles containing the pushed metrics, so to check if you are looking at a "stock" profile or not (this only applies to if you find a .pro file)

adb shell busybox md5sum /path/to/profile.pro

If it matches one of these md5sums (number on the left) it is likely a known "stock" profile.

e37a4a8e3ea6d6aa4b7423a462541fa9 att-galaxy-s2-defaultProfile.pro
2618eaa2e3310ec36e1b86f8b643c5fa htc-amaze-tmob-defaultProfile.pro
a6886135d2d1ea423d4edde389fe1794 htc-evo-sprint-iqprofile.pro
2618eaa2e3310ec36e1b86f8b643c5fa tmob-defaultProfile.pro

SUBMITTING PROFILES

If you would like to submit your profile or archive.img to EFF so CarrierIQ metrics on what operator can be better understood, please send that data off to iqiq@eff.org . It would be very helpful to them to include phone model and network it was pulled from as well.

Thanks for all your help guys!
[Android General] azrienoch:
Portal article coming up. Thanks TrevE, and Jered and the EFF. We really appreciate everything you do.
[Android General] tommytomatoe:
Woot woot! Keep up the good work Treve!
[Android General] guitardoc64:
Thanks again TrevE!! I have some friends with OG EVOS that are stock that can help and a buddy on ATT with an Atrix. I'll get them involved if possible. It's great to see them collecting evidence.
[Android General] AdamOutler:
Any other names?

adam@Adam-Desktop:~/IQIQ$ adb shell
/ # busybox find / -iname "*.pro"
/data/data/pl.aygorund.littlepiano.pro
/ #

I'm sure "little piano" which was free on amazon 2 days ago is not Carrier IQ related.

Note:This is an AT&T Infuse 4G with a 4 month old custom kernel.
[Android General] waiters:
maybe something like this (which could take hours to run...)

grep -r "CONSTANT PROPID" /
[Android General] Orical:
I'm just glad it's out in the open, there's enough crap in the world to have to worry about, if your banking from your phone (no chance I would have in the first place but) there's a real threat and a legit reason to go in personally to change all of your accounts and who in the world has had all pass access to look at it for how long.

Thanks for the post man, this is that great eye opener that needed to happen I just hope nothing happens to anyone because of Carrier IQ. It's good to see there are others that are taking this seriously.
[Android General] edanono:
Will root explorer find this for us?
[Android General] sgt. slaughter:
tried this on my sprint evo3d and got a odd response...

when I go into adb shell and type: busybox find / -iname "*.pro"

I get the following:

/system/etc/iqprofile.pro

find: /data/DxDrm/fuse: Permission denied

Thought this was odd since im fully rooted s-off here and would be denied access to something, no? The first few times I did it straight from the comand line and that resulted in only outputting the "...permission denied" line shown above. When I did "adb shell" first by itself and then did the second part of the command it gave me the iqprofile.pro....

any thoughts?
[Android General] PromaneX:
I had to go into a shell, do the su command to gain root, and THEN do the search.
[Android General] TrevE:
Not sure, does the Infuse have CIQ?

That system/etc/iqprofile.pro might get updated on the e3d, were not sure yet how that works. Can you run busybox md5sum on that file?

I will post known hashes for stuff EFF has already (default profiles).
[Android General] sgt. slaughter:
just to be sure i did this right, heres what i did:

adb shell

/system/xbin/busybox md5sum system/etc/iqprofile.pro

this output the following:

a6886135d2d1ea423d4edde389fe1794 system/etc/iqprofile.pro

and btw im running the stock OTA with full CIQ and stock kernel here.
[Android General] shaky153:
TrevE Im not sure if you mentioned this but I have been working on kernel and I came across code in kernel to do with carrier IQ
[Android General] TrevE:
I made a quick and dirty android app for searching / sending profiles. I will be polishing it up more, anyone else feel free to jump in here to. The more grep strings or other locations these profiles are found will be added in here.

Right now it works only for tmobile flawlessly. But it will work for non rooted tmobile users, so if you feel comfortable send this to everyone on tmo you know to get the ball rolling. There is a compiled copy in bin\

https://github.com/TrevE/IQTool
[Android General] guitardoc64:
I hope it will work on unrooted stock. I have some candidates.

Sent from my Clean ROM 3D
[Android General] TrevE:
for tmo? it should the files are world readable - but it was quick and dirty code :) If not let me know and ill have er fixed up. Ill be cleaning up more in the next hours, just havta run up for a bit so i pushed my first checkpoint
[Android General] snowdream86:
Thank you.

i will have a try too.
[Android General] sgt. slaughter:
cool deal.

you get a chance to see my response to you about md5sum you asked about?

just wanna know if i should send this file off or should i look for something else. lemme know if you want me to compare to anything you have on ur 3D. like i said mine is all stock kernel and all just on last OTA and rooted s-off.
[Android General] guitardoc64:
For Sprint and ATT. 2 og EVOs and an Atrix.

Sent from my Clean ROM 3D
[T-Mobile LG G2x] jacob808:
I've read on many a thread to not buy the next Nvidia quadcore equipped phones, just because Nvidia isn't very open with their source code.

I say Hell No! If anyone, especially in the cellphone business knows about gaming, it's Nvidia.

Video games are the state of the art in software. It the most demanding kind of software on computer hardware, and Nvidia is an expert in this area.

The G2x was meant to be first and foremost a GAMING device disguised as a super smartphone, and almost a year after it's release it's showing it's potential with PSP looking graphics in games such as GTA 3, Frontline Commando, 9 MM, and many new releases in the month of Nov/Dec.

Most of the other phones that don't have a Nvidia chip has massive framerate drops and struggle to play these new breed of cellphone games smoothly.

Also, when the G2x does get the ICS update it'll extend the life of the phone maybe another year or 2 just because now the operating system should fully support multiple cores thereby tapping into every bit of power the Tegra 2 has to run more smartly future videogames adapted to take advantage of the phones particular architecture.

Me, I'm staying with Nvidia and I suggest anyone that wants a powerful phone for entertainment (not just videogames but any sort of media) fully support Nvidia and their upcoming powerhouse Tegra chips.
[T-Mobile LG G2x] JaiaV:
Sorry, but between NVIDIA's insane attitude towards open source software and companies like LG who are utterly inept at properly coding, maintaining and updating Android devices...no. Without the necessary code to update and improve the phone ourselves without tons of necessary workaround coding that is having to be done to port updates (like ICS) to our phones, dev support for Tegra 2 devices will be slim in the next year or so. There is absolutely no doubt that LG will delay, waffle and even possibly just never update the G2x in the first place, and that's hoping that update won't be another buggy mess that has to be cleaned up by independent developers in the first place like the Froyo and Gingerbread builds were.

I'll pass on buying another NVIDIA product. I stopped buying their PC products a LONG time ago because of their poor support and quality (don't even get me started on their Linux drivers), and I should have stayed away from their mobile processors too.

That's not even beginning to counter your assertions that most of the games you mention only run on Tegra processors well. The Tegra 2 isn't the best mobile processor by any means (although it is nice), almost no games were ever made to actually take advantage of the GPU any more than any other Android device's GPU and in fact the Tegra zone games run just fine on other processors once you remove the device check. O yeah, and it handles almost all native high resolution videos poorly, even in YouTube for crying out loud.
[T-Mobile LG G2x] Evofusion:
None of the games you mention are Tegra games...not one... Have you seen tegra zone? Its a disgrace. BTW the reason tegra games don't work on other phones is not because our phones are that much superior, but because the game checks to see if your phone has a tegra chip, if it doesn't, it won't even boot, but people have made an app that bypasses this check so even people using the SGS2 can play our so called "unique to tegra" games.

Sorry, not to be rude or offensive, but I don't think you know what you're talking about =\

Also, this quote:

^^ is pretty funny hehe

EDIT: I might be wrong about the way the phone plays Tegra games, I believe Tegra specific games are coded differently (NOT STANDARD) so that's why other phones can't play them. Some smart people made an app that can read this code and play the game on phones like the Galaxy S2, I remember reading about it when the SGS2 barely came out.

Either way...
[T-Mobile LG G2x] psychoace:
a lot of your complaints seem to not be g2x specific. everyone is having the exact same problems the g2x is having with ICS because its the same problem across all boards. a lot of drivers are not open sourced. even in the source release drivers are compiled and presented as modules. the reason it seems worse for the g2x is because there are so few tegra 2 devices to rip files from. there are tons of snap dragon and hummingbird devices out there that its easier to fix some problems. what your asking from

nvidia is more then what you will get from anyone else you might jump too
[T-Mobile LG G2x] JaiaV:
Believe it or not, Qualcomm releases open source for their chips (and devs like the CM team are having an extremely easy time of porting ICS to many of those devices for that very same reason, OMAP4 is just first as the Galaxy Nexus is OMAP4 itself and therefore faster to port). Your post appeared to be about the Tegra 2 chipset, which is why you got a response about the Tegra 2 chipset. While my complaints are not necessarily G2x specific, they are very much so about the Tegra 2, and by extension all are also directly related to the G2x as all of them exist on the G2x also due to it being run by the very same chipset...
[T-Mobile LG G2x] mustangtim49:
Its amazing how when someone comes along with an educated opinion, how many omniscient people are members here. We should change the name from xda to I know everything and everyone else is wrong. Opinions are like assholes, everybody has one. Nice point bro, nvidia does do games well. They don't have to make them, their job is to run them efficiently.
[T-Mobile LG G2x] JaiaV:
On a positive note:

1. The Tegra 2 really is a nice chipset, it does some graphics very well and while some of the Adreno models are superior they don't tend to have well coded drivers on some phones.

2. NVFlash - While in some ways this is limiting (not being able to flash the recovery otherwise, Rom Manager isn't really great on Tegra 2 devices as you cannot use the recovery on the actual recovery partition), this pretty much makes the phone nearly unbrickable and far easier to root than many other devices being as you can just NVFlash CWM and flash away.

3. I love my G2x and would love to use it as my daily driver for at least another year or so, unless some phone comes out that has both amazing processing power AND vastly improved battery technology. Batteries are really what is holding mobile computing back at this time, and current battery technology is why the battery life on new cell phones and devices like the 3DS and (reportedly) the Vita suffer, as while processing power has increased exponentially, battery life has not really improved at all.

Now, it does have some issues:

1. NVIDIA is not likely to ever release source, which pretty much spells death for the phone as far as updates are concerned as soon as LG stops updating the phone or NVIDIA has a change of heart. LG's update history is pretty sordid. Devs WILL get tired of trying to find workarounds to get basic hardware functions like hardware acceleration, sound, and bluetooth working when other phones have hardware that is far easier to code for (and often with open sourced drivers, at least enough to update the drivers to run on newer versions of Android).

2. There are issues with certain (many) formats of video running in 720/1080p that the Tegra 2 chipset stutters on. As there is no hardware acceleration for these video formats in the Tegra 2, it all has to be done in software which these devices seem to have trouble with.

3. The Tegra 2 has a limited amount of bus to work with, while not a problem most of the time and helped by the decent amount of cache on the chip, this occasionally causes some traffic jams.

4. The G2x has a somewhat limited amount of RAM with only 512MB total and a good chunk of that dedicated to the GPU. That said, that's really not that bad considering just how recent being able to have that much RAM was possible in such a small device really is.

Note: I am not an expert by any means and what I say is not to be taken as the absolute truth. I am merely a device enthusiast who happens to really love his Asus Transformer and G2x (both Tegra 2 devices) and I try to spend time learning about how they work and hoping that those with far more skill than I will ever have continue to love and support our wonderful toys.
[T-Mobile LG G2x] fcisco13:
I'll cut my left nut before i buy another NVIDIA phone, even LG is not getting my business, had ti buy 3 TVs for xmas, guess what was one of my choices ....LG and guess what even tho they were cheap i did not buy them, same thing goes with the next rig i build NO NVIDIA. They did us soo bad with the ****ty GB update, if it wasn't because of the lawsuit we would have never received it, thanks cm team, the hell with NVIDIA & LG. If i want cool graphics ill play 360 or ps3, i want my phone fast, smooth and trouble free.

NVIDIA NEVER AGAIN!
[T-Mobile LG G2x] JaiaV:
Off topic, but if you don't want to deal with NVIDIA, AMD makes some very nice video cards that have been properly trouncing NVIDIA's offerings for over a year now, AND they have far more open source and Linux support available to boot! They also designed the GPUs in the Wii and the 360 (the 360's GPU was actually fairly impressive at the time and Nintendo buys older, cheaper and more stable hardware on purpose, which the GPU in the Wii is). Also, all Adreno GPUs are actually based on the ATi Imageon GPUs.
[T-Mobile LG G2x] ZeroT| Demented:
I say stay with nvidia its not out fault cyanogen mod is so stubborn that they have to not use the original drivers just so there build.prop says 2.3.7 instead of 2.3.3 there's no difference
[T-Mobile LG G2x] mjm128:
I'll stick to boycotting nvidia. I'd rather my money goes to a company that will support me. I'd also rather own a phone that isn't a nightmare for devs to upgrade that devs will probably lose interesting in developing for sooner.
[T-Mobile LG G2x] fcisco13:
+1 you mean sooner as in lost interest now...

NVIDIA NEVER AGAIN!
[T-Mobile LG G2x] d12unk13astard:
G2X is king.

We will have ICS soon enough once cyanogenmod team releases it. What our devs did on here was kind as we got to play around on the beta rom, nobody should be complaining about any issues on it.

Bottom line= our phones our the best gaming mobiles til tegra 3 comes out( or intel releases their mobile with AMD gpu?) Ya some phones can play some of our exclusive THD games but not all of them, i bet not as smoothly either.
[T-Mobile LG G2x] mackster248:
Oh man... So much fail in this statement.
[T-Mobile LG G2x] besthijacker:
I will never get another nvidia powered mobile device. Even if you pay me. Such a crap.

I dont care for CM much.

What I want I real support from both LG and Nvidia. Not half assed responses and their blame game.

While, I admint its good gaming platform, it underperforms with playing of 1080p videos.
[T-Mobile LG G2x] tombaker1:
1. A bunch of yahoos seem to want to be the first to get ICS to boot on any device....this exercises is a complete waste. Your panties are all bunched up just to keep up with the Jones' down the street. Congrats

2. ICS is coming to the G2x in the first batch of devices that did not ship with it. Yes that is not fast enough for babies. Waaaa Waaaa They could just buy a device that ships with ICS. ".....crying out loud." awwwww

3. According to your logic, ever device and supplier to every device, sucks because a custom ROM was generated. Those superhero DEVs had to swoop in and fix everything.

4. The device that was perfect on its day of release that did not need a custom ROM is? What is the name of this unicorn of unicorn devices?

5. I don't believer there is a better game phone on the market right now....maybe some peers but nothing better of any significance. Are you upset about the ability of the G2X to run any games?

6. G2x not going through the menus fast enough for you? Really?

7. Just another complaint for the sake of benchmarking, or saying you have ICS first? Or is their something your phone is not doing for you? Something broken, that all these ROM available dispite LG and Nvidia is still having you want?

9. Any apps that you can not run?

I know you want to toss a tantrum on not having ICS, or scared that daddy won't provide you with ICS fast enough, but why so slow with your protest? Why not just get the heck out of here? Terminate your contract with T-Mobile, sell your phone for 200 easy, pay the 200 terminate fee, and go out and buy the phone that fills that empy void. Its does not cost you a thing to leave.

Or perhaps buying ICS is beyond your ability to not complain? Boycott this or that. Whinnnnnnnnne. The world of angst solved by just buying the phone you wanted for Christmas....how hard is this?

Yes ICS is not on the G2x when it released 8 months ago....it is so very terrible for this to be done to you. Nvidia and LG so very terrible to you. You poor little thing you.
[T-Mobile LG G2x] AbsoluteDesignz:
Are you aware how off topic your reply was? His complaint is with the fact that nvidia and/or LG make it unnecessarily difficult to make custom roms for the g2x due to a lack of embracing the Android community and open source.

It's why cm7 and MIUI run smoother on the vastly underpowered nexus one than the G2x.

An ics port would be done by now if not for the devs having to work around these restrictions.
[T-Mobile LG G2x] Core Memory:
I am not bothering to argue why boycotting NVIDIA is necessary, there's plenty of reasons to. The G2X is the last anything with NVIDIA I buy and probably LG too.
[T-Mobile LG G2x] jacob808:
Again video game software has always influenced computer manufacturers to develop more powerful hardware, and modern day cell phones are minature PCs.

Most of the people advocating boycotting Nvidia don't realize this trend that smartphones will advance in power because of gaming apps.

I have always been a gamer and creating virtual worlds has been about how much polygons a CPU/GPU can generate per second. This is crucial.

Nvidia has been in this game a very long time on the PC side and their years of R&D has minaturized a power efficient dual and now quadcore chip that can run for hours on battery.

The Nvidia Tegra has put the Android platform on the map and made it here to stay. They will be the leader as far as the future of cellphones are concerned.

I'm sure I'm not the only one who actually found it the right time to upgrade to a smartphone such as the G2x because it was marketed as a TRUE GAMING superphone that could generate xbox like graphics.

I'll bet alot of hardcore PC gamers who are tired of lugging around their laptop will invest in Nvidia Tegra cellphones. Nvidia knows this and will capture this huge market.
[T-Mobile LG G2x] GenghisKhan67:
I'm calling bull**** on this. I was a Nexus One user and my G2X feels much faster running custom ROMs, including CM7 or MIUI.
[T-Mobile LG G2x] monkor:
Well you might feel dumb after this one...