yashsingh1/xdaforums huggingface dataset:
[Android Software/Hacking General [Developers Only]] FFU5y:
That's ok. I've actually tracked down the reason for the slow-down in the camera: I fixed a bug which had been around since the original PDroid, but the 'correct' behaviour was to open and close the database which stores the privacy settings a lot (which is very slow). I'm working what the behaviour should be, as well as adding caching which will reduce database load anyway.

In brief, you can expect an improvement in camera performance in the next release, which I imagine will be fairly soon.

I agree that it would be better to have more developers on board, because having one mission critical developer is risky. (With OpenPDroid, mateor and wbedard are both involved so if I were to spontaneously combust it wouldn't be so bad). Also, as you suggest there is great value in being able to bounce ideas off each other. Where I program as a day job, I am currently the sole developer and I really do miss that 'collaboration' - luckily here I've got mateor and wbedard.

Whether PDroid 2.0 and OpenPDroid will continue separate long-term I don't know. From the PDroid 2.0 thread, it looks like CollegeDev may have been (or be?) planning to adopt a more collaborative development approach, and if that happens then the need for both projects may disappear a bit. I think there will probably remain two forks, just because I think CollegeDev and we have different ideas about what should be included in a privacy control system.

However, if we are both drawing on bugfixes from each other's codebase, and the code remains largely similar we gain a lot of the advantages of having a single line of development anyway (in my opinion). Time will tell, I suppose, but hopefully the outcome will be great PDroid implementations for users =)
[Android Software/Hacking General [Developers Only]] nsmart:
I did some more testing. Phone becomes slow when I start "GPS Status" and compass rotation is very slow. alogcat is attached. I am using cm-10.1-20130111-NIGHTLY-mako.zip.
[Android Software/Hacking General [Developers Only]] asicman:
Current pdroid2.0 user here. There is an issue with PD2.0 blocking all requestions to get location via cell tower:

http://xdaforums.com/showpost.php?p=36312056&postcount=523

Has this been fixed in OpenPDroid?
[Android Software/Hacking General [Developers Only]] g.armour:
I've been running this for the past day or so on AOKP Mr1 with Toro and I'm impressed. It seems as though everything is working flawlessly. I think there was a bug though when I first installed, I installed ROM, flashed gapps, flashed patch, booted ROM, and my MMS app was MIA. So, I wiped, reflashed ROM and GAPPs, rebooted, set up phone, went back to recovery and applied patch and everything was there. Odd, but anyway, thanks for this!

I'm curious, is there a way that we can make an app think we are on WiFi when we are really on a network connection with this?
[Android Software/Hacking General [Developers Only]] tonyp:
This does look great indeed! Thanks FFU5y, mateorod & Wbedard!

I really love the github approach.

As the topic of including PDroid Manager came up - another way to include it would be to add the source to the manifest (or local manifest) and change the makefile to build & include it as well.

I will wait a little while longer until it stabilizes, but then I'm going to include it in my releases as well.
[Android Software/Hacking General [Developers Only]] TamCore:
local_manifest.xml

<project name="wsot/pdroid-manager" path="packages/apps/PDroidManager" revision="master" />

vendor/cm/config/common.mk

PRODUCT_PACKAGES += \
PDroidManager

Already included in OpenSensation2. But it fails to compile currently (fix is submitted via github)
[Android Software/Hacking General [Developers Only]] FFU5y:
No, it hasn't.

I've lodged it as a bug against OpenPDroid here so it doesn't get lost.

Thanks for the report.

Thanks for that notification. I believe that this problem is related to the 'lagging camera' problem, and so will be resolved by the same fix.

I've lodged an issue at Github here so you can track it.
[Android Software/Hacking General [Developers Only]] M66B:
You won't have market updates this way, because the signatures will be different.
[Android Software/Hacking General [Developers Only]] TamCore:
I know, but that shouldnt be a problem on nightly builds :)
[Android Software/Hacking General [Developers Only]] wbedard:
Hey FFU5y,

I just tested this out on my Nexus7 while recording an extended logcat. After reviewing same, I wonder if this issue is related to the GPS and camera issues by all going back to the database hammering you described. When I turned on notifications for Play Store and Play Services, just starting up the Play Store and bringing up my installed app caused the toast for accessing Account List to basically stay on continuously. When I went into the logcat, sure enough, there were huge runs of the following:

V/PDroidAlternative(32624): NotificationHandler: Notification for: com.android.vending:accountsList:0

Even when it wasn't logging 4-5 events in a row, that line was present in at least every 3-5 lines throughout the relevant sections of the logcat (i.e. when I was using the store). If this is true, a caching solution would probably help here, as well. Oh...and yes, the Market was very sluggish while I was conducting this test.

R/

wbedard
[Android Software/Hacking General [Developers Only]] FFU5y:
Hi,

Yeah, every line on which you see a notification there in your log, the database is being opened, and closed. I ran some performance testing, and discovered that while using the camera OpenPDroid core was eating ~30% of the CPU time, and (literally) 99% of this was opening the database repeatedly.

I've added caching code to the openpdroid-devel branches of all 4.2.1-based roms in github (http://github.com/wsot/platform_frameworks_base) and am doing test builds now. I've already done some very basic testing of it, and the camera went from unusable to working just fine, and I think the impact will be the same on Google Play and GPS/compass.
[Android Software/Hacking General [Developers Only]] wian:
Did you guys see the paper presented at the ACSAC 2012 conference on android permissions? It doesn't mention pdroid, but it does show that pdroid is a real necessity!

Unfortunately I'm not allowed to post links, but if you google for "ACSAC 2012 Permission Evolution in the Android Ecosystem" you'll find it.
[Android Software/Hacking General [Developers Only]] L30nh4rt:
Hi, i have the same issues as you guys using openpdroid on Paranaoid Android. I belive the both issues are releated together.

Hopefully it will be fixed soon.

After flashing the restore zip, both gps and location through cell network are back to normal
[Android Software/Hacking General [Developers Only]] FFU5y:
Some user feedback suggests Paranoid Android may be having a wider range of issues than other roms (because the source only just became available, there was less extensive testing on it), and I am presently investigating this.

Updates to very substantially improve the performance of the camera are in testing, and will almost certainly affect GPS/compass as well (and Google Play store).

(For those who are building themselves or experimenting, the updates are changes in 'http://github.com/wsot/platform_frameworks_base' - they are currently in the *-openpdroid-devel branches for all 4.2.1-based ROMs: cm-10.1, aokp-jb-mr1, pa-jellybean, and jb-mr1-release; although PA may have other bugs).
[Android Software/Hacking General [Developers Only]] M66B:
I would like to test for CM10 ...
[Android Software/Hacking General [Developers Only]] FFU5y:
Cool. Just cherry-picked the relevant patches onto the CM10 branch (i.e. cm-jellybean-openpdroid-devel). That is the cutting edge: for CM10 'stable', there is the cm-jellybean-stable-openpdroid-devel which matches the jellybean-stable branch in the CM repositories.

You can pull from http://github.com/wsot/platform_frameworks_base to get the changes.

Also, you can download my experimental 'OpenPDroid debug setting' app, which basically just allows you to switch on and off a couple of settings inside the current OpenPDroid development version: http://digitalfeed.net/android/OpenPDroid_Debug.apk

Source for that is at https://github.com/wsot/openpdroid_debug_control

If you want to build the openpdroid_debug_control, you either need to remove the android/privacy files from it, or make sure they are up to date from the frameworks/base/privacy/... used in your framework. I've just included them for convenience so that I can build without having to always update my frameworks.jar file on Eclipse.

If the pre-built version is crashing, you'll probably need to build it yourself. I just slapped it up last night, so it isn't exactly robust (the debug controller tool, that is).

Let me know how it goes.
[Android Software/Hacking General [Developers Only]] M66B:
Thank you very much for this prompt action!

How can I generate a patch set from cm-jellybean-openpdroid-devel?

Maybe it is an idea to document this in the start post.
[Android Software/Hacking General [Developers Only]] FFU5y:
Good question: if you pull the cm-jellybean-openpdroid-devel branch into an existing repository you have, you can then create a patch file using the command:

git diff <SHA of earlier commit> <SHA of later commit> but that does require you to pull into your repository, or clone your repository and then pull into that.

I've basically done exactly that to create the attached patch. In practice, the best way to get out development-stage code will be by having a repository you can pull OpenPDroid code into. If you already have a clone of the CM10.1 android_frameworks_base repository, you can make a clone off that and then checkout the 'cm-10.1-openpdroid' and '...-openpdroid-devel' branches from the OpenPDroid platform_frameworks_base.

Hope that helps.

Edit: (I wish there were a good way to do it in github, but there isn't).

Edit2: that patch is to go from cm-10.1-openpdroid to cm-10.1-openpdroid-devel - that is, you need to use this patch on code you've already patched with the previous OpenPDroid patches.
[Android Software/Hacking General [Developers Only]] M66B:
I did it this way:

git clone http://github.com/wsot/platform_frameworks_base
cd platform_frameworks_base
git checkout cm-jellybean-openpdroid-devel
diff -rupN -x \.git ~/android/system/frameworks/base . >openpdroid.patch
[Android Software/Hacking General [Developers Only]] FFU5y:
That'll do it too =)

The only downside of doing it that way is that if you have newer updates in your ~/android/system/frameworks/base (because there is a delay between CM releasing an update and me merging it into the OpenPDroid repos) then you can end up with that showing up in patches.

Rather than using the SHA of the commit, you can also use branches, e.g. in the platform_frameworks_base, you can do:

git diff cm-10.1-openpdroid cm-10.1-openpdroid-devel > openpdroid-devel.patch

or

git diff cm-10.1 cm-10.1-openpdroid-devel > openpdroid-devel.patch

(to get the full patch, rather than one which requires the openpdroid patch to already be included).
[Android Software/Hacking General [Developers Only]] M66B:
I have compared the previous patch set with the development patch set as obtained as described above and I am missing:

build/core/pathmap.mk
build/target/product/core.mk
build/target/product/security/privacy.x509.pem
libcore/luni/src/main/java/java/lang/PrivacyInputStream.java

and I have extra

telephony/java/com/android/internal/telephony/SonyQualcommRIL7x27a.java

Any idea why? Was this changed?

---------- Post added at 02:23 PM ---------- Previous post was at 02:16 PM ----------

I am going to use:

git diff remotes/origin/cm-jellybean-openpdroid remotes/origin/cm-jellybean-openpdroid-devel

(syntax is a little different than given before)
[Android Software/Hacking General [Developers Only]] FFU5y:
Hi,

Those first 3 files are provided by JB_build.patch

The fourth is provided by JB_libcore.patch

I'm not sure what's going on with the SonyQualcommRIL7x27a.java file...

The 'platform_frameworks_base' is only one of a number of repositories.

Basically, the relationship between the patches attached to the OP and the repositories are:

JB_framework.patch combines changes to platform_frameworks_base and platform_frameworks_opt_telephony

JB_build.patch is the changes based on platform_build

JB_libcore.patch is the changes based on platform_libcore

JB_mms.patch is the changes based on platform_packages_apps_mms

Each of these repositories has *-openpdroid-devel branches, but right now they only actually contain changes for platform_frameworks_base, no others.

With the git diff commant you listed earlier, you'll need to apply all the CM10 patches, and then apply the patch generated by that git diff.

It should then work.

Sorry this is fiddly: we're still working out the best way to distribute patches, provide access to the development code, etc.
[Android Software/Hacking General [Developers Only]] M66B:
Thanks for the explanation. I already deduced the same myself.

The extra file is probably because your are building for another device.

IMHO git diff between branches is a good way. The kernel patch sets work the same way, for each point release there is a patch set. So you could keep the base patch set as available in the start post now and publish upgrade patch sets for each new release. You will have to make a branch for each new release (make the development branch the new release branch and create a new development branch). In this way there will be a clear history in git too.

I am building right now and will let you know how things go later.
[Android Software/Hacking General [Developers Only]] M66B:
The build was okay and everything seems to work, including the provided OpenPDroid_Debug.apk. The slowness of the compass is gone. I didn't notice other slowness before (not for the Play Store and not for the Camera). Maybe this is version dependent. The Xperia 2011 devices use the legacy (ICS) camera, which could make the difference.

What is the idea behind not closing the database?

And why did you make an extra notification switch?

Maybe it is a good idea to include some or all of these settings in the next PDroid Manager.
[Android Software/Hacking General [Developers Only]] dirtyreturn:
@FFU5y or anyone with an answer. GSM gal.nex. /cyanogen 4.1.2/nov. Release I think. I have the rom patched with good old pdroid and a market search brought up openpdroid. If i recall correctly that if the search did not bring up openpdroid that there is a compatibility issue. Does openpdroid require an all new rom patch? Or, does it need the rom to have been patched with pdroid2.0? ... I think that is an OK question to ask. I guess, if not, flame away.
[Android Software/Hacking General [Developers Only]] FFU5y:
The idea behind not closing the database is that in previous code (i.e. PDroid/PDroid 2.0) there was a bug which would have prevented the database from closing when Syvat (the original developer) had intended it to. The effect of that was actually to make PDroid a lot faster, since opening the database is a processor-heavy operation. I fixed the bug, and so the database started opening and closing a lot which slowed things down. Basically those switches are just for testing, so I could work out where the performance impact was coming from, and how much difference caching makes. The additional notification switch was to stop notifications being sent at all so that I could check if the notifications were causing the slow-down.

Right now, I don't think these will make it into PDroid Manager, for a couple of reasons. First, there is no persistence mechanism for those settings - when you reboot, they are reset. Second, there will be a particular configuration which provides the best performance (probably cache activated, database closing deactivated).

I'm not sure why some people experience lag in some places and others don't, but to be honest the lag in this original release was probably mostly attributable to the same underlying problem, regardless of where the lag was appearing.

I'm glad the slowness of the compass has gone: that means the new caching mechanism is working =)
[Android Software/Hacking General [Developers Only]] FFU5y:
OpenPDroid is a whole new patch: you can't apply it to a ROM already using Pdroid or Pdroid 2.0.

With GSM galaxy nexus and CM 4.1.2 (cm10), you should be able to patch with OpenPDroid using auto_patcher.

You may want to wait until the next release (which I'll be discussing with Mateor soon, to get integrated into auto_patcher) because we're fixing a slow-down bug affecting camera, compass/GPS, and for some people possibly Google Play.

I hope that helps.
[Android Software/Hacking General [Developers Only]] FFU5y:
I know it isn't great form to be posting right after another post by me, but:

Just wanted to let everyone know that I've updated the OP to include more info about the various repositories and branches, as well as linking to the key posts in this thread so far. If there are specific things people feel should be included, please let me know. I hope to move some of this to the wiki at some point so it can be community-edited, and have a proper table of contents.
[Android Software/Hacking General [Developers Only]] wkwkwk:
Hi,

It looks like the Llama bug in PD2.0 is present in OpenPDroid.

With the original PDroid LLama was working fine but now it is not detecting a CellID at all, not even immediately after starting. No rights have been revoked, it has full access.

CollegeDev was saying on his thread that he fixed the problem but never made it public (so far).

Thanks.

Edit: Looks like issue #1 has already been opened for this.
[Android Software/Hacking General [Developers Only]] pulser_g2:
This is really good to see. Unfortunately though, it feels awfully "diverged" - I wanted to apply this patch onto CM10.1, but the path from code submission to retrieval seems a bit convoluted.

The reason is that there's about 5 different places to get the code from - the attached patches, your git, the other guy's git. And within those github profiles, there's often one containing patches, and one containing the full trees.

If someone wishes to submit code, where do they submit it to? The patches? The main trees? etc... You get the idea :)

BTW, the attached CM10.1 patch seems to be missing the MMS changes, as the file is empty, but the AOSP 4.2.1 patch applies cleanly) ,
[Android Software/Hacking General [Developers Only]] FFU5y:
Yeah, this is a bit of a tangle at the moment because we're still working out what the 'best' structure is.

The goal is (roughly):

Provide patch files of the latest stable release for each of the major ROMs, in Mateor's github repository and linked from the first post (rather than attached to the first post). That means users can patch relatively easily without having to mess about with repositories.

Provide repositories for the complete codebase (e.g. platform_frameworks_base, etc) so users can see the code in greater context, fork it, and then submit pull requests for it. This is where all the development should happen, and with the exception of patches attached to specific issues this is where input into the code should be made.

Provide a single repository against which all bugs can be lodged: this is the openpdroid_support repository. This doesn't hold any code; it simply provides a place to lodge issues for OpenPDroid (because there are so many different repositories, and lodging issues against individual ones would rapidly become a great big mess). A patch to the source code to fix an issue can be submitted/attached in the issue and then one of us will merge it into the code itself.

So I think each of these has its place, and there isn't necessarily a one-size-fits-all. Basically, the patch repository is for convenience, but the individual repositories (e.g. platform_frameworks_base, platform_build, etc) will always be the go-to place the detail, for making changes, and for bleeding-edge code.

Edit: I should note that I'm mulling over the idea of actually creating a 'platform_manifest' repository, and having modified version of the manifests for CM, AOKP, AOSP which include OpenPDroid. I'm not sure yet whether it is a good idea, or a terrible idea.

Edit 2: I should also note that I'm open to suggestions on managing this: the issue of 'many branches' is a difficult one, especially if people start adding code to just one devel branch, and it needs cherry-picking into all the others.

Yes, it is missing the MMS changes. I have re-created and reuploaded the patch for cm-10.1.
[Android Software/Hacking General [Developers Only]] mateorod:
Yes, this is something I have been thinking about as well. There is a lot to consider about the best practices here, as pulsar_g2 says and including your reply FFU5y. I have actually made the patches as of a couple days ago, but have yet to fully compare them (i.e. do we actually need different patch trees for things like PA and AOSP and CM. For 4.1.2, there were only two patches needed, AOSP and other. The only difference was the framework split, which isn't needed for 4.2.1. You get the idea.).

The patches will probably end up being where many people interact with the project, if only because it is a non-destructive way to integrate OpenPDroid into your rom project as well as an easy place to examine only OpenPDroid-germane code. I also fully support the idea of a continuing devel branch, with a periodic release schedule, at least for these early days. Otherwise we will have too many people running too many different versions of the project and debugging will be impossible. My goal would be to keep the non-devel branch, the patches and the auto-patcher all in congruence, so that we could mostly know where people were.

As the project matures and the distribution channels improve (read: Auto-Patcher. Everything else is likely in it's final form, distribution-wise) we can start instituting some of these ideas that get the latest main-branch pushes to end-users immediately.

I will put some time into the patches today. Maybe an OpenPDroid github site would be a decent ultimate landing place. Let me know if any of these ideas do not mesh with yours FF...I am open to discussion on any of it.

As for the manifest, perhaps a local_manifest.xml would be an easy place to start. It has been deprecated, I believe, so perhaps not a final solution, but one that could be explored until we have a more fit answer. I think people expect a little less from an overlay than they do from a full rom, and maintaining an entire manifest can be a bit of work, especially if you intend to host for each rom type.
[Android Software/Hacking General [Developers Only]] tonyp:
I think it really would be the best idea if someone (FFU5y) creates a github "organization" called OpenPDroid instead of having the source files and patches lying around at different Github profiles.

This might even help with the problem you described at having issues at different repos.

Organizations got an own issues Dashboard which aggregates all issues within this Organisation from all branches.

I'm not sure if this will be better than having an own issues repo - but it's worth an idea.

Once you crate the Organization it should be available here: https://github.com/organizations/OpenPDroid/dashboard/issues

It might get messy with lots of branches indeed - but I think it's the best solution.

A big problem might be that people send in pull requests to the most different branches (as they work with different codebases).

My idea would be a main development branch. Instead of accepting the pull requests in the different branches you merge/cherry-pick them to the main development one.

Afterwards they can get merged to the different release branches again (including the one where it got initially sent).

This might reduce the complexity of managing different branches for different ROMs and makes it easier to follow the new development, as that's only happening at the main dev / wip branch.

If everything gets documented well at a new Github Wiki (or at least within the README.mkdn files) then it should be managable.