yashsingh1/xdaforums huggingface dataset:
[Android Apps and Games] shivadow:
Hi. Being a big fan of the original app, as soon as I saw this I'm on board. Perusing this thread I saw the link for the beta testing so I signed up in the hopes we can get an issue resolved that I've found with this app.

I have multiple apps installed that all have "draw over apps" permission. When inspecting the permissions of those apps via System2 that permission has a line through the middle of it. I cannot find any information about what the line means so I presume it means that the permission is for some reason revoked. The problem is that all those apps have that permission allowed.

This came about when conferring with the developer of the app "GrammarPal". When launching GrammarPal the floating icon to enable the app is not present on the screen. All permissions are enabled however System2 shows the "draw over apps" permission as what I presume is "revoked", with the line through it. All other apps using this permission have no issues but the permission still appears with a line through it in system2 for some these apps as well.

I cannot fathom if the problem with GrammarPal is with the app or the system itself, or both, as System2 is possibly giving me an unreliable result, or is it?. Maybe system2 is actually right and there's a problem with my device, which is an HTC M9 Wwe (region 401) rooted and stock firmware.

Since signing up for the beta testing yesterday I have not received any form of beta version or update to my app.

Can anyone help investigate please?.

Beamed in by telepathy.
[Android Apps and Games] tliebeck:
You have to grant the draw-over-apps permission specifically. On a stock Android, go to Settings, Apps, tap "Gear" icon in action bar. Tap "Special Access" (last option on mine), then "Draw Over Apps", and then enable the app you want.

A permission is strikethrough is one the app supports/has requested but that has not been granted.
[Android Apps and Games] shivadow:
Thanks for the info!. I enabled the permission within android settings/accessibility, clearly it says enabled. Going to system2 and checking the permission under grammarpal it has a strike through. So, this concludes that the app is requesting it and it is not granted. But why?. Do you know of any reason why the permission would not be granted if it shows as enabled in system settings?.

Beamed in by telepathy.
[Android Apps and Games] AlphaUMi:
[MENTION=2954544]tliebeck[/MENTION]: I just purchased this app because I find it amazing! I never heard ystemPanel v1 before, but it was just my ignorance, because now I learned it was one of the most famous app back in the days! My compliments for reviving such a useful all-in-one app for monitoring your own device and keep up your work!

Greetings and compliments,

Alpha.
[Android Apps and Games] tliebeck:
Another status update in lieu of a release...

I can't stand the way the plots in the release version of SystemPanel work. There's just too much data for the plotting engine to handle. The big problem here is that I've yet to find a plotting engine that's designed to deal with 1,680 data points (10 data points per hour, 168 hours) with several plots that have up to 6 stacked bars in them. Some screens have 20,000+ data points plotted. The problem is made worse than you've all seen by the power-use charts coming in the next beta.

My daily driver is still a Nexus 6-not-P and I'm just not happy with performance. It's okay on my newer test devices, but 2-year old stuff like the one I use everyday suffers.

So I've written a purpose-built plotting engine for SystemPanel rather than using a commodity library. It's faster and prettier, and it's been a colossal pain in the behind to get it right. It still needs a bit of work for feature parity to the old version, but it'll be there soon, with a beta shortly thereafter.

Next release is a big one, thanks for your patience and apologies for the delay.
[Android Apps and Games] tgp1994:
I tried posting in the G+community and didn't hear a peep, I should have known there'd be an XDA thread. :laugh:

I'm trying to figure out why my phone has sometimes been slow as molasses. If I open up CPU usage, more often than not "other processes" is taking up the majority of the pie chart. What is this "other processes"?
[Android Apps and Games] yochananmarqos:
What does this have to do with Adguard?
[Android Apps and Games] jcmm11:
Where did Adguard come into the picture?
[Android Apps and Games] yochananmarqos:
Whoops, wrong thread. :silly:
[Android Apps and Games] tliebeck:
Are you Android 7 or later? If so, SystemPanel won't show you what the active processes are unless you have root access.

You can plug your phone into your computer and do "adb shell top -m 10" if you don't have root.
[Android Apps and Games] tliebeck:
SystemPanel 2.0.b8 now available in the beta channel.

2.0b8/R2050 is a major overhaul of the SystemPanel 2 beta.

ENHANCED STATISTICS: SystemPanel is now recording system-estimated per-app and per-hardware-resource battery drain and wakelock usage data over time. This data is plotted alongside CPU usage, battery consumption, app/display activity, and Wi-Fi/Cell signal data.

Enhanced statistics works WITHOUT ROOT on Android 7 devices, but you must grant special permissions to monitor system stats using a computer with ADB. With root, enhanced statistics can be enabled directly on the device.

NEW PLOT ENGINE: The plotting library that was being used for previous versions of SystemPanel had been pushed beyond its limits. The new plot engine was written from scratch, and is designed to deal with very large datasets. The plots display more granular data when zoomed in. Pinch-to-zoom is easier because the gesture no longer needs to be done within the confines of a single plot. The plots are more space efficient on the screen. And the new plot engine reduces SystemPanel’s memory footprint on the device.

ADDITIONAL UPDATES: There have been “veritably superlatively gargantuan plethoric myriad” updates to this release. Changes include:

Major UI Updates.

Added “Battery” tab in history to show top battery consuming hardware resources and apps.

Options to sort top apps by battery and wake lock usage when viewing history plots.

Running processes can now be sorted alphabetically. Idle processes are now always sorted alphabetically rather than by PID.

Cellular reception is now recorded using a new mechanism. Phone and Location permissions are no longer required to record cellular reception. (They are still used, if desired, for the “Location” and “Network” tabs in the “Device” section.)

Many bugs removed.

One bug added.

The new "Power Use" plot showing device calculated battery usage of applications:

The "Battery" tab shows a breakdown of battery use of any given time range by application / hardware resource:

Wake lock data is now plotted on the "Activity" tab as well:

The "Live" process view has received major UI updates (and can now be sorted alphabetically using a menu option).

The beta channel is for those that want early access to new releases, you can opt-in here: https://play.google.com/apps/testing/nextapp.sp

The product itself is of course beta, but opting into the beta channel will get you new releases earlier.
[Android Apps and Games] Wapu:
You Rock!!! Can't wait to try this out when I get home from work. Thanx for this addition. :)

-----

Edit: Where is the documentation for the ADB commands?

---

Edit 2: Nevermind... Just got the update and the instructions are inside the app. Easy to follow & flawless implementation. Nice work!
[Android Apps and Games] peterbonge:
That (finally) seems to work, but the cell signal on the live page needs too much time (3-4s) to be displayed. Other apps are showing this value immediately.
[Android Apps and Games] tliebeck:
Thanks, and may add a Windows app in the future to do this.... technically inclined/XDA people will probably be able to follow the current instructions, but they're probably a bit much for every user.
[Android Apps and Games] tliebeck:
It's currently being queried via a PhoneStateListener, so it has to wait for a broadcast to receive it. Looks like last service state is available via "dumpsys telephony.registry" directly, but wonder if there might be a more universal way to get at it. There's probably something in /sys or /proc, or they're calling a hidden API via reflection. Developer documentation says you have to wait for it...but Android developer documentation is almost an oxymoron. :D
[Android Apps and Games] peterbonge:
I hope you find a solution. Sometimes it takes even more than 5 seconds. That's not OK.

And if you switch to another page and then back to the live page you have a gap in the small graph besides the cell value. The small WiFi graph is always full. Even after a restart of the app.

If I open the app "Network Cell Info" it immediately shows the cell signal:

https://play.google.com/store/apps/details?id=com.wilysis.cellinfo

Maybe you can contact the developer and ask him how he did it.

Apart from all this the new beta seems to work really fine. Good work!
[Android Apps and Games] tliebeck:
One more quick note...this was planned but didn't make it in time for beta8, will be in beta9 (not final yet, working on it now).
[Android Apps and Games] Dark3n:
[MENTION=2954544]tliebeck[/MENTION]

I'm interested in your solutions to detect running apps on Android O, facing the same issues (https://github.com/d4rken/sdmaid-public/issues/946#issuecomment-308485998).

What kind of accuracy are you getting from the usage stats manager? Last time I looked at the data it was not optimal to determine "currently" running apps, as the data is still often ~30s behind, even on the highest interval setting.

The permission `PACKAGE_USAGE_STATS` can be granted via system settings, similar to notification access, why do it via ADB?

Kudos for the idea to use `dumpsys` to detect running apps, have not thought about that.

I assume for long running stats, for your history, you are using the usage stats data and for the currently, "right now", running apps, you use `dumpsys activity processes` periodically?
[Android Apps and Games] tliebeck:
Currently SystemPanel is ONLY using dumpsys batterystats. Most of the usage stats data does appear to be very delayed. Seeing a proper live process listing on Android 7+ requires root access at present.

PACKAGE_USAGE_STATS is needed on Android 8 in order for dumpsys batterystats to work. I'm asking for it on Android 7 as well to make things future proof, even though it's not necessary. I'd rather make everyone do it from the beginning rather than dealing with devices upgrading from 7 to 8 and having things break.

The system settings activity doesn't appear to actually grant PACKAGE_USAGE_STATS. If you query the permission state with it only enabled in the system settings activity, it'll come back false. Allowing it via the system settings activity only allows UsageStatsManager to provide your app with its almost worthless data. (This is based on observation, I can't find exact documentation to verify).

I eventually hope to use dumpsys cpuinfo to get running process data for 7+ users, even if it's delayed a bit. Do you know if the interval setting on this is configurable? On Pixel C on Android 8, I get a 30 second window within 30 seconds of present time. On an S7 Edge, It's showing me 5 minute intervals within the last 5 minutes of present time. In the case of SystemPanel, I'd love to get both the 30 second window for pseudo-live display and the 5 minute interval for recording to history.

I've not looked into it tremendously, but dumpsys procstats might be a solution for you on 8. Running app data seems to appear immediately, but it stays after apps are closed. I haven't looked into it enough yet to see if it's immediately determinable if an app has closed.
[Android Apps and Games] Dark3n:
> Seeing a proper live process listing on Android 7+ requires root access at present.

Yeah, I'm using `getRunningServices()` as workaround without root on 7, as I'm not really interested in apps that are still running as part of a not yet killed activity (most apps shouldn't do that anyway). Though I'm probably going to use dumpsys if it's available (i.e. the user did the work).

> PACKAGE_USAGE_STATS is needed on Android 8 in order for dumpsys batterystats to work.

Interesting, didn't know that.

> I'm asking for it on Android 7 as well to make things future proof, even though it's not necessary. I'd rather make everyone do it from the beginning rather than dealing with devices upgrading from 7 to 8 and having things break.

Yeah... I'm not unfamiliar with API dependent behavior :D, I assume you have something like this too ;).

> The system settings activity doesn't appear to actually grant PACKAGE_USAGE_STATS. If you query the permission state with it only enabled in the system settings activity, it'll come back false. Allowing it via the system settings activity only allows UsageStatsManager to provide your app with its almost worthless data. (This is based on observation, I can't find exact documentation to verify).

That's weird.

AppOpsManager appOps = (AppOpsManager) getSystemService(Context.APP_OPS_SERVICE);
int mode = appOps.checkOpNoThrow(AppOpsManager.OPSTR_GET_USAGE_STATS, android.os.Process.myUid(), getPackageName());

returns ALLOWED, but trying it via dumpsys still returns

V/SDM:Shell:Static:Executor: dumpsys batterystats
V/SDM:Shell:Static:Executor: Output: Permission Denial: can't dump BatteryStatsService from from pid=18568, uid=10440 due to missing android.permission.PACKAGE_USAGE_STATS permission
V/SDM:Shell:Static:Executor: Exitcode: 0

Maybe this is a bug and should be reported?

> I eventually hope to use dumpsys cpuinfo to get running process data for 7+ users, even if it's delayed a bit. Do you know if the interval setting on this is configurable? On Pixel C on Android 8, I get a 30 second window within 30 seconds of present time. On an S7 Edge, It's showing me 5 minute intervals within the last 5 minutes of present time. In the case of SystemPanel, I'd love to get both the 30 second window for pseudo-live display and the 5 minute interval for recording to history.

Not that I know of.

> I've not looked into it tremendously, but dumpsys procstats might be a solution for you on 8. Running app data seems to appear immediately, but it stays after apps are closed. I haven't looked into it enough yet to see if it's immediately determinable if an app has closed.

procstats looks interesting `dumpsys activity processes` also looks good, though much more verbose, will have to compare if there is any discrepany between listed processes.
[Android Apps and Games] tliebeck:
I'm hesitant to report anything as a bug in this space. They might "fix" it. Google seems to be almost at war with system apps lately, and though my best guess is that they despise the various wildly-popular-near-billion-downloads "snake oil" boost/clean utilities, I'm still not certain what they're up to. They've already removed documented functionality in maintenance releases, with no mention in changelogs (e.g. ActivityManager in the Lollipop release that also was the first attempt to fix stagefright).

Their claim is security via side-channel attacks, but they refuse to provide an alternative method for developers to create user-friendly tools to investigate device issues. The fact that we have to ask users to download/use ADB in order to find and fix problems with their devices is insane. See https://issuetracker.google.com/issues/37140047 or https://issuetracker.google.com/issues/37091475 . They're opposed to providing a new permission because bad apps could ask for it too. The fact that bad apps can ask for access to your contacts and then email them all viruses isn't seen as a threat, but getting access to /proc/stat somehow is (even if it had to be enabled manually and had a giant warning like an accessibility service).

The kicker for me is that they still refuse to fix the "any app can launch on top at any time and impersonate another app with no notification to the user" security issue. It's even mentioned in some of the security articles they cite in 37091475. I reported it myself in 2016 as a security issue (and thus hidden from public view). They marked it as a duplicate of 37140047 a year later. I responded immediately that that was wildly incorrect, have received no response.
[Android Apps and Games] tliebeck:
SystemPanel 2.0.b9 now available in the beta channel.

Added "Wakelocks" tab to History section to show complete list of applications using wakelocks, including time and wakelock count. (Screenshot #1)

The "Battery", "CPU", and "Wakelocks" history sections now feature a plot showing the relevant statistic they are analyzing. (Screenshots #1, #2, and #3)

Updated directions for enabling enhanced statistics to show appropriate commands on Windows vs Unix (i.e, putting a ./ (dot-slash) in front of commands).

Various bugfixes for stuff reported via Play Store.

Slight aesthetic change to pie plots.

New History Wakelocks screen; updated History Battery screen:

.

Updated History CPU screen; updated Setup screen:

.

This release has been initially uploaded to the beta channel. It will be placed in production soon assuming further testing proceeds well.

The beta channel is for those that want early access to new releases, you can opt-in here: https://play.google.com/apps/testing/nextapp.sp

The product itself is of course beta, but opting into the beta channel will get you new releases earlier.
[Android Apps and Games] tliebeck:
Anyone seeing a crash in the new Battery/CPU/Wakelock tabs on beta9 (released above in beta channel only)?

I've received a couple crash reports on Google Play (5 total, affecting 2 users). They make no sense to me though, the reported point of the failure is nonsensically impossible. Could be an issue with my deobfuscation files (which I've now pulled). If you have seen a crash, please post!
[Android Apps and Games] tliebeck:
Minor update 2.0.b9.1 Released to Beta Channel, I'll move this to production in a few hours.

This just fixes a crash bug in 2.0.b9 when mashing on the history UI. I figured out the crash issue in prior post. Deobfuscated stack traces are so much fun. :D
[Android Apps and Games] FlyingHorseGA:
Are advanced statistics only available for Android 7 and above?
[Android Apps and Games] tliebeck:
For the moment, yes, but it may be possible to allow Android 6 users with root to use the feature as well. Will require changes to SystemPanel, looking into it.
[Android Apps and Games] DLC956:
Request for charts to go to 100% and mah chart to go higher than 800 mah cause mine are getting cut off. I'm running a Sony Z4 tablet.
[Android Apps and Games] peterbonge:
[MENTION=2954544]tliebeck[/MENTION]

1. Any progress with the cell signal on the live and device pages?

2. A graph with the battery temperature would be nice.

3. The power use (mA) is really nice, but I miss the other side of the graph. The charging power.

4. On the live page a small graph with the charging power would also be nice. And maybe a small info how much power the current cable can offer, just like this app:

https://play.google.com/store/apps/details?id=com.gombosdev.ampere

That can help to identify dangerous USB Type C cables.
[Android Apps and Games] tliebeck:
Is SystemPanel missing any data compared to that app? Looking at the Play Store screenshots that's already reported on Device->Power page.

The charge/discharge current data is poorly supported on Android phones. Data comes from BatteryManager, by reading the properties "BATTERY_PROPERTY_CURRENT_NOW" and "BATTERY_PROPERTY_CURRENT_AVERAGE". The data is rough. Samsung arbitrarily hands it back in milliamps instead of microamps. Some phones only provide the "now" data which changes so rapidly as to be useless, and is likely inaccurate because it's by definition being queried only when a significant moment of processor use is occurring.

Charge rate (based on battery percentage) is currently plotted in the battery plot, may adapt that data into the power plot during charging times. Power plot definitely needs to indicate periods of charge vs periods of no/low-discharge.

Thanks for suggestion on battery temp plot, do like that idea a lot.

Focus of development at immediate moment is on streamlining installation of enhanced statistics (e.g. building desktop app) and incorporating them throughout the app (e.g. analysis section, which needs much work).
[Android Apps and Games] peterbonge:
It seems that the screenshots and the description are really outdated.

Here is an article with a newer screenshot and more info:

http://ampereapp.blogspot.de/2015/11/how-to-test-usb-c-cables-with-ampere.html

And I forgot one more idea:

Wouldn't it be useful if the cell and WiFi graphs would show different connection types with different colors? (2G/3G/4G and 2,4/5GHz)
[Android Apps and Games] tliebeck:
Just looked up the max current/voltage data. It's being sent in an intent, though technically it's "hidden" according to the Android source. Drives me crazy how much they like to "hide" things. Another example is LTE signal. It's hidden. Why? Because. In any case, now showing this data when it's present, along with the "invalid charger" flag (next version.
[Android Apps and Games] MKairys:
At the risk of repeating myself ;)
[Android Apps and Games] tliebeck:
I've written a trivial Windows application to enable enhanced statistics. This app does not need to be installed, just unzip it, connect phone, run it, and throw it away when finished.

App can be downloaded here: http://sp.nextapp.com

Next version of SystemPanel will be updated is being updated (see last screenshot) to provide instructions for getting/using the app. But it works just fine with the current version, just run the app instead of invoking the ADB commands (it does it for you).

Greatly appreciate any testing/feedback on this!

The app is truly trivial, just a thin wrapper to execute the ADB commands, check output, and provide help as needed.
[Android Apps and Games] tliebeck:
This actually will be somewhat difficult to implement well. Right now stats are thrown after 10 days (though only last 7 days are displayed). Saving high resolution data for very long periods would be huge and wreck the database performance.

To this right I'd probably want to have some type of aggregator that takes logs overall statistics before they're deleted, and then implement a new system to compare aggregate stats over time. It'd be a neat feature, but would be a decent size subproject to implement.
[Android Apps and Games] sparksd:
Nice add - thanks! Used it on my S7 Edge and S2 Tab.
[Android Apps and Games] MKairys:
Understood, thanks.

Understood, thanks. For now screenshots will do ;)

How about saving snapshots of the data and pulling them back into the displays? It would be kind of like taking screenshots but much easier on the user ;)