yashsingh1/xdaforums huggingface dataset:
[Android Apps and Games] cyansmoker:
Thanks a lot, guys :)

yuripave:

Seriously? It rebuilds the lib directory for all the apps you moved? I am...speechless.
[Android Apps and Games] yuripave:
Yes, even I do not move anything using your app, on boot, logcat shows me:

INFO/PackageManager(271): Unpacking native libraries for /data/app/[app].apk

It does this too ALL my apps.

I just didn't know if this happen until I upgraded to cm7, or if it was like that at the beginning....

Do you think it's related to I am using a2sd on it? All my apps are on /sd-ext and the ext2 partition are mounted on boot by Darktremor's script.

I am using CM7 Nightly #46, Darktremor a2sd 2.7.5.2-1.
[Android Apps and Games] cyansmoker:
Aww, crap.

It would seem that Google has introduced this new behaviour sometime after Froyo. CM7 is based on Gingerbread, right?

I just ran a 'adb logcat -v' while booting my Honeycomb-running Nook Color, and there they were, these hideous messages from PackageManager :(

Note that people do not seem very pleased with it; Froyo introduced longer boot times due to "Move to SD Card" and now this:

http://xdaforums.com/showpost.php?p=11361127&postcount=1085

http://forum.cyanogenmod.com/topic/16056-cm7-rc1-restart-takes-10minutes-dalvik-cache/
[Android Apps and Games] yuripave:
Digging into the framework source, indeed it is introduced in Gingerbread.

On /framework/services/java/com/android/server/PackageManagerService.java

if (isSystemApp(pkg) && !isUpdatedSystemApp(pkg)) {
/*
* Upgrading from a previous version of the OS sometimes
* leaves native libraries in the /data/data/<app>/lib
* directory for system apps even when they shouldn't be.
* Recent changes in the JNI library search path
* necessitates we remove those to match previous behavior.
*/
if (NativeLibraryHelper.removeNativeBinariesFromDirLI(nativeLibraryDir)) {
Log.i(TAG, "removed obsolete native libraries for system package " + path);
}
} else if (nativeLibraryDir.getParent().equals(dataPathString)) {
/*
* If this is an internal application or our
* nativeLibraryPath points to our data directory, unpack
* the libraries. The native library path pointing to the
* data directory for an application in an ASEC container
* can happen for older apps that existed before an OTA to
* Gingerbread.
*/
Slog.i(TAG, "Unpacking native libraries for " + path);
mInstaller.unlinkNativeLibraryDirectory(dataPathString);
NativeLibraryHelper.copyNativeBinariesLI(scanFile, nativeLibraryDir);
} else {
Slog.i(TAG, "Linking native library dir for " + path);
mInstaller.linkNativeLibraryDirectory(dataPathString,
pkg.applicationInfo.nativeLibraryDir);
}

The PackageManagerService.java on Froyo doesn't extract the libraries.

Meaning your current app won't work on 2.3+:(
[Android Apps and Games] usn.mustanger:
Well, perhaps not for moving apps/app data, but I presume it's still good for informational purposes, correct?
[Android Apps and Games] yuripave:
Correct. It can let us know which app is taking up the space on /data/data.

cyansmoker did a great job, it's just android just keep changing its behavior and this change makes my booting process significantly slower than on froyo!
[Android Apps and Games] usn.mustanger:
Well, I haven't tried my luck at moving any apps/data, and given what y'all are saying above, I probably won't, but I can happily report that NES is working as it should (information-wise) on my DInc, which is useful enough for me.

Great work, cyansmoker, thanks!
[Android Apps and Games] yuripave:
Oh there is a good news!

Today I found out that Gingerbread actually improved the native app to sd, it actually move the lib files to the asec container which resides on sd card!

In froyo, app to sd just move the apk itself. In Gingerbread, it moves the apk, dalvik-cahce as well as libs files of the app to the sd card, BIG difference!

Thanks cyansmoker for you hard work! Great to have information which app is taking up space and move to sd!
[Android Apps and Games] usn.mustanger:
If I understand what you're saying, this truly is fantastic news. So does this mean that, by moving apps to SD using the built in "Move to SD" function of GB, we're no longer constrained to the small 150MB of data/data space for app data? So I can now, theoretically, install as many apps as I want so long as I have space on my SD?

Because that 150MB limitation seemed like the biggest software design blunder on Android phones.

Thanks for the update!

And thanks again to cyansmoker for his continuing efforts on this great app--it will always have a place on my phone!
[Android Apps and Games] cyansmoker:
V1.0.0

"maybe?"

This version now allows you to move a full application, not just its subdirectories. I believe that this should present an acceptable workaround to Gingerbread re-creating the lib files, because they will be re-created in the new location.

Also, an online FAQ is under construction.

***

Very very interesting news regarding Gingerbread. I guess this means that the new containers that Google creates for relocated apps will have to be non-VFAT; rather something like EXT4. This is indeed great news!

-C.
[Android Apps and Games] cmgrant:
I was just pointed to this thread from a question I posted. I have a DINC running CM 7.0-RC1. I was wondering if there was a way the NES partition on the emmc or even create an ext2/3/4 partition that /data/data could be mapped to

Thanks,

CG
[Android Apps and Games] cyansmoker:
Well, the NES partition works on CM7.

However, replacing /data/data/ with it or remapping the data partition would be a bad idea.

See why here: http://www.voilaweb.com/nes_faq#nespartdownsides

You may be able to make this work using a different ROM, though.
[Android Apps and Games] cmgrant:
Thanks for the info. I was thinking there was a way (maybe manually) to edit the boot scripts the mount this new emmc partition in place of the /data/data at the correct time but maybe not.

CG
[Android Apps and Games] JohnBPRice:
Some apps don't move to NES partition or cache

AWESOME app, finally a way to diagnose the low on space issue.

Works great on some apps (current v 1.0, but start back in 0.5.2). Aidiko e-reader is a fairly big hog and NES works fine for it. I've never been able to move FireFox (big hog of data in the "mozilla" and "lib" folder), is there some reason or trick to move it?
[Android Apps and Games] hgoldner:
Reporting some erratic behavior in 1.0

Chris, here's what I'm currently experiencing. (See signature for ROM info, etc).

Firefox for Android - \files are "blue" (indicating, I think that NES has modified), but only options are move to NES Partition and Move to Cache. I uninstalled and reinstalled and it came back "blue" on listing.

Internet (i.e. stock browser) - also blue (I had moved cache to NES cache), cache is "white." On long press --> restore I get following message: unable to retrieve info:[/system/xbin/busybox cat /app-cache.com.browser/cache/noensp.oid]

Wyse Pocket Cloud (I had moved library to NES partition) - also blue; lib is "white" as above, when I attempt restore, I get following message: unable to retrieve info:[/system/xbin/busybox cat /data/nespart/data/noensp/com.wyse.pocketcloudfull/lib/noensp.oid.]

Internet and Firefox are behaving, but Pocketcloud force closes. I need to get that back and I'm not sure whether to simply remove Firefox, PocketCloud and NES and start over or what.

BTW, even NES force closes periodically when trying to do the restores.

Your suggestions?
[Android Apps and Games] cmgrant:
.so files in the cache directory

scummvm has an .so file kicking around in its cache directory. I know NES has told me about .so files in a lib directory but it did not say anything in this case. When trying to move this directly to the emmc or sdcard it obviously doesn't work but it would be good if NES could detect instances of these and only allow it to be moved to cache or NES partition.

CG
[Android Apps and Games] cyansmoker:
Wow. I wonder what exactly they do with that .so file; I mean, it is supposed to be available to the application itself in its lib directory. Also, it is supposed to be executable. I wonder what it is exactly that they do with it.

Speaking of, that's what NES does: it looks for executables and that's how it disables moving to emmc/sdcard. I guess I could improve the app and have it look for the .so extensions while it's at it.
[Android Apps and Games] cyansmoker:
Well, this does not sound like a success story at all :(

I think I could fix the error messages ("...oid") by relying on busybox even less.

But a blue file showing "Move to ..." rather than "Restore" menu items is quite puzzling.

Edit: oops, I see it. Totally a last-minute bug. :/
[Android Apps and Games] cmgrant:
If it helps it seems the permissions for this file is 075. I know if it gets deleted it will be re-created.
[Android Apps and Games] cyansmoker:
I think I will soon push NES to the market.

But before that I will include the reboot capability in NES itself and add a "Fair EULA" (google it if you don't know what it is!)

BTW, NES has already proved helpful as an organ donor: http://xdaforums.com/showthread.php?t=977154 ("Alternate 3") :)
[Android Apps and Games] hgoldner:
Chris, I'll look forward to seeing it in the market.

Is it your recommendation to restore all changes made by NES prior to flashing a new ROM (and does it matter whether it's a flash-over without a wipe, or a flash with a wipe)?

Thanks!
[Android Apps and Games] overground:
+1 on the market release. I flash over without wiping cache (as that's where I move stuff to) all the time. I only wipe dalvik. So you'll be fine on a "no-wipe" same ROM upgrade. If you switch ROM's (i.e. going from a Froyo to a GB), you'll need a full wipe anyway. As far as the NES partition goes, I'm pretty sure a complete data wipe will kill it, but you'll be starting from scratch anyway. Wonder how Titanium will handle a restore in these cases? There is a "force restore to same location" option in TB. I'd restore NES first thing (recreating the NES partition, if that's what you use), then try restoring your apps with data using TB. Anyhow, restoring a nandroid+ext backup should get you back, if things don't go right.
[Android Apps and Games] cyansmoker:
Finally! I was starting to think that the app would never make it to the market. I'll bet I wasn't the only one ;)

I tried to set the price point as low as possible for you guys but Google won't let me go below $.99

I also had to enter the currency values manually for all the countries currently supported, learning in the process that Google won't allow an exact conversion either at it demands the price to be at least 6KRR, etc.

Anyway, I just updated this thread's original post:

Update 03/08/11

NotEnoughSpace has finally graduated to the Android Marketplace!

Beta testers can currently get it at its "thank you" price of $0.99.

I will set the regular price in a week so hurry :)
[Android Apps and Games] usn.mustanger:
Just bought it. THANKS! Much appreciated.
[Android Apps and Games] cyansmoker:
You're welcome!

Oh, I almost forgot: if you guys have the beta version on your phone, I highly recommended uninstalling first as you may not be able to install the Market version until you do so (certificates conflict)
[Android Apps and Games] hgoldner:
Congrats; downloaded it and looking forward to never seeing the 'low app space' warning again.
[Android Apps and Games] cantare:
Bought. Downloaded. Installed. Nirvana! Feels as if you were jabbing at a hex bolt with lineman's pliers, and someone stepped up and handed you a 12-point socket wrench. Nothing like having the right tool for the job...GINORMOUS thank-yous, CS!
[Android Apps and Games] Elemetrix:
Just spotted this on here. Saw the very early betas posted on Android forums and they seemed to work then started giving funny results.

Just bought this and it's showing Shazam to be using 2GB of space and Facebook Checkin to be using 480Mb.

Almost all of the other apps are shown as 4KB

I don't think so..

I noticed however that it didn't request root permission from superuser.

HTC Desire running MoDaCo R9
[Android Apps and Games] cyansmoker:
Yes it definitely should have requested permission. The Desire is one of the phones that should run well, actually, so that's disappointing.

What happens when you touch one of those '4kb' apps? Does it show any subdirectories?
[Android Apps and Games] hgoldner:
FYI, NES immediately requests root on initial run on an HTC Incredible.
[Android Apps and Games] fuzzynco:
NES Partition

I created the partition on /emmc. Will it automount on a reboot,

or do I need to run the app to do the mount?

Will Titanium Backup Pro follow the link to backup the data

in the new location?
[Android Apps and Games] cyansmoker:
Yes NES mounts the partition upon boot. Have a look at the FAQ, there is a more in depth explanation there.

Titanium Backup...well it should. I haven't tested that yet.
[Android Apps and Games] cyansmoker:
v1.0.2

* Cleaner 'info' menu for apps

* Apps stoplist to avoid maiming important apps

* Fixed: sorting after crawling
[Android Apps and Games] cyansmoker:
(Me again)

Of course, I will keep working on this app.

But in the meantime, I could totally use your input on this new app for which I got the idea while working on NES:

http://xdaforums.com/showthread.php?t=990274

Cheers!

-Chris.
[Android Apps and Games] Elemetrix:
Weird, well it didn't ask for permission so I guess it was never going to work.

The 4kb apps did show subdirectories but I can't remember their content.

To be honest I have followed this app from the first beta published online and never saw it working again after those initial early betas so when the full release didn't work after a few minutes mucking about it was quickly refunded.

With only 15 minutes to play with you don't have a lot of choice if it doesn't work immediately.
[Android Apps and Games] ned4spd8874:
I just started using the app "notenoughspace" and am immediately having problems with it. When I use the app to move a program to the NES partition, it disappears after a reboot. And then I have problems trying to reinstall the app. Eventually I'm able to remove it from the Android applications menu and then able to re-install.

Anyone else notice this while using this app?

I'm using it on an Inc. rooted with unRevoked/Virtuous. And have tried to create the NES partition on both the SD card and the internal emmc partition. Both cause this problem.
[Android Apps and Games] Doxola:
Nifty App

I have been using NES on my Dinc v1 (rooted w/ unRevoked/Virtuous) for about 2 months. Haven't had any major issues with it (nothing involving the problem as described above).

I use SkyRaider Sense 4.3, and I have a 16GB microSD card in the phone. Per the instructions, I have mainly only added the larger folders to the NES partition as opposed to whole apps. In some instances, there was only 1 folder, yet I still moved the folder (not the app) to the NES partition.
[Android Apps and Games] hgoldner:
I started using NES during its beta-testing phase and the only time I encountered any issues was during the early phases when Gingerbread was being leaked for the Incredible. I used it continuously after that until this week when Jermaine's exploit to move \data\data to an "unlimited" partition solved the problem altogether.

If you aren't willing to flash Jermaine's ext4 for all, NES is the only way to ensure you don't get "low memory" warnings due to less than 10% of the \data\data partition being free, as the problem persists in both Froyo and Gingerbread ROMs.
[Android Apps and Games] fuzzynco:
How to uninstall NES and move data back?

Since I cannot undo a NES move to NES partition.

I've tried repeatedly and restore just doesn't work.

If I uninstall the apps that were moved, then

I uninstall the NES app (so it doesn't mount its partition),

and reinstall the moved apps, will that put the files / directories

back in /data/data?

I'm thinking of using the Ext4 fix that puts /data/data into

/data to remove the 150mb limit. I want be sure when I reinstall

the apps they are returned to /data/data which will be in /data?

thanks
[Android Apps and Games] hgoldner:
Yes. That's preciselyl how I did things. (1) Uninstalled the "stuck" apps; (2) moved back everything I could; (3) deleted NES; also (4) destroyed NES partition(s). Double checked in \data\nespart to make sure nothing was there.

Then wiped cache, dalvik, and flashed ext4 (normal dalvik) in CWM 5.0.2.0.
[Android Apps and Games] usn.mustanger:
Would one of you be so kind as to point me in the right direction for what you're talking about here? Perhaps I'm not searching for the right thing...? If what fuzzynco is talking about actually removes the 150MB data/data limit, then there's really no reason to go through the NES gymnastics, and the inevitable problems that arise. (Why the f*ck is that limit even still in Android???)

TIA!
[Android Apps and Games] hgoldner:
Can't see what kind of phone you have while on xda premium on the phone, but it's in the Incredible development forum. Search for "ext4 for all" by Jermaine151
[Android Apps and Games] usn.mustanger:
Awesome, thank you. Found it. And I am on an Incredible.

Thanks again!
[Android Apps and Games] mmillmor:
I just get 4.0k for everything too I'm afraid. I'm running on a nexus one with CM6.1 RC1. I tried to get the debug log, but got a message bad status code #1003 when it was generating the log.

Martin
[Android Apps and Games] BrooklynAvi:
Hey,

I am the developer of CacheMate for Root Users. One of the planned next steps for CacheMate was exactly this. Since you have already done it, I will not bother. I probably couldn't do it better anyway. I noticed that your next step was to be able to actually remove cache and files. I have spent over a year analyzing hundreds of apps (maybe thousands) and how their data is stored on the /data/data.

I have pretty much reached a point where it has become very difficult to keep up with development of CacheMate. I have many projects I'm working on and have decided to make CacheMate open source, if I can get some people to help me make the code palatable and ready for public consumption. Would you be interesting in helping with this? Our apps deal attempt to solve the same problem which is the abysmal storage Android provides for app data. Please PM me if you are interested. This goes for anyone interested in helping.
[Android Apps and Games] Osleg:
Desire with CM6.1RC1, N1Tables, same issue with not showing correctly sizes and free space.

Debug stuck at "Please Wait".
[Android Apps and Games] brisssou:
lot's of NaN and error 1002

I get to see only the Total and Free, everything else is NaN'd and I get Error #1002

On a Desire running LeeDroid2.2f, s-on.
[Android Apps and Games] teorouge:
That would be nice, cross-development! ;)
[Android Apps and Games] cyansmoker:
Worth thinking about.

I hope it didn't kill this thread, though.
[Android Apps and Games] overground:
I get the same thing, although running Miui 11.26 with app, app-private, and dalvik moved to ext and mounted in /data. My numbers as reported by NES:

TOTAL: 196.2MB

USED: NaNKB

FREE: 194.3MB

ALL OTHER FIGURES: NaNKB

All data sizes listed @ 4.0kb

df -h reports:

Filesystem Size Used Available Use% Mounted on
tmpfs 203.3M 0 203.3M 0% /dev
tmpfs 203.3M 0 203.3M 0% /mnt/asec
/dev/block/mtdblock3 145.0M 133.6M 11.4M 92% /system
/dev/block/mtdblock5 196.3M 171.8M 24.5M 88% /data
/dev/block/mtdblock4 95.0M 7.1M 87.9M 7% /cache
/dev/block/mmcblk0p2 800.8M 379.1M 379.0M 50% /data/sd
/dev/block/vold/179:1
14.0G 10.0G 4.0G 72% /mnt/sdcard
/dev/block/vold/179:1
14.0G 10.0G 4.0G 72% /mnt/secure/asec

With many additional .AndroidSecure listings

As you can see, I could really benefit from this app. ;)
[Android Apps and Games] mrBira:
TOTAL: 147.6MB

USED: NaNKB

FREE: 146.1MB

ALL OTHER FIGURES: NaNKB

HTC Desire, LeeDroid 2.2f, S-OFF
[Android Apps and Games] cyansmoker:
Please, guys, post what the debug screen has to say here.

Alternatively, overgound, you seen comfortable with the command-line.

Can you post here the output of:

ls -l /data/data/com.android.vending/databases/ | head -1

Thanks!
[Android Apps and Games] cyansmoker:
You know what? I think that's OK.

It looks like these custom roms use busybox by default; simple as that.

I am adapting the code to support both cases. I'll let you know.
[Android Apps and Games] overground:
Debug gives me the error code # 1003

Swapping sdcards ATM, but if you haven't figured it out by the time I check back, I'll post my output for ya. Hope you get it going. Thanks

# ls -l /data/data/com.android.vending/databases/ | head -1
ls -l /data/data/com.android.vending/databases/ | head -1
unknown option -- 1BusyBox v1.16.2androidfull (2010-08-01 14:57:25 EDT) multi-ca
ll binary.

Usage: head [OPTIONS] [FILE]...

Print first 10 lines of each FILE (or stdin) to stdout.
With more than one FILE, precede each with a header giving the file name.

Options:
-n N Print first N lines instead of first 10
[Android Apps and Games] cyansmoker:
Update 12/04/10

The application should now display correct stats for phones/ROMs that use Busybox. If you have already tested it and it said "NaN" please try it again.

Unable to debug?

If you were getting "Error #xxxx" messages when accessing the debug screen, please try again: this should be fixed.

Thank you

...for your patience and your help debugging this app on your platform.

-Chris.
[Android Apps and Games] overground:
Everything seems to be reporting correctly now. I'll nandroid tomorrow and try moving some data. Where, by default, on /sdcard does it move the relocated data, and how is it linked, symlink? Thanks for your work so far.
[Android Apps and Games] cyansmoker:
Excellent!

When you move a directory, it goes to <volume>/data/noensp/

Note that I wrote <volume> because it can be /sdcard or /cache or the /ext partition if you have one.

The advantage of using /cache is that you are still writing to a file system that fully preserves all files permissions but of course it's much smaller than the other two.

Yes, it is linked using symlink. Good work there, Android, supporting symlinks between partitions!
[Android Apps and Games] dadurf:
Good Job

I just started this apk on a Droid1, and I see no problems with it. I have an accurate count of apps, how big they are, what space I've used, and what space I have left. I like it so far, it works! :D
[Android Apps and Games] overground:
Cool. My cache partition is essentially empty. I will try there first. I also have ext3, I will give that a shot too. The biggest hogs are apps I like to have, but are not too important (i.e. Google Earth, games, etc.). I think I will move some of these non-essential app's data, and this should help a ton. Thank you.
[Android Apps and Games] OOMatter:
Just started testing on a stock 2.2 rooted Dinc. So far it seems non-game apps work fine moved to SD, but games are force closing. I was wondering if you could add /emmc (the Dinc's internal storage) to possible locations to move data. Or even better, allow the user to add a custom location. Thanks.
[Android Apps and Games] overground:
Debug:

1:bb
2:/dev/block/mtdblock5 200960 168764 32196 84% /data
3:134934	/data/data
4:null
5:4096/15488/183463

Tried moving Google Earth's lib folder to cache. Got these errors:

Unable to Parse " as integer
Bad Status code: #2001.

Same errors when moved to sdcard.

EDIT: Also same errors whilst attempting to move stock browser cache.
[Android Apps and Games] cyansmoker:
Do you mean that with other applications, it just worked, but with these two it's failing, or are these the two you tested?

Cheers
[Android Apps and Games] cyansmoker:
Yes. As you may or may not have guessed, it is very likely that games fail because native binaries and libraries cannot be executed when on the SD Card.

Indeed, the ability to move them to /emmc would be welcome. I've just uploaded an updated version that does that.

Long term, I may allow for seemingly "arbitrary" destination partitions but I want to proceed very carefully at least until I know for sure what partitions exist on various phones and which ones are OK to use.