yashsingh1/xdaforums huggingface dataset:
[Nook Color Android Development] inportb:
Thanks for the tip. If it's a matter of making a static build of busybox, we could simply tweak the build script for "anise" and pull it from Debian for "basil". It's a significant bit of baggage, so I'm hesitant to include it in the package. Even though only package-specific versions are installed in /data/data, the .apk itself would include some 1MB~2MB for ARM, x86, and MIPS, adding 2MB~4MB of unnecessary bulk. But smaller binaries could easily be bundled without much trouble; the init program is included as libinit.so (because the SDK doesn't like bundling executables), which gets copied to the bootstrap location as needed.

During bootstrap, we call either /system/bin/su or /system/xbin/su, depending on which is available (for some reason, PATH does not work with the embedded terminal emulator). Then we extract pkg.zip, copy libinit.so->init, and chmod it 04755 (when invoked by a non-superuser, the init binary only works within its parent directory, and drops privileges right after setup). busybox is never involved, unless we're performing a loop-mounted installation (in which case it's used to mount the disk image).

Would you please tell me what you see while you try to bootstrap? Does su return an error, or does pkg.zip not run, or is it something else? On my G1, I had a problem with insufficient storage. The code is at https://github.com/jyio/botbrew-gui

Regarding packages from repository-botbrew-native: the repackaging effort has just begun, and what you find in that repository are pretty much the original opk's renamed to debs. None of the services work because the runscript directories are different (though the binaries themselves should run without trouble). The idea is to repackage these things by hand and move them into the main repo when they work normally. BotBrew "basil" runs on dpkg so if you want, you could install stuff from Debian as a stopgap measure. Meanwhile, you should probably remove repository-botbrew-native (and I should probably add a stronger warning against using it...)
[Nook Color Android Development] AdamOutler:
This is from my Samsung Captivate.

Still working on it.. just wanted to give you this screenshot.
[Nook Color Android Development] inportb:
Oh, I know. Your mkdir (from toolbox, most likely) does not support the -p parameter. I guess it's not really needed in most cases, so I'll get rid of it. Even better, I'll code up that functionality in Java...

/edit: here you go... I didn't do a version bump yet, so you'd want to install it like:

adb install -r BotBrewBasil.apk
[Nook Color Android Development] AdamOutler:
that got through the installation. although there was a problem.. there was a ton of errors with lsmod, maybe insmod? no such file or directory.. I couldn't read them because they went so fast.

I tried to reinstall and I could not reproduce....

So i went to install botbrew-core and I got this:

# which botbrew
$ su
# botbrew
botbrew: not found
# which botbrew
# cd /data/botbrew-basil

# cd bin
# pwd
/data/botbrew-basil/bin
# ls
busybox
bash
dash
tar
sh
# ./bash
./bash: not found

I just realized this test was invalid because I had manually created and unzipped the files into that folder earlier.
[Nook Color Android Development] AdamOutler:
I deleted /data/botbrew-basil.. then I reinstalled

here's the errors I was seeing whizing across the screen:

Busybox should be good..

# busybox --help
BusyBox v1.19.3 (Debian 1:1.19.3-7) multi-call binary.
Copyright (C) 1998-2011 Erik Andersen, Rob Landley, Denys Vlasenko
and others. Licensed under GPLv2.
See source distribution for full notice.

Now it's downloaded and listed all the packages, however I'm getting this...

Maybe you could busybox everything...

# busybox ls
acct init sbin
cache init.goldfish.rc sdcard
config init.herring.rc sys
d init.rc system
data lib tmp
dbdata lpm.rc ueventd.goldfish.rc
default.prop mnt ueventd.herring.rc
dev proc ueventd.rc
efs recovery vendor
etc recovery.rc
fota.rc res
# busybox chmod
BusyBox v1.19.3 (Debian 1:1.19.3-7) multi-call binary.

Usage: chmod [-Rcvf] MODE[,MODE]... FILE...

Each MODE is one or more of the letters ugoa, one of the
symbols +-= and one or more of the letters rwxst

-R	Recurse
-c	List changed files
-v	List all files
-f	Hide errors

#
[Nook Color Android Development] inportb:
No, it's not a "real" error; the program is just reporting that it's creating a symlink to a nonexistent file (which is true; the target only exists while BotBrew is working), so there's no way to chmod it. The symlinks are created just fine and the targets end up having the correct modes; therefore, it exits cleanly and we progress to the next screen. Busybox would also have a problem with chmodding nonexistent files if you tried using it to unzip the payload ;)

Meanwhile, the problem with "error retrieving information" is a real bug. You could probably work around it by backing out of the app, relaunching it, and hitting the refresh button. But it annoys the heck out of everyone and needs to be fixed.

Thanks for the reports!

/edit: opened an issue for this
[Nook Color Android Development] AdamOutler:
Well, backing out allowed me to go back in...

terminal> botbrew no file found

I found a /system/bin/botbrew2 so I renamed it to botbrew, chmodd'd it and was able to launch it, but then I could not find dpkg, apt-get or any others. they are launching from within the program, however the program is not able to complete the installation of botbrew-core...

botbrew-core: depends: botbrew wrapper but it is not going to be installed...

botbrew-wrapper-- unable to make a backup link of `./system/bin/botbrew2` before installing new version: Operation not permitted.

Why is it "./system"? shouldn't it be /system/bin/botbrew.. not the "." static reference?

My terminals are not getting environmental variables for botbrew after launching the botbrew application, and botbrew wraper/core are not installing properly. I can't get a screenshot until tonight ,but I can type out any text you need at 120wpm :)
[Nook Color Android Development] inportb:
You could also have symlinked it, too. Actually, it should have symlinked it automatically if /system/bin/botbrew did not exist before. The contents of botbrew2 should be something like /data/botbrew-basil/init -- "$@" depending on where you bootstrapped.

When invoked from the command line, I think apt-get would refuse to install anything that does not target armel or all. The program passes lots of configuration parameters to apt-get to make it work. A possible improvement might involve writing all the configuration to a file for seamless command line package management.

Maybe it could not find /system/bin/botbrew2 ;)

I guess you could touch that path to fool the package manager.

Because the cwd is /, ./system points to Android's system directory at /system. All paths in packages (tarballs, really) are relative, and files are not directly extracted to / during installation (so it could, for example, rollback changes if needed).

Here are my environment variables... http://pastebin.com/YbHJqNLG

In particular, the ones that BotBrew sets for its subprocesses are

BOTBREW_PREFIX=/data/botbrew-basil
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games:/botbrew/bin:/usr/lib/busybox:/sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin

But no worries... I probably can't do much debugging until tonight, too.
[Nook Color Android Development] AdamOutler:
I'm not getting any environmental variables.. for sure.

I think I know what's going on here... why your app works sometimes and terminal works others...Linux shells have two places to load environmental variables from, one is for a login terminal and one is for a non-login terminal. the /etc/profile on my system is empty and I'm not loading any botbrew variables on my system.
[Nook Color Android Development] inportb:
Right, so I assumed that /etc/profile is bash-specific, since it usually contains references to /etc/bash.bashrc. When invoked as sh (without --login), even bash should not source these files. How do you spawn sh (and Superuser.apk's su) so it does not reset its environment?
[Nook Color Android Development] AdamOutler:
/etc/profile is global variables for all shells.

If youd like, i can set up adbwireless and open a port for my device later tonight you can use it with adb connect and see whats up.
[Nook Color Android Development] AdamOutler:
I sent you a PM with my google talk. If you're interested, i can set up my device in the DMZ on my router, run wirelessADB and a VNC server. Then you will be able to connect up with ADB to debug and VNC into my device and then run my device remotely like a virtual machine.
[Nook Color Android Development] AdamOutler:
AH HAH!!!! I think I found the problem with botbrew failing to install.. It went through the entire setup and worked.

I noticed that there were no available loops, so here is a way to universally create a new loop file in bash... it even tests to make sure the next "free" loop does not exist before creating it :)

# ls /dev/loop?
/dev/loop0
# FreeLoop=`losetup -f`;FreeLoop=`losetup -f`;test ! -e "$FreeLoop" && mknod "$FreeLoop" b 7 "${FreeLoop#${FreeLoop%?}}"
# ls /dev/loop?
/dev/loop0
/dev/loop1

however, it will not install packages still..

adam@adam-Desktop:~$ adb shell
$ su
# botbrew
# apt-get
apt-get: not found
# exit

and the environmental variables are not there

# botbrew
# printenv
ANDROID_ROOT=/system
LD_LIBRARY_PATH=/vendor/lib:/system/lib
PATH=/sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin
LOOP_MOUNTPOINT=/mnt/obb
ASEC_MOUNTPOINT=/mnt/asec
EXTERNAL_STORAGE2=/mnt/sdcard/external_sd
BOOTCLASSPATH=/system/framework/core.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/android.policy.jar:/system/framework/services.jar:/system/framework/core-junit.jar
ANDROID_BOOTLOGO=1
ANDROID_ASSETS=/system/app
EXTERNAL_STORAGE=/mnt/sdcard
ANDROID_DATA=/data
USBHOST_STORAGE=/mnt/sdcard/usbStorage
ANDROID_PROPERTY_WORKSPACE=13,32768
#

So I tried setting environmental variables..

# find -name apt-get
./mnt/sdcard/botbrew-basil/usr/bin/apt-get
./data/botbrew-basil/usr/bin/apt-get
# PATH=$PATH:/data/botbrew-basil/usr/bin:/data/botbrew-basil/bin:/data/botbrew-basil/usr/local/bin
# apt-get
apt-get: not found
# /data/botbrew-basil/usr/bin/apt-get

and that did not work either. I'm having a problem running the applications from within the botbrew package.. however botbrew is able to run apt-get.

This is the kind of thing I am facing.. it bothers me..

# chmod 777 ./bash
# ls -l
-rwxr-xr-x root root 1425732 2012-05-31 19:16 busybox
-rwxrwxrwx root root 28 2012-05-31 19:16 bash
-rwxr-xr-x root root 88336 2012-05-31 19:16 dash
-rwxr-xr-x root root 264924 2012-05-31 19:16 tar
lrwxrwxrwx root root 2012-05-31 19:16 sh -> /bin/dash
# ./bash

Why is botbrew able to use all of the apps, but the shell is not?
[Nook Color Android Development] inportb:
Thanks for offering to do a debugging session with me. I couldn't find the message, but I think we could work just fine asynchronously; after all, good ideas take time to percolate :)

Now, it seems strange that you don't have PATH set or access to apt... So I've a few questions, if you don't mind:

Where did you bootstrap?

I get the idea that you installed to a disk image, since you speak of loopback mounting.

What's inside your /system/bin/botbrew?

It should refer to your bootstrap directory.

While you're inside the botbrew shell, what do you see in the root directory?

It should show a bit of extra stuff such as /usr and /var.

I'll let you know if I think of more things to ask. Thanks for your time!

Also, we've determined that something's wrong with wget (or busybox's wget), because apt-get is able to do domain name resolution just fine. There have been similar reports involving android+busybox+wget, and the workaround seems to be to bypass the domain name.
[Nook Color Android Development] AdamOutler:
I downloaded the source and have been attempting to locate required libraries.

Would you please move your libraries into your source, then link to them from there?

These are the classes I need in order to compile from source.. They are not included in your source.

com.actionbarsherlock.app.Actionbar

com.actionbarsherlock.app.SherlockFragmentActivity

com.actionbarsherlock.app.SherlockDialogFragment

com.actionbarsherlock.view.Menu

com.actionbarsherlock.view.MenuInflater

com.actionbarsherlock.view.MenuItem

jackpal.androidterm.emulatorview.ColorScheme

jackpal.androidterm.emulatorview.EmulatorView

jackpal.androidterm.emulatorview.TermSession

org.acra.ReportingInteractionMode

org.acra.annotation.ReportsCrashes

----These are not included with my version of the ADK.. maybe I used the wrong version?----

android.support.v4.app.Fragment

android.support.v4.app.FragmentManager

android.support.v4.app.FragmentPagerAdapter

android.support.v4.view.ViewPager

android.support.v4.app.LoaderManager

android.support.v4.content.CursorLoader

android.support.v4.content.Loader

android.support.v4.widget.CursorAdapter

android.support.v4.widget.SimpleCursorAdapter

android.support.v4.app.LoaderManager

android.support.v4.content.AsyncTaskLoader

android.support.v4.content.Loader

If you are using proprietary binaries, it is best to include them in the source, like I make a /libraries/ folder and include it in the revision control. eg https://code.google.com/p/android-casual/source/browse/#svn/trunk/GUI/src/Libraries/BSAFramework

This way anyone who wants to contribute can do so easily. Either way, the actionbarsherlock site is down.
[Nook Color Android Development] AdamOutler:
I found an img file and I noticed alot of functionality was missing, so I assumed that you were mounting an image... Must have been something else I did.

/system/bin/botbrew is an executable

$ su
# ls /system/bin/botbrew
/system/bin/botbrew
# ls /system/bin/botbrew/
/system/bin/botbrew/: Not a directory
# /system/bin/botbrew && echo $?
0
# rm /system/bin/botbrew
# ls /system/bin/botbrew
/system/bin/botbrew: No such file or directory
# ls /
efs
config
cache
dbdata
sdcard
acct
mnt
vendor
d
etc
data
proc
dev
sbin
ueventd.goldfish.rc
recovery.rc
ueventd.rc
fota.rc
tmp
res
recovery
lib
init.rc
lpm.rc
init.goldfish.rc
init
ueventd.herring.rc
sys
init.herring.rc
system
def
#

I removed botbrew and did a rebootstrap. Nothing changed. I've seriously been working on this for entirely too long and hitting a bunch of dead-ends.

I have a operational version of ARM busybox with working wget. let me know if you need it.
[Nook Color Android Development] AdamOutler:
It is failing on this when attempting to install botbrew-core

$ su
# exec /data/botbrew-basil/init -- -oapt::architectures='device-sgh-i897,cpu-abi2-armeabi,baord-sgh-i897,android,sgh-i897,armeabi-v7a,armeabi' -oapt::clean-installed='1' -oapt::get::download='1' -oapt::get::assume-yes='1' -oapt::get::show-upgrade='1' -oapt::get::purge='1' -oapt::get::allowunauthenticated='1'- odpkg::options::='--ignore-depends=libgcc1' install botbrew-core
whoops: cannot run `-oapt::architectures=device-sgh-i897,cpu-abi2-armeabi,baord-sgh-i897,android,sgh-i897,armeabi-v7a,armeabi -oapt::clean-installed=1 -oapt::get::download=1 -oapt::get::assume-yes=1 -oapt::get::show-upgrade=1 -oapt::get::purge=1 -oapt::get::allowunauthenticated=1- odpkg::options::=--ignore-depends=libgcc1 install botbrew-core'
[Nook Color Android Development] E:V:A:
What env.vars. were you expecting?

I think you're missing part of the complete path (depending on where you executed the above command. Try this instead:

# find / -iname "apt-get"

Then after you have set the path variable you actually have to enable it. By "sourcing" it. This is done by re-running the login script.

# source <path+name of login script>

(or just try it with "./<name>", but then you risk duplicating the envrionment variables.) The normal linux names are usually something like ".bashrc", ".bash_profile" or ".csh" etc etc. So I don't know what they would be in your case.

Probably because a running script has its own set of env vars...depending on what shell its running under. (Ash, Csh, Bash etc)..
[Nook Color Android Development] AdamOutler:
well.. i installed a custom ICS based ROM on my Captivate (ICSSGS) and Botbrew worked. So, you can add Captivates running ICS. It may have to do with the MTD over BML.. i have no clue really. I do know that it is not launching the botbrew environment from stock.

I cannot compile your source. The source is missing dependencies which I cannot locate.

I have another spare captivate which i have no problem with toasting the firmware and dropping it into my DMZ with ADB and VNC enabled for your testing purposes if youd like to get it operational on the entire Galaxy S series.
[Nook Color Android Development] inportb:
Just checking in to say that I'm removing the busybox requirement for loop-mounted installations. I haven't tested extensively, but I'll poke around before generating another binary package.

Incidentally, most of the libraries that we use could be had from GitHub, which was operational the last time I checked. These are Android library projects that you should add to your Eclipse workspace and reference from BotBrew. The support package from Google provides the libraries that you're missing, such as android.support.v4.*; this one goes in the libs directory, along with acra. I'd like to keep binaries (other than graphics) out of the source tree, so feel free to drop 'em in -- they won't be versioned.
[Nook Color Android Development] AdamOutler:
Hrm... i was able to install an additional repo then install apache2. When i run it, it does a user check... now the two filesystems are out of sync i think... my system toolbox knows how to properly handle Android names, but busybox does not?

id -u returns proper results.

whoami does not.

Busybox and toolbox should probly come from within your package in order to maintain standard linux compatibility... or some other crafty method...

Anyway.. mounted system rw then i added passwd, group, and shadow entries to no avail.
[Nook Color Android Development] inportb:
If by Android names, you mean stuff like app_xyz, that sounds about right. Android does not respect /etc/passwd et al, but it does query its own database. Busybox, on the other hand, only loads entries out of /etc/passwd. With libmusl, I have some custom functions that search first in /etc and then query Android if they don't find anything; but even then, busybox uses its own methods instead of libc's (for portability... or a moderate dose of NIH? ;)).

But hang on... these might help --

within botbrew, /system is always writable (so you don't need to remount anything)

it might be useful to edit the files in /data/botbrew-basil/etc

Thanks for your efforts! Meanwhile, on my end: it looks like the loop mount stuff is working just fine. The idea is to make the gap as small as possible, so /sdcard users don't feel left out. You probably ran into some of these issues earlier with your loop-mounted installation. Now we have a more capable init :)
[Nook Color Android Development] AdamOutler:
inportb: I need to contact you directly. I'm sending a PM. If you do not receive it, please contact me at my username @gmail. I'm working on an XDATV episode.
[Nook Color Android Development] inportb:
Hey folks; it's been a while :)

I've been busy with the chroot side of things. There was one "feature" that kept bugging me; in particular, mount points did not stay in sync. What does that mean?

The fix was surprisingly complicated, given that Android uses rootfs as its primary filesystem, and that lots of stuff has already been mounted by the time BotBrew comes into action:

bind-mount everything under /mnt into a tmpfs named /data/.botbrew

unmount everything under /mnt, keeping /data/.botbrew intact

bind-mount /data/.botbrew onto /mnt

unmount everything under /data/.botbrew, keeping /mnt intact

You might be thinking -- wtf? that just moves everything from /mnt to /data/.botbrew and back again!

Right, but notice that we've surreptitiously inserted a little shim /data/.botbrew in-between /mnt and all the mount points. This lets us propagate all the mount activity over to BotBrew, which is awesome. Wow, that's a lot of lines.

Please expect a new build soon :)
[Nook Color Android Development] rocker2344:
passwd
[Nook Color Android Development] shinji257:
I setup the bootstrap to go to /mnt/sdcard/external_sd. What's the best way to go about increasing the size of the loopfs? I did it for space reasons. /data is only 2GB on my phone and is already 50%. Rather leave it for android. My external SD on the other hand is 32GB and even the internal one is 16GB (minus some stuff for the phone itself like the 2GB /data)
[Nook Color Android Development] inportb:
Thanks for all the comments, questions, bug reports, and other messages! I happen to be out of my usual country at the moment, but I'd like to note that I'm listening and will continue working when I get back in a few days.
[Nook Color Android Development] hiddenhandgun:
This may or may not be OT. Here's a weird issue I had hoped the BotBrew binaries would have fixed for me, but no luck. I have been using statically compiled versions of many of the basic gnu utils over ssh (droidsshd), but have had problems with name resolution. Under AOKP the default busybox utils can resolve, but little else can.

Installed and ran up BotBrew's dropbear and lynx, and tried with that:

$ ssh -t root@android /botbrew/bin/bbsh
root@android's password:
root@android:/botbrew/root # lynx -dump http://google.com

Looking up google.com
Unable to locate remote host google.com.
Alert!: Unable to connect to remote host.

lynx: Can't access startfile http://google.com/

but...

root@android:/botbrew/root # ls -l /system/xbin/nslookup
lrwxrwxrwx root root 2012-06-25 13:50 nslookup -> busybox
root@android:/botbrew/root # nslookup google.com
Server: 0.0.0.0
Address 1: 0.0.0.0

Name: google.com
Address 1: 173.194.37.66 atl14s08-in-f2.1e100.net
Address 2: 173.194.37.71 atl14s08-in-f7.1e100.net
Address 3: 173.194.37.68 atl14s08-in-f4.1e100.net
Address 4: 173.194.37.69 atl14s08-in-f5.1e100.net
Address 5: 173.194.37.72 atl14s08-in-f8.1e100.net
Address 6: 173.194.37.67 atl14s08-in-f3.1e100.net
Address 7: 173.194.37.70 atl14s08-in-f6.1e100.net
Address 8: 173.194.37.73 atl14s08-in-f9.1e100.net
Address 9: 173.194.37.65 atl14s08-in-f1.1e100.net
Address 10: 173.194.37.64 atl14s08-in-f0.1e100.net
Address 11: 173.194.37.78 atl14s08-in-f14.1e100.net
Address 12: 2607:f8b0:4002:802::1005 atl14s08-in-x05.1e100.net

Like some others, I find that getprop returns empty. I have valid nameservers in resolv.conf in every damn /etc/ dir I can find.

strace shows this very similar weirdness:

bind(3, {sa_family=AF_INET, sin_port=htons(28750), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
sendto(3, "n\t\1\0\0\1\0\0\0\0\0\0\6google\3com\0\0\1\0\1"..., 28, 0, NULL, 0) = 28
clock_gettime(CLOCK_REALTIME, {1340733838, 381744387}) = 0
clock_gettime(CLOCK_REALTIME, {1340733838, 381927492}) = 0
select(4, [3], NULL, NULL, {4, 999817}) = 1 (in [3], left {4, 999817})
getsockopt(3, SOL_SOCKET, SO_ERROR, [111], [4]) = 0

The really unamusing part is that this works on a local term in connectbot.

I'm convinced this is a dropbear issue, and it seems to run across at least three builds. It _could_ be a libc thing. I really really don't know at this point.

tl;dr Does anyone know how to get name resolution working properly in an ssh session?

root@android:/botbrew/root # setprop net.tiwlan0.dns1 8.8.8.8

root@android:/botbrew/root # getprop

root@android:/botbrew/root #