yashsingh1/xdaforums huggingface dataset:
[Android Software Development] dragonnn:
Hmmm that will by grate, but after I extract you files and will launch an binary i get this:

localhost / # /sd-ext/external_linux/bin/getcap
bash: /sd-ext/external_linux/bin/getcap: No such file or directory

Now I'm confused, I have try to set the permissions to 777 but I doesn't help:confused:. The files are 100% in this folder:

-rwxr-xr-x 1 system 100 33018 Jan 27 19:42 capsh
-rwxrwxrwx 1 system 100 13438 Jan 27 19:42 getcap
-rwxr-xr-x 1 system 100 10821 Jan 27 19:42 getpcaps
-rwxr-xr-x 1 system 100 13826 Jan 27 19:42 setcap
[Android Software Development] smitna:
I think I found your problem:

pam_cap.c:19:34: fatal error: security/pam_modules.h: No such file or directory

Please prepend PAM_CAP=no to make

should be:

PAM_CAP=no $MAKE CC="$CC_NAME" LD="$CC_NAME -shared" CFLAGS="$CFLAGS_DEFAULT" LDFLAGS="$LDFLAGS_DEFAULT -lattr" INCDIR="$SYSROOT/include"

in line 5049 and try again.
[Android Software Development] smitna:
I have set this:

TARGET_SYSROOT="/sd-ext/external_linux"

is this the right path on your phone?

But please try first a new run, see my new post.
[Android Software Development] dragonnn:
Yes, this is right patch o my phone. I think I have found the problem why I can not launch you binaries.

Here is readelf from you binary:

ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: ARM
Version: 0x1
Entry point address: 0x9660
Start of program headers: 52 (bytes into file)
Start of section headers: 81852 (bytes into file)
Flags: 0x5000000, Version5 EABI
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 7
Size of section headers: 40 (bytes)
Number of section headers: 25
Section header string table index: 24

And here is from my phone from sh binary:

ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: ARM
Version: 0x1
Entry point address: 0x8964
Start of program headers: 52 (bytes into file)
Start of section headers: 8676 (bytes into file)
Flags: 0x5000002, has entry point, Version5 EABI
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 8
Size of section headers: 40 (bytes)
Number of section headers: 38
Section header string table index: 35

And from mempdroid binary too (an Android exploit, can run it):

ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: ARM
Version: 0x1
Entry point address: 0x89c0
Start of program headers: 52 (bytes into file)
Start of section headers: 31620 (bytes into file)
Flags: 0x5000002, has entry point, Version5 EABI
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 7
Size of section headers: 40 (bytes)
Number of section headers: 33
Section header string table index: 30

Can you see the difference? You binaries don't have "has entry point" and the flag is different - my have 0x5000002 yours have 0x5000000. For now I am rebuilding the binaries by adding PAM_CAP=no, but I think we need to solve this problem too.
[Android Software Development] dateno1:
i think

you can add checksum function for downloaded source archive in build.sh (if not matched redownload again)

sometime it will prevent critical error

i already add boot and run time md5checksum on my recovery (i can't add redownload and reinstall yet)
[Android Software Development] smitna:
Yes, that's not ok.

But I read this:

readelf -h sources/libcap-2.22/libcap_dragonn/getcap

ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: ARM
Version: 0x1
Entry point address: 0x8964
Start of program headers: 52 (bytes into file)
Start of section headers: 8676 (bytes into file)
Flags: 0x5000002, has entry point, Version5 EABI
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 8
Size of section headers: 40 (bytes)
Number of section headers: 38
Section header string table index: 35
[Android Software Development] dragonnn:
WTF:confused: how can this happened? And another WTF, I have try build self this library and what I get when I make readelf on my binary build:

ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
Class: ELF64
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: Advanced Micro Devices X86-64
Version: 0x1
Entry point address: 0x400e0c
Start of program headers: 64 (bytes into file)
Start of section headers: 12520 (bytes into file)
Flags: 0x0
Size of this header: 64 (bytes)
Size of program headers: 56 (bytes)
Number of program headers: 8
Size of section headers: 64 (bytes)
Number of section headers: 38
Section header string table index: 35

WTF "Advanced Micro Devices X86-64" and I can run this binary on my desktop:mad:. Now i try to bulid this binary for armv5te, the memdroid binary has been build for it too and it works. But I think I have problem witch the compiler:confused:.
[Android Software Development] smitna:
If you only need a ssh client, I can offer you a static build of ssh (OpenSSH). Can you please post your phone specs (the optimizer settings)?
[Android Software Development] dragonnn:
Is this to me? At this moment I have:

TARGET_MARCH="armv5te"
TARGET_MTUNE="arm6"
TARGET_MFPU="vfp"
TARGET_MFLOAT="softfp"
# default: OPTIMIZATIONS="-g -Os -marm -march=$TARGET_MARCH -mtune=$TARGET_MTUNE -mfpu=$TARGET_MFPU -mfloat-abi=$TARGET_MFLOAT"

OPTIMIZATIONS="-g -Os -marm -march=$TARGET_MARCH -mtune=$TARGET_MTUNE -mfpu=$TARGET_MFPU -mfloat-abi=$TARGET_MFLOAT"

But previous builds I had removed the MTUNE from OPTIMIZATIONS. Now waiting for the effects.
[Android Software Development] Droidzone:
Thank you. I'll get the error logs tomorrow (too late today). I'd be very happy if you can provide a static ssh binary... :)

Hope these are the specs you need..

Edit: Just found what you asked for.
[Android Software Development] smitna:
Only one question:

What's about your other binaries, can you run python -V for example?
[Android Software Development] Droidzone:
Damn! It works! Thanks a lot!
[Android Software Development] :
Hello guys,

i've got this thing running under Archlinux but you've to cp the autoconf-2.13 binary:
[Android Software Development] Droidzone:
Successfully compiled openssh, and it runs too. Maybe that too needed a two pass run, or maybe singing failed to build the first time. Now I just need to figure out how to resolve the host name to Sourceforge. Added a rule in /etc/hosts, but apparently the ip's wrong.

Swyped on my Htc Desire running on Supernova Extreme - Gingerbread goodness with 2 GB of Internal memory
[Android Software Development] dateno1:
thanks but i think cp is waste

if you use it copied binary not be updated so if you update something it will be not work well (and waste of space)

i think symlink is best solution
[Android Software Development] smitna:
If you don't need PHP, comment out the check for it in build.sh I think, I will remove it in the next version.
[Android Software Development] Droidzone:
smitna, thanks a lot...

There are a few things to correct in the toolchain, but ought to work soon..

Swyped on my Htc Desire running on Supernova Extreme - Gingerbread goodness with 2 GB of Internal memory
[Android Software Development] smitna:
Please don't use it for the kernel compilation, I am not sure about it. You should use the none eabi version. Look at the version, that Gokhan is using for his kernel.
[Android Software Development] Droidzone:
No problem. I have my own toolchain for kernel compiling, with latest gcc and glibc.. Was just testing if the Desire can do it. But for actual compilation, I need to cross compile my cross compiler.. wish you had a script for that! ;)

Swyped on my Htc Desire running on Supernova Extreme - Gingerbread goodness with 2 GB of Internal memory
[Android Software Development] smitna:
No, it's not good to symlink to another version. If the old autoconf is missing, it is better to find a safer solution for it.
[Android Software Development] smitna:
I'm impressed!
[Android Software Development] dragonnn:
Not really:mad:, I get this:

localhost bin # ./python
bash: ./python: No such file or directory

For every binary. Only I can call mempodroid:

localhost bin # ./mempodroid
usage: mempodroid <exit> <call> <args...>
exit: address in memory to exit function
call: address in memory of setresuid call
args: command to run, including arguments

Galaxy Nexus 4.0.2: 0xd7f4 0xad4b
Transformer Prime 4.0.3: 0xd9ec 0xaf47

concrete implementation by Jay Freeman (saurik)
based on exploit by Jason A. Donenfeld (zx2c4)
more information at: http://blog.zx2c4.com/749
original kernel exploit reported by Jüri Aedla
localhost bin #

I put this binary in the same directory as other binaries, I found it here https://github.com/saurik/mempodroid . I think the problem can by found in the readelf log. I have put here http://dl.dropbox.com/u/1664131/Binares.tar.bz2 the original bash binary, mempodroid and the build python2.7 and all logs of readelf from each binary, you should look at the end, the original and mempodroid binary are don't have:

Tag_DIV_use: Not allowed
Tag_ABI_align_preserved: 8-byte, except leaf SP

I don't know what is really it, but I think it should by possible to configure the compile process to get binaries without this Tags.
[Android Software Development] dateno1:
localhost bin # ./python
bash: ./python: No such file or directory

it is simple (binary can't found ld-linux.so.3 or something like interpreter)

i think you must extract your all file on prefix and overwrite share files
[Android Software Development] smitna:
Edit: Sorry wrong quote...
[Android Software Development] smitna:
The problem is the following:

On an unknown point of the script (or from the beginning) the settings from file prepare_codesourcery.sh gets lost.

readelf -d python2.7

rpath ok

readelf -A python2.7

arch: 5TE

VFP: VFPv2

are this your settings from the config?

readelf -d bash

Shared libraries in libs, this is also not ok, because bash is static and build with NDK

readelf -d mempodroid

no rpath

Do you compile in a bash console (that's important!!) and what's about the file permissions of prepare_codesourcery.sh, prepare_ndk_static.sh and unset_crosscompiler.sh

For setcap/getcap we have two passes:

The first generates and executes the binary to generate the headers

The second builds the binaries for the phone

So there is a switch from prepare_codesourcery.sh to unset_crosscompiler.sh and back

Please try this:

in build.sh before or right after line

# DO NOT CHANGE ANYTHING FROM HERE IF YOU DO NOT KNOW WHAT YOU ARE DOING

insert:

showvars

exit

now start the build.sh as usual and look at the output.

You should see some lines with LDFLAGS, CFLAGS etc:

for example:

CROSS_COMPILE: [PATH TO YOUR BUILDROOT]/arm-2011.09/bin/arm-none-linux-gnueabi-

and

LDFLAGS: -L... -L... -Wl,-rpath=[PATH ON YOUR PHONE]/lib

If this vars are not set, then the files are not included for some reason and we have to check why.

I would also recommend, that you clear the build folder complete and start from a fresh state with the latest version 2.1.

The files from cs-sysroot.tar.gz must be in place on your phone on the ext3 or ext4 partition (all in /sd-ext/external_linux/), so you find the ld-linux.so.3, libc-2.13 and others in /sd-ext/external_linux/lib and some others in /sd-ext/external_linux/usr/lib

Sorry for your problems!
[Android Software Development] dragonnn:
What a shame:eek:, I forget to extract cs-sysroot.tar.bz2 (I have think I don't really need it). But the libcap binaries are still x86. Hover this are now try to run you libcap binaries and the working, but kernel don't support it:

Failed to set capabilities on file `/system/bin/dhcpcd' (Operation not supported)

Now I am will try to find out how to get kernel support for this:D. Really big thanks for helping;).

EDIT

Btw the bash binary isn't from you script, it's original from my rom.
[Android Software Development] smitna:
/system/bin is normally read only (I don't know your phone settings). Better try and make the test with a dummy binary in /data

But yes, I think there is no capabilities support.

Please read also the README as I say, there should all described (CS sysroot etc.)
[Android Software Development] dragonnn:
Yes, I didn't read the complete README, sorry. I know that /system/bin is normal mount ro, have remount it before tried to make it.
[Android Software Development] smitna:
I added a small android app (ScriptStarter) to start services and other shell scripts easily from the GUI.

See post #1 for details: http://xdaforums.com/showthread.php?t=1444792
[Android Software Development] dateno1:
i try to make develope pack

but it not well....

i can't find gcc or g++ on /bin :(

build log attached

this is my phone's cpu info

this is my setting

i already has static comiled GNUMAKE, STRACE(maybe from cm7) so i don't need they but this is full package for native develop :)

i remove all symlink on /sddata/gcc and all files&folders from my built directory and run remove_all_files.sh before running build.sh (i tried ./build.sh 2>&1 | tee /built/build.log)
[Android Software Development] Droidzone:
Is there an easy way to find out whether the binary I've compiled is static or dynamic with a link to a lib?

Damn...I just tried building bash 4.2 statically the manual way. I had to apply no fewer than 20 patches! Phew... You've done a great job with the scripts.. :)
[Android Software Development] smitna:
To check dynamic linking use

readelf -d [FILE]

output for static:

There is no dynamic section in this file.

output for dynamic something like this:

0x00000001 (NEEDED) Shared library: [libdl.so.2]

...

The patches are only partly from me or just collected from other sources and modified, for example for busybox and iptables.

Some of them are modified makefiles after compiling manually over and over again.
[Android Software Development] Droidzone:
Would it be possible to easily switch between static and dynamic compilation in your script (maybe in a future release)?
[Android Software Development] smitna:
Hm, after a quick view I can see only a little problem at the end of the log:

unrecognized option '-Qy' when executing arm-none-linux-gnueabi/bin/as

I cannot say, why the makefile or configure suddenly is setting this switches. I will look at that. -Q is useless and -y unknown. If you find this switches in one of the configure files, it should be safe to delete them.

Edit: I found this:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32829

Please check your PATH var, perhaps it will help.
[Android Software Development] smitna:
No, I tried this and if it is working, the results are very big files.

There is also an other problem with the static builds. The libc has still an old bug when calling system functions (check user/groups). I don't know, why CodeSourcery had not patched this, like the Debian devs. But I think, this is also a Android problem (no real users/groups).

It' possible to build partly static, and it's only a little change in the prepare_codesourcery.sh (CFLAGS switch -static and remove of the dynamic part of LDFLAGS)

The static one is prepare_codesourcery_static.sh in the script set.

With NDK you can build static (see for example bash), but sometimes there is the need to patch a lot of files because of the limited support from the bionic glibc.
[Android Software Development] Droidzone:
Hmm..makes	a lot of sense. That may be partly the reason why Ganymedes suggested making our own toolchain. Anyway in some cases like bash, a static binary is only 3 MB, and really useful.

It took me some time to realize that the bug I was getting was actually a part of bash 4.2 itself rather than something I was doing wrong. :) Reading through your script, I found many references to patches, and got the hint that the patches were there in the gnu bash tree itself.
[Android Software Development] dateno1:
new share library not work :(

this is my cpu info

i try CS_SYSROOT_VARIANT="thumb2"

then overwrite new cs-sysroot.tar.bz2 to my old built mysql

it output error messsage

Segmentation fault

then i try strace -o error.txt ./mysql_embedded

it end by killed by SIGSEGV

my old mysql built by 1.7

setting is

same as new shared library
[Android Software Development] smitna:
I'm using thumb2 with a cortex-a9 and have tested ./mysql_embedded successful. I guess it's not compatible with your phone. You should use the v5TE version.
[Android Software Development] dateno1:
no it strange

if i overwrite /lib it not work but /usr/lib is ok
[Android Software Development] Droidzone:
While trying to run your script in an ssh envt where I dont root have access to, I'm facing some issues. The script requires glib2. I tried compiling glib2, got an error that libffi>3 is not found. Compiled libffi successfully, and built to ~/bin, but apparently I cant make glib2 "see" my version of libffi... I thought it may be due to pkg-config, but on trying to compile pkg-config, apparently it needs glib2! Chicken and egg all over again..

Error message:

No package 'libffi' found

I have a very slow net connection at home, which is why I try to do this in an ssh envt. Do you know how to pass the location of libffi to glib2 while configuring?

Edit: Solved. Had to set some envt vars

export LIBFFI_LIBS=-lffi
export LIBFFI_CFLAGS=-I//home/users/d/dr/droidzone/hostusr/lib/libffi-3.0.10/include

where /home/users/d/dr/droidzone/hostusr was the prefix dir of my libffi install

Now,

./configure --prefix=/home/users/d/dr/droidzone/hostusr

for Glib2 works

I know it has nothing to do with your project, but it may turn up in search for the same
[Android Software Development] smitna:
You don't need pkg-config, look at Line 1490 in build.sh:

ffi.h isn't copied when installing (and glib then don't find ffilibs)

That was my problem. If the CFLAGS var is equipped with the right include folder to the ffi.h and the configure/Makefile can find the libs (LDFLAGS) it should work without problems. But good, you solved it yourself and thanks for the information.
[Android Software Development] Droidzone:
Thank you. By the way, is it possible to lie to the lib files regarding the location they share. In my ssh envt, I have a path to the binary sysroot of /home/u/users/project/d/dr/droidzone

I cant access root path.. So ulitmately I have to create such a folder in my phone's root! :) Any way to build it at /home/u/users/project/d/dr/droidzone, but ultimately install at /data/bin, without creating symlinks in root? I suppose the only way out would be to create a chroot envt?

This is ubercool. My git repos always on my phone.
[Android Software Development] smitna:
Try it without the symlinks. Some packages will fail, because they want to use the install folders for the make install, others will install without problems.

A not so nice solution would be this:

use a subfolder in your user dir /home/u/users/project/d/dr/droidzone as TARGET_SYSROOT

On the phone create this as symlink to the real installation directory (if necessary at boot time with a script). The real directory can be in /data or somewhere on an extX partition.

You can also add more directories to the rpath (TARGET_RPATH) but some binaries are looking for configuration files only in the hard coded paths at build time.

There are two alternative methods as I know:

fakeroot and chroot

But I don't tried this methods with the scripts because I don't want to make the use more complicated as it needs to be. And the script needs read access to a few headers and files from the host, for example Python. An other way is to build all first for the host and then use this local binaries for the target build.

The buildroot devs using one of these methods I guess, but I haven't looked in the source code.

If someone knows a good solution, please let me know.
[Android Software Development] smitna:
Good to know. I put it on my todo list to make first a check for "tic".
[Android Software Development] Droidzone:
I edited my post and added a few requests/comments at the end. :)
[Android Software Development] smitna:
Hello Droidzone,

first thank you very much for your feedbacks, this is really very useful for me.

For xz the tar has the switch xJf, but xz must be installed on the system. More and more devs are switching to the xz packer, because the packages are significant smaller.

Thank you for the patch file. I have tried to use the FTP variant, if a maintainer or dev offers both http and ftp because ftp is usually a little bit faster (less overhead). But I can of course replace the ftp links with http when possible.

Why? It's the original authors site, but unfortunately only FTP, yes...

Of course! I will check the build specs, but it seems to be simple, only a Makefile :)

No! After an initial build of all "needed packages" you can set them to "no" and "forget" them ;) New programs will find the eventual needed headers always in the sysroot folder. To force a "never rebuild", please make comments in the dependency list, for example for ImageMagick.
[Android Software Development] Droidzone:
Thank you for the clear answers. :)

I have compiled the latest version of xz and installed it. But apparently it doesnt allow tar to have j switch for xz..

Resolving ftp.gnome.org... 130.239.18.138, 130.239.18.163, 2001:6b0:e:2018::163, ...
Connecting to ftp.gnome.org|130.239.18.138|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6040856 (5.8M) [application/x-tar]
Saving to: `glib-2.31.6.tar.xz'

100%[===============================================================================>] 6,040,856 2.84M/s in 2.0s

2012-02-01 11:22:00 (2.84 MB/s) - `glib-2.31.6.tar.xz' saved [6040856/6040856]

[droidzone@shell-23004 sources]$ tar xvvfj glib-2.31.6.tar.xz
bzip2: (stdin) is not a bzip2 file.
tar: Child returned status 2
tar: Error is not recoverable: exiting now
[droidzone@shell-23004 sources]$ which xz
~/hostusr/bin/xz
[droidzone@shell-23004 sources]$ xz --help
Usage: xz [OPTION]... [FILE]...
Compress or decompress FILEs in the .xz format.

-z, --compress force compression
-d, --decompress force decompression
-t, --test test compressed file integrity
-l, --list list information about .xz files
-k, --keep keep (don't delete) input files
-f, --force force overwrite of output file and (de)compress links
-c, --stdout write to standard output and don't delete input files
-0 ... -9 compression preset; default is 6; take compressor *and*
decompressor memory usage into account before using 7-9!
-e, --extreme try to improve compression ratio by using more CPU time;
does not affect decompressor memory requirements
-q, --quiet suppress warnings; specify twice to suppress errors too
-v, --verbose be verbose; specify twice for even more verbose
-h, --help display this short help and exit
-H, --long-help display the long help (lists also the advanced options)
-V, --version display the version number and exit

With no FILE, or when FILE is -, read standard input.

Report bugs to <lasse.collin@tukaani.org> (in English or Finnish).
XZ Utils home page: <http://tukaani.org/xz/>
[droidzone@shell-23004 sources]$ xz --version
xz (XZ Utils) 5.0.3
liblzma 5.0.3

Whats interesting is that a two command extract works:

[droidzone@shell-23004 sources]$ unxz glib-2.31.6.tar.xz ; tar -xvvf glib-2.31.6.tar

If a tar file is put in the sources directory, or extracted and kept there, will the script detect it and use it, rather than redownloading?