kevinweil/hadoop-lzo — Issue #33: Build fails on Ubuntu 11.10 (changed ld default behavior)
Author: miguno
Created: 2011-10-27 10:16:59+00:00
State: open

I noticed that the build of hadoop-lzo fails on Ubuntu 11.10. The reason is that the default behavior of ld was changed in 11.10.

From [Ubuntu 11.10 Release Notes](https://wiki.ubuntu.com/OneiricOcelot/ReleaseNotes#GCC_4.6_Toolchain):

> The compiler passes by default two additional flags to the linker:
>
> [...snipp...]
>
> -Wl,--as-needed with this option the linker will only add a DT_NEEDED tag for a dynamic library mentioned on the command line if if the library is actually used.

This was apparently planned to be changed already back in 11.04 but was eventually reverted in the final release. From [11.04 Toolchain Transition](https://wiki.ubuntu.com/NattyNarwhal/ToolchainTransition#Indirect_Linking_for_Shared_Libraries):

> Also in Natty, ld runs with the --as-needed option enabled by default. This means that, in the example above, if no symbols from libwheel were needed by racetrack, then libwheel would not be linked even if it was explicitly included in the command-line compiler flags. NOTE: The ld --as-needed default was reverted for the final natty release, and will be re-enabled in the o-series.

Now in hadoop-lzo this manifests itself in the script {{src/native/configure}}. The following check will fail because of the changed ld default behavior:

```bash
echo 'int main(int argc, char **argv){return 0;}' > conftest.c
if test -z "`${CC} ${CFLAGS} ${LDFLAGS} -o conftest conftest.c -llzo2 2>&1`"; then
if test ! -z "`which otool | grep -v 'no otool'`"; then
ac_cv_libname_lzo2=\"`otool -L conftest | grep lzo2 | sed -e 's/^ *//' -e 's/ .*//'`\";
elif test ! -z "`which objdump | grep -v 'no objdump'`"; then
ac_cv_libname_lzo2="`objdump -p conftest | grep NEEDED | grep lzo2 | sed 's/\W*NEEDED\W*\(.*\)\W*$/\"\1\"/'`"
elif test ! -z "`which ldd | grep -v 'no ldd'`"; then
ac_cv_libname_lzo2="`ldd conftest | grep lzo2 | sed 's/^[^A-Za-z0-9]*\([A-Za-z0-9\.]*\)[^A-Za-z0-9]*=>.*$/\"\1\"/'`"
else
as_fn_error $? "Can't find either 'objdump' or 'ldd' to compute the dynamic library for '-llzo2'" "$LINENO" 5
fi
else
```

This line compiles a dummy C script and tells gcc to link it to the liblzo library (the native LZO library).

```bash
${CC} ${CFLAGS} ${LDFLAGS} -o conftest conftest.c -llzo2
```

Because of the changed ld behavior in 11.10, however, `conftest` will not link liblzo2. Hence the subsequent check will fail; since I do not have `otool` installed on my local dev box, the following code is run in my case:

```bash
ac_cv_libname_lzo2="`objdump -p conftest | grep NEEDED | grep lzo2 | sed 's/\W*NEEDED\W*\(.*\)\W*$/\"\1\"/'`"
```

This command will return an empty string and assign it to the variable `ac_cv_libname_lzo2`. The eventual result is that the symbol `HADOOP_LZO_LIBRARY` will be assigned an empty string, too.

```bash
cat >>confdefs.h <<_ACEOF
#define HADOOP_LZO_LIBRARY ${ac_cv_libname_lzo2}
_ACEOF
```

Without a proper value for `HADOOP_LZO_LIBRARY`, however, the build of hadoop-lzo will fail.

* How to fix*

In our build setup we are using `LDFLAGS` to pass this option when invoking `ant` like so:

```bash
$ env LDFLAGS="-Wl,--no-as-needed" ant ...
```

In general though I'd think it would be best to fix this in `build.xml`:

```xml

<== ADD THIS LINE
...

```

Best,
Michael