Bug: Fail to build hadoop-common from source on Fedora
Status: Open | Priority: Major | Resolution: 
Created: 18/May/21 13:08

Dear I tried to build hadoop from source with a vanilla fedora 34

{code:bash}
dnf group install -y "Development Tools" \
 && dnf install -y java-1.8.0-openjdk-devel fuse-devel snappy-java snappy-devel jansson-devel protobuf zlib-devel libzstd-devel \
                   maven-1:3.6.3 cmake gcc-c++ ant protobuf-compiler protobuf-java slf4j 

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-0.fc34.x86_64/
export MAVEN_OPTS="-Xms2048M -Xmx4096M"
export PATH="/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-0.fc34.x86_64/bin/:$PATH"
export CC=/usr/bin/gcc
export CXX=/usr/bin/g++

curl -LO https://apache.mediamirrors.org/hadoop/common/hadoop-3.2.2/hadoop-3.2.2-src.tar.gz
tar xf hadoop-3.2.2-src.tar.gz && cd hadoop-3.2.2-src
mvn package -Pdist,native -Drequire.snappy=true  -DskipTests -Dtar
{code}

But I have this error

{code:java}
    at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.runMake (CompileMojo.java:229)                                                                                                                                                                                                   
    at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.execute (CompileMojo.java:98)                                                                                                                                                                                                    
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)                                                                                                                                                                                       
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)                                                                                                                                                                                                         
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)                                                                                                                                                                                                         
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)                                                                                                                                                                                                         
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)                                                                                                                                                                                
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)                                                                                                                                                                                 
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)                                                                                                                                                                   
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)                                                                                                                                                                                                 
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)                                                                                                                                                                                                                          
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)                                                                                                                                                                                                                          
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)                                                                                                                                                                                                                            
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)                                                                                                                                                                                                                                
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)                                                                                                                                                                                                                                 
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)                                                                                                                                                                                                                                   
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)                                                                                                                                                                                                                             
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)                                                                                                                                                                                                           
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)                                                                                                                                                                                                   
    at java.lang.reflect.Method.invoke (Method.java:498)                                                                                                                                                                                                                                        
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)                                                                                                                                                                                                     
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)                                                                                                                                                                                                             
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)                                                                                                                                                                                                   
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)                                                                                                                                                                                                               
[ERROR]                                                                                                                                                                                                                                                                                         
[ERROR]                                                                                                                                                                                                                                                                                         
[ERROR] For more information about the errors and possible solutions, please read the following articles:                                                                                                                                                                                       
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException                                                                                                                                                                                                        
[ERROR]                                                                                                                                                                                                                                                                                         
[ERROR] After correcting the problems, you can resume the build with the command                                                                                                                                                                                                                
[ERROR]   mvn <args> -rf :hadoop-common                                                             
{code}

If we take a look to cmake log
I have:


{code:c}
gmake[1]: *** [CMakeFiles/cmTC_82239.dir/build.make:106: cmTC_82239] Error 1                                                                                                                                                                                                                    
gmake[1]: Leaving directory '/hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp'                                                                                                                                                                           
gmake: *** [Makefile:140: cmTC_82239/fast] Error 2                                                                                                                                                                                                                                              
                                                                                                                                                                                                                                                                                                
                                                                                                                                                                                                                                                                                                
Source file was:                                                                                                                                                                                                                                                                                
#include <pthread.h>                                                                                                                                                                                                                                                                            
                                                                                                                                                                                                                                                                                                
static void* test_func(void* data)                                                                                                                                                                                                                                                              
{                                                                                                                                                                                                                                                                                               
  return data;                                                                                                                                                                                                                                                                                  
}                                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                                
int main(void)                                                                                                                                                                                                                                                                                  
{                                                                                                                                                                                                                                                                                               
  pthread_t thread;                                                                                                                                                                                                                                                                             
  pthread_create(&thread, NULL, test_func, NULL);                                                                                                                                                                                                                                               
  pthread_detach(thread);                                                                                                                                                                                                                                                                       
  pthread_cancel(thread);                                                                                                                                                                                                                                                                       
  pthread_join(thread, NULL);                                                                                                                                                                                                                                                                   
  pthread_atfork(NULL, NULL, NULL);                                                                                                                                                                                                                                                             
  pthread_exit(NULL);                                                                                                                                                                                                                                                                           
                                                                                                                                                                                                                                                                                                
  return 0;                                                                                                                                                                                                                                                                                     
}                                                                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                                                
Determining if the function pthread_create exists in the pthreads failed with the following output:                                                                                                                                                                                             
Change Dir: /hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp                                                                                                                                                                                             
                                                                                                                                                                                                                                                                                                
Run Build Command(s):/usr/bin/gmake cmTC_2b9ff/fast && /usr/bin/gmake  -f CMakeFiles/cmTC_2b9ff.dir/build.make CMakeFiles/cmTC_2b9ff.dir/build                                                                                                                                                  
gmake[1]: Entering directory '/hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp'                                                                                                                                                                          
Building C object CMakeFiles/cmTC_2b9ff.dir/CheckFunctionExists.c.o                                                                                                                                                                                                                             
/usr/bin/cc   -DCHECK_FUNCTION_EXISTS=pthread_create -o CMakeFiles/cmTC_2b9ff.dir/CheckFunctionExists.c.o -c /usr/share/cmake/Modules/CheckFunctionExists.c                                                                                                                                     
Linking C executable cmTC_2b9ff                                                                                                                                                                                                                                                                 
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_2b9ff.dir/link.txt --verbose=1                                                                                                                                                                                                              
/usr/bin/cc  -DCHECK_FUNCTION_EXISTS=pthread_create -rdynamic CMakeFiles/cmTC_2b9ff.dir/CheckFunctionExists.c.o -o cmTC_2b9ff  -lpthreads                                                                                                                                                       
/usr/bin/ld: cannot find -lpthreads                                                                                                                                                                                                                                                             
collect2: error: ld returned 1 exit status                                                                                                                                                                                                                                                      
gmake[1]: *** [CMakeFiles/cmTC_2b9ff.dir/build.make:106: cmTC_2b9ff] Error 1                                                                                                                                                                                                                    
gmake[1]: Leaving directory '/hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp'                                                                                                                                                                           
gmake: *** [Makefile:140: cmTC_2b9ff/fast] Error 2 
{code}

and if I create the C code shown above and build with `-lpthread` that works fine

So as I use cmake 3.19.7, is it an old cmake bug ?


Thanks for your help


best regards