twitter/hadoop-lzo — pull request #78 by cnauroth (2013-09-18 00:02:37+00:00)
Title: Support Windows native build for hadoop-lzo.

This patch updates the lzo codec for compatibility with Windows.

Summary of changes in this patch:

1. Add Visual Studio project files.
2. Split pom.xml Ant tasks into 2 platforms-specific paths: native-non-win (existing build) and win (new Windows logic to call msbuild).
3. Update build instructions with some Windows-specific information.
4. Update C code to enable calling Windows dynamic library loading functions.
5. Change call to lzo1x_999_compress_level to use dynamic linking. Otherwise, I'd get a linker error on Windows.
6. The MS compiler doesn't support C99, so I made a lot of small changes to satisfy the compiler, such as moving variable declarations to the top of functions.
7. Some minor refactoring to lift function pointer specs into typedefs in a header.
8. Added a log4j.properties to see log output during test runs.

I ran a full build and all unit tests passed on Windows and Linux with both default-profile and hadoop-old. To run the tests on Windows, you'll need to set the C_INCLUDE_PATH environment variable to the location of the lzo headers. (This is consistent with the existing convention for the Linux build.) You'll also need to set PATH to include the location of your lzo2.dll. Additionally, you'll also need to set the HADOOP_HOME environment variable so that winutils.exe can be found while running the tests. I documented all of this in README.md.

I also ran end-to-end testing of some word count and grep map-reduce jobs using the codec and both indexers.