Installing Apache Pig 0.17.0 on Windows 10
By: Hadi Fadlallah
Tags: Hadoop, Hadoop 3, Apache Pig, Big Data

This article is a part of a series that we are publishing on TowardsDataScience.com that aims to illustrate how to install Big Data technologies on Windows operating system.

Previously published:

In this article, we will provide a step-by-step guide to install Apache Pig 0.17.0 on Windows 10.

1. Prerequisites

1.1. Hadoop Cluster Installation

Apache Pig is a platform build on the top of Hadoop. You can refer to our previously published article to install a Hadoop single node cluster on Windows 10.

Note that the Apache Pig latest version 0.17.0 supports Hadoop 2.x versions and still facing some compatibility issues with Hadoop 3.x. In this article, we will only illustrate the installation since we are working with Hadoop 3.2.1

1.2. 7zip

7zip is needed to extract .tar.gz archives we will be downloading in this guide.

2. Downloading Apache Pig

To download the Apache Pig, you should go to the following link:

Figure 1 — Apache Pig releases directory

If you are looking for the latest version, navigate to “latest” directory, then download the pig-x.xx.x.tar.gz file.

Figure 2 — Download Apache Pig binaries

After the file is downloaded, we should extract it twice using 7zip (using 7zip: the first time we extract the .tar.gz file, the second time we extract the .tar file). We will extract the Pig folder into “E:\hadoop-env” directory as used in the previous articles.

3. Setting Environment Variables

After extracting Derby and Hive archives, we should go to Control Panel > System and Security > System. Then Click on “Advanced system settings”.

Figure 3 — Advanced system settings

In the advanced system settings dialog, click on “Environment variables” button.

Figure 4 — Opening environment variables editor

Now we should add the following user variables:

Figure 5 — Adding user variables

PIG_HOME: “E:\hadoop-env\pig-0.17.0”

Figure 6 — Adding PIG_HOME variable

Now, we should edit the Path user variable to add the following paths:

%PIG_HOME%\bin

Figure 7 — Editing Path variable

4. Starting Apache Pig

After setting environment variables, let's try to run Apache Pig.

Note: Hadoop Services must be running

Open a command prompt as administrator, and execute the following command

pig -version

You will receive the following exception:

'E:\hadoop-env\hadoop-3.2.1\bin\hadoop-config.cmd' is not recognized as an internal or external command,

operable program or batch file.

'-Xmx1000M' is not recognized as an internal or external command,

operable program or batch file.

Figure 8 — Pig exception

To fix this error, we should edit the pig.cmd file located in the “pig-0.17.0\bin” directory by changing the HADOOP_BIN_PATH value from “%HADOOP_HOME%\bin” to “%HADOOP_HOME%\libexec”.

Now, let's try to run the “pig -version” command again:

Figure 9 — Pig installation validated

The simplest way to write PigLatin statements is using Grunt shell which is an interactive tool where we write a statement and get the desired output. There are two modes to involve Grunt Shell:

Local: All scripts are executed on a single machine without requiring Hadoop. (command: pig -x local) MapReduce: Scripts are executed on a Hadoop cluster (command: pig -x MapReduce)

Since we have installed Apache Hadoop 3.2.1 which is not compatible with Pig 0.17.0, we will try to run Pig using local mode.

Figure 10 — Starting Grunt Shell in local mode

5. References