Hive — How to install in 5 Steps in Windows 10
By: Shashank Singhal
Tags: Big Data, Easy, MySQL, Apache, Installation

4. Editing Hive

Once we have configured the environment variables next step is to configure Hive. It has 7 parts:-

4.1 Replacing bins

First step in configuring the hive is to download and replace the bin folder.

* Go to this GitHub Repo and download the bin folder as a zip.

* Extract the zip and replace all the files present under bin folder to %HIVE_HOME%\bin

Note:- If you are using different version of HIVE then please search for its respective bin folder and download it.

4.2 Creating File Hive-site.xml

Now we need to create the Hive-site.xml file in hive for configuring it :-

(We can find these files in Hive -> conf -> hive-default.xml.template)

We need to copy the hive-default.xml.template file and paste it in the same location and rename it to hive-site.xml. This will act as our main Config file for Hive.

Fig. 11:- Creating Hive-site.xml

4.3 Editing Configuration Files

4.3.1 Editing the Properties

Now Open the newly created Hive-site.xml and we need to edit the following properties

<property>

<name>hive.metastore.uris</name>

<value>thrift://<Your IP Address>:9083</value>



<property>

<name>hive.downloaded.resources.dir</name>

<value><Your drive Folder>/${hive.session.id}_resources</value>



<property>

<name>hive.exec.scratchdir</name>

<value>/tmp/mydir</value>

Replace the value for <Your IP Address> with the IP Address of your System and replace <Your drive Folder> with the Hive folder Path.

4.3.2 Removing Special Characters

This is a short step and we need to remove all the character present in the hive-site.xml file.

4.3.3 Adding few More Properties

Now we need to add the following properties as it is in the hive-site.xml File.

<property>

<name>hive.querylog.location</name>

<value>$HIVE_HOME/iotmp</value>

<description>Location of Hive run time structured log file</description>

</property> <property>

<name>hive.exec.local.scratchdir</name>

<value>$HIVE_HOME/iotmp</value>

<description>Local scratch space for Hive jobs</description>

</property> <property>

<name>hive.downloaded.resources.dir</name>

<value>$HIVE_HOME/iotmp</value>

<description>Temporary local directory for added resources in the remote file system.</description>

</property>

Great..!!! We are almost done with the Hive part, for configuring MySQL database as Metastore for Hive, we need to follow below steps:-

4.4 Creating Hive User in MySQL

The next important step in configuring Hive is to create users for MySQL.

These Users are used for connecting Hive to MySQL Database for reading and writing data from it.

Note:- You can skip this step if you have created the hive user while SQOOP installation.

Firstly, we need to open the MySQL Workbench and open the workspace(default or any specific, if you want). We will be using the default workspace only for now.

Fig 12:- Open MySQL Workbench

Now Open the Administration option in the Workspace and select Users and privileges option under Management.

Fig 13:- Opening Users and Privileges

Now select Add Account option and Create an new user with Login Name as hive and Limit to Host Mapping as the localhost and Password of your choice.

Fig 14:- Creating Hive User

Now we have to define the roles for this user under Administrative Roles and select DBManager , DBDesigner and BackupAdmin Roles

Fig 15:- Assigning Roles

Now we need to grant schema privileges for the user by using Add Entry option and selecting the schemas we need access to.

Fig 16:- Schema Privileges

I am using schema matching pattern as %_bigdata% for all my bigdata related schemas. You can use other 2 options also.

After clicking OK we need to select All the privileges for this schema.

Fig 17:- Select All privileges in the schema

Click Apply and we are done with the creating Hive user.

4.5 Granting permission to Users

Once we have created the user hive the next step is to Grant All privileges to this user for all the Tables in the previously selected Schema.

Open the MySQL cmd Window. We can open it by using the Window’s Search bar.

Fig 18:- MySQL cmd

Upon opening it will ask for your root user password(created while setting up MySQL).

user password(created while setting up MySQL). Now we need to run the below command in the cmd window.

grant all privileges on test_bigdata.* to 'hive'@'localhost';

where test_bigdata will be you schema name and hive@localhost will be the user name @ Host name.

4.6 Creating Metastore

Now we need to create our own metastore for Hive in MySQL..

Firstly, we need to create a database for metastore in MySQL OR we can use the one which used in previous step test_bigdata in my case.

Now Navigate to the below path

hive -> scripts -> metastore -> upgrade -> mysql and execute the file hive-schema-3.1.0.mysql in MySQL in your database.

Note:- If you are using different Database, select the folder for same in upgrade folder and execute the hive-schema file.

4.7 Adding Few More Properties(Metastore related Properties)

Finally, we need to open our hive-site.xml file once again and make some changes their, these are related to Hive metastore that’s why did not add them in starting so as to distinguish between the different set of properties.

<property>

<name>javax.jdo.option.ConnectionUserName</name>

<value>hive</value>

<description>Username to use against metastore database</description>

</property>



<property>

<name>javax.jdo.option.ConnectionURL</name>

<value>jdbc:mysql://localhost:3306/<Your Database>?createDatabaseIfNotExist=true</value>

<description>

JDBC connect string for a JDBC metastore.

To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.

For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.

</description>

</property>



<property>

<name>hive.metastore.warehouse.dir</name>

<value>hdfs://localhost:9000/user/hive/warehouse</value>

<description>location of default database for the warehouse</description>

</property>



<property>

<name>javax.jdo.option.ConnectionPassword</name>

<value><Hive Password></value>

<description>password to use against metastore database</description>

</property>



<property>

<name>datanucleus.schema.autoCreateSchema</name>

<value>true</value>

</property>

<property>

<name>datanucleus.schema.autoCreateTables</name>

<value>True</value>

</property>



<property>

<name>datanucleus.schema.validateTables</name>

<value>true</value>

<description>validates existing schema against code. turn this on if you want to verify existing schema</description>

</property>

Replace the value for <Hive Password> with the hive user password that we created in MySQL user creation. And <Your Database> with the database that we used for metastore in MySQL.