Package: hivejdbc
hivejdbchivejdbc isdb-api-2.0compliantApache Hivedriver that supportskerberossslservice discovery via zookeeperhost-connection listand all other jdbc driver optionsinstallationpip3install hivejdbcCursorshivejdbccan use adictonarycursor if desired.fromhivejdbcimportconnect,DictCursorconn=connect('example.com','default',cursor=DictCursor)Cursors supportwithfromhivejdbcimportconnectconn=connect('example.com',database='default')withconn.cursor()ascursor:cursor.execute('select * from test.persons')rows=cursor.fetchall()Cursors are iterablefromhivejdbcimportconnectconn=connect('example.com',database='default')cursor=conn.cursor()cursor.execute('select * from test.persons')forrowincursor:print(row[0])cursor.close()Cursors Supportfetchone()fetchmany()fetchall()fromhivejdbcimportconnectconn=connect('example.com',database='default')cursor=conn.cursor()cursor.execute('select * from test.persons')cursor.fetchone()# fetch first row or Nonecursor.fetchmany(5)# fetch next 5 rowscursor.fetchall()# fetch remaining rows or empty listcursor.close()Connection Stringshivejdbcfeatures manyconnectfunction arguments. Many of these arguments can be ignored
and are simply present to offer the full options provided by theHivejdbc driver.To import thehivejdbcconnect function:fromhivejdbcimportconnectUnsecured Hive Instanceto connect to an unsecured hive instance listening on the default port10000, and thedefaultdatabase:conn=connect('example.com','default')unless all requiredhive-jarsare on the classpath already you'll need to define the driver pathJava usesjarfiles to combine many libraries into one. We'll use ourfatjarto provide all the required
dependencies in one place.Make sure you're using the correct driver for yourHiveversion.conn=connect('example.com','default',driver='hive-client-hive-2.1.1-hdfs-3.0.3-fatjar.jar')to connect with a custom port of10015conn=connect('example.com','default',port=10015)Username and Passwordconn=connect(host='example.com',database='default',port=10015,user='hive_user',password='secret')SSLIf the hive-server hassslenabled you'll need to provide ajkstrust store that contains the servers public
certificate.conn=connect(host='hive2.example.com',port=10015,database='default',driver='hive-client-hive-2.1.1-hdfs-3.0.3-fatjar.jar',ssl=True,trust_store='./truststore.jks',trust_password='changeit',principal='hive/hive2.example.com@EXAMPLE.COM',user_principal='hive/hive2.example.com',user_keytab='hive.keytab',realm='EXAMPLE.COM',kdc='kerberosdc.example.com:88')KerberosAuthenticating with kerberos can be done a few ways:get valid kerberos credentials viakinitbefore runninghivejdbcrely onhivejdbcto obtain kerberos credentials via auser-principalanduser-keytabprovided
to the program.Operating SystemkinitConnect to...asslenabled clustera secured cluster (kerberos)using the operating systems kerberos configuration
default locations are searched depending on platformusing the operating systemkinittokendefault locations for thetoken-cacheare searchedifkinithas not been performed, or atoken-cachecannot be found an exception will be thrownconn=connect(host='hive2.example.com',port=10015,database='default',driver='hive-client-hive-2.1.1-hdfs-3.0.3-fatjar.jar',ssl=True,trust_store='./truststore.jks',trust_password='changeit',principal='hive/hive2.example.com@EXAMPLE.COM')hivejdbcdoes thekinitviakeytaband a customkrb5.confconnect to...asslenabled clustera secured cluster (kerberos)using the operating systems kerberos configuration krb5.confusing akeytabfor authenticationthe keytab will be used to login via java's built-in kerberos implementation
avoiding the need for any operating system dependencywe will provide thekdcandrealmvia thekrb5_confargument
if we didn't providekrb5_confargument default locations would be searched within various system pathsconn=connect(host='hive2.example.com',port=10015,database='default',driver='hive-client-hive-2.1.1-hdfs-3.0.3-fatjar.jar',ssl=True,trust_store='./truststore.jks',trust_password='changeit',principal='hive/hive2.example.com@EXAMPLE.COM',krb5_conf='kerberos/custom_krb5.conf',user_principal='hive/hive2.example.com',user_keytab='user.keytab')hivejdbcdoes thekinitviakeytabwith nokrb5.confconnect to...an ssl enabled clustera secured cluster (kerberos)not using the operating system or relying on any of its configurationsmanually setting the realm, and the kerberos "kdc" to authenticate tousing a keytab for authenticationthis configuration is the most portable...conn=connect(host='hive2.example.com',port=10015,database='default',driver='hive-client-hive-2.1.1-hdfs-3.0.3-fatjar.jar',ssl=True,trust_store='./truststore.jks',trust_password='changeit',principal='hive/hive2.example.com@EXAMPLE.COM',user_principal='hive/hive2.example.com',user_keytab='hive.keytab',realm='EXAMPLE.COM',kdc='kerberosdc.example.com:88')Queries and ParametersFor these examples we'll setup atestdatabase with apersonstable...cursor=conn.cursor()cursor.execute('CREATE DATABASE IF NOT EXISTS test')cursor.execute('DROP TABLE IF EXISTS test.persons')cursor.execute('CREATE TABLE test.persons (name VARCHAR(64), age INT, address STRING, ''first TIMESTAMP, balance DECIMAL(12,2))')Our table sql will have 5 columns defined in the above statement:CREATETABLEtest.persons(nameVARCHAR(64),ageINT,addressSTRING,firstTIMESTAMP,balanceDECIMAL(12,2))single insertLet's insert a single record:cursor.execute('''INSERT INTO TABLE test.persons (name, age, address, first, balance)VALUES ('john doe', 35, '1583 Whistling Pines Dr, Redstone CO 80612', '08-22-1981 00:00:00', '100.10')''')positionalparameterized sql queryInsert a single record, using paramterized arguments that will automatically be escaped.This prevents sql injection as wellcursor.execute('''INSERT INTO TABLE test.persons (name, age, address, first, balance)VALUES (%s,%s,%s,%s,%s)''',['Kevin Jones',28,'802 1st st, Raleigh NC','12-23-2020 00:00:00',85.25])The signature ofexecuteis:defexecute(sql,params=None):""sqlis the sql statementparamsarenamed (dict)orpositional (sequence)arguments used by the sql statement for variable
substitutionnamedparameterized sql queryINSERTwith named parametersIn addition to positional parameters using%swe supportnamed parametersas well.You can see the named arguments are defined below in thesqlstatement as:(:name, :age, :addr, :dt, :bal)The second parameter to theexecutemethod is adictionarywhere the keys are equal to the parameters defined in the sqlcursor.execute('''INSERT INTO TABLE test.persons (name, age, address, first, balance)VALUES (:name, :age, :addr, :dt, :bal)''',{'name':'Bob Clark','age':41,'addr':'348 W Dickinson Rd, Norfolk VA','dt':'12-23-2020 00:00:00','bal':200.20})UsingexecutemanyYou can execute many queries in one python statement usingexecutemanyNote that this is for programmer ease of use; hive'sjdbcdriver does not supportbatch-mode, so this functionality is faked and is no more
efficient than executing 3 statements individually.cursor.executemany('''
INSERT INTO TABLE test.persons (name, age, address, first, balance)
VALUES (%s, %s, %s, %s, %s)
''', [
    ('john doe', 35, '1583 Whistling Pines Dr, Redstone CO 80612', '08-22-1981 00:00:00', 100.10),
    ('Kevin Jones', 28, '802 1st st, Raleigh NC', '12-23-2020 00:00:00', 85.25),
    ('Bob Clark', 41, '348 W Dickinson Rd, Norfolk VA', '12-23-2020 00:00:00', 200.20)
])