Package: pychbase
pychbaseThis is a Python C wrapper for MapRDB and HBase using thelibhbase C
API.pychbaseis modeled after the HappyBase API, but it does not usethrift, and is ideal for MapRDB.pychbaseis tested on Python 2.7 and MapR 5.1.LD_LIBRARY_PATHTo compile as well as importpychbase, yourLD_LIBRARY_PATHmust
have the directory withlibjvm.soon it, normally in either:$JAVA_HOME/lib/amd64/server$JAVA_HOME/jre/lib/amd64/serverIf you are using this with MapR, you must also have/opt/mapr/libon
yourLD_LIBRARY_PATHE.g,export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_HOME/lib/amd64/server:/opt/mapr/libInstallationInstallation on a MapR environmentNormally, the only environment variable to worry about on a MapR
environment is $PYCHBASE_LIBJVM_DIRexport PYCHBASE_LIBJVM_DIR=/usr/lib/jvm/jre-1.7.0/lib/amd64/server
virtualenv pychbase
cd pychbase
source bin/activate
pip install pychbase

# Or build it from source
git clone https://github.com/mkmoisen/pychbase.git
cd pychbase
python setup.py installInstallation on a Non-MapR environmentPlease see the end of the readme for Cloudera intallation notes.Run the testsTheconfig.pyfile in thetestsdirectory has two constants,ZOOKEEPERandTABLE_NAME, that probably won’t work if you run
the tests without modificationCreate atests/local_config.pyfile like the following:ZOOKEEPERS = 'localhost:7222'
TABLE_NAME = 'testpychbase'To run the tests, make sure to be in thetestsdirectory, or else
you will face import problems:cd tests
python tests.pyCurrentlynosetestswill not work without facing an import issue.UsageI have attempted to mimic the great HappyBase API as closely as
possible.Make sure to set the LD_LIBRARY_PATH environment variable:# MapR
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_HOME/jre/lib/amd64/server:/opt/mapr/lib

# Non MapR
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/jvm/jre-1.7.0/lib/amd64/server::/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/lib/native
export HBASE_LIB_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/lib/
# I've only gotten it to work on CDH4. If you are on CDH5 you'll need to mess around with the classpath some moreImports:from pychbase import Connection, Table, BatchTo create a connection:# On MapR, you don't need to specify the CLDBS/zookeepers:
connection = Connection()

# On Non-MapR environments, you'll need to specify the zookeepers:
connection = Connection('zookeeper-one:2181",zookeeper-two:2181",zookeeper-three:2181"')To create and delete tables:# Create a table named 'test_table' with a single column family named 'f' with no additional attributes
connection.create_table('test_table', {'f': {}})
connection.delete_table('test_table')To get a table to operate on:table = connection.table('test_table')To put, delete, and get from a table:# Put a row with the current timestamp
table.put('rowkey1', {'f:foo': 'bar', 'f:hai': 'bai'})
data = table.row('rowkey1')
assert data == {'f:foo': 'bar', 'f:hai': 'bai'}

# Get a row and only include a single column
data = table.row('rowkey1', columns=('f:foo',))
assert data == {'f:foo': 'bar'}

# Delete the row
table.delete('rowkey1')
data = table.row('rowkey1')
assert data == {}

# Put a row with a given timestamp
table.put('rowkey1', {'f:foo': 'bar'}, timestamp=1000)
table.put('rowkey1', {'f:foo': 'BAR'}, timestamp=10000)

# Get a row with a given timestamp and include its timestamp
data = table.row('rowkey1', timestamp=1000, include_timestamp=True)
assert data == {'f:foo': ('bar', 1000)}To scan:# Full table scan:
for row, data in table.scan():
    pass

# Scan with a start and stop:
for row, data in table.scan('foo', 'bar'):
    pass

# Scan with a row prefix:
for row, data in table.scan(row_prefix='baz'): # E.g., start='baz', stop='baz~'
    pass

# Scan with a filter: # Check out tests.py on how to use all of the filters
for row, data in table.scan(filter="SingleColumnValueFilter('f', 'foo', =, 'binary:foo')":
    pass

# Scan a table but return only row keys, no rows:
table.put('foo', {'f:foo': 'foo'}
table.put('foo1', {'f:foo': 'foo'}
table.put('foo2', {'f:foo': 'foo'}

rows = list(table.scan(only_rowkeys=True))
assert rows == ['foo', 'foo1', 'foo2']To count the number of rows in a table:# Full table count:
count = table.count()

# Count all rows with a start and stop
count = table.count('foo', 'bar')

# Count all rows whose row key starts with a row prefix:
count = table.count(row_prefix='baz') # E.g., start='baz', stop='baz~'

# Count all rows with a filter:
count = table.count(filter="SingleColumnValueFilter('f', 'foo', =, 'binary:foo')")To batch put:batch = table.batch()
datas = [
    ('foo', 'a', 'b'),
    ('foo1', 'a1', 'b1'),
    ('foo2', 'a2', 'b2'),
]

for data in datas:
    batch.put(data[0], {'f:foo': data[1], 'f:bar': data[2]})

errors = batch.send()

assert errors == 0To batch delete:batch = table.batch()
rows = ['foo', 'foo1', 'foo2']
for row in rows:
    batch.delete(row)

errors = batch.send()

assert errors == 0Note thatbatch.send()returns the number of errors that occurred,
if any. It is up to the client to ignore this or raise an exception.batch = table.batch()
batch.put('foo', {'f:foo', 'bar'})
batch.put('foo', 'invalid')

errors = batch.send()
assert errors == 1An additional helper method istable.delete_prefix(row_prefix),
which deletes all rows containing starting with the prefix.table.put('foo', {'f:foo', 'foo'}
table.put('foo1', {'f:foo', 'foo'}
table.put('foo2', {'f:foo', 'foo'}
table.put('foo3', {'f:foo', 'foo'}
table.put('bar', {'f:bar', 'bar'}

number_deleted = table.delete_prefix('foo')
assert number_deleted == 3

assert table.count() == 2Note that attempting to batch/put unescaped null terminators will result
in them being stripped. Attempting to use a row key with an unescaped
null terminator will raise a TypeException. It is the user’s duty to
escape null terminators before attempting to batch/put data.table.put('foo', {'f:foo\0bar': 'baz\0bak'})
data = table.row('foo')
assert data == {'f:foo': 'baz'}

table.put('bar', {'f:foo\\0bar': 'baz\00bak'})
data = table.row('foo')
assert data == {'f:foo\\0bar': 'baz\00bak'}HappyBase compatibilityOne goal of this library is to maintain compatibility with the APIs in
HappyBase.Check out__init__.pyto understand which features of HappyBase I
have not yet implemented.In the future, I will force print warnings to stderr in the event a user
uses an unimplemented feature.Non-MapR Installation and Environment Variables GuideI have not testedpychbaseheavily on Cloudera. I couldn’t get it
working on CDH5 due to a classpath issue withlibhbase, and and
while I was able to get it up and running with CDH4, some of the tests
are failing. It seems to me thatlibhbaseis not fully compatible
outside of MapR.export PYCHBASE_IS_MAPR=FALSE
export PYCHBASE_LIBJVM_DIR=/usr/lib/jvm/jre-1.7.0/lib/amd64/server
export PYCHBASE_INCLUDE_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/include
export PYCHBASE_LIBRARY_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/lib/native
virtualenv pychbase
cd pychbase
source bin/activate
pip install pychbase

# Or build it from source
git clone https://github.com/mkmoisen/pychbase.git
cd pychbase
python setup.py installPlease note that the following environment variables must be set in
order to installpychbasecorrectly:PYCHBASE_IS_MAPRPYCHBASE_LIBJVM_DIRPYCHBASE_INCLUDE_DIRPYCHBASE_LIBRARY_DIRPYCHBASE_IS_MAPRThis defaults toTRUE. IF you are using Cloudera/etc, make sure to:export PYCHBASE_IS_MAPR=FALSEPYCHBASE_LIBJVM_DIRThis is the directory that houses thelibjvm.sofile. Normally it is
in either:$JAVA_HOME/lib/amd64/server$JAVA_HOME/jre/lib/amd64/serverIfPYCHBASE_LIBJVM_DIRis not set, the installer will check ifJAVA_HOMEhas been set, and then try each of the above directories.
IfJAVA_HOMEis not set, it will attempt to default to/usr/lib/jvm/jre-1.7.0/.Example:export PYCHBASE_LIBJVM_DIR=/usr/lib/jvm/jre-1.7.0/lib/amd64/serverPYCHBASE_INCLUDE_DIRThis houses the/hbase/hbase.hand otherlibhbaseC header
files.IfPYCHBASE_IS_MAPRis true, this defaults to/opt/mapr/include.For Non-MapR environments, this must be set or the installation will
fail.Example on Cloudera:export PYCHBASE_INCLUDE_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/includePYCHBASE_LIBRARY_DIRThis houses either thelibMapRClient.sofile on MapR environments,
or thelibhbase.sofile on Non-MapR environments.IfPYCHBASE_IS_MAPRis true, this defaults to/opt/mapr/lib.For Non-MapR environments, this must be set or the installation will
fail.Example on Cloudera:export PYCHBASE_LIBRARY_DIR=/home/matthew/libhbase/target/libhbase-1.0-SNAPSHOT/lib/nativeAuthorMatthew MoisenLicenseMIT