update hadoop-wordcount test to be able to run on hadoop 2.x. The hadoop and mapreduce library are no longer hard coded so they can be specified to point to the right path. The relative paths hdfs are changed to absolute paths.
A sample command to run the test on hadoop 2.x is
TESTDIR=/home/test HADOOP_HOME=/usr/lib/hadoop HADOOP_MR_HOME=/usr/lib/hadoop-mapreduce sh workunits/hadoop-wordcount/test.sh starting hadoop-wordcount test

Signed-off-by: rootfs <hchen@redhat.com>
osd: add config for osd_max_object_name_len = 2048 (was hard-coded at 4096)

Previously we had a hard coded limit of 4096.  Objects > 3k crash the OSD
when running on ext4, although they probably work on xfs.  But rgw only
generates objects a bit over 1024 bytes (maybe 1200 tops?), so let set a
more reasonable limit here.  2048 is a nice round number and should be
safe.

Add a test.

Fixes: #8174
Signed-off-by: Sage Weil <sage@redhat.com>
os: add ObjectStore::get_max_attr_name_length()

Most importantly, capture that attrs on FileStore can't be more than about
100 chars.  The Linux xattrs can only be 128 chars, but we also have some
prefixing we do.

Signed-off-by: Sage Weil <sage@redhat.com>
osd: add config for osd_max_attr_name_len = 100

Set a limit on the length of an attr name.  The fs can only take 128
bytes, but we were not imposing any limit.

Add a test.

Reported-by: Haomai Wang <haomaiwang@gmail.com>
Signed-off-by: Sage Weil <sage@inktank.com>
utf8: export encode_utf8() and decode_utf8()

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: improve delmited listing of bucket

If found a prefix, calculate a string greater than that so that next
request we can skip to that. This is still not the most efficient way to
do it. It'll be better to push it down to the objclass, but that'll
require a much bigger change.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: add NextMarker param for bucket listing

Partially fixes #8858.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: account common prefixes for MaxKeys in bucket listing

To be more in line with the S3 api. Beforehand we didn't account the
common prefixes towards the MaxKeys (a single common prefix counts as a
single key). Also need to adjust the marker now if it is pointing at a
common prefix.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: list extra objects to set truncation flag correctly

Otherwise we end up returning wrong truncated value, and no data on the
next iteration.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: dump prefix unconditionally

As part of issue #8858, and to be more in line with S3, dump the Prefix
field when listing bucket even if bucket is empty.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Revert "debian: move ceph-dencoder to ceph from ceph-common"

This reverts commit b37e3bde3bd31287b11c069062280258666df7c5.
Revert "ceph.spec: move ceph-dencoder to ceph from ceph-common"

This reverts commit 95f5a448b52db545a2b9bbad47fdb287254f93ea.
libosd_types, libos_types, libmon_types

Signed-off-by: Sage Weil <sage@redhat.com>
rgw: move a bunch of stuff into rgw_dencoder

This will help out ceph-dencoder ...

Signed-off-by: Sage Weil <sage@redhat.com>
ceph-dencoder: don't link librgw.la (and rados, etc.)

Signed-off-by: Sage Weil <sage@redhat.com>
osd/MemStore: handle collection_move_rename within the same collection

Signed-off-by: Sage Weil <sage@redhat.com>
os/MemStore: fix wrlock ordering checks

We can't compare the shared_ptrs themselves; we need to compare the
addresses of the actual objects.

Signed-off-by: Sage Weil <sage@redhat.com>
os/MemStore: copy attrs on clone

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
ceph_test_objectstore: test memstore

Signed-off-by: Sage Weil <sage@redhat.com>
os/FileStore: fix max object name limit

Our max object name is not limited by file name size, but by the length of
the name we can stuff in an xattr.  That will vary from file system to
file system, so just make this 4096.  In practice, it should be limited
via the global tunable, if it is adjusted at all.

Signed-off-by: Sage Weil <sage@redhat.com>