util: remove unused functions

If we need it future, we can bring them back on demand from git repo. No need
to keep it in the source file.

Signed-off-by: Liu Yuan <namei.unix@gmail.com>
Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep/md: change process of for_each_object_in_wd() to multi-threads

In our test environment, we upload 6TB data and kill one node, then the sheep
daemons on each server will try to rename files from 'data' to '.stale', but
the rename operations cost almost half an hour. And in this half an hour, the
whole cluster can't be read or write.

To accelerate the speed of renames, we change for_each_object_in_wd() to
multi-threads, which is one thread for one disk.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
dog/farm: register vdi for 'snapshot load operation'

This will git rid of need that we have to manually check copy_policy of requests
in the plain store.

Reviewed-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: kill is_erasure_obj()

Since we already register vdi for 'cluster snapshot load', we don't need
ugly is_erasure_obj() any more. Now we can use is_erasure_oid() to reliably
check if oid is ec object.

Reviewed-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: store ec_index in the pathname instead of in xattr

This patch solves a performance issue that in the recovery, update_epoch() is
called to iterate all the objects to move stale objects into stale directories
this is fast for replication objects because oid_stale() doesn't access disks
at all. But for erasure objects, we have to read xattr for each one. This is
considerably slow for massive data deployment. For e.g, it might take more than
one hour for 6T data in 6 nodes cluster. Since update_epoch() is required to
run in the main thread, this means we can't process any requests during the
first phase of recovery.

Instead of storing 'ec_index' in the xattr of the object, we directly store it
in the pathname of ec objects, e.g, xxxoidxxx_1 indicate its ec index is 1 and
it is an ec object. By this way, we

- simplify the read/write path a bit
- no need to read disks at all for update_epoch()

Thus largely boost the update_epoch() and also boost normal read/write a bit.

With this patch, the disk layout for ec objects is changed from

oid       --> oid_index       [data  object]
oid.epoch --> oid_index.epoch [stale object]

Reviewed-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: reload config after successful migration

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: migrate format from v3 to v4

The commit 79706e07a068 introduced incompatibility of store format
(store ec index in path instead of xattr). This patch produces
migration function for this change.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>