lib: change xmalloc() to xvalloc() in sd_inode.c

The default_read_from_path() returned "Invalid argument" when we running
sheepdog cluster in test environment with sheep option "-D". The reason
is that the allocating memory for reading and writing btree-object is not
aligned. Therefore we should use xvalloc() instead of xmalloc() to allocate
aligned memory.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: go to panic if using unaligned memory in direct-io mode

When startup sheep with option "-D", the plain_store will check alignment of
read/write buffer, but if the buffer is not aligned, it will do nothing if
it dose not choose "--enable-debug" for configuration. Then sheep will open
file with "O_DIRECT" and call pread() with unaligned buffer, it will cause
error io and remove the disk from md which is totally wrong behavior.
So, we should make the check strong enough: sheep will panic directlly if
finding unaligned buffer.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: cleaning vdi deletion process, round 2

This patch cleans the vdi deletion process of sheep. Current vdi
deletion code creates one work for one deletion. It seems that there
is no reason to create so many work. This patch lets the code to
create only one work for vdi deletion request.

With this change, this patch improves simplicity of the code. For
example, the queue of struct deletion_info and related code are
removed.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: fix some compile errors

Both MIN/MAX and roundup are provided by glibc and would cause compile error
when we #include <sys/param.h>. Just use GNU version.

Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
add INFO level operation logging on each node.

Administrator couldn't track operation on each node without this function.
This function logs only administrative operation at rx_main and tx_main.
Operations that is not related to administrative operation are not
logged in this function. such as read/write etc.

Signed-off-by: Yoshinori Matsuo <matsuo.yoshinori@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: save dw->finish_fd to local variable

We cannot access dw after we call delete_vdis_done() because it may be
already freed.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
ops: add NULL check of req->op

req->op can be NULL when the requested opcode is invalid.  It's safe
to add NULL checks before accessing req->op in ops.c.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: add helper function to make sure that req->data is string

There is no guarantee that req->data is a string.  Actually, the
current code can cause a buffer overrun when, e.g.,
SD_OP_FORCE_RECOVER is requested.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
tests/functional: fix valgrind test

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
recovery: fix buffer overflow of rinfo->oids

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: initialize object cache info

This silences valgrind errors.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
fec: fix buffer overrun

After all the parities are set to out[], p will increase beyond
ctx->dp.  This adds a check for it.

This also adds assert() to make sure that we don't overrun the buffer.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: use correct error code in rebae_vdi()

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: fix vdi clone operation

- pass correct array size
- return FULL_VDI instead of wrong NO_BASE_VDI

Signed-off-by: Liu Yuan <namei.unix@gmail.com>
Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: changed end of auto-recovery message to notice

Sheep output the end of auto-recovery log message (all nodes are recovered)
at DEBUG level.
When sheep nodes are working in the INFO level or upper,
the operator could not get this message
even if operators want to know the completion of auto-recovery.

This patch raised the log message to the NOTICE level.

Signed-off-by: Yoshinori Matsuo <matsuo.yoshinori@lab.ntt.co.jp>
Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
lib: Remove executable stack comming from sha1_ssse3.S

Executable stack can be security hole, EX stack prevents working of NX.
More information can be found here:
https://wiki.gentoo.org/wiki/Project:Hardened/GNU_stack_quickstart
https://wiki.gentoo.org/wiki/Hardened/PaX_Utilities#Introduction

i686 architecture wasn't affected.

Signed-off-by: Marcin Mirosław <marcin@mejor.pl>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: fix error in sheepdog cluster recovery

Sheepdog failed to recover object when we running it on 5 servers cluster with
about 20G data by erasure-code mode.

The reason is in default_create_and_write(): it rename() obj to data-directory
and then set xattr of ec-index for it, this will leave a time-window for another
process to read the data-object but can't get xattr of ec-index. Then the
process will report get-xattr fail and remove the disk as it think it's an
io-error event.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
lib: fix compiler error in logger.c

When build sheepdog in centos6, it will report:

    logger.c: In function ‘gdb_cmd’:
    logger.c:746: error: ‘ARG_MAX’ undeclared (first use in this function)
    logger.c:746: error: (Each undeclared identifier is reported only once
    logger.c:746: error: for each function it appears in.)
    logger.c:746: warning: unused variable ‘cmd_str’
    logger.c: In function ‘sd_backtrace’:
    logger.c:797: error: ‘ARG_MAX’ undeclared (first use in this function)
    logger.c:797: warning: unused variable ‘cmd’

the reason has been explained here: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=518858
and the standard revolution is using 'sysconf()'.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: fix memory leak in _invert_mat()

First run swift interface on sheepdog, thean kill a node and wait for
recovery. After it, the RSS of sheep daemon will increase to about 8GB
which is a sign of memory leak.

Then we use valgrind to check sheepdog daemon and finally find out the
leak point: _invert_mat() in lib/fec.c

Signed-off-by: Robin DOng <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: fix memory leak in local_req_async_main()

We find out a memory-leak point in local_req_async_main() by valgrind.
After fix it, the RSS of sheep daemon could keep stablely for press-testing.

Signed-off-by: Robin DOng <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>