Cache-inode-avl-hashing

Add murmur3 public domain files and test_mh_avl.c.
Update blocked in.  Some adjustment to invalidation should still be done.
Also, lock splitting and fill abstraction required.
Also, bugfix:  do not cast NFSv3|4 coookies to unsigned int in nfs3_op_Readdirplus
and nfs4_op_readdir.
Prevent assignment of 0, 1, or 2 as the key of any dirent in the
cache inode avl (however unlikely).
Remove avl_inline_lookup.  Replace a temporary LogCrit printing
cache_inode_readdir results with LogFullDebug.
Derive export cache_inode_policy in nfs_FhandleToCache.
Correct error handling on failure of nfs_Get_export_by_id, still
assuming that an error is fatal.
Cache Inode Rewrite

Rewrite the cache_inode layer extensively.  This rewrite accomplishes
several goals:

  * Replace the call to cache_inode_renew_entry on every entry with a
    single heavy-weight validation on cache_inode_get, and 'trust'
    bits that can be cleared by upcalls from the FSAL.

  * Replace the LRU/garbage collection functions with a new two-level,
    multilane LRU that doesn't impose so much performance overhead.

  * Allow greater concurrency through finer grained locking and
    eliminating the write-lock-then-downgrade-on-every-call pattern.

  * Introduce a weakref scheme, to allow references to entries
    that do not prevent recycling or freeing (such as in directories.)

  * Remove the parent list, vastly simplifying validation.

  * Create a more capable and flexible file descriptor cache and
    method of reaping FDs.

  * Implement a callback function in readdir so as to avoid allocating
    structures and passing them up or requiring the caller to free the
    lock on the directory explicitly.

In order to make rebasing and merging this thing less of an exercise
in tribulation and get rid of double-commits, I have squeezed all the
patches together.  Therefore, here are the contributors:

Matt Benjamin <matt@linuxbox.com> who originated the two-level LRU
design and wrote the initial implementation, hit upon the idea of
using weakrefs and implemented support for them, and doing the
directory integration.

Eitan Ben-Amos <eitanb@il.ibm.com> who provided sharp-eyed code
review and feedback.

Max Matveev <makc@gmx.co.uk> who provided very helpful feedback and
fixed a quite embarrassing bug in readdir.

Frank S. Filz <ffilzlnx@us.ibm.com> who took the time to integrate the
LRU pinning code into the State Abstraction Layer, debugged locking
functionality, and provided feedback in fixing lock ordering in
cache_inode_lru

Venkateswararao Jujjuri (JV) <jvrao@linux.vnet.ibm.com> and Rong Zeng
<rongzeng@us.ibm.com> who did extensive testing, audited lock use
throughout, and provided a largep ortion of the bug fixes.

Signed-off-by: Adam C. Emerson <aemerson@linuxbox.com>
Strict compile fix.

Signed-off-by: Venkateswararao Jujjuri "<jvrao@linux.vnet.ibm.com>
Make cache_inode_lru_unref lock the queue when decrementing from 1

This function was intended to lock the queue, but due to an oversight
it finds the queue and unlocks the queue, but never actually acquires
the mutex on it.

Thanks to Frank S. Filz for finding this.
Simple patch to fix FSAL_UP tests.

Signed-off-by: Lance Russell
Signed-off-by: Venkateswararao Jujjuri "<jvrao@linux.vnet.ibm.com>
Fix release_openstate to free share/deny modes

I pulled the copy of this function back out of next and changed it to
use the new locking.
fix differences found in the merged branch
 - remove share reservations when a client expires
 - pass an fd to FSAL_truncate
 - read and write use new check_special_stateid routine
 - fix for renaming directories
Take and release references in release_openstate

Signed-off-by: Adam C. Emerson <aemerson@linuxbox.com>
Make cache_inode_inc/dec_pin_ref also take/release an LRU ref

The caller should have his own LRU ref, so dec_pin_ref asserts
LRU ref is > 1. Not a perfect assert.

Signed-off-by: Frank Filz <ffilzlnx@us.ibm.com>
Added atomic_int_inc and atomic_int_dec premitives and used
those to inc/dec  open_fd_count. Also added the inc/dec in other
places where the fd is opened/closed.

Signed-off-by: Venkateswararao Jujjuri "<jvrao@linux.vnet.ibm.com>
Signed-off-by: Jeremy Bongio <jbongio@us.ibm.com>
Signed-off-by: Jim Wahlig <jgwahlig@us.ibm.com>
Update documentation for readdir_populate
Fix the rm -rf bug

cache_inode_new_entry takes an argument giving additional information
about a file: such as special device numbers, symlink content, and
whether the directory is newly created.  In this case, a new directory
created through ganesha is known to be empty and can be marked as such
in the cache.  cache_inode_readdir_populate had a single create_arg
used throughout any given invocation.  It was initialized to mark a
found (not created) directory at the beginning, but if a symlink was
encountered, the link content was copied into it.  As the create_arg
is a union and the content was never cleared, this caused directories
to be spuriously treated as new creates.

To fix it, we set the flag indicating a newly created directory to
false for every entry that is not a symbolic link.
Add static specifier to FORCE_INLINE macro in murmur3.c.

Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Fix deadlock pointed out by Eitan Ben-Amos

Re-insert the get_located functionality for use in lookupp.  The
deadlock came about when a FSAL_lookup of ".." returned the same
handle as the entry being queried but the entry wasn't marked as a
root.  This would cause a lock order inversion in which
cache_inode_get would acquire the attribute lock while
cache_inode_lookupp held the content lock.
Fix build failure in support/nfs_client_id.c. Further issues at line:511

This is an obvious error, probably from a confused merge.
The following code starting at line 511.  It does look like a similar
merge confusion, then again it may not.  It needs to be checked.

Signed-off-by: Jim Lieb <jlieb@panasas.com>
Add cache_inode_puts in the fsal_up locking functions.

Signed-off-by: Venkateswararao Jujjuri "<jvrao@linux.vnet.ibm.com>