IB/qib: MADs should return failure with misset mkeys

If a MAD is sent directly to the local HCA rather than placed on a QP and the MAD fails mkey checks,
there is no means to generate a timeout for the client, which may hang.  Instead we report
IB_MAD_RESULT_FAILURE, which operates the same for on-the-wire packets, but will generate a send
failure back to the client.

Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Jim Foraker <foraker1@llnl.gov>
IB/qib: correct ordering of reregister vs. port active event

When a port first goes active with SMA Set(PortInfo) and reregister bit set, the
driver sends up the reregister event followed by a port active event.

The problem is that in response to reregister event most apps try to issue a SA
query of some sort, but that fails because port is not active.

The qib driver needs to a trivial change to correct this behavior.

This issue has been there for a while, however the recent serdes
work has probably made the delay between the reregister event and the active
event larger and hence opened the race far enough so that its being seen more
often.

This fragment in subn_set_portinfo needs to move:

if (pip->clientrereg_resv_subnetto & 0x80) {
        clientrereg = 1;
        event.event = IB_EVENT_CLIENT_REREGISTER;
        ib_dispatch_event(&event);
}

This patch moves the code to just before the nested call to subn_get_portinfo():

    TO HERE
    ret = subn_get_portinfo(smp, ibdev, port);

The patch also changes the clientrereg local to a u8 and saves off the rereg bit into
it.  The code following the nested subn_get_portinfo() now restores that bit per o14-12.2.1
with a logical from that copy.

Reviewed-by: Ram Vepa <ram.vepa@qlogic.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@qlogic.com>
IB/qib: Change driver default for wc_pat to 2 for cross CPU inconsistencies

The fix for this problem has been committed by Suresh Siddha of Intel to 2.6.38.3
(https://lkml.org/lkml/2011/4/13/232) but there could be kernels without the patch
backported. This patch works around this bug without any intervention by the
customer or our support.

Signed-off-by: Ramkrishna Vepa <ram.vepa@qlogic.com>
IB/qib: fix DMA_nnBIT_MASK warnings

Change the code to use DMA_BIT_MASK(32) and DMA_BIT_MASK(64)
instead of DMA_32_BIT_MASK and DMA_64_BIT_MASK respectively.

Reviewed-by: Ram Vepa <ram.vepa@qlogic.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@qlogic.com>
IB/qib: include slab.h as necessary for newer kernels

Fix syntax errors in newer kernels when slab.h isn't
included properly.

Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
IB/qib: optimize pio ack buffer allocation

This patch optimizes pio buffer allocation in the kernel.

For the qib, kernel pio buffers are used for sending acks.  The code
to allocate the buffer would always start at 0 until it found a buffer.

This means that an average of 64 comparisions were done on each allocate, since
the busy bit won't be cleared until the bits are refreshed when buffers are
exhausted.

This patch adds two new fields in the devdata struct, last_pio and
min_kernel_pio.  last_pio is the last buffer that was allocated.
min_kernel_pio is the lowest potential available buffer.

min_kernel_pio is modifed as contexts are allocated and
deallocted.

Reviewed-by: Ram Vepa <ram.vepa@qlogic.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@qlogic.com>
IB/qib: Fix QLE734X link cycling

The SERDES was using the incorrect Frequency Loop
Bandwidth setting causing the link to cycle through
the Physical link negotiation state machine.
Fixing the Frequency Loop Bandwidth setting in the
SERDES helps the link come up faster and more
reliably.

Signed-off-by: Mitko Haralanov <mitko@qlogic.com>
IB/qib: Display correct value for number of contexts

A "fix" for a bug with the number of contexts on a
single-port board caused the calculation to be off by
one, which causes problems with the upper layers. The
same problem exists for number of free contexts, which
is also fixed here.

Signed-off-by: Mitko Haralanov <mitko.haralanov@intel.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
IB/qib: patch for ordering issue with 16 bytes writes at tail of move

This patch corrects an issue where a 16 byte move at the tail of a bypass move
will store the last 8 bytes prior to the preceding 8 bytes.

OpenMPI must do some memory polling on memory being updated with an RDMA write
and the sequencing causes issues with the subsequent processing.

The flaw exists in both memcpy_cachebypass() and memcpy_cachebypass2().

Reviewed-by: Mike Heinz <michael.william.heinz@intel.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
IB/qib: patch for bypass selection logic

This patch corrects the selection logic for using the two bypass functions to
insure that the memcpy_cachebypass2() is only called when the pd is known and
it is NOT a user pd.

The old code might mistakenly call the write bypass copy and foil memory
polling when the pd is NULL.

Reviewed-by: Mike Heinz <michael.william.heinz@intel.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
IB/qib: fix UC MR refs for immediate operations

An MR reference leak exists handling UC RDMA writes with
immediate data by manipulating the reference counts as if the
operation had been a send.

This patch moves the last_imm label so that the RDMA write
operations with immediate data converge at the cq building code.
The copy/mr deref code is now done correctly prior to the branch
to last_imm.

Reviewed-by: Edward Mascarenhas <edward.mascarenhas@intel.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
IB/qib: Avoid returning EBUSY from MR deregister

A timing issue can occur where qib_mr_dereg can
return -EBUSY if the MR use count has is not zero.

This can occur if the MR is deregistered while RDMA
read response packets are being progressed from the
SDMA ring.  The suspicion is that the peer sent an RDMA
read request, which has already been copied across to the
peer.  The peer sees the completion of his request and
then communicates to the responder that the MR is not
needed any longer.  The responder tries to deregister the
MR, catching some responses remaining in the SDMA ring holding
the MR use count.

The code now uses a get/put paradigm to track MR use
counts and coordinates with the MR de-registration
process using a completion when the count has reached
zero.  A timeout on the delay is in place to catch
other EBUSY issues.

The reference count protocol is as follows:
- The return to the user counts as 1
- A reference from the lk_table or the qib_ibdev counts as 1.
- Transient I/O operations increase/decrease as necessary

Additional cleanup:
- Push dma_mr key allocation into qib_keys.c
- Simply interface to alloc/dealloc lkey routines

Reviewed-by: Ram Vepa <ram.vepa@qlogic.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
IB/qib: RCU locking for MR validation

Profiling indicates that MR validation locking is expensive.  The
MR table is largely read-only and is a suitable candidate for RCU
locking.

The patch uses RCU locking during validation to eliminate one
lock/unlock during that validation.

Reviewed-by: Mike Heinz <michael.william.heinz@intel.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
IB/qib: fix compliance test failure C14-024#06_LocalPortNum

Commit 7b123f8b ("IB/qib: MADs with misset M_Keys should return failure")
introduced a return code assignment that unfortunately introduced
an unconditional exit for the routine due to the lack of braces.

This patch adds the braces to correct the original patch.

Reviewed-by: Dean Luick <dean.luick@intel.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
IB/qib: fix local access validation for user MRs

Commit fb7ff000 ("IB/qib: RCU locking for MR validation") introduced
an error that broke user post sends.  The proper validation of the MR
was lost in the patch.

This patch corrects that validation.

Reviewed-by: Dean Luick <dean.luick@intel.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
IB/qib: Add a qib driver version

Signed-off-by: Dean Luick <dean.luick@intel.com>