IB/qib: Reduce sdma_lock contention

Profiling has shown that sdma_lock is proving a bottleneck for
performance. The situations include:
- RDMA reads when krcvqs > 1
- post sends from multiple threads

For RDMA read the current global qib_wq mechanism runs on all CPUs
and contends for the sdma_lock when multiple RMDA read requests are
fielded on differenct CPUs. For post sends, the direct call to
qib_do_send() from multiple threads causes the contention.

Since the sdma mechanism is per port, this fix converts the existing
workqueue to a per port single thread workqueue to reduce the lock
contention in the RDMA read case, and for any other case where the QP
is scheduled via the workqueue mechanism from more than 1 CPU.

For the post send case, This patch modifies the post send code to
test for a non empty sdma engine.   If the sdma is not idle the
(now single thread) workqueue will be used to trigger the send engine
instead of the direct call to qib_do_send().

Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
IB/qib: Add support for per-device/per-port parameters

Add support for per-device/per-port driver parameters
allowing users to specify different values for different
ports/devices. All converted parameters will behave as
expected if the old parameter format is used.

New per-device/per-port parameter format:
    <param>=[<d>,]<u>[:<p>]=<v>[,<u>[:<p>]=<v>]
where:
    <param> - driver parameter name
    <d> - default value applicable to all devices/ports
    <u> - unit number (zero-based)
    <p> - port number (1-based)
    <v> - numeric value

Signed-off-by: Mitko Haralanov <mitko@qlogic.com>
IB/qib: Convert krvqs module parameter to per-port

Convert the krcvqs module paratemter from a per-unit
parameter to a per-port one. This allows the user to
specify kernel receive contexts allocation on each
individual port.

Signed-off-by: Mitko Haralanov <mitko@qlogic.com>
IB/qib: Added congestion control agent implementation

Added a congestion control agent in the driver that handles
gets and sets from the congestion control manager in the
fabric for the Performance Scale Messagin (PSM) library.

Signed-off-by: Ramkrishna Vepa <ram.vepa@qlogic.com>
IB/qib: Fix message rate degradation on AMD Magny Cours & Interlagos

The optimal message rate performance with PSM on AMD Magny Cours and
Interlagos systems is acheived when the receive header queue, eager
buffers and the send pioavail register are allocated on the local
NUMA node. The driver allocates these memories on the NUMA node on
which the process is running. The fix is to allow for both these
memory allocation schemas.
If the numa_aware loadable parameter is not set which would be the
case for the AMD systems the driver allocates these memories on
the HCA's local NUMA node which can be overriden by a loadable
parameter, numa_node. A configuration of an invalid NUMA node
causes the driver to allocate memories on the local NUMA node.
If the numa_aware loadable parameter is set by the user or is
set automatically by the driver then the receive buffers are
allocated on the numa node on which the process is running and
the send pioavail register is allocated on the local NUMA node.

Reviewed-by: Mitko Haralanov <mitko.haralanov@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Ramkrishna Vepa <ramkrishna.vepa@intel.com>
IB/qib: Remove unnecessary syslog entries from driver

Driver was filling up the syslog with entries such as the following
every time a PSM process started up:
Apr  3 13:19:32 romley-02 kernel: ib_qib 0000:07:00.0: [39672] unit=0 CPU=0 NUMA id=0
Apr  3 13:21:52 romley-02 kernel: ib_qib 0000:07:00.0: [39741] unit=0 CPU=0 NUMA id=0
Apr  3 13:22:32 romley-02 kernel: ib_qib 0000:07:00.0: [39768] unit=0 CPU=0 NUMA id=0
...
Created a new macro, qib_dev_trace, that writes into the trace buffer only in these
cases.

Reviewed-by: Mitko Haralanov <mitko.haralanov@intel.com>
Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Ramkrishna Vepa <ramkrishna.vepa@intel.com>
IB/qib: Add dual rail NUMA awareness for PSM processes

The driver currently selects a HCA based on the algorithm that
PSM chooses, contexts within a HCA or across. The HCA can also
be chosen by the user. Either way, this patch assigns a CPU on
the NUMA node local to the selected HCA. This patch also tries
to select the HCA closest to the NUMA node of the CPU assigned
via taskset to PSM process. If this HCA is unusable then
another unit is selected based on the algorithm that is
currently enforced or selected by PSM - round robin
context selection 'within' or 'across' HCA's.

Fixed a bug wherein contexts are setup on the NUMA node on
which the processes are opened (setup_ctxt()) and not on the
NUMA node that the driver recommends the CPU on.

Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Ramkrishna Vepa <ramkrishna.vepa@intel.com>
IB/qib: Fix an incorrect log message

There is a cut-and-paste typo in the function qib_pci_slot_reset()
where it prints that the "link_reset" function is called rather than
the "slot_reset" function.  This makes the message misleading.

Signed-off-by: Betty Dall <betty.dall@hp.com>
IB/qib: optimize CQ callbacks

The current workqueue implemention has the following performance
deficiencies on QDR HCAs:
- The CQ call backs tend to run on the CPUs processing the
receive queues
- The single thread queue isn't optimal for multiple HCAs

This patch adds a dedicated per HCA bound thread to process CQ callbacks.

Reviewed-by: Ramkrishna Vepa <ramkrishna.vepa@intel.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
IB/qib: This patch improves SDMA performance.

1. The code accepts chunks of messages, and splits the chunk into
packets when converting packets into sdma queue entries, adjacent
packets will use user buffer pages smartly to avoid pinning the
same page multiple times.

2. Instead of discarding all the work when SDMA queue is full, the
work is saved in a pending queue, whenever there is enough SDMA
queue free entries, pending queue is directly put onto SDMA queue.

3. An interrupt handler is used to progress this pending queue.

Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: CQ Tang <cq.tang.@intel.com>
IB/qib: Branding changes to change Qlogic to Intel/Qib.

Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Vinit Agnihotri <vinit.abhay.agnihotri@intel.com>
IB/qib: Add tunings for new Dell backplane

The Dell blade chassis got an updated backplane which requires
new transmitter tuning settings.

Signed-off-by: Mitko Haralanov <mitko.haralanov@intel.com>
IB/qib: use %u to print out unsigned stats

Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Dean Luick <dean.luick@intel.com>
IB/qib Unconditionally log SDMA errors

Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Dean Luick <dean.luick@intel.com>
IB/qib: Fix module-level leak

The vzalloc()'ed field physshadow is leaked on module unload.
This patch adds vfree after the sibling page shadow is freed.

Reported-by: Dean Luick <dean.luick@intel.com>
Reviewed-by: Dean Luick <dean.luick@intel.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Roland Dreier <roland@purestorage.com>
IB/qib: Change SDMA progression mode depending on single- or mulit-rail.

Improve performance by changing the behavour of the driver when all SDMA
descriptors are in use, and the processes adding new descriptors are
single- or multi-rail.

For single-rail processes, the driver will block the call and finish
posting all SDMA descriptors onto the hardware queue before returning
back to PSM.  Repeated kernel calls are slower than blocking.

For multi-rail processes, the driver will return to PSM as quick as
possible so PSM can feed packets to other rail.  If all hardware queues
are full, PSM will buffer the remaining SDMA descriptors until notified
by interrupt that space is available.

This patch builds a red-black tree to track the number rails opened by
a particular PID. If the number is more than one, it is a multi-rail
PSM process, otherwise, it is a single-rail process.

Reviewed-by: Dean Luick <dean.luick@intel.com>
Reviewed-by: John A Gregor <john.a.gregor@intel.com>
Reviewed-by: Mitko Haralanov <mitko.haralanov@intel.com>
Signed-off-by: CQ Tang <cq.tang@intel.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
IB/qib: Convert qib_user_sdma_pin_pages() to use get_user_pages_fast()

qib_user_sdma_queue_pkts() gets called with mmap_sem held for
writing. Except for get_user_pages() deep down in
qib_user_sdma_pin_pages() we don't seem to need mmap_sem at all.  Even
more interestingly the function qib_user_sdma_queue_pkts() (and also
qib_user_sdma_coalesce() called somewhat later) call copy_from_user()
which can hit a page fault and we deadlock on trying to get mmap_sem
when handling that fault.

So just make qib_user_sdma_pin_pages() use get_user_pages_fast() and
leave mmap_sem locking for mm.

This deadlock has actually been observed in the wild when the node
is under memory pressure.

Cc: <stable@vger.kernel.org>
Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Roland Dreier <roland@purestorage.com>
IB/qib: Correct style issues in qib_iba7322.c

Signed-off-by: Mike Marciniszyn <mike.marciniszyn@qlogic.com>
IB/qib: Fix error return code in qib_init_7322_variables()

Convert a 0 error return code to a negative one, as returned elsewhere
in the function.

A simplified version of the semantic match that finds this problem is
as follows: (http://coccinelle.lip6.fr/)

// <smpl>
@@
identifier ret;
expression e,e1,e2,e3,e4,x;
@@

(
 if (\(ret != 0\|ret < 0\) || ...) { ... return ...; }
  |
 ret = 0
)
... when != ret = e1
 *x = \(kmalloc\|kzalloc\|kcalloc\|devm_kzalloc\|ioremap\|ioremap_nocache\
 |devm_ioremap\|devm_ioremap_nocache\)(...);
... when != x = e2
when != ret = e3
*if (x == NULL || ...)
 {
  ... when != ret = e4
  *  return ret;
 }
// </smpl>

Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Acked-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Roland Dreier <roland@purestorage.com>
IB/qib: remove ioremap_wc()

It was a RHEL5 backport N/A for RHEL6.

Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>