softirq: reduce latencies

In various network workloads, __do_softirq() latencies can be up
to 20 ms if HZ=1000, and 200 ms if HZ=100.

This is because we iterate 10 times in the softirq dispatcher,
and some actions can consume a lot of cycles.

This patch changes the fallback to ksoftirqd condition to :

- A time limit of 2 ms.
- need_resched() being set on current task

When one of this condition is met, we wakeup ksoftirqd for further
softirq processing if we still have pending softirqs.

Using need_resched() as the only condition can trigger RCU stalls,
as we can keep BH disabled for too long.

I ran several benchmarks and got no significant difference in
throughput, but a very significant reduction of latencies (one order
of magnitude) :

In following bench, 200 antagonist "netperf -t TCP_RR" are started in
background, using all available cpus.

Then we start one "netperf -t TCP_RR", bound to the cpu handling the NIC
IRQ (hard+soft)

Before patch :

RT_LATENCY,MIN_LATENCY,MAX_LATENCY,P50_LATENCY,P90_LATENCY,P99_LATENCY,MEAN_LATENCY,STDDEV_LATENCY
MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET
to 7.7.7.84 () port 0 AF_INET : first burst 0 : cpu bind
RT_LATENCY=550110.424
MIN_LATENCY=146858
MAX_LATENCY=997109
P50_LATENCY=305000
P90_LATENCY=550000
P99_LATENCY=710000
MEAN_LATENCY=376989.12
STDDEV_LATENCY=184046.92

After patch :

RT_LATENCY,MIN_LATENCY,MAX_LATENCY,P50_LATENCY,P90_LATENCY,P99_LATENCY,MEAN_LATENCY,STDDEV_LATENCY
MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET
to 7.7.7.84 () port 0 AF_INET : first burst 0 : cpu bind
RT_LATENCY=40545.492
MIN_LATENCY=9834
MAX_LATENCY=78366
P50_LATENCY=33583
P90_LATENCY=59000
P99_LATENCY=69000
MEAN_LATENCY=38364.67
STDDEV_LATENCY=12865.26

Change-Id: I0f04d2f06e08a896fa4abb62734983804b854c94
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: David Miller <davem@davemloft.net>
Cc: Tom Herbert <therbert@google.com>
Cc: Ben Hutchings <bhutchings@solarflare.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
vmscan: add block plug for page reclaim

per-task block plug can reduce block queue lock contention and increase
request merge.  Currently page reclaim doesn't support it.  I originally
thought page reclaim doesn't need it, because kswapd thread count is
limited and file cache write is done at flusher mostly.

When I test a workload with heavy swap in a 4-node machine, each CPU is
doing direct page reclaim and swap.  This causes block queue lock
contention.  In my test, without below patch, the CPU utilization is about
2% ~ 7%.  With the patch, the CPU utilization is about 1% ~ 3%.  Disk
throughput isn't changed.  This should improve normal kswapd write and
file cache write too (increase request merge for example), but might not
be so obvious as I explain above.

Change-Id: Ic02b467c30a1e924026e05dd488f631ba70702e0
Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Minchan Kim <minchan.kim@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
params: <level>_initcall-like kernel parameters

This patch adds a set of macros that can be used to declare
kernel parameters to be parsed _before_ initcalls at a chosen
level are executed.  We rename the now-unused "flags" field of
struct kernel_param as the level.  It's signed, for when we
use this for early params as well, in future.

Linker macro collating init calls had to be modified in order
to add additional symbols between levels that are later used
by the init code to split the calls into blocks.

Change-Id: I67d518a3f2aab8c90cd2b4f347450b30fcec4f5c
Signed-off-by: Pawel Moll <pawel.moll@arm.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Blechd0se <alex.christ@hotmail.de>
Fix lockup related to stop_machine being stuck in __do_softirq.

commit 34376a50fb1fa095b9d0636fa41ed2e73125f214 upstream.

The stop machine logic can lock up if all but one of the migration
threads make it through the disable-irq step and the one remaining
thread gets stuck in __do_softirq.  The reason __do_softirq can hang is
that it has a bail-out based on jiffies timeout, but in the lockup case,
jiffies itself is not incremented.

To work around this, re-add the max_restart counter in __do_irq and stop
processing irqs after 10 restarts.

Thanks to Tejun Heo and Rusty Russell and others for helping me track
this down.

This was introduced in 3.9 by commit c10d73671ad3 ("softirq: reduce
latencies").

It may be worth looking into ath9k to see if it has issues with its irq
handler at a later date.

The hang stack traces look something like this:

    ------------[ cut here ]------------
    WARNING: at kernel/watchdog.c:245 watchdog_overflow_callback+0x9c/0xa7()
    Watchdog detected hard LOCKUP on cpu 2
    Modules linked in: ath9k ath9k_common ath9k_hw ath mac80211 cfg80211 nfsv4 auth_rpcgss nfs fscache nf_nat_ipv4 nf_nat veth 8021q garp stp mrp llc pktgen lockd sunrpc]
    Pid: 23, comm: migration/2 Tainted: G         C   3.9.4+ #11
    Call Trace:
     <NMI>   warn_slowpath_common+0x85/0x9f
      warn_slowpath_fmt+0x46/0x48
      watchdog_overflow_callback+0x9c/0xa7
      __perf_event_overflow+0x137/0x1cb
      perf_event_overflow+0x14/0x16
      intel_pmu_handle_irq+0x2dc/0x359
      perf_event_nmi_handler+0x19/0x1b
      nmi_handle+0x7f/0xc2
      do_nmi+0xbc/0x304
      end_repeat_nmi+0x1e/0x2e
     <<EOE>>
      cpu_stopper_thread+0xae/0x162
      smpboot_thread_fn+0x258/0x260
      kthread+0xc7/0xcf
      ret_from_fork+0x7c/0xb0
    ---[ end trace 4947dfa9b0a4cec3 ]---
    BUG: soft lockup - CPU#1 stuck for 22s! [migration/1:17]
    Modules linked in: ath9k ath9k_common ath9k_hw ath mac80211 cfg80211 nfsv4 auth_rpcgss nfs fscache nf_nat_ipv4 nf_nat veth 8021q garp stp mrp llc pktgen lockd sunrpc]
    irq event stamp: 835637905
    hardirqs last  enabled at (835637904): __do_softirq+0x9f/0x257
    hardirqs last disabled at (835637905): apic_timer_interrupt+0x6d/0x80
    softirqs last  enabled at (5654720): __do_softirq+0x1ff/0x257
    softirqs last disabled at (5654725): irq_exit+0x5f/0xbb
    CPU 1
    Pid: 17, comm: migration/1 Tainted: G        WC   3.9.4+ #11 To be filled by O.E.M. To be filled by O.E.M./To be filled by O.E.M.
    RIP: tasklet_hi_action+0xf0/0xf0
    Process migration/1
    Call Trace:
     <IRQ>
      __do_softirq+0x117/0x257
      irq_exit+0x5f/0xbb
      smp_apic_timer_interrupt+0x8a/0x98
      apic_timer_interrupt+0x72/0x80
     <EOI>
      printk+0x4d/0x4f
      stop_machine_cpu_stop+0x22c/0x274
      cpu_stopper_thread+0xae/0x162
      smpboot_thread_fn+0x258/0x260
      kthread+0xc7/0xcf
      ret_from_fork+0x7c/0xb0

Change-Id: I55b099f11238b38150cbf65ed23c1f2d6282eb16
Signed-off-by: Ben Greear <greearb@candelatech.com>
Acked-by: Tejun Heo <tj@kernel.org>
Acked-by: Pekka Riikonen <priikone@iki.fi>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: czechop <czechop@example.com>
Staging: TIDSPBRIDGE: Use vm_iomap_memory for mmap-ing instead of remap_pfn_range

This fixes the following bug:

----  Bug Report ----

source file: drivers/staging/tidspbridge/rmgr/drv_interface.c
issue      : mapping of physical memory without address range checks

259 static int bridge_mmap(struct file *filp, struct vm_area_struct *vma)
260 {
261         u32 status;
262
263         /* VM_IO | VM_DONTEXPAND | VM_DONTDUMP are set by remap_pfn_range() */
264         vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
265
266         dev_dbg(bridge, "%s: vm filp %p start %lx end %lx page_prot %ulx "
267                 "flags %lx\n", __func__, filp,
268                 vma->vm_start, vma->vm_end, vma->vm_page_prot,
269                 vma->vm_flags);
270
271         status = remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff,
272                                  vma->vm_end - vma->vm_start,
273                                  vma->vm_page_prot);
274         if (status != 0)
275                 status = -EAGAIN;
276
277         return status;
278 }

The function provides an interface to remap physical memory to user space, but
does not provide any checks to ensure that the memory is within the region that
should be accessible.

Reported-by: Nico Golde <nico@ngolde.de>
Reported-by: Fabian Yamaguchi <fabs@goesec.de>

Signed-off-by: Ivaylo Dimitrov <freemangordon@abv.bg>
Signed-off-by: Pavel Machek <pavel@ucw.cz>
[steven@steven676.net: make apply to omapzoom p-android-omap3-3.0 branch]

Change-Id: Ia3e8b4ff1237f70e79f5b965a0b7a574171cfbee
tidspbridge: fix last patch to map same region of physical memory as before

Commit 559c71fe5dc3 ("Staging: TIDSPBRIDGE: Use vm_iomap_memory for
mmap-ing instead of remap_pfn_range") had the effect of inadvertently
shifting the start of the physical memory area mapped by
pdata->phys_mempool_base.  Correct this by subtracting that shift before
calling vm_iomap_memory() and adding it back afterwards.

Change-Id: I66e05a9f1ed8ea031ea8087c21d5e6f62670ba69
Reported-by: Dheeraj CVR <cvr.dheeraj@gmail.com>
Signed-off-by: Steven Luo <steven@steven676.net>
omap: mmc-twl4030: move clock input selection prior to vcc test

otherwise it is not executed on systems that use non-twl regulators

Change-Id: Ib23292e2c15ec48f81c0edeef2ff65026b96c9d4
omap3: Increase limit on bootarg 'mpurate'

The value of mpurate is currently expected to be less
than 1000 when specified in MHz.

This patch raises this limit to 2000 to support 1GHz
capable processors. The new limit should be reasonable
for quite some time.

Change-Id: I8848de92a86564f7ecd0ff10aa83692e7698c2af
Signed-off-by: Robert Nelson <robertcnelson@gmail.com>
OMAP3: SR: Added runtime check to disable smartreflex-class3 driver.

With OMAP3630 onwards, a new SmartReflex AVS
class of operation Class 1.5 is introduced.

Change-Id: I3e34a9c51722f6d259e6c6faa944221665a38633
Signed-off-by: bharath <x0172602@ti.com>
Signed-off-by: Evisceration <eviscerationls@gmail.com>
OMAP: DMA: Set thread reservation for high-priority channels

Merge from TI:

> From edc44e73e33d63b423778782ea1fb2c77b311e34 Mon Sep 17 00:00:00 2001
> From: Misael Lopez Cruz <misael.lopez@ti.com>
> Date: Sat, 31 Mar 2012 15:01:53 -0500
> Subject: [PATCH 1/4] OMAP: DMA: Set thread reservation for high-priority channels
>
> Reserve one thread for write and one for read for high-priority
> channels. This configuration helps latency critical synchronized
> transfers: memory-peripheral and/or peripheral-memory.
>
> Change-Id: Ib8ecf32db126937820b1da04b183568c44143d64
> Signed-off-by: Misael Lopez Cruz <misael.lopez@ti.com>

Change-Id: I3a22e85bab6f79c14c9657104817e7fd9dc013ca
Signed-off-by: John Grossman <johngro@google.com>
(cherry picked from commit b05b13b9ba90e6c5e194a6149c1ea6cf46427d09)
OMAP: hwmod: Don't update sysc cache on soft-reset

When the module is resetted using via SYSCONFIG.SOFTRESET the following
sequence causes SYSC cache value to be updated:
omap_hwmod_softreset()
 |-->_write_sysconfig()

_ocp_softreset
 |-->_write_sysconfig()

As result, when the module is enabled next time the SYSC cache value
with SYSCONFIG.SOFTRESET=1 will be used by _enable_sysc() and that
will cause unexpected reset of module. This commit prevents SYSC
cache updating during soft reset.

New static function _write_sysconfig_raw() is introduced for writing
value to the module's OCP_SYSCONFIG register without cache updating.

Change-Id: I0058740c2888c23648a498597142a5c0d1b550ba
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
usb: ehci: fix the host wakeup state machine.

usb: ehci: fix the host wakeup state machine.

Current code enables host clocks and disables immediately
which can have side effects if port status change is not propagated
and clocks are cut, so enable clocks if ports are suspended
in all wakeup paths either from io-pad wakeup or through
bus resume in suspend path.

Change-Id: I5a03fcf46160176289d41cda6574c0c261a13e94
Signed-off-by: Govindraj.R <govindraj.raja@ti.com>
Signed-off-by: Ruchika Kharwar <ruchika@ti.com>
omap: remove unused omaplfb platform device

Change-Id: Ic7d8aa22868c0cf2396ae499a1a320903e9feb31
Signed-off-by: Tony Lofthouse <a0741364@ti.com>
omap: Adding opp functions to gpu platform data.

This is required for providing opp framework access to SGX kernel module.

Change-Id: Ie968a6c51ac636b70909257be4c51430c4cc3d48
Signed-off-by: Rogelio Garcia <rgarcia@ti.com>
smartreflex changes

Change-Id: Ie7506e15a595d2b39af41b621c320c8dda150288
USB: OHCI: do not poll root hub status if HC is not running

USB: OHCI: do not poll root hub status if HC is not running

If the OHCI root hub is suspended the clocks may be
not provided to HW and we get errors on OHCI registers
access. So do not poll it in such cases (like is done
for EHCI)

Change-Id: Ic81833a87c479822fe3c7f23792ee7ac7154e248
Signed-off-by: Ruslan Bilovol <ruslan.bilovol@ti.com>
OMAP: I2C: Wait till functional part of the I2C controller completely becomes valid

Bus busy status bit is not effective when functional part of the I2C controller
is not valid, and bus busy detection in omap_i2c_wait_for_bb() will be skipped
sometimes. So we should wait till functional part of the I2C controller
completely becomes valid, after setting the I2Ci.I2C_CON[15] I2C_EN bit to 1.
For doing that we have to wait till RDONE bit of SYSS register reach "1".

For details see TRM, 23.1.4.3 HS I2C Software Reset.

Change-Id: I989855d6cb2b1aa0098e11245dacb77583fe6647
Signed-off-by: Semen Protsenko <semen.protsenko@ti.com>
OMAP: I2C: Disable interrupts in omap_i2c_bus_clear()

Interrupts that occur during omap_i2c_bus_clear() can cause to
erroneous behaviour in interrupt handler, i.e. multiple printing of
"RRDY IRQ while no data requested" message. So disable interrupts
in omap_i2c_bus_clear() function since they can't be handled correctly.

Change-Id: If4525c93284766d0a2705d7ca4d1f962e3cc4b15
Signed-off-by: Semen Protsenko <semen.protsenko@ti.com>
OMAPDSS: DISPC: Fix rounding error in scaling decision

When checking if we are downscaling by more than a factor,
we were dividing the source dimensions instead of multiplying
back the target dimensions.  This allowed downscaling by
slightly more than the max supported downscaling, resulting
in out-of-range register settings and ultimately SYNC_LOST.

Change-Id: Ic039789c5011d64da870121eeb227a803af1fa83
Signed-off-by: Lajos Molnar <lajos@ti.com>
OMAP: clock: fix race in disable all clocks

clk_disable_unused is invoked when CONFIG_OMAP_RESET_CLOCKS=y.
Since clk_disable_unused is called as lateinitcall, there can
be more than a few workqueues executing off secondary CPU(s).
The current code does the following:
a) checks if clk is unused
b) holds lock
c) disables clk
d) unlocks

Between (a) and (b) being executed on CPU0, It is possible to
have a driver executing on CPU1 which could do a get_sync->clk_get
(and increase the use_count) of the clock which was just about
to be disabled by clk_disable_unused.

We ensure instead that the entire list traversal is protected by
the lock allowing for parent child clock traversal which could be
potentially be done by runtime operations to be safe as well.

Change-Id: Iea5201233d89b8417576df4ed9026e1c05001596
Reported-by: Todd Poynor <toddpoynor@google.com>
Signed-off-by: Nishanth Menon <nm@ti.com>