config: Make packing present on TC2

The scheduler will default packing to disabled, but this includes the
feature so that we can test it more easily.

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
sched: hmp: Fix potential task_struct memory leak

We use get_task_struct to increment the ref count on a task_struct
so that even if the task dies with a pending migration we are still
able to read the memory without causing a fault.

In the case of non-running tasks, we forgot to decrement the ref
count when we are done with the task.

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
HMP: Restrict irq_default_affinity to hmp_slow_cpu_mask

This patch limits the default affinity mask for all irqs to the cluster of
the little cpus.

This patch has the positive side effect that an irq thread which has its
IRQTF_RUNTHREAD set inside irq_thread() -> irq_wait_for_interrupt() will
not overwrite its struct task_struct->cpus_allowed with a full cpu mask of
desc->irq_data.affinity in irq_thread_check_affinity() essentially reverting
patch "HMP: experimental: Force all rt tasks to start on little domain."
for this irq thread.

Signed-off-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
Signed-off-by: Abhijeet Dev <abhijeet@abhijeet-dev.net>
HMP: Fix rt task allowed cpu mask restriction code on 1x1 system

There is an error scenario where on a 1x1 HMP system (weight of the
hmp_slow_cpu_mask is 1) the short-cut of restricting the allowed cpu mask
of an rt tasks leads to triggering a kernel bug in the rt sched class
set_cpus_allowed function set_cpus_allowed_rt().

In case the task is on the run-queue and the weight of the required cpu mask
is 1 and this is different to the p->nr_cpus_allowed value, this back-end
function interprets this in such a way that a task changed from being
migratable to not migratable anymore and decrements the rt_nr_migratory
counter.  There is a BUG_ON(!rq->rt.rt_nr_migratory) check in this code
path which triggers in this situation.

To circumvent this issue, set the number of allowed cpus for a task p to
the weight of the hmp_slow_cpu_mask before calling do_set_cpus_allowed()
in __setscheduler(). It will be set to this value in do_set_cpus_allowed()
after the call to the sched class related backend function any way.  By
doing this, set_cpus_allowed_rt() returns without trying to update the
rt_nr_migratory counter.

This patch has been tested with a test device driver requiring a threaded
irq handler on a TC2 system with a reduced cpu mask (1 Cortex A15, 1
Cortex A7).

Signed-off-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
sched: hmp: Change small task packing defaults for all platforms

All platforms other than TC2 default to enabling packing. Since TC2
shows no performance or energy degradation with this feature enabled
make it default enabled the same as everyone else.
Likewise, vendors have been including TC2 support in multi-machine
kernel builds so they expect the default thresholds to remain the
same when the TC2 #ifdef is removed.

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
hmp: sched: Clean up hmp_up_threshold checks into a utility fn

In anticipation of modifying the up_threshold handling, make all
instances use the same utility fn to check if a task is eligible
for up-migration. This also removes the previous difference in
threshold comparison where up-migration used '!<threshold' and
idle pull used '>threshold' to decide up-migration eligibility.
Make them both use '!<threshold' instead for consistency, although
this is unlikely to change any results.

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
sched: hmp: unify active migration code

The HMP active migration code is functionally identical to the CFS
active migration code apart from one flag check. Share the code
and make the flag check optional.

Two wrapper functions allow the flag check to be present or not.

Thanks to tixy@linaro.org for pointing out the build break and a
good solution in an earlier version.

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
Signed-off-by: Abhijeet Dev <abhijeet@abhijeet-dev.net>
hmp: Use idle pull to perform forced up-migrations

When a normal forced up-migration takes place we stop the task to
be migrated while the target CPU becomes available. This delay can
range from 80us to 1500us on TC2 if the target CPU is in a deep idle
state.

Instead, interrupt the target CPU and ask it to pull a task.
This lets the current eligible task continue executing on the
original CPU while the target CPU wakes. Use a pinned timer to
prevent the pulling CPU going back into power-down with pending
up-migrations.

If we trigger for a nohz kick, it doesn't matter about triggering
for an idle pull since the idle_pull flag will be set when we
execute the softirq and we'll still do the idle pull.

If the target CPU is busy, we will not pull any tasks.

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
Signed-off-by: Abhijeet Dev <abhijeet@abhijeet-dev.net>
hmp: dont attempt to pull tasks if affinity doesn't allow it

When looking for a task to be idle-pulled, don't consider tasks
where the affinity does not allow that task to be placed on the
target CPU. Also ensure that tasks with restricted affinity
do not block selecting other unrestricted busy tasks.

Use the knowledge of target CPU more effectively in idle pull
by passing to hmp_get_heaviest_task when we know it, otherwise
only checking for general affinity matches with any of the CPUs
in the bigger HMP domain.

We still need to explicitly check affinity is allowed in idle pull
since if we find no match in hmp_get_heaviest_task we will return
the current one, which may not be affine to the new CPU despite
having high enough load. In this case, there is nothing to move.

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
sched: hmp: fix out-of-range CPU possible

If someone hotplugs all the little CPUs while another CPU is handling
a wakeup, we can potentially return new_cpu == NR_CPUS from
hmp_select_slower_cpu (which is called internally by
hmp_best_little_cpu as well). We will use this to deref the
per_cpu rq array in hmp_next_down_delay which can go boom.

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
HMP: use per cpu cpuidle driver to fix deadlock in hmp_idle_pull

Using per cpu cpuidle driver to fix deadlock in hmp_idle_pull.
Otherwise a deadlock happened when do bl_idle_init.

[  113.878664] other info that might help us debug this:
[  113.878667]  Possible unsafe locking scenario:
[  113.878667]
[  113.878670]        CPU0
[  113.878673]        ----
[  113.878681]   lock(cpuidle_driver_lock);
[  113.878684]   <Interrupt>
[  113.878691]     lock(cpuidle_driver_lock);
[  113.878693]
[  113.878693]  *** DEADLOCK ***
[  113.878693]
[  113.878697] 1 lock held by ksoftirqd/4/28:
[  113.878719]  #0:  (hmp_force_migration){+.....}, at: [<c0054da5>]
hmp_idle_pull+0x49/0x508

This patch is just a quick/cheap workaround for cpuidle_driver_lock
deadlock. It works for TC2 and any other platform where the idle
driver cannot be changed at runtime.

Signed-off-by: Alex Shi <alex.shi@linaro.org>
Signed-off-by: Jon Medhurst <tixy@linaro.org>