sched: hmp: unify active migration code

The HMP active migration code is functionally identical to the CFS
active migration code apart from one flag check. Share the code
and make the flag check optional.

Two wrapper functions allow the flag check to be present or not.

Thanks to tixy@linaro.org for pointing out the build break and a
good solution in an earlier version.

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
Signed-off-by: Abhijeet Dev <abhijeet@abhijeet-dev.net>
hmp: Use idle pull to perform forced up-migrations

When a normal forced up-migration takes place we stop the task to
be migrated while the target CPU becomes available. This delay can
range from 80us to 1500us on TC2 if the target CPU is in a deep idle
state.

Instead, interrupt the target CPU and ask it to pull a task.
This lets the current eligible task continue executing on the
original CPU while the target CPU wakes. Use a pinned timer to
prevent the pulling CPU going back into power-down with pending
up-migrations.

If we trigger for a nohz kick, it doesn't matter about triggering
for an idle pull since the idle_pull flag will be set when we
execute the softirq and we'll still do the idle pull.

If the target CPU is busy, we will not pull any tasks.

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
Signed-off-by: Abhijeet Dev <abhijeet@abhijeet-dev.net>
hmp: dont attempt to pull tasks if affinity doesn't allow it

When looking for a task to be idle-pulled, don't consider tasks
where the affinity does not allow that task to be placed on the
target CPU. Also ensure that tasks with restricted affinity
do not block selecting other unrestricted busy tasks.

Use the knowledge of target CPU more effectively in idle pull
by passing to hmp_get_heaviest_task when we know it, otherwise
only checking for general affinity matches with any of the CPUs
in the bigger HMP domain.

We still need to explicitly check affinity is allowed in idle pull
since if we find no match in hmp_get_heaviest_task we will return
the current one, which may not be affine to the new CPU despite
having high enough load. In this case, there is nothing to move.

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
Signed-off-by: Abhijeet Dev <abhijeet@abhijeet-dev.net>
sched: hmp: fix out-of-range CPU possible

If someone hotplugs all the little CPUs while another CPU is handling
a wakeup, we can potentially return new_cpu == NR_CPUS from
hmp_select_slower_cpu (which is called internally by
hmp_best_little_cpu as well). We will use this to deref the
per_cpu rq array in hmp_next_down_delay which can go boom.

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
Signed-off-by: Abhijeet Dev <abhijeet@abhijeet-dev.net>
HMP: use per cpu cpuidle driver to fix deadlock in hmp_idle_pull

Using per cpu cpuidle driver to fix deadlock in hmp_idle_pull.
Otherwise a deadlock happened when do bl_idle_init.

[  113.878664] other info that might help us debug this:
[  113.878667]  Possible unsafe locking scenario:
[  113.878667]
[  113.878670]        CPU0
[  113.878673]        ----
[  113.878681]   lock(cpuidle_driver_lock);
[  113.878684]   <Interrupt>
[  113.878691]     lock(cpuidle_driver_lock);
[  113.878693]
[  113.878693]  *** DEADLOCK ***
[  113.878693]
[  113.878697] 1 lock held by ksoftirqd/4/28:
[  113.878719]  #0:  (hmp_force_migration){+.....}, at: [<c0054da5>]
hmp_idle_pull+0x49/0x508

This patch is just a quick/cheap workaround for cpuidle_driver_lock
deadlock. It works for TC2 and any other platform where the idle
driver cannot be changed at runtime.

Signed-off-by: Alex Shi <alex.shi@linaro.org>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
Signed-off-by: Abhijeet Dev <abhijeet@abhijeet-dev.net>
HMP: Do not fork-boost tasks coming from PIDs <= 2

System services are generally started by init, whilst kernel threads
are started by kthreadd. We do not want to give those tasks a head
start, as this costs power for very little benefit. We do however
wish to do that for tasks which the user launches.

Further, some tasks allocate per-cpu timers directly after launch
which can lead to those tasks being always scheduled on a big CPU
when there is no computational need to do so. Not promoting services
to big CPUs on launch will prevent that unless a service allocates
their per-cpu resources after a period of intense computation, which
is not a common pattern.

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
Signed-off-by: Abhijeet Dev <abhijeet@abhijeet-dev.net>
hmp: Restrict ILB events if no CPU has > 1 task

Frequently in HMP, the big CPUs are only active with one task per
CPU and there may be idle CPUs in the big cluster. This patch avoids
triggering an idle balance in situations where none of the active
CPUs in the current HMP domain have > 1 tasks running.

When packing is enabled, only enforce this behaviour when we are
not in the smallest domain - there we idle balance whenever a CPU
is over the up_threshold regardless of tasks in case one needs to
be moved.

Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
Signed-off-by: Abhijeet Dev <abhijeet@abhijeet-dev.net>