cpufreq: Manage only online cpus

cpufreq core doesn't manage offline cpus and if driver->init() has returned
mask including offline cpus, it may result in unwanted behavior by cpufreq core
or governors.

We need to get only online cpus in this mask. There are two places to fix this
mask, cpufreq core and cpufreq driver. It makes sense to do this at common place
and hence is done in core.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: Retain only online cpus in managed_policy->cpus

cpufreq's handling of cpu online event can incorrectly initialize
managed_policy->cpus to include offline cpus. This causes cpufreq
governors to include offline cpus in their idle time computation, which
causes frequency to shoot up beyond normal levels (on systems where 2 or
more cpus need to be synchronized wrt frequency change).

This frequency shootup was not seen prior to commit 21111be8 which
changed idle time accounting for offline cpus. Prior to commit 21111be8,
idle time statistics for offline cpus would include their offline time
as well. Post commit 21111be8, offline time is not included in a cpu's
idle time statistics.

As a result, ondemand (or other cpufreq) governors would see unchanged
idle time for offline cpus, causing them to interpret those (offline)
cpus as 100% busy, which is wrong.

This patch fixes the issue by modifying cpufreq's hotplug handling to
retain only online cpus in managed_policy->cpus field. cpufreq governors
would then skip such offline cpus from idle time calculations.

Change-Id: Id2eba8c0ec3367609369b2e42a4e1386ff2e7ab5
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpuidle: move field disable from per-driver to per-cpu

Andrew J.Schorr raises a question.  When he changes the disable setting on
a single CPU, it affects all the other CPUs.  Basically, currently, the
disable field is per-driver instead of per-cpu.  All the C states of the
same driver are shared by all CPU in the same machine.

The patch changes the `disable' field to per-cpu, so we could set this
separately for each cpu.

Signed-off-by: ShuoX Liu <shuox.liu@intel.com>
Reported-by: Andrew J.Schorr <aschorr@telemetry-investments.com>
Reviewed-by: Yanmin Zhang <yanmin_zhang@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpuidle: Quickly notice prediction failure for repeat mode

The prediction for future is difficult and when the cpuidle governor prediction
fails and govenor possibly choose the shallower C-state than it should. How to
quickly notice and find the failure becomes important for power saving.

cpuidle menu governor has a method to predict the repeat pattern if there are 8
C-states residency which are continuous and the same or very close, so it will
predict the next C-states residency will keep same residency time.

There is a real case that turbostat utility (tools/power/x86/turbostat)
at kernel 3.3 or early. turbostat utility will read 10 registers one by one at
Sandybridge, so it will generate 10 IPIs to wake up idle CPUs. So cpuidle menu
 governor will predict it is repeat mode and there is another IPI wake up idle
 CPU soon, so it keeps idle CPU stay at C1 state even though CPU is totally
idle. However, in the turbostat, following 10 registers reading is sleep 5
seconds by default, so the idle CPU will keep at C1 for a long time though it is
 idle until break event occurs.
In a idle Sandybridge system, run "./turbostat -v", we will notice that deep
C-state dangles between "70% ~ 99%". After patched the kernel, we will notice
deep C-state stays at >99.98%.

In the patch, a timer is added when menu governor detects a repeat mode and
choose a shallow C-state. The timer is set to a time out value that greater
than predicted time, and we conclude repeat mode prediction failure if timer is
triggered. When repeat mode happens as expected, the timer is not triggered
and CPU waken up from C-states and it will cancel the timer initiatively.
When repeat mode does not happen, the timer will be time out and menu governor
will quickly notice that the repeat mode prediction fails and then re-evaluates
deeper C-states possibility.

Below is another case which will clearly show the patch much benefit:

volatile int * shutdown;
volatile long * count;
int delay = 20;
int loop = 8;

void usage(void)
{
	fprintf(stderr,
		"Usage: idle_predict [options]\n"
		"  --help	-h  Print this help\n"
		"  --thread	-n  Thread number\n"
		"  --loop     	-l  Loop times in shallow Cstate\n"
		"  --delay	-t  Sleep time (uS)in shallow Cstate\n");
}

void *simple_loop() {
	int idle_num = 1;
	while (!(*shutdown)) {
		*count = *count + 1;

		if (idle_num % loop)
			usleep(delay);
		else {
			/* sleep 1 second */
			usleep(1000000);
			idle_num = 0;
		}
		idle_num++;
	}

}

static void sighand(int sig)
{
	*shutdown = 1;
}

int main(int argc, char *argv[])
{
	sigset_t sigset;
	int signum = SIGALRM;
	int i, c, er = 0, thread_num = 8;
	pthread_t pt[1024];

	static char optstr[] = "n:l:t:h:";

	while ((c = getopt(argc, argv, optstr)) != EOF)
		switch (c) {
			case 'n':
				thread_num = atoi(optarg);
				break;
			case 'l':
				loop = atoi(optarg);
				break;
			case 't':
				delay = atoi(optarg);
				break;
			case 'h':
			default:
				usage();
				exit(1);
		}

	printf("thread=%d,loop=%d,delay=%d\n",thread_num,loop,delay);
	count = malloc(sizeof(long));
	shutdown = malloc(sizeof(int));
	*count = 0;
	*shutdown = 0;

	sigemptyset(&sigset);
	sigaddset(&sigset, signum);
	sigprocmask (SIG_BLOCK, &sigset, NULL);
	signal(SIGINT, sighand);
	signal(SIGTERM, sighand);

	for(i = 0; i < thread_num ; i++)
		pthread_create(&pt[i], NULL, simple_loop, NULL);

	for (i = 0; i < thread_num; i++)
		pthread_join(pt[i], NULL);

	exit(0);
}

Get powertop V2 from git://github.com/fenrus75/powertop, build powertop.
After build the above test application, then run it.
Test plaform can be Intel Sandybridge or other recent platforms.

We will find that deep C-state will dangle between 40%~100% and much time spent
on C1 state. It is because menu governor wrongly predict that repeat mode
is kept, so it will choose the C1 shallow C-state even though it has chance to
sleep 1 second in deep C-state.

While after patched the kernel, we find that deep C-state will keep >99.6%.

Signed-off-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Youquan Song <youquan.song@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>

Conflicts:

	kernel/time/tick-sched.c

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpuidle: Quickly notice prediction failure in general case

The prediction for future is difficult and when the cpuidle governor prediction
fails and govenor possibly choose the shallower C-state than it should. How to
quickly notice and find the failure becomes important for power saving.

The patch extends to general case that prediction logic get a small predicted
residency, so it choose a shallow C-state though the expected residency is large
. Once the prediction will be fail, the CPU will keep staying at shallow C-state
for a long time. Acutally, the CPU has change enter into deep C-state.
So when the expected residency is long enough but governor choose a shallow
C-state, an timer will be added in order to monitor if the prediction failure.

When C-state is waken up prior to the adding timer, the timer will be cancelled
initiatively. When the timer is triggered and menu governor will quickly notice
prediction failure and re-evaluates deeper C-states possibility.

Signed-off-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Youquan Song <youquan.song@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpuidle: Get typical recent sleep interval

The function detect_repeating_patterns was not very useful for
workloads with alternating long and short pauses, for example
virtual machines handling network requests for each other (say
a web and database server).

Instead, try to find a recent sleep interval that is somewhere
between the median and the mode sleep time, by discarding outliers
to the up side and recalculating the average and standard deviation
until that is no longer required.

This should do something sane with a sleep interval series like:

	200 180 210 10000 30 1000 170 200

The current code would simply discard such a series, while the
new code will guess a typical sleep interval just shy of 200.

The original patch come from Rik van Riel <riel@redhat.com>.

Signed-off-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Youquan Song <youquan.song@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpuidle: Fix finding state with min power_usage

Since cpuidle_state.power_usage is a signed value, use INT_MAX (instead
of -1) to init the local copies so that functions that tries to find
cpuidle states with minimum power usage works correctly even if they use
non-negative values.

Signed-off-by: Sivaram Nair <sivaramn@nvidia.com>
Reviewed-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpuidle: remove the power_specified field in the driver

We realized that the power usage field is never filled and when it
is filled for tegra, the power_specified flag is not set causing all
of these values to be reset when the driver is initialized with
set_power_state().

However, the power_specified flag can be simply removed under the
assumption that the states are always backward sorted, which is the
case with the current code.

This change allows the menu governor select function and the
cpuidle_play_dead() to be simplified.  Moreover, the
set_power_states() function can removed as it does not make sense
any more.

Drop the power_specified flag from struct cpuidle_driver and make
the related changes as described above.

As a consequence, this also fixes the bug where on the dynamic
C-states system, the power fields are not initialized.

[rjw: Changelog]
References: https://bugzilla.kernel.org/show_bug.cgi?id=42870
References: https://bugzilla.kernel.org/show_bug.cgi?id=43349
References: https://lkml.org/lkml/2012/10/16/518
Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>

Conflicts:

	drivers/cpuidle/cpuidle.c
	drivers/cpuidle/driver.c

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: Use dedicated high-priority workqueues

In the process of scaling CPU frequencies, cpufreq and the APIS is
calls may hold resources that will prevent threads critical for
system stability from running.

Specifically, thermal-mitigation software may need to respond
quickly to high-temperature conditions and throttle the CPU speed
in an effort to cool down. If the system is already in the middle
of a CPU frequency switch being executed on behalf of a cpufreq
governor, thermal mitigation will be blocked until the scaling
action has completed. If the system is under heave load of medium
or high-priority threads, this may take a long while and the system
may potentially surpass thermal limits in the meantime.

Resolve this by using high priority workqueues for all ondemand
and conservative governor scaling operations.  This will allow them
to complete promptly and release their hold on resources necessary
for maintaining system stability.

Change-Id: I2f56052c131442838036cf4cdd8059f7c09bb805
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
Signed-off-by: Ajay Dudani <adudani@codeaurora.org>
cpufreq: ondemand: change freq sync code to use per-CPU kthreads

The migration notifier may run in a context where it is not safe to
enqueue a work item. For example:

(__queue_work+0x1f4/0x40c)
(queue_work_on+0x34/0x44)
(dbs_migration_notify+0x2c/0x40)
(notifier_call_chain+0x38/0x68)
(__atomic_notifier_call_chain+0x34/0x44)
(atomic_notifier_call_chain+0x14/0x18)
(try_to_wake_up+0x350/0x36c)
(autoremove_wake_function+0xc/0x34)
(__wake_up_common+0x48/0x7c)
(__wake_up+0x3c/0x50)
(trustee_thread+0x1d0/0x528)
(kthread+0x80/0x8c)

The trustee code already takes a lock internal to the workqueue
implementation and is not expecting a wake up to come around and
enter the workqueue code again, where the same lock will be needed.
Instead of relying on a workqueue, use a per-CPU kthread to do
the frequency syncing.

Change-Id: I7555ee40867792fa8ec4ea8f9a6309323775e797
Signed-off-by: Steve Muckle <smuckle@codeaurora.org>
Signed-off-by: Ajay Dudani <adudani@codeaurora.org>
cpufreq: ondemand: disable export of utilisation to userspace, we don't use this.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
mach-msm: subsystem_restart: Fix CPU detection

Initialization function could not detect the correct CPU type
(apq8064). Therefore it could not register correctly subsystem
restart order for the panic_handler.

Signed-off-by: Stratos Karafotis <stratosk@semaphore.gr>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: ondemand: Fix store_powersave_bias() race with hotplug

store_powersave_bias() assumes that CPUs will not be taken offline or
brought online while executing its for_each_online_cpu() loops. Fix
this race by blocking hotplug while executing in these loop.

CRs-Fixed: 494407
Change-Id: I0d5520444ec21a8d9b12fb73c06ac8442449fa85
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: ondemand: Fix locking in store_powersave_bias

store_powersave_bias() assumes that dbs_info->cur_policy will
not change while it is executing. If it does, memory corruption
can potentially result. Fix this by acquiring the dbs_lock.

Most noticeably, this resolves timer-related list corruptions
encountered when queue_delayed_work_on() is called from within
store_powersave_bias() while a CPU is being hotplugged in and
out.

CRs-Fixed: 494407
Change-Id: I0fd5cbef6834545274cad9481e574ec50b37b871
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: ondemand: Fix locking issue in store_powersave_bias

store_powersave_bias takes timer_mutex before calling dbs_timer_exit
which tries to cancel the delayed work do_dbs_timer which in turn
tries to take the same lock. This can cause a lock recursion under
the race condition where can cancel_delayed_work_sync is called
when the work has already started executing.

This can be avoided by taking that lock after calling
dbs_timer_exit.

Change-Id: I7f862286e66f1ddc1e13e4eeee369dd188fc10d5
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: ondemand: Disable freq sync feature in store_powersave_bias

Turn off frequency synchronization of CPUs on thread migrations
when powersave bias is enabled. This is done to prevent re-arming
of the dbs timer work (which is cancelled by store_powersave_bias)
by the sync_thread.

Change-Id: I165dd591845c1d66d01a14e8dfc44c767c677b0d
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>

Conflicts:
	drivers/cpufreq/cpufreq_ondemand.c
cpufreq: ondemand: Boost CPU frequency only for touch input

Originally CPU frequency were boosted for any input event.
In some case, sensor sends a lot of input event, which
keep CPU in high frequency.  CPU frequency only need to
be boosted when real user interaction happens.

Change-Id: Ia3ad755b98d8363a17729926610b5dd6f0075288
CR-Fixed: 507519
Signed-off-by: Tingwei Zhang <tingwei@codeaurora.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: ondemand: add input_boost interface

Currently Ondemand governor handles any input event
like touch by scaling the CPU frequency to maximum
available on the target.

This change adds a new sysfs interface "input_boost"
whereby the CPU will scale to this frequency on input events.
The value of this sysfs is user defined so input events
can now be handled by scaling the CPU to lower frequencies
than target max.

Change-Id: I5428fd8797c9984b17a66b01a44557f2160e8b68
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>

Conflicts:
	drivers/cpufreq/cpufreq_ondemand.c
cpufreq: ondemand: Fix update_sampling_rate race with hotplug

update_sampling_rate has a for loop which goes through each
online cpu and possibly queue up the ondemand work for them.
But while doing this it doesnt take any hotplug lock which
could potentially cause a race condition where ondemand work
is queued after the hotplug code (which sets the policy to NULL)
in the governor has cancelled any pending work. This could cause
a crash while trying to access the NULL policy in dbs_check_cpu.

Protecting the for_each_online_cpu loop with get_online_cpus()
and put_online_cpus().

Change-Id: Ia3f43ca7e4bed542834ab03ca1191d728f13311c
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: ondemand: Fix hotplug deadlock with store_powersave_bias

store_powersave_bias() acquires the hotplug lock and the dbs_mutex
lock, but does so in the wrong order.  Deadlocks like the following
can result.

Thread A:
    get_online_cpus+0x3c/0x5c           <- acquires 'cpu_hotplug.lock'
    store_powersave_bias+0x80/0x3f4     <- acquires 'dbs_mutex'
    kobj_attr_store+0x14/0x20
    sysfs_write_file+0x108/0x13c
    vfs_write+0xb0/0x128
    sys_write+0x38/0x64

Thread B:
    cpufreq_governor_dbs+0x7c/0x55c     <- acquires 'dbs_mutex'
    __cpufreq_governor+0x90/0xe0
    __cpufreq_set_policy+0x1b0/0x258
    cpufreq_add_dev_interface+0x2cc/0x334
    cpufreq_add_dev+0x514/0x580
    cpufreq_cpu_callback+0x88/0x9c
    notifier_call_chain+0x38/0x68
    __cpu_notify+0x28/0x40
    _cpu_up+0xe4/0x118                  <- acquires 'cpu_hotplug.lock'
    cpu_up+0x64/0x80
    store_online+0x48/0x78
    dev_attr_store+0x18/0x24
    sysfs_write_file+0x108/0x13c
    vfs_write+0xb0/0x128
    sys_write+0x38/0x64

Fix this by flipping the order in which the locks are acquired and
released in store_powersave_bias so that it is the same as in the
hotplug path.

Change-Id: Idc59fb29d60b8f7fceb8ed0f2bb9eff4670abda7
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>