ARM: mm: implement LoUIS API for cache maintenance ops

ARM v7 architecture introduced the concept of cache levels and related
control registers. New processors like A7 and A15 embed an L2 unified cache
controller that becomes part of the cache level hierarchy. Some operations in
the kernel like cpu_suspend and __cpu_disable do not require a flush of the
entire cache hierarchy to DRAM but just the cache levels belonging to the
Level of Unification Inner Shareable (LoUIS), which in most of ARM v7 systems
correspond to L1.

The current cache flushing API used in cpu_suspend and __cpu_disable,
flush_cache_all(), ends up flushing the whole cache hierarchy since for
v7 it cleans and invalidates all cache levels up to Level of Coherency
(LoC) which cripples system performance when used in hot paths like hotplug
and cpuidle.

Therefore a new kernel cache maintenance API must be added to cope with
latest ARM system requirements.

This patch adds flush_cache_louis() to the ARM kernel cache maintenance API.

This function cleans and invalidates all data cache levels up to the
Level of Unification Inner Shareable (LoUIS) and invalidates the instruction
cache for processors that support it (> v7).

This patch also creates an alias of the cache LoUIS function to flush_kern_all
for all processor versions prior to v7, so that the current cache flushing
behaviour is unchanged for those processors.

v7 cache maintenance code implements a cache LoUIS function that cleans and
invalidates the D-cache up to LoUIS and invalidates the I-cache, according
to the new API.

Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Shawn Guo <shawn.guo@linaro.org>

Change-Id: I561c3b187b5b87b25de07ea82a4ef73a42761e27
Signed-off-by: Karthik Parsha <kparsha@codeaurora.org>
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
clocksource: arm_arch_timer: Use clocksource for suspend timekeeping

The ARM architected timers keep counting during suspend so we can
mark this clocksource with the CLOCK_SOURCE_SUSPEND_NONSTOP. This
flag will indicate that this clocksource can be used for
calculating suspend time and injecting sleep time into the
timekeeping core. This will usually be more accurate than using
an external RTC or architecture specific persistent clock.

CRs-fixed: 536881
Change-Id: I882bf19a87ceb642aa069bbf26802c66c96b4426
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
msm: pm-8x60: Fix suspend time tracking

Calls to sched_clock() after syscore devices have been suspend
will return the same value until the syscored devices are
resumed. Furthermore, the value won't jump forward to account for
the time left in suspend so we can't use it to calculate how much
time we spent in suspend.

We could expose the hardware directly to the pm code but it's
better to use generic interfaces. Move to using the timekeeping
APIs to determine how long we were in suspend. Unfortunately, we
can't use the timekeeping APIs this early in the resume path, so
we must read the current time of day before we suspend the
timekeeping core and after we resume the timekeeping core. Move
the calls to the suspend_late() and wake() callbacks so we can
properly read the amount of suspend time.

CRs-fixed: 536881
Change-Id: I3b47511afb75926e51e82fc83929f405404fbf07
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
msm: acpuclock-krait: Program both high- and low-performance sources

The latest LDO/BHS switching sequence employed in the 8974
krait-regulator driver involves mandatory operations that cause
the CPUs to switch to the low-performance clock source when the
CPU is powered from the LDO, and to the high-performance clock
source when powered from the BHS.

This results in potential undervolting of the CPUs since the
acpuclock driver does not program the low-performance source,
and the clock source it selects be default is running at the
same speed as DDR (which can be very fast).

Fix this by programming both the high and low-performance clock
sources to be the same at all times.

CRs-Fixed: 538629
Change-Id: Iaa39cca9165b892e52acd8ec3aac5d0784112623
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
msm: acpuclock-krait: Fix HFPLL regulator voting in regulator_init()

Simply incrementing l2_vreg_count is not enough to ensure the
regulators necessary for the L2 HFPLL are enabled. Instead,
enable_l2_regulators() must be called to place votes for the
regulators.

This problem will only result in incorrect behaviour (a missing
vote for the HFPLL regulators) if the L2 rate detected at boot is
not in the frequency table. In all other cases, the vote will be
applied as part of the hfpll_enable call when init_clock_sources()
is called for the L2.

Fix the code so that the voting works correctly even if the initial
L2 rate is not in the frequency table.

Change-Id: I8e84cd81de17a0efcc91281848180cf34ef9bb57
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
msm: acpuclock-krait: Update 'unknown rate' debug prints to warnings

While not fatal, boot performance will be suboptimal if the L2 or
CPU frequencies at boot are not in the frequency tables. This is
because acpuclock 'plays it safe' by applying the lowest possible
clock rate in such situations to ensure clocks are not set higher
than boot-up voltages can support.

Change-Id: I07fcff70d742b2590fa996ea4ca7a1f1c560eee3
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
Merge branch 'bcmdhd-update' from CM-11.0 Kernel!
msm_sdcc: remove 5-second mdelay on command timeout

Signed-off-by: Iliyan Malchev <malchev@google.com>
mmc: sdhci: add auto command error interrupt support

The auto command error interrupt is not enabled now. Hence,
if there occurs a timeout when auto-cmd is in progress,
no data timeout interrupt would occur and the driver would
only timeout after the software timer expires.

This patch enables the auto command error interrupt and lets the
higher layers know of such an error, if any.

CRs-Fixed: 515513
Change-Id: I69880a72ece7730645dcfe699d58271d60cab33d
Signed-off-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Krishna Konda <kkonda@codeaurora.org>
mmc: sdhci: fix issue with auto cmd err detection

As per specification, auto cmd error status register is valid only
when auto cmd error bit is set in Error interrupt status register.

Change-Id: Id1013e1705d8efdba0171dcad14f783607d38ef3
Signed-off-by: Sahitya Tummala <stummala@codeaurora.org>
Signed-off-by: ChangGyun Kim <cg.kim@lge.com>
Signed-off-by: Devin Kim <dojip.kim@lge.com>
mmc: sdhci: ignore the hardware timeout when requested

Few card commands can be interrupted (using HPI command) if it takes
longer time to complete so in such case its acceptable to get the
timeout errors hence ignore them when MMC core layer request for it.

Change-Id: I347a2171e21746643e5f39eb927394d55cbca46e
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Maya Erez <merez@codeaurora.org>
mmc: core: expose HPI capability to SWITCH commands

Some of the time consuming operations such as BKOPS, SANITIZE, CACHE
flush/off use the SWITCH command (CMD6) but as these operations don't
have card specification defined timeout for completion, we may see
timeout errors if card doesn't complete the operation within the SW
defined timeout. If SW defined timeout is hit, above operations are
considered to be failed and no real recovery mechanism is implemented
after timeout.

Most of the above operations (BKOPS/SANITIZE/CACHE flush/off) can be
interrupted by sending the HPI (High Priority Interrupt) command to card
if they taking longer than expected. This change adds the base support
which will these operations to be HPIed after timeout.

Change-Id: I5b4bfd9fed30f5e4ad57cd4f06c85ae5c5ba169d
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Maya Erez <merez@codeaurora.org>
mmc: msm_sdcc: change the bkops_busy flag check to ignore_timeout

MMC core layer have made the "bkops_busy" flag in the mmc_command
structure absolute, use "ignore_timeout" instead.

Change-Id: Ib3c01dafce3f099895cadf5cf29cc370a6a15fc2
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Maya Erez <merez@codeaurora.org>
mmc: core: handle flush requests timeout

According to the eMMC 4.5 spec "Flushing a large amount of cached data may
take very unpredictably long time". Therefore the timeout for FLUSH should
be increased to prevent timeouts.

In case the timeout occurs HPI issued.

Change-Id: Ib4f7e43d32d14f5d4e112057c7427f4b97957518
CRs-Fixed: 500874
Signed-off-by: Maya Erez <merez@codeaurora.org>
Signed-off-by: Konstantin Dorfman <kdorfman@codeaurora.org>
mmc: core: hpi in TRAN state error handling

There could be a race condition in the stop transmission request.
1. Urgent read request comes in.
2. Stop the ongoing write request by calling stop_request MMC host op.
3. Send the stop command (CMD12) to move the card from the receive state
   to programming state.
4. Send the status command (CMD13) to know if the card is in programming
   state or not. If yes, then go to step.5 or else go to step.6.
5. Send the HPI command to bring the card out of programming state.
6. Now send the urgent read request.

In above sequence in step.4, we check if the card is in programming or
not and if it is then we go ahead and send the HPI in step.5 but what if
after step.4 is completed and before the step.5, card moves out of the
programming state so in that case HPI command would timeout as we are
sending it in the wrong state. In that case no command response (by eMMC
protocol) causes mmc_send_hpi_cmd to fail (with -ETIMEOUT error).  In
case the card indeed in TRAN state, no error is reported.

Change-Id: Ia6f81f6cc4ec933f2e8423029f7236401aea04df
Signed-off-by: Konstantin Dorfman <kdorfman@codeaurora.org>
mmc: sdhci-msm: Ignore AHB bus transfer stop request

The latest SDHCI controller core v3.1.0 can handle reset in middle of an
ongoing transfer without the need to stop AHB bus transaction
separately.

Change-Id: I34d2359dd2b22696a70cc146652811cb866a1d3c
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
mmc: core: dont send PON during suspend

During suspend, eMMC VCC can be turned off. So instead of sending
a power off notification, send a sleep command. According to the
eMMC 4.5 specification, this is the only time where its clearly
mentioned that the VCC regulator can be turned OFF.

This has been clarified in eMMC 5.0 specification and this applies
even if PON with sleep is not sent to the card.

Change-Id: I6c424bed3158132af7f9c2a2a701af7369fd5ec7
Signed-off-by: Krishna Konda <kkonda@codeaurora.org>
mmc: core: Enable multiple urgent requests notification

At the moment if the block layer issues an URGENT request notification
before receiving a complete on the URGENT request currently handled by
the device driver - the system crashes with a BUG_ON().
This change enables multiple block layer urgent request notification to
be handled by MMC layer by removing single notification enforcement.

CRs-fixed: 517724
Change-Id: Idfa410d0c2286146f8d58a38b5abd6886036317c
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
mmc: sdhci: Add a mechanism to disable HPI

Certain cards might cause a timeout when used with HPI mode
due to card specific issues. Provide a mechanism where HPI
mode can be turned OFF to workaround such issues.

Signed-off-by: Naveen Ramaraj <nramaraj@codeaurora.org>
mmc: sdhci-msm: Enable controller power save feature

Enable power save feature within controller by setting bit 1
in vendor specific register (0x10C). This allows controller to
disable SD clock when bus is idle to save power.

https://www.codeaurora.org/cgit/quic/la/kernel/msm/commit/drivers/mmc?id=efad7a79d65e9fe73e28efd96392a6e48ab26529

Bug: 10796623
Change-Id: I916a5a414adb3f21dc3a75f3f86c3a81d6956dc8
Signed-off-by: Sahitya Tummala <stummala@codeaurora.org>
Signed-off-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Devin Kim <dojip.kim@lge.com>