msm: Improve qpnp-vibrator scheduling

 * Skip the workqueue and use spinlocks to improve interactivity
 * This vastly improves "recent apps" in Android

Change-Id: I77da17b4f1ce2f213772c41ddbac21250432279e
msm: kgsl: Skip dispatcher mutex owner check

Skip dispatcher mutex owner checking in adreno_idle
when we know that current thread doesn't own it.

Change-Id: I4e9afa104884a1a0505582058829b7bb780d33cf
Signed-off-by: Rajesh Kemisetti <rajeshk@codeaurora.org>
msm: kgsl: Remove io_fraction as it is no longer used

io_fraction was originally used as a way to influence the CPU
frequency governor.  Due to governor updates and timing changes
no targets use io_fraction now and there is no plan to do so in
the future.

Change-Id: Ib2739393d83a197d18d4446e6afcb889196f479b
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>
msm: kgsl: Return an error for failure in _cmdbatch_create_legacy

Callers of _cmdbatch_create_legacy expect an error to be returned in
the style of ERR_PTR.  Fix an error path wherein NULL was returned
instead.

Change-Id: Ic0dedbadcbc73f6320869c44df9599d754cd139a
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
msm: kgsl: Fix a sign mismatch in memdesc_sg_virt

get_user_pages returns a signed long but we do all of our math
with unsigned longs so cast the return value to the appropriate
type for a comparison.

Change-Id: Ic0dedbad0c931c2440929c615879a288d2f13828
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
msm: kgsl: Do not return invalid power stats when the device is off

adreno_power_stats() was returning unitialized data when the device
state was not active. Leave early from the function if the device
is not active and return 0s instead.

Change-Id: Ic0dedbadf6201cadb4472327c4ce7f9d9d32b82d
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>

Conflicts:
	drivers/gpu/msm/adreno.c
msm: kgsl: Fix enum indexing in the adreno CP parser

enum adreno_cp_addr_regs defines a dummy entry (ADRENO_CP_ADDR_MAX)
to specify the maximum allowable address.  Most notably the value
is used to initalize an array of register addresses. Several functions
incorrectly allowed ADRENO_CP_ADDR_MAX to be used as index which
would overstep the bounds of the array.

Change-Id: Ic0dedbadd993c93aef7c80e0621ea022738e772a
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
defconfig: Update (performance)

Signed-off-by: Kostas K <mpokwsths@gmail.com>
msm: kgsl: Remove the "global" locking from the ioctl path

Remove the lock code from the generic ioctl handler and move the
mutex lock/unlock code to the places that really need it.  In
particular try not to hold a lock while doing user memory functions
(copy_to_user/copy_from_user) to avoid the potential of a deadlock
with the memory semaphore.

[mpokwsths: Also remove some obsolete and unused code]

CRs-Fixed: 671080
Change-Id: Ic0dedbad6f7325b53d71a5dbbed50f92483871c8
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Kostas K <mpokwsths@gmail.com>
msm: kgsl: Fix some variable issues in the perfcounter code

Fix two potential issues in the A3XX performance counter code:

* Some compilers might get worried that the 'in' variable might be
  used uninitialized in the perfcounter read functions. It isn't;
  but not really worth arguing over.

* It is possible for the result of the macro ADRENO_PERFCOUNTERS to
  be NULL if the GPU device isn't define.  Correctly check for a
  NULL pointer to avoid a dereference.

Change-Id: Ic0dedbad5a74101233ced2402e98e36d1acb2d71
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>

Conflicts:
	drivers/gpu/msm/adreno_a3xx.c
msm: kgsl: There is no need to map global PT entries for NOMMU

There is some tortured logic preventing a NULL pointer from being
dereferenced deep in the depths of kgsl_mmu_map if kgsl_mmu_type is
KGSL_MMU_TYPE_NONE but there isn't any reason to even go down the
path in the first place.

Change-Id: Ic0dedbad94c5374e62de3fd7e9c03a404c954014
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
msm: kgsl: Use irqsave/irqrestore for cmdbatch locks

Because the cmdbatch spin lock can be taken in a timer we need
to protect ourselves by turning off interrupts for a bit in
in the various places the lock is taken during nominal operation.

Change-Id: Ic0dedbad3eedba9427933eb5b3ea359f05a018b1
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>

Conflicts:
	drivers/gpu/msm/adreno_debugfs.c
	drivers/gpu/msm/adreno_dispatch.c
msm: kgsl: fix type mismatches in memdesc_sg_virt()

If the loop end condition is a long, the iterator should be
a long too.

Change-Id: I9bffe845af72a7524c6918517d6c2d0a24439547
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
msm: kgsl: Don't update devfreq if the device is in slumber

There is no need to notify devfreq with the device statistics
if the device is power collapsed.

Change-Id: I57af414fa66417226dcb682ac3fcb088503fa5ab
Signed-off-by: Suman Tatiraju <sumant@codeaurora.org>
word-at-a-time: provide generic big-endian zero_bytemask implementation

Whilst architectures may be able to do better than this (which they can,
by simply defining their own macro), this is a generic stab at a
zero_bytemask implementation for the asm-generic, big-endian
word-at-a-time implementation.

On arm64, a clz instruction is used to implement the fls efficiently.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
ARM: 7927/1: dcache: select DCACHE_WORD_ACCESS for big-endian CPUs

With commit 11ec50caedb5 ("word-at-a-time: provide generic big-endian
zero_bytemask implementation"), the asm-generic word-at-a-time code now
provides a zero_bytemask implementation, allowing us to make use of
DCACHE_WORD_ACCESS on big-endian CPUs, providing our
load_unaligned_zeropad function is endianness-clean.

This patch reworks the load_unaligned_zeropad fixup code to work for
both big- and little-endian CPUs, then removes the !CPU_BIG_ENDIAN check
when selecting DCACHE_WORD_ACCESS.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

Conflicts:
	arch/arm/Kconfig
ARM: 7928/1: kconfig: select HAVE_EFFICIENT_UNALIGNED_ACCESS for CPUv6+ && MMU

Modern ARM CPUs can perform efficient unaligned memory accesses in
hardware and this feature is relied up on by code such as the dcache
word-at-a-time name hashing.

This patch selects HAVE_EFFICIENT_UNALIGNED_ACCESS for these cores and
reworks the kconfig select logic for DCACHE_WORD_ACCESS to use the new
symbol.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

Conflicts:
	arch/arm/Kconfig
msm: kgsl: Don't allow userspace to pass in an invalid size

We currently allow userspace to pass in size zero for our cache
range operations without any checks. Calling dma cache range
operations with zero size is very problematic and results in a
kernel panic.

Change-Id: Ic60d622c06b26e55296aed584e8c87bc9ad3ca37
Signed-off-by: Shrenuj Bansal <shrenujb@codeaurora.org>
msm: kgsl: fix device->mutex recursions

kgsl_mutex_lock() is a workaround for complicated locking related
to invalidating the iommu TLB during kgsl_mmu_map() and
kgsl_mmu_unmap().  But it worked around this problem by adding
the possibility of lock recursion, which risks many other problems.

The locking here was complicated because global memory regions,
like the ringbuffer were often mapped and unmapped with the
device mutex held, whereas userspace (or other non-global)
memory regions were mapped and unmapped with the device->mutex
NOT held.

However, there is no reason to invalidate the TLB for global
mappings. These mappings are only mapped or unmapped when
creating or destroying pagetables. And when this happens,
the pagetable being (un)mapped is NOT the active pagetable.
Additionally, the global mappings are supposed to be the
same in every pagetable, so there's no point in invalidating
their TLB entries at any point. When a pagetable becomes
active or inactive, the TLB will still be flushed by
the setstate code.

Addtionally, there were some places where userspace
buffers could get unmapped with the device mutex held,
which would deadlock on the device mutex. Fix these
by making sure that kgsl_mem_entry_put() is called
with the mutex open.

Change-Id: I17864bbcf8829d661d0709bd66feb4acaa3f134c
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>

Conflicts:
	drivers/gpu/msm/kgsl.c
	drivers/gpu/msm/kgsl_iommu.c

Conflicts:
	drivers/gpu/msm/kgsl_device.h
word-at-a-time: avoid undefined behaviour in zero_bytemask macro

The asm-generic, big-endian version of zero_bytemask creates a mask of
bytes preceding the first zero-byte by left shifting ~0ul based on the
position of the first zero byte.

Unfortunately, if the first (top) byte is zero, the output of
prep_zero_mask has only the top bit set, resulting in undefined C
behaviour as we shift left by an amount equal to the width of the type.
As it happens, GCC doesn't manage to spot this through the call to fls(),
but the issue remains if architectures choose to implement their shift
instructions differently.

An example would be arch/arm/ (AArch32), where LSL Rd, Rn, #32 results
in Rd == 0x0, whilst on arch/arm64 (AArch64) LSL Xd, Xn, #64 results in
Xd == Xn.

Rather than check explicitly for the problematic shift, this patch adds
an extra shift by 1, replacing fls with __fls. Since zero_bytemask is
never called with a zero argument (has_zero() is used to check the data
first), we don't need to worry about calling __fls(0), which is
undefined.

Cc: <stable@vger.kernel.org>
Cc: Victor Kamensky <victor.kamensky@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>