msm: kgsl: Squashed update from msm-3.4

msm: kgsl: add IOCTL_KGSL_GPUMEM_ALLOC_ID

Previously, the gpu address has been used to uniquely
identify each memory allocation. Upcoming patches will
introduce cases where an allocation does not always
have a gpu address, so an additional id is needed.

IOCTL_KGSL_GPUMEM_ALLOC_ID allocates pages and returns
an id.

IOCTL_KGSL_GPUMEM_FREE_ID frees an id. KGSL_SHAREDMEM_FREE
can still be used to free by GPU address, if it exists.

The id can also be passed to mmap(), shifted left by
PAGE_SIZE to get a CPU mapping for the buffer.

IOCTL_KGSL_GPUMEM_ALLOC_GET_INFO can be called to retrieve
the id and other information about the buffer.

Change-Id: I4b45f0660cb9d4a5fb1323ccc6c4aa360791c1ec
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>

msm: kgsl: Support user-specified caching hints

The user allocating memory can provide hints for the caching
settings when said buffer is mapped on the user side. Try
to obey the cache settings when applicable.  The user will
be responsible for handling cache management both to and
from the GPU so add a new ioctl IOCTL_KGSL_GPUMEM_CACHE_SYNC to
support both directions (the old IOCTL_KGSL_SHAREDMEM_FLUSH_CACHE)
only handled flushing, not invalidating. The legacy ioctl still
works it just does what it says it does - a cache flush.

Change-Id: Ic0dedbad55ce82f2b01ebc56de30d4649e2e7311
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>

msm: kgsl: disable use of iommu TTBR1

The v1 iommu only supports splitting between TTBR0 and TTBR1
on a power of two boundary. Cutting off the userspace address
at 2G (0x80000000) is inconvienient, as the GPU userspace
address space should align with the CPU address space.

This requires changing how global allocations are managed,
since there is no longer a separate pagetable for TTBR1.
The default pagetable is still the master of these allocations
and maintains the gen_pool for allocating global addresses.
But now, these regions are mapped into each process pagetable
by calling kgsl_setup_pt(). This requires kgsl_mmu_map
and kgsl_mmu_unmap to be able to handle mapping without
virtual address allocation.

Change-Id: I94e2d63dc7e6a7ef576f993770725b6b7ba14228
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>

msm: kgsl: Add memfree_history debug feature

A kgsl user can free up an allocation of GPU memory with
kgsl_ioctl_sharedmem_free() function, but the GPU address
might still be in use resulting in a GPU page fault.
A bug of this kind is hard to find because the driver keeps
no trace of kgsl_ioctl_sharedmem_free calls.
This change implements a "memfree history" debug feture that
allows to see an information about a last couple of hundred
memory free calls in kernel/debug/kgsl/kgsl-3d0/memfree_history
file.

Change-Id: I12e0e3702db56d99d5de644739b364dea4cc37b1
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>

msm: kgsl: clean up use of memdesc->hostptr

Some external memory types were using hostptr to store
a userspace virtual address, but other code assumes it is a
kernel virtual address. Make memdesc->hostpr always
be the kernel virtual address and add a useraddr
field for the userspace virtual address.

Change-Id: Id4580a2ff34aeb15f2c1b26a7134f0fd4ec52a6e
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>

msm: kgsl: initialize kgsl_sync_timeline_ops properly

Change-Id: Ib8dccb60ead6fcecc0f9beaf271ba28199d404a2
Signed-off-by: Iliyan Malchev <malchev@google.com>
Signed-off-by: Jeff Boody <jboody@quicinc.com>

msm: kgsl: Add CP_WAIT_FOR_IDLE packet before updating timestamp

HW workaround to resolve MMU pagefault caused by memory
freeing early before GPU completes it.

This is race condition where timestamp is getting updated
and the event handler runs before the GPU actually got done
with the timestamp, hence freeing the buffer early while GPU
still not completed with it.

Change-Id: I6d338d7dca680f57f2c39ed5a14381b261bf8bce
Signed-off-by: Anshuman Dani <adani@codeaurora.org>

msm: kgsl: Return correct timestamp for consumed timestamp reads

adreno_readtimestamp() would return a register value when asked to
get the KGSL_TIMESTAMP_CONSUMED value.  With per-context timestamps
this no longer holds valid since the value in the register could
correspond to a different contexts timestamp.  Return the start of
packet timestamp for the specified context now that the start of
timestamp values are correctly submitted.

CRs-fixed: 382888
Change-Id: Iad56d07eb4a2973a5906d989c96dd117ff902cea
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>

msm: kgsl: Add identifier for internal command submissions

Add an identifier to label internal commands being sent to the
ringbuffer.  This will be used to further enable debugging
and hang recovery.

Change-Id: I4b3c10f0c3d290ae092182cbb450abe65e9dda80
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>

msm: kgsl: Optimize clock call order

Eliminate the possibility of duplicate RPM calls while using
set_rate in addition to prepare/unprepare.

Change-Id: I74905c2f79b7f78047798b6e8022ded2fce5bf33
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>

msm: kgsl: Change bus frequency before GPU frequency

On frequency increase the GPU may be able to overflow the
bandwidth it has requested.  Increase the bus first to be
sure that isn't an issue.

Change-Id: Ief21ce5263196e0e0c537901c91986b178fb8910
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>

msm: kgsl: Handle a possible ringbuffer allocspace error

In the GPU specific start functions, account for the possibility
that ringbuffer allocation routine might return NULL.

Change-Id: Ic0dedbadf6199fee78b6a8c8210a1e76961873a0
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>

msm: kgsl: Using ion allocation instead of pmem for GEM

In GEM object allocations that are requested to be in
physical memory, pmem was used in the past.  Since pmem
has been replaced with ion, we have updated the code to
use ion to manage memory in DRM/GEM that needs to be
shared with MDP.

Change-Id: I245b863ada6778d2206baf59a98124d39ee89d4a
Signed-off-by: Michael Street <mstreet@codeaurora.org>

msm: kgsl: Added DRM_IOCTL_KGSL_GEM_GET_ION_FD ioctl to DRM device.

To allow for user space ion based zero copy operations
(such as camera preview) on X11 allocated buffers (e.g. Pixmaps).
This ioctl returns a sharable ION file descriptor given a GEM object.

Change-Id: I055712b8fa469c99a224933a1f2fd62d2d811536
Signed-off-by: Michael Street <mstreet@codeaurora.org>

msm: kgsl: Added IOMMU support for GEM allocations in DRM

To support chipsets that use IOMMU, added ION
allocations from IOMMU heap and map them into
the GPU. This is used in the DRM driver for MSM.
Makefile change for include location changes in 3.7.

Change-Id: I32d71e749c6269261957c58324105a15d11ce104
Signed-off-by: Michael Street <mstreet@codeaurora.org>

msm: kgsl: Added DRM_IOCTL_KGSL_GEM_CREATE_FROM_ION to DRM

This ioctl allows a userspace program to allocate an ION buffer
and create a GEM handle that uses this memory so that it can
be shared with programs that utilize DRM/GEM sharing protocols
such as X11.

CRs-Fixed: 453003
Change-Id: I5dab19451b98017d9076b7688bcb1314dcc75944
Signed-off-by: Michael Street <mstreet@codeaurora.org>

msm: kgsl: use readonly mappings on the iommu

Userspace passes a flag KGSL_MEMFLAGS_GPUREADONLY when
the gpu should not write to a memory region. Use this
flag to control IOMMU_WRITE permissions on the buffer.

Change-Id: I5d3fc615dc36687252e2242f63fe74d6ce1c4fbc
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>

msm: kgsl: Print the nearest active GPU buffers to a faulting address

Print the two active GPU memory entries that bracket a faulting GPU
address. This will help diagnose premature frees and buffer ovverruns.

Check if the faulting GPU address was freed by the same process.

Change-Id: Ic0dedbadebf57be9abe925a45611de8e597447ea
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>

msm: kgsl: Remove an uneeded register write for A3XX GPUs

A3XX doesn't have the MH block and so the register at 0x40 points
somewhere else. Luckily the write was harmless but remove it anyway.

Change-Id: Ic0dedbadd1e043cd38bbaec8fcf0c490dcdedc8c
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>

msm: kgsl: Idle GPU core before programming SMMU from CPU

Always idle the GPU core before programming SMMU from CPU for
SMMU-v1. GPU core was already being idled before programming
the pagetable register, make sure that it's also idle before
programming the tlb invalidate registers. This is required to
prevent a deadlock from happening at the bus level.

Change-Id: Ie901b92028b289fc546ab6186eedd01411d0727e
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>

msm: kgsl: Lock ringbuffer translation in TLB

Lock ringbuffer virtual address translation in TLB of IOMMU. This
is required to prevent a hardware table walk from happening when
GPU executes commands that program the IOMMU registers. If a hardware
table walk happens when GPU is programming IOMMU register then it
can cause a deadlock on the bus.

Change-Id: I99428be7879a5210ba816f5f96864f94714649d7
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>

msm: kgsl: allow consistent CPU and GPU mappings

KGSL_GPUMEM_ALLOC_ID now takes a flag,
KGSL_MEMFLAGS_USE_CPU_MAP. When set, the GPU
mapping will be set up to match the CPU mapping
during mmap().  This feature is only supported when
using per process pagetables with the IOMMU. The
flags field of KGSL_GPUMEM_ALLOC_ID is copied back
to userspace and KGSL_MEMFLAGS_USE_CPU_MAP will
be cleared when this feature is not supported.

The IOMMU virtual address space has been adjusted
when perprocess pagetables is enabled so that the
entire userpace address range (0 to TASK_SIZE) can
have equivalent mappings on the IOMMU. For buffers
that do not have equivalent mappings, the address
range from PAGE_OFFSET to KGSL_IOMMU_GLOBAL_MEM_BASE
is used.

Change-Id: Ib61c03aa7453c3dd901c41e8fd297f66d402ae1a
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>

msm: kgsl: clean up iommu/gpummu protflag handling

Make kgsl_memdesc_protflags() return the correct type of flags
for the type of mmu being used. Query the memdesc with this
function in kgsl_mmu_map(), rather than passing in the
protflags. This prevents translation at multiple layers of
the code and makes it easier to enforce that the mapping matches
the allocation flags.

Change-Id: I2a2f4a43026ae903dd134be00e646d258a83f79f
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>

msm: kgsl: remove kgsl_mem_entry.flags

The two flags fields in kgsl_memdesc should be enough for
anyone.  Move the only flag using kgsl_mem_entry, the
FROZEN flag for snapshot procesing, to use kgsl_memdesc.priv.

Change-Id: Ia12b9a6e6c1f5b5e57fa461b04ecc3d1705f2eaf
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>

msm: kgsl: don't fault in cached allocations

Make sure cache operations don't hit pagefaults by
backing the entire vma in mmap() instead of faulting
in pages as they're touched. Otherwise, there's a
chance that a later cache operation on the mapping
could trigger an unhandled page fault leading to
a kernel panic.

Change-Id: Ia73c8aaed2708c5b9ef46ed50fb0f5cf1ad2450c
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>

msm: kgsl: Clean up warnings and a mismerge

Change-Id: I087e11ae6ea492e4fc1be35716bacc73cef549c0

msm: kgsl: avoid vmap failure while allocating big buffers

Limit the amount of pages vmapped at one time during allocation,
to avoid requesting more vmalloc space than is likely available.

CRs-Fixed: 445005
Change-Id: Icee5912687edf4da585d88308ee0e1c971964785
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>

msm: kgsl: Copy data back to user in case of error

In case of an error, copy back any data that KGSL might
have changed on purpose.  This is to prevent the user from
ignoring the error and using the values that were passed in.
This means that in error cases, KGSL needs to be mindful
and reset certain values (such as a buffer size to 0) before
returning the error to the user

Change-Id: I9201ce4361c9fd1d6477f18c67f5c651e8408bd2
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>

msm: kgsl: Check for zero length buffers with ION mappings

When ION buffers are mapped into GPU virtual address space,
trigger an error if a buffer of size zero is requested to be mapped.

Change-Id: I59d9dcee0b6eaf38f886a6b5b84b5fb7b16c4c06
Acked-by: Sunil Kumar <sdevalok@qualcomm.com>
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>

msm: kgsl: Align ION buffer mappings from userspace to 4k

When userspace allocates an ION buffer and maps it to GPU virtual space,
adjust the buffer size to be 4k aligned before mapping it to GPU virtual
space. GPU virtual space map routine expects the buffer to be 4k aligned.

Change-Id: Ibdb95597c24e2b5d0c3390d2c8550abb6b4b730b
Acked-by: Rama Vaddula <rvaddula@qualcomm.com>
Signed-off-by: Tarun Karra <tkarra@codeaurora.org>

msm: kgsl: Use ERR_PTR to return errors from kgsl_create_context()

Normally kgsl_create_context() returns a pointer to a new context.
In case of errors it just returns NULL pointer, and that doesn't allow
to propagate the right error code back to the user via
kgsl_ioctl_drawctxt_create()
This fix modifies kgsl_create_context to use ERR_PTR macro for passing
error codes to its caller.

Change-Id: I447c1765828912b0994bcae67a73864e62eef9b6
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>

msm: kgsl: prevent multiple or partial mmaps of a buffer

These cases have never been a normal operation pattern of the
userspace driver. Sharing buffers through multiple mappings
is better left to dma-buf or ion.

Change-Id: I7e7658137937c96b9505d0f912dcb262d652e0c3
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>

msm: kgsl: Add a ftrace event for GPU busy

Log a ftrace event when the GPU statistics are collected so that
we can keep track of GPU busy along with the other statistics
in the ftrace stream.

Change-Id: Ic0dedbadb58b9bef6e8669a7634bf9ca87073bcb
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>

msm: kgsl: Set a useful pm_qos default value

Also allow the value to be updated via sysfs.

CRs-Fixed: 434556
Change-Id: Ia69cafd44f48895b9d71e184efdb3cd253cec75b
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>

msm: kgsl: Track GPU device resets

Keep track of the number of times the GPU power rails have been
cycled and provide said number to the user in the device sysfs
directory.

Change-Id: Ic0dedbad4489524a98e52b9993131257068ae2ef
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>

msm: kgsl: Revert low latency power collapse

Standalone PC causes a watchdog bark on some hardware
during boot.  Disable it until the issue is resolved.

Change-Id: I9eb7be66cdb001db10849424a59064ad41f9db22
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>

msm: kgsl: Lower bounds check the number of power levels

A misconfigured board file could cause an out-of-bounds
array access through this value.

Change-Id: I64bbad0c096a2efe65376991537d810141370d72
CRs-Fixed: 439230
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>

msm: kgsl: Idle timeout should be reported in msec

Remove the restrictions on the low end of the idle timeout.
Accept the idle timeout in msec and also display it in msec.

Change-Id: I8ca4cd7ade503ed24424e4e429243e921eb6fb0f
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>

msm: kgsl: Set a useful pm_qos default value

Set the value to allow low latency power collapse.
This reverses commit 2d65af80aa2ed831d875beec4f3c3223e6be5a06
which disabled low latency PC due to a boot up stability
issue that was resolved in TZ.

Change-Id: I4ecfd9e1bfeed90c82eeec9e1dcb99c8083745e9
Signed-off-by: Patrick Cain <pcain@codeaurora.org>

msm: kgsl: Add intermediate power levels

Make use of the two independent power domains to
add intermediate levels.  Attempt to just increase
the bus frequency before also raising the GPU
frequency.

Change-Id: Ie2109df001f96e36e03959288aa957d63c0ef4a7
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>

msm: kgsl: Log retired timestamps on device wake

Knowing the retired timestamp on wake-up is useful to debug hangs that
are caused due to clock instability. When we request for clock ON, the
clock may/may-not be turned ON, but we go ahead assuming it is ON. This
can be caught by printing the timestamp on wake. The timestamp we print,
can be compared against the timestamp in postmortem dump (assuming a
hang happens).

If there is a difference, we can say that GPU has run for sometime after
wake. So, we needn't suspect clocks.

If there is no difference, we can say that GPU has hung immediately after
wake. So, we can positively suspect the clocks.

Change-Id: Iedf49977adbb0e61a9b71bbe23acc81f2a75162d
Signed-off-by: Ranjhith Kalisamy <ranjhith@codeaurora.org>

msm: kgsl: pwrscale functions should be run while GPU is ACTIVE

Base the decision to call power scaling functions on the current
state, not the requested state.  The GPU may not transition to the
requested state; call through based on current GPU state instead.

Change-Id: I1524bc92c908dc50a60b5815f2221ae3feef59be
CRs-Fixed: 441847
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>

msm: kgsl: add ftrace for cache operations

Add the event kgsl_mem_sync_cache. This event is
emitted when only a cache operation is actually
performed. Attempts to flush uncached memory,
which do nothing, do not cause this event.

Change-Id: Id4a940a6b50e08b54fbef0025c4b8aaa71641462
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>

msm: kgsl: Add a ftrace event for register writes

It is interesting to know when and how we program the hardware so
add a tracepoint to monitor register writes.  Register reads are
not traced since they are done much more frequently and are somewhat
less interesting from a debug perspective.

Change-Id: Ic0dedbad8ee85f154b9ba8731fe2c14603351243
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>

msm: kgsl: Replace 10secs timeout with Z180_IDLE_TIMEOUT for 2D.

Use Z180_IDLE_TIMEOUT macro for 2D hang detection instead
of hardcoded value of 10 secs.

Change-Id: I39f9d3ff9ef00df2f3d399454279045fe68e471a
CRs-Fixed: 454439
Signed-off-by: Vikram Orre <vorre@codeaurora.org>

msm: kgsl: Remove ts_notifier_list

Remove ts_notifier_list from the driver since it is not being
used and is causing extra work to be done in the interrupt
handlers for A2XX, A3XX and Z180.

Change-Id: I5512e36f1e807f3a3e62aeac54cfd3075d4cf7a4
Signed-off-by: Carter Cooper <ccooper@codeaurora.org>

msm: kgsl: Fix warnings

Change-Id: Ie741fea8932043dba53683531b717f257b0c2db2

msm: kgsl: Do not dump the istore on A3XX cores

The instruction store on A3XX GPU cores isn't really an instruction store
as on A2XX and it doesn't give us any debugging value. On top of all
that it can be unstable to read in some situations. Since it is of
questionable value and questionable stability pull it from the snapshot
for all A3XX cores and move the functionality to the A2XX specific code.

Change-Id: Ic0dedbada32d8b6b9b93768d148c2624faf3ef30
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>

msm: kgsl: Correct the initial idle_timer value

The gpu idle_timer initial value is incorrect in the DT for 8974

Change-Id: I07cd42c93070162bf520cfc9cfa7e15a46b2c2e8
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>

msm: kgsl: Remove all reference to CONFIG_ANDROID_PMEM

CONFIG_ANDROID_PMEM is completely deprecated. Remove all
references to it.

Change-Id: I34039709b0dc6cd5bb96497a8df971f420836023
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>

msm: kgsl: snapshot: Don't keep parsing indirect buffers on failure

Stop parsing an indirect buffer if an error is encountered (such as
a missing buffer). This is a pretty good indication that the buffers
are not reliable and the further the parser goes with a unreliable
buffer the more likely it is to get confused.

Change-Id: Ic0dedbadf28ef374c9afe70613048d3c31078ec6
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>

msm: kgsl: Add the buffer user address to the debugfs memory list

For each buffer in the debugfs memory list print the address it is
mapped to in the user process.  This is useful for matching up the
user side allocations with what the kernel thinks it has.

Change-Id: Ic0dedbadd07a8f704c0bf73b183aaeeddbbdd129
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>

msm: kgsl: snapshot: Only push the last IB1 and IB2 in the static space

Some IB1 buffers have hundreds of little IB2 buffers and only one of them
will actually be interesting enough to push into the static space.  Only
push the last executed IB1 and IB2 into the static space.

Change-Id: Ic0dedbad26fb30fb5bf90c37c29061fd962dd746
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>

msm: kgsl: An error condition could cause an array overflow

In kgsl_setup_pt() if the MMU specific setup_pt function fails
the code will attempt to unwind the setup but it will use an
out of bounds index to do so.

Change-Id: Ic0dedbad8b23f65d4b52cb5ba01810ff26b19b89
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>