gpu: ion: create separate heap and client debugfs directories

It can be slightly annoying to figure out which files under the ion
debugfs directory are heap debug files and which ones are client debug
files. Create separate subdirectories under ion to hold the different
types of debug files.

Change-Id: Ic773ab619ef94b9b4d0f3794def9d37645c7c212
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: don't use "vmalloc" for system heap readable name

The human-readable name of the system heap is currently "vmalloc", which
is misleading since we don't use vmalloc at all for that heap, we use
alloc_pages. Make the human-readable name of the heap "system".

Change-Id: Iefb92ea8adc8d76b8eccc504010aa9bc2fa5c50c
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: optimize zero'ing of pages

Instead of using __GFP_ZERO, vmap high-order pages for zero'ing all at
once (or in a few large chunks).

Test results show a ~25% speedup for allocations that are fulfilled from
lowmem, and ~100% speedup for allocations that are fulfilled from
highmem.

Change-Id: I57270361ff195780370e1c6f458985bdbb641efe
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: add page pooling to cached allocations from the system heap

Add a page pool for cached allocations from the system heap.

Change-Id: I6d7688026ff4ee65efa047a164652da1427e72c4
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: make sure all clients are exposed in debugfs

Currently, if multiple Ion clients are created with the same name, only
the first one shows up in debugfs. Rectify this by adding a
monotonically-increasing serial number to the debug names of Ion
clients.

CRs-Fixed: 566001
Change-Id: I000e45055d5029c7bccd88c36b238736929da3a5
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: Support probe deferal for client creation

Ion is treated like any other device in the system. Clients
may try to initialize before Ion is setup. If the Ion driver
has not yet probed, return -EPROBE_DEFER to try again later.
Since the flow is being restructured to support probe deferal,
also include some better error handling for when Ion returns
an error while probing.

Change-Id: I7ee6dde2c837fba8bba0ba4aaddee3af37b2fec1
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
gpu: ion: fix zero'ing of cached buffers in system heap

Currently, in the system heap we zero out buffers at free-time when they
came from uncached allocations. The reason for this is that uncached
allocations come from the page pools, which require this free-time
zero'ing. However, the assumption that a buffer came from the page pools
iff it was an uncached allocation is no longer valid since [0204b38
"gpu: ion: add page pooling to cached allocations from the system heap"]
introduced page pooling for cached allocations as well. This results in
us not zero'ing out cached buffers before putting them back in the page
pools, which is a security bug (we could potentially leak sensitive
information).

Fix this by zero'ing out all buffers that are going back into the page
pools (cached and uncached).

Change-Id: I5c51eca75c8cc3b3823df1494bb0eef295c77b86
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: kmalloc_heap: flush cpu caches at allocation time

There's currently a problem with the kmalloc heap where buffers
occasionally come through unzero'd when mmap'd to userspace, which is a
security hole. We actually are zero'ing the buffer but that data isn't
immediately visible to folks with other mappings to the memory (e.g. the
mapping set up for userspace by the mmap handler). Fix this by flushing
the cpu caches at alloc time.

Change-Id: I3895c6bc163e2d065b48cae66e99825c5c0a865b
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: system_heap: invalidate cache on cached pool allocations

We need to invalidate the cache when allocating pages for the cached
page pools to avoid the possibility of a stale cached value from a
different mapping being written back after we allocate our page.

Change-Id: I9b0ebfd143850cf4a4ef309a264f947a57abff77
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: use a list instead of a tree for heap debug memory map

Currently we use an rb tree to store information about the memory map
which gets passed to the heap print_debug functions. The reason for
using a tree instead of a simple list is to maintain sortedness as we
build the memory map. However, it can be necessary to store multiple
entries for the same address in the memory map since there can be
multiple clients with handles to the same buffer. This information is
interesting and useful but we currently can't store and display it since
the rb tree requires that the key used for sorting (the physical address
in this case) be unique. Fix this by replacing the rb tree with a linked
list. In order to maintain sorted output of the print_debug functions,
sort the list by physical address after fully building the list.

This also has the positive side-effect of simplifying the code and
making future print_debug methods less error-prone.

CRs-Fixed: 571918
Change-Id: I5b129fd809fb53c66042eab10d096238a34c2b20
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: fix locking issues in debug code

There are a few places in Ion where we are iterating over volatile rb
trees without proper locking. In some places the proper locking cannot
be added since it would require us to take locks in a different order
than they are taken in other places in Ion. Fix this by re-working some
of the debug code so that we can take locks in an allowed order.

One side-effect of the re-work is that the memory maps will now show
every client that has a handle to a particular region of memory, rather
than just showing the first one that we encounter. This will allow for
more accurate accounting and will give better insight as to who is
actually using the memory.

CRs-Fixed: 571918
Change-Id: Ia43e4dbc412cd480c828173f8c20b5095d87d858
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: Explictly mark DMA allocations as GFP_KERNEL

For legacy reasons, the GFP allocation for dma allocations is currently
marked as 0, which corresponds to atomic allocations. These allocations
should not be marked as atomic since the atomic pool is very small.
Switch to properly using GFP_KERNEL to always allocate from the
correct pool.

Change-Id: Id3ce68b538a3d6dbdc8857109cca9ec8706ca4ef
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
gpu: ion: Flush the chunk list before passing it to trustzone

Previously, the function scm_call used to take care of flushing
the caches to ensure all data was written back to memory before
going into the secure world. The behavior of scm_call is changing
such that clients are expected to flush caches themselves. Since
the address that the chunk list is written to is cached, we need
to ensure that the data is actually written back. Add the appropriate
cache operations on the chunk list.

Change-Id: I05e6fa868409255e66f1f37f98199c5a7e81882c
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
msm: kgsl: Protect against a potential overflow in kgsl_sg_alloc

Avoid allocating a scatterlist so large that it overflows the size
passed to the memory allocators.

CRs-fixed: 564448
Change-Id: Ic0dedbad2ab421ddec8f3be38d61c9bdf9ae5bd4
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
iommu: msm: Refactor use of remote spinlock

The remote spinlock was originally meant to synchronize access between
GPU and CPU accessing SMMU. However, there are more use cases that
this remote spinlock serves. This change decouples the spinlock from
GPU and adds support for taking the spinlock only for SMMUs that are
specified in the device tree.

CRs-fixed: 517873
Change-Id: Ic50992d0d1a102fbd05855e09e254e627f99ec33
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>

Conflicts:
	arch/arm/mach-msm/include/mach/iommu.h
	drivers/iommu/msm_iommu-v0.c
kgsl: Implement conservative scaling policy

This behaves similarly to the conservative cpufreq governor,
which is similar to how most Mali GPU scaling algorithms behave.

The "trustzone" policy that is default relies on an undocumented
black box to make scaling decisions.

How to enable:
echo conservative > /sys/devices/platform/kgsl-3d0.0/kgsl/kgsl-3d0/pwrscale/policy

Conservative attr group:
/sys/devices/platform/kgsl-3d0.0/kgsl/kgsl-3d0/pwrscale/policy_config/conservative

Polling interval (10-1000):
/sys/devices/platform/kgsl-3d0.0/kgsl/kgsl-3d0/pwrscale/policy_config/conservative/polling_interval

Enable stats printing (causes microlags on low polling intervals):
echo 1 > /sys/devices/platform/kgsl-3d0.0/kgsl/kgsl-3d0/pwrscale/policy_config/conservative/print_stats

NOTE:
- perms are set to 666 for testing purposes

Change-Id: I9d854a73981515fc9850104302804928b04c4687
Signed-off-by: Andrew Dodd <atd7@cornell.edu>
Simple GPU governor: fix a merge derp

Signed-off-by: Paul Reioux <reioux@gmail.com>
gpu: ion: system_heap: add support for 2MB allocations

Enable 2MB (order 9) allocations from the system heap. This improves the
performance of some heavyweight use cases. The benefit is especially
large for use cases where the buffer is mapped into an IOMMU since we
can leverage the 2MB IOMMU page table mappings.

Change-Id: I869733089b9ccba8090bbca80aac7b5f248b335e
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
ion: system_heap: only use low gfp flags for order-0 pages

By trying a little harder to get high-order pages (by virtue of the
liberal gfp flags used for high-order allocations) we can actually
optimize performance since high-order pages seem to help a lot. Only use
the low-order flags as a last resort, when trying to allocate an order-0
page.

Change-Id: If8d8f4b1b4503b0c7546cf5a4bcdcaecd2bb5905
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
msm: kgsl: Remove sscanf() from kgsl_pwrctrl.c

Replace sscanf() in kgsl_pwrctrl.c with a generic function to safely
convert 32 bit unsigned ints.

CRs-fixed: 563585
Change-Id: Ic0dedbad10e2264ec8fccab622d094d123b49b77
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>

Signed-off-by: Paul Reioux <reioux@gmail.com>