gpu: ion: create separate heap and client debugfs directories

It can be slightly annoying to figure out which files under the ion
debugfs directory are heap debug files and which ones are client debug
files. Create separate subdirectories under ion to hold the different
types of debug files.

Change-Id: Ic773ab619ef94b9b4d0f3794def9d37645c7c212
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: don't use "vmalloc" for system heap readable name

The human-readable name of the system heap is currently "vmalloc", which
is misleading since we don't use vmalloc at all for that heap, we use
alloc_pages. Make the human-readable name of the heap "system".

Change-Id: Iefb92ea8adc8d76b8eccc504010aa9bc2fa5c50c
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: optimize zero'ing of pages

Instead of using __GFP_ZERO, vmap high-order pages for zero'ing all at
once (or in a few large chunks).

Test results show a ~25% speedup for allocations that are fulfilled from
lowmem, and ~100% speedup for allocations that are fulfilled from
highmem.

Change-Id: I57270361ff195780370e1c6f458985bdbb641efe
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: add page pooling to cached allocations from the system heap

Add a page pool for cached allocations from the system heap.

Change-Id: I6d7688026ff4ee65efa047a164652da1427e72c4
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: make sure all clients are exposed in debugfs

Currently, if multiple Ion clients are created with the same name, only
the first one shows up in debugfs. Rectify this by adding a
monotonically-increasing serial number to the debug names of Ion
clients.

CRs-Fixed: 566001
Change-Id: I000e45055d5029c7bccd88c36b238736929da3a5
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: Support probe deferal for client creation

Ion is treated like any other device in the system. Clients
may try to initialize before Ion is setup. If the Ion driver
has not yet probed, return -EPROBE_DEFER to try again later.
Since the flow is being restructured to support probe deferal,
also include some better error handling for when Ion returns
an error while probing.

Change-Id: I7ee6dde2c837fba8bba0ba4aaddee3af37b2fec1
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
gpu: ion: fix zero'ing of cached buffers in system heap

Currently, in the system heap we zero out buffers at free-time when they
came from uncached allocations. The reason for this is that uncached
allocations come from the page pools, which require this free-time
zero'ing. However, the assumption that a buffer came from the page pools
iff it was an uncached allocation is no longer valid since [0204b38
"gpu: ion: add page pooling to cached allocations from the system heap"]
introduced page pooling for cached allocations as well. This results in
us not zero'ing out cached buffers before putting them back in the page
pools, which is a security bug (we could potentially leak sensitive
information).

Fix this by zero'ing out all buffers that are going back into the page
pools (cached and uncached).

Change-Id: I5c51eca75c8cc3b3823df1494bb0eef295c77b86
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: kmalloc_heap: flush cpu caches at allocation time

There's currently a problem with the kmalloc heap where buffers
occasionally come through unzero'd when mmap'd to userspace, which is a
security hole. We actually are zero'ing the buffer but that data isn't
immediately visible to folks with other mappings to the memory (e.g. the
mapping set up for userspace by the mmap handler). Fix this by flushing
the cpu caches at alloc time.

Change-Id: I3895c6bc163e2d065b48cae66e99825c5c0a865b
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: system_heap: invalidate cache on cached pool allocations

We need to invalidate the cache when allocating pages for the cached
page pools to avoid the possibility of a stale cached value from a
different mapping being written back after we allocate our page.

Change-Id: I9b0ebfd143850cf4a4ef309a264f947a57abff77
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: use a list instead of a tree for heap debug memory map

Currently we use an rb tree to store information about the memory map
which gets passed to the heap print_debug functions. The reason for
using a tree instead of a simple list is to maintain sortedness as we
build the memory map. However, it can be necessary to store multiple
entries for the same address in the memory map since there can be
multiple clients with handles to the same buffer. This information is
interesting and useful but we currently can't store and display it since
the rb tree requires that the key used for sorting (the physical address
in this case) be unique. Fix this by replacing the rb tree with a linked
list. In order to maintain sorted output of the print_debug functions,
sort the list by physical address after fully building the list.

This also has the positive side-effect of simplifying the code and
making future print_debug methods less error-prone.

CRs-Fixed: 571918
Change-Id: I5b129fd809fb53c66042eab10d096238a34c2b20
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: fix locking issues in debug code

There are a few places in Ion where we are iterating over volatile rb
trees without proper locking. In some places the proper locking cannot
be added since it would require us to take locks in a different order
than they are taken in other places in Ion. Fix this by re-working some
of the debug code so that we can take locks in an allowed order.

One side-effect of the re-work is that the memory maps will now show
every client that has a handle to a particular region of memory, rather
than just showing the first one that we encounter. This will allow for
more accurate accounting and will give better insight as to who is
actually using the memory.

CRs-Fixed: 571918
Change-Id: Ia43e4dbc412cd480c828173f8c20b5095d87d858
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
gpu: ion: add locking to traversal of volatile rb tree

In ion_debug_heap_show we're iterating over an rb tree (dev->clients)
that could change while we're iterating. Fix this by taking the lock
that is used to control access to this tree.

CRs-Fixed: 571918
Change-Id: I6832e1e98e2d2a69fc653451d3752d43ec3ef269
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>