Revert "cma: Remove potential deadlock situation"

This reverts commit b46a91befe5afacac03f721513e5813a922672af.
cma: Allow option to use strict memblock_reserved memory

Despite all the performance optimizations, some clients are
still unable to use CMA because of the allocation latency.
Rather than make those clients use a separate set of APIs,
extend the CMA code to allow clients to keep the memory out of
the buddy allocator. Since the pages never actually go to the
buddy allocator, allocation and freeing is only based on the bitmap
allocator to find the appropriate region.

Change-Id: Ia31bb1212fd7b19280361128453c8d25369ce592
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
cma: Remove potential deadlock situation

A report was given of a deadlock situation on the cma_mutex:

mutex_lock(cma_mutex)   (*2)
dma_release_from_contiguous
ion_secure_cma_free_chunk
ion_secure_cma_shrinker
shrink_slab
try_to_free_pages
migrate_pages
alloc_contig_range
mutex_lock(cma_mutex)  (*1)
dma_alloc_from_contiguous
ion_alloc

We may need to free CMA allocations while a current CMA allocation is in
progress if CMA is freed from a shrinker. cma_mutex currently protects
two things: the bitmap indicating which pages are allocated/free and
serialization of isolation/migration calls on allocation. There is
no need to take the mutex on free calls though as the pages are
freed back into the system via the regular __free_page call. Move
the free_contig_range call outside the cma_mutex to break the
chain dependency. We can safely free the pages back into the system
before changing the bitmap without any risk of races.

Change-Id: I4989eb2891e502b08db8117a51bd86652e902778
CRs-Fixed: 619644
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
cma: Increase the default number of regions

The total number of CMA regions that can be created is statically
decided at compile time via a config option. Incoming memory map
changes increase the number of CMA regions significantly across
almost all targets. Rather than make every target set the CONFIG,
just increase the default number. The additional overhead is very
small.

Change-Id: I2f60ee56b43c256f3bfa4d1e3ab95a8f378568ab
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
drivers: cma: Fix wrong CMA selected region size default value

Kconfig lists CMA_SIZE_SEL_ABSOLUTE as the default value fo the CMA
selected region size, but that option isn't available in the defined
choices. Set the default to CMA_SIZE_SEL_MBYTES instead.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
drivers: Add option to reserve default CMA region

CMA provides good utilization of memory but for some use cases, the
allocation time is too costly. Add a Kconfig option to allow the
default region to be permanently set aside for contiguous use cases.

Change-Id: I1eef508d37cf6ae3b7b7a652fc59391b186fc122
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
ARM: 7430/1: sizes.h: move from asm-generic to <linux/sizes.h>

sizes.h is used throughout the AMBA code and drivers, so the header
should be available to everyone in order to driver AMBA/PrimeCell
peripherals behind a PCI bridge where the host can be any platform
(I'm doing it under x86).

At this step <asm-generic/sizes.h> includes <linux/sizes.h>,
to allow a grace period for both in-tree and out-of-tree drivers.

Signed-off-by: Alessandro Rubini <rubini@gnudd.com>
Acked-by: Giancarlo Asnaghi <giancarlo.asnaghi@st.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Cc: Alan Cox <alan@linux.intel.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
cma: Add support for different size_cells and address_cells

Currently, the CMA flat device tree code does not take into account
targets that may specify size_cells and address_cells > 1. This will
lead to unsuccessful parsing. Add support for taking into account
nodes that may specify size_cells and address_cells explicitly.

Change-Id: I9ea63b8c34e903b186c29ec6555dd7a5c317c602
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
cma: Add support for memory limits

Currently, when dynamically placing regions CMA will allow the memory
to be placed anywhere, including highmem. Due to system restrictions,
regions may need to be placed in a smaller range. Add support to
devicetree to allow these regions to have an upper bound on where they
will be placed.

Change-Id: Ib4ae194cbb6389e1091e7e04cfd331e9ab67ad05
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
cma: print physical addresses correctly

Physical addresses can be greater than an unsigned long on
systems with LPAE. Print these addresses properly with %pa

Change-Id: If9f1572bb9dadcb14f2834647a4421509e90096a
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
cma: Make the default CMA region not reserved by default

Due to a bug in code logic, the default CMA region is
reserved when CONFIG_CMA_RESERVE_DEFAULT_AREA is NOT set
instead of when it is set. Fix this logic.

Change-Id: I476594c1e3745386741f2aba1b978a436b60c7a4
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
cma: Add support for status in DT nodes

CMA scans the flattened devicetree to reserve memory early. Check
for the status of a DT node to possibly skip initialization.

Change-Id: I58760ee5ff241a1ce3a93955b1e507d506f92162
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
msm: mdss: Allocate fb memory under 4GB physical address space

Linux kernel framework of framebuffer APIs do not support LPAE systems.
These APIs are using "unsigned long" data type instead of "phys_addr_t"
for physical addresses which doesn't work with LPAE systems. Until
framebuffer APIs are fixed, allocate fb memory on MSM chip-sets using
dma_alloc_coherent and GFP_KERNEL to get the memory from below 4GB
physical address space.

Change-Id: Ieca41cae21c956a8b4cf1d5729d287bb0bb30f8d
Signed-off-by: Ujwal Patel <ujwalp@codeaurora.org>
ARM: dts: msm: Move audio heaps over to use reserved CMA regions

Carveout regions are being deprecated. Convert the audio heaps over
to use a reserved CMA region instead of a carveout. At a later time,
the audio regions may be converted over to a fully functional CMA region.

Change-Id: I70d9c671ff118400ee60e669062a7b2092931536
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
ARM: dts: msm: Add reservation for framebuffer memory

The framebuffer needs contiguous memory to work. Add a
dedicated CMA region for framebuffer. This region is
specified with a default size which works for most targets
and overridden for targets that have a larger screen and
therefore require a large framebuffer.

Change-Id: If5f23884f6779ed325be15ee2e16e3bd23ba3698
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
ARM: dts: msm: Update size of modem pil heap

The modem pil heap size was incorrect. Updating the
size to reflect the correct size.

Change-Id: I1ec2a907ec1e1b74e31cbe1c23817b25057237bb
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
cma: Change to reserve-contiguous-region

Currently, devices are associated with CMA regions via phandle to the
CMA region. The phandle name 'linux,contiguous-region' is the same as
that which is used to designate a CMA region. This can lead to issues
of trying to treat a device only using a CMA region as an actual CMA
region. Rather than continuing to rely on this handle and the depth
to differentiate CMA regions, just create separate DT binding
linux,reserve-contiguous-region to indicate that the node is
an actual CMA node for reserving rather than a client referencing
via phandle.

Change-Id: I88b2d86054525b0569efc424da87974509ce9b25
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
cma: Add support for removed regions

In addition to reserving memory from the system, there may
be uses where memory should be completely removed from control
of the linux page allocator. Add the appropriate information to
be able to remove the memory.

Change-Id: Ia2e959e0858fb240ab5c0deee49b0d6e4aecfc00
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
drivers: Add dma-removed dma_ops

The current set of dma_ops only supports memory that's actually
backed by struct pages. In the general case this is what we want,
but there are some use cases where it's useful to allocate
removed memory through dma APIs (through CMA region for example).
Add a limited set of dma operations to accomplish this. This is
only the allocate/free version of the APIs which is all that is
necessary for these purposes.

Change-Id: I518e6965d63f3c95697f59c089b95219f8e6d96e
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>

ARM: DMA mapping: fix bad atomic test

Realview fails to boot with this warning:
BUG: spinlock lockup suspected on CPU#0, init/1
 lock: 0xcf8bde10, .magic: dead4ead, .owner: init/1, .owner_cpu: 0
Backtrace:
[<c00185d8>] (dump_backtrace+0x0/0x10c) from [<c03294e8>] (dump_stack+0x18/0x1c) r6:cf8bde10 r5:cf83d1c0 r4:cf8bde10 r3:cf83d1c0
[<c03294d0>] (dump_stack+0x0/0x1c) from [<c018926c>] (spin_dump+0x84/0x98)
[<c01891e8>] (spin_dump+0x0/0x98) from [<c0189460>] (do_raw_spin_lock+0x100/0x198)
[<c0189360>] (do_raw_spin_lock+0x0/0x198) from [<c032cbac>] (_raw_spin_lock+0x3c/0x44)
[<c032cb70>] (_raw_spin_lock+0x0/0x44) from [<c01c9224>] (pl011_console_write+0xe8/0x11c)
[<c01c913c>] (pl011_console_write+0x0/0x11c) from [<c002aea8>] (call_console_drivers.clone.7+0xdc/0x104)
[<c002adcc>] (call_console_drivers.clone.7+0x0/0x104) from [<c002b320>] (console_unlock+0x2e8/0x454)
[<c002b038>] (console_unlock+0x0/0x454) from [<c002b8b4>] (vprintk_emit+0x2d8/0x594)
[<c002b5dc>] (vprintk_emit+0x0/0x594) from [<c0329718>] (printk+0x3c/0x44)
[<c03296dc>] (printk+0x0/0x44) from [<c002929c>] (warn_slowpath_common+0x28/0x6c)
[<c0029274>] (warn_slowpath_common+0x0/0x6c) from [<c0029304>] (warn_slowpath_null+0x24/0x2c)
[<c00292e0>] (warn_slowpath_null+0x0/0x2c) from [<c0070ab0>] (lockdep_trace_alloc+0xd8/0xf0)
[<c00709d8>] (lockdep_trace_alloc+0x0/0xf0) from [<c00c0850>] (kmem_cache_alloc+0x24/0x11c)
[<c00c082c>] (kmem_cache_alloc+0x0/0x11c) from [<c00bb044>] (__get_vm_area_node.clone.24+0x7c/0x16c)
[<c00bafc8>] (__get_vm_area_node.clone.24+0x0/0x16c) from [<c00bb7b8>] (get_vm_area_caller+0x48/0x54)
[<c00bb770>] (get_vm_area_caller+0x0/0x54) from [<c0020064>] (__alloc_remap_buffer.clone.15+0x38/0xb8)
[<c002002c>] (__alloc_remap_buffer.clone.15+0x0/0xb8) from [<c0020244>] (__dma_alloc+0x160/0x2c8)
[<c00200e4>] (__dma_alloc+0x0/0x2c8) from [<c00204d8>] (arm_dma_alloc+0x88/0xa0)[<c0020450>] (arm_dma_alloc+0x0/0xa0) from [<c00beb00>] (dma_pool_alloc+0xcc/0x1a8)
[<c00bea34>] (dma_pool_alloc+0x0/0x1a8) from [<c01a9d14>] (pl08x_fill_llis_for_desc+0x28/0x568)
[<c01a9cec>] (pl08x_fill_llis_for_desc+0x0/0x568) from [<c01aab8c>] (pl08x_prep_slave_sg+0x258/0x3b0)
[<c01aa934>] (pl08x_prep_slave_sg+0x0/0x3b0) from [<c01c9f74>] (pl011_dma_tx_refill+0x140/0x288)
[<c01c9e34>] (pl011_dma_tx_refill+0x0/0x288) from [<c01ca748>] (pl011_start_tx+0xe4/0x120)
[<c01ca664>] (pl011_start_tx+0x0/0x120) from [<c01c54a4>] (__uart_start+0x48/0x4c)
[<c01c545c>] (__uart_start+0x0/0x4c) from [<c01c632c>] (uart_start+0x2c/0x3c)
[<c01c6300>] (uart_start+0x0/0x3c) from [<c01c795c>] (uart_write+0xcc/0xf4)
[<c01c7890>] (uart_write+0x0/0xf4) from [<c01b0384>] (n_tty_write+0x1c0/0x3e4)
[<c01b01c4>] (n_tty_write+0x0/0x3e4) from [<c01acfe8>] (tty_write+0x144/0x240)
[<c01acea4>] (tty_write+0x0/0x240) from [<c01ad17c>] (redirected_tty_write+0x98/0xac)
[<c01ad0e4>] (redirected_tty_write+0x0/0xac) from [<c00c371c>] (vfs_write+0xbc/0x150)
[<c00c3660>] (vfs_write+0x0/0x150) from [<c00c39c0>] (sys_write+0x4c/0x78)
[<c00c3974>] (sys_write+0x0/0x78) from [<c0014460>] (ret_fast_syscall+0x0/0x3c)

This happens because the DMA allocation code is not respecting atomic
allocations correctly.

GFP flags should not be tested for GFP_ATOMIC to determine if an
atomic allocation is being requested.  GFP_ATOMIC is not a flag but
a value.  The GFP bitmask flags are all prefixed with __GFP_.

The rest of the kernel tests for __GFP_WAIT not being set to indicate
an atomic allocation.  We need to do the same.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: dma-mapping: Small logical clean up

Skip unnecessary operations if order == 0. A little bit easier to
read.

Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: Refrain noisy console message

With many IOMMU'able devices, console gets noisy.

Tegra30 has a few dozen of IOMMU'able devices.

Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: add support for DMA_ATTR_SKIP_CPU_SYNC attribute

This patch adds support for DMA_ATTR_SKIP_CPU_SYNC attribute for
dma_(un)map_(single,page,sg) functions family. It lets dma mapping clients
to create a mapping for the buffer for the given device without performing
a CPU cache synchronization. CPU cache synchronization can be skipped for
the buffers which it is known that they are already in 'device' domain (CPU
caches have been already synchronized or there are only coherent mappings
for the buffer). For advanced users only, please use it with care.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: Kyungmin Park <kyungmin.park@samsung.com>

ARM: add coherent dma ops

arch_is_coherent is problematic as it is a global symbol. This
doesn't work for multi-platform kernels or platforms which can support
per device coherent DMA.

This adds arm_coherent_dma_ops to be used for devices which connected
coherently (i.e. to the ACP port on Cortex-A9 or A15). The arm_dma_ops
are modified at boot when arch_is_coherent is true.

Change-Id: I5f4f022c9374a6a6ab4e447bb213b3e94e9544ae
Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

common: DMA-mapping: add DMA_ATTR_SKIP_CPU_SYNC attribute

This patch adds DMA_ATTR_SKIP_CPU_SYNC attribute to the DMA-mapping
subsystem.

By default dma_map_{single,page,sg} functions family transfer a given
buffer from CPU domain to device domain. Some advanced use cases might
require sharing a buffer between more than one device. This requires
having a mapping created separately for each device and is usually
performed by calling dma_map_{single,page,sg} function more than once
for the given buffer with device pointer to each device taking part in
the buffer sharing. The first call transfers a buffer from 'CPU' domain
to 'device' domain, what synchronizes CPU caches for the given region
(usually it means that the cache has been flushed or invalidated
depending on the dma direction). However, next calls to
dma_map_{single,page,sg}() for other devices will perform exactly the
same sychronization operation on the CPU cache. CPU cache sychronization
might be a time consuming operation, especially if the buffers are
large, so it is highly recommended to avoid it if possible.
DMA_ATTR_SKIP_CPU_SYNC allows platform code to skip synchronization of
the CPU cache for the given buffer assuming that it has been already
transferred to 'device' domain. This attribute can be also used for
dma_unmap_{single,page,sg} functions family to force buffer to stay in
device domain after releasing a mapping for it. Use this attribute with
care!

Change-Id: I0735afb1a22922db459234c60a52812f43d03b5a
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: Kyungmin Park <kyungmin.park@samsung.com>

ARM: dma-mapping: add support for dma_get_sgtable()

This patch adds support for dma_get_sgtable() function which is required
to let drivers to share the buffers allocated by DMA-mapping subsystem.

Generic implementation based on virt_to_page() is not suitable for ARM
dma-mapping subsystem.

Change-Id: I34cc245abe7c368d90d429b5b3650f47c9722542
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: Kyungmin Park <kyungmin.park@samsung.com>

common: dma-mapping: add support for generic dma_mmap_* calls

Commit 9adc5374 ('common: dma-mapping: introduce mmap method') added a
generic method for implementing mmap user call to dma_map_ops structure.

This patch converts ARM and PowerPC architectures (the only providers of
dma_mmap_coherent/dma_mmap_writecombine calls) to use this generic
dma_map_ops based call and adds a generic cross architecture
definition for dma_mmap_attrs, dma_mmap_coherent, dma_mmap_writecombine
functions.

The generic mmap virt_to_page-based fallback implementation is provided for
architectures which don't provide their own implementation for mmap method.

Change-Id: I5e8b74846511e745f0f98a055add06e52ad64c6f
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: Kyungmin Park <kyungmin.park@samsung.com>

ARM: DMA-Mapping: add function for setting coherent pool size from platform code

Some platforms might require to increase atomic coherent pool to make
sure that their device will be able to allocate all their buffers from
atomic context. This function can be also used to decrease atomic
coherent pool size if coherent allocations are not used for the given
sub-platform.

Change-Id: I5d272e55fbf0be1c875b8b05b56abe4dd67a95a7
Suggested-by: Josh Coombs <josh.coombs@gmail.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: remove init_consistent_dma_size() stub

Since commit e9da6e9905e639b0 ("ARM: dma-mapping: remove custom consistent
dma region") setting consistent dma memory size is not longer required. All
calls to this function has been already removed, so the
init_consistent_dma_size() stub can also be gone.

Change-Id: I2d8cc0915b601e7bd4deb23f0c2e9fb0df1b8d47
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

common: dma-mapping: introduce dma_get_sgtable() function

This patch adds dma_get_sgtable() function which is required to let
drivers to share the buffers allocated by DMA-mapping subsystem. Right
now the driver gets a dma address of the allocated buffer and the kernel
virtual mapping for it. If it wants to share it with other device (= map
into its dma address space) it usually hacks around kernel virtual
addresses to get pointers to pages or assumes that both devices share
the DMA address space. Both solutions are just hacks for the special
cases, which should be avoided in the final version of buffer sharing.

To solve this issue in a generic way, a new call to DMA mapping has been
introduced - dma_get_sgtable(). It allocates a scatter-list which
describes the allocated buffer and lets the driver(s) to use it with
other device(s) by calling dma_map_sg() on it.

This patch provides a generic implementation based on virt_to_page()
call. Architectures which require more sophisticated translation might
provide their own get_sgtable() methods.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: Kyungmin Park <kyungmin.park@samsung.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>

ARM: dma-mapping: atomic_pool with struct page **pages

struct page **pages is necessary to align with non atomic path in
__iommu_get_pages(). atomic_pool() has the intialized **pages instead
of just *page.

Change-Id: Ia7a0dc13bb8584c6182dbc311a9777da788e1d5b
Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: DMA-mapping: add missing GFP_DMA flag for atomic buffer allocation

Atomic pool should always be allocated from DMA zone if such zone is
available in the system to avoid issues caused by limited dma mask of
any of the devices used for making an atomic allocation.

Change-Id: If6741c60c27cfb6754c441248b8f163a19eb6897
Reported-by: Krzysztof Halasa <khc@pm.waw.pl>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Stable <stable@vger.kernel.org>	[v3.6+]

ARM: dma-mapping: add support for CMA regions placed in highmem zone

This patch adds missing pieces to correctly support memory pages served
from CMA regions placed in high memory zones. Please note that the default
global CMA area is still put into lowmem and is limited by optional
architecture specific DMA zone. One can however put device specific CMA
regions in high memory zone to reduce lowmem usage.

Change-Id: Ia19124ea842df45413485fe28db8a32c9529fdb8
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Acked-by: Michal Nazarewicz <mina86@mina86.com>

cma: use pfn instead of pages for argument passing

The CMA code is generic enough that it can be expanded out to track
regions of memory that aren't officially managed by the regular page
allocator. This memory can't be referenced via struct page. Change the
CMA apis to track using pfn for allocation/free instead. The pfn can
be converted to a struct page as needed elsewhere.

Change-Id: I5ac3fa5e2169b2101a738177f1654faa401f7604
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>

ARM: another fix for the DMA mapping checks

Peter reports that OMAP audio broke with the recent fix for these
checks, caused by OMAP audio using a long DMA mask.  We should
allow long DMA masks even with dma_addr_t if we can be sure
the amount of RAM we have won't allow the dma_addr_t to
overflow.  Unfortunately, the checks to detect overflow were not
correct.

Change-Id: I90d92ccf96eea7cad36739022c5179d6b98473de
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Git-commit: 9f28cde0bc643186d32744ce69d12222e8588fe7
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
[ilialin@codeaurora.org: resolve trivial merge conflicts]
Signed-off-by: Ilia Lin <ilialin@codeaurora.org>

ARM: dma-mapping: Allow highmem pages to not have a mapping

The DMA_ATTR_NO_KERNEL_MAPPING is used to make sure that CMA
pages have no kernel mapping. Add support to make sure that
highmem pages have no mapping.

Change-Id: Ife76df126ecfedf0dba81a35e0de8a1787355b3d
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>

ARM: dma-mapping: Refactor out to introduce __in_atomic_pool

Check the given range("start", "size") is included in "atomic_pool" or not.

Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: add support for DMA_ATTR_NO_KERNEL_MAPPING attribute

This patch adds support for DMA_ATTR_NO_KERNEL_MAPPING attribute for
IOMMU allocations, what let drivers to save precious kernel virtual
address space for large buffers that are intended to be accessed only
from userspace.

This patch is heavily based on initial work kindly provided by Abhinav
Kochhar <abhinav.k@samsung.com>.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: Kyungmin Park <kyungmin.park@samsung.com>

ARM: dma-mapping: Introduce __atomic_get_pages() for __iommu_get_pages()

Support atomic allocation in __iommu_get_pages().

Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>
[moved __atomic_get_pages() under #ifdef CONFIG_ARM_DMA_USE_IOMMU
 to avoid unused fuction warning for no-IOMMU case]
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: IOMMU allocates pages from atomic_pool with GFP_ATOMIC

Make use of the same atomic pool as DMA does, and skip a kernel page
mapping which can involve sleep'able operations at allocating a kernel
page table.

Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: Fix potential memory leak in atomic_pool_init()

When either of __alloc_from_contiguous or __alloc_remap_buffer fails
to provide a valid pointer, allocated memory is freed up and an error
is returned. 'pages' was however not freed before returning error.

Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Sachin Kamat <sachin.kamat@linaro.org>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: fix atomic allocation alignment

The alignment mask is calculated incorrectly. Fixing the calculation
makes strange hangs/lockups disappear during the boot with Amstrad E3
and 3.6-rc1 kernel.

Signed-off-by: Aaro Koskinen <aaro.koskinen@iki.fi>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

arm: mm: fix DMA pool affiliation check

The __free_from_pool() function was changed in
e9da6e9905e639b0f842a244bc770b48ad0523e9. Unfortunately, the test that
checks whether the provided (start,size) is within the DMA pool has
been improperly modified. It used to be:

  if (start < coherent_head.vm_start || end > coherent_head.vm_end)

Where coherent_head.vm_end was non-inclusive (i.e, it did not include
the first byte after the pool). The test has been changed to:

  if (start < pool->vaddr || start > pool->vaddr + pool->size)

So now pool->vaddr + pool->size is inclusive (i.e, it includes the
first byte after the pool), so the test should be >= instead of >.

This bug causes the following message when freeing the *first* DMA
coherent buffer that has been allocated, because its virtual address
is exactly equal to pool->vaddr + pool->size :

WARNING: at /home/thomas/projets/linux-2.6/arch/arm/mm/dma-mapping.c:463 __free_from_pool+0xa4/0xc0()
freeing wrong coherent size from pool

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Lior Amsalem <alior@marvell.com>
Cc: Maen Suleiman <maen@marvell.com>
Cc: Tawfik Bayouk <tawfik@marvell.com>
Cc: Shadi Ammouri <shadi@marvell.com>
Cc: Eran Ben-Avi <benavi@marvell.com>
Cc: Yehuda Yitschak <yehuday@marvell.com>
Cc: Nadav Haklai <nadavh@marvell.com>
[m.szyprowski: rebased onto v3.6-rc5 and resolved conflict]
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: fix build warning in __dma_alloc()

Fix build warning in __dma_alloc() as below:

arch/arm/mm/dma-mapping.c: In function '__dma_alloc':
arch/arm/mm/dma-mapping.c:653:29: warning: 'page' may be used uninitialized in this function [-Wuninitialized]

Signed-off-by: Jingoo Han <jg1.han@samsung.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: use himem for DMA buffers for IOMMU-mapped devices

IOMMU can provide access to any memory page, so there is no point in
limiting the allocated pages only to lowmem, once other parts of
dma-mapping subsystem correctly supports himem pages.

Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: fix error path for memory allocation failure

This patch fixes incorrect check in error path. When the allocation of
first page fails, the kernel ops appears due to accessing -1 element of
the pages array.

Reported-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: modify condition check while freeing pages

WARNING: at mm/vmalloc.c:1471 __iommu_free_buffer+0xcc/0xd0()
Trying to vfree() nonexistent vm area (ef095000)
Modules linked in:
[<c0015a18>] (unwind_backtrace+0x0/0xfc) from [<c0025a94>] (warn_slowpath_common+0x54/0x64)
[<c0025a94>] (warn_slowpath_common+0x54/0x64) from [<c0025b38>] (warn_slowpath_fmt+0x30/0x40)
[<c0025b38>] (warn_slowpath_fmt+0x30/0x40) from [<c0016de0>] (__iommu_free_buffer+0xcc/0xd0)
[<c0016de0>] (__iommu_free_buffer+0xcc/0xd0) from [<c0229a5c>] (exynos_drm_free_buf+0xe4/0x138)
[<c0229a5c>] (exynos_drm_free_buf+0xe4/0x138) from [<c022b358>] (exynos_drm_gem_destroy+0x80/0xfc)
[<c022b358>] (exynos_drm_gem_destroy+0x80/0xfc) from [<c0211230>] (drm_gem_object_free+0x28/0x34)
[<c0211230>] (drm_gem_object_free+0x28/0x34) from [<c0211bd0>] (drm_gem_object_release_handle+0xcc/0xd8)
[<c0211bd0>] (drm_gem_object_release_handle+0xcc/0xd8) from [<c01abe10>] (idr_for_each+0x74/0xb8)
[<c01abe10>] (idr_for_each+0x74/0xb8) from [<c02114e4>] (drm_gem_release+0x1c/0x30)
[<c02114e4>] (drm_gem_release+0x1c/0x30) from [<c0210ae8>] (drm_release+0x608/0x694)
[<c0210ae8>] (drm_release+0x608/0x694) from [<c00b75a0>] (fput+0xb8/0x228)
[<c00b75a0>] (fput+0xb8/0x228) from [<c00b40c4>] (filp_close+0x64/0x84)
[<c00b40c4>] (filp_close+0x64/0x84) from [<c0029d54>] (put_files_struct+0xe8/0x104)
[<c0029d54>] (put_files_struct+0xe8/0x104) from [<c002b930>] (do_exit+0x608/0x774)
[<c002b930>] (do_exit+0x608/0x774) from [<c002bae4>] (do_group_exit+0x48/0xb4)
[<c002bae4>] (do_group_exit+0x48/0xb4) from [<c002bb60>] (sys_exit_group+0x10/0x18)
[<c002bb60>] (sys_exit_group+0x10/0x18) from [<c000ee80>] (ret_fast_syscall+0x0/0x30)

This patch modifies the condition while freeing to match the condition
used while allocation. This fixes the above warning which arises when
array size is equal to PAGE_SIZE where allocation is done using kzalloc
but free is done using vfree.

Change-Id: I4c60aa727cea6b35d10d5b00208bb62f276205db
Signed-off-by: Prathyush K <prathyush.k@samsung.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: Set arm_dma_set_mask() for iommu->set_dma_mask()

struct dma_map_ops iommu_ops doesn't have ->set_dma_mask, which causes
crash when dma_set_mask() is called from some driver.

Change-Id: Ia68590d30f5453104fdcad55d69ae4254e7fb95e
Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

Revert "ARM: dma-mapping: remove dmac_clean_range and dmac_inv_range"

This partially reverts commit
702b94bff3c50542a6e4ab9a4f4cef093262fe65. Some MSM drivers still
use the dmac_clean and dmac_inv_range APIs. Bring back the
defines and exports for v7 CPUs.

Change-Id: I69017d73da1065a5eeb9c87c899b6a51be5ebfe6
Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org

ARM: 7479/1: mm: avoid NULL dereference when flushing gate_vma with VIVT caches

The vivt_flush_cache_{range,page} functions check that the mm_struct
of the VMA being flushed has been active on the current CPU before
performing the cache maintenance.

The gate_vma has a NULL mm_struct pointer and, as such, will cause a
kernel fault if we try to flush it with the above operations. This
happens during ELF core dumps, which include the gate_vma as it may be
useful for debugging purposes.

This patch adds checks to the VIVT cache flushing functions so that VMAs
with a NULL mm_struct are flushed unconditionally (the vectors page may
be dirty if we use it to store the current TLS pointer).

Cc: <stable@vger.kernel.org> # 3.4+
Reported-by: Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org>
Tested-by: Uros Bizjak <ubizjak@gmail.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: Fix bad merge bd1274dc005 (Merge branch 'v6v7' into devel)

Commit 774c096bf9e49 (ARM: v6/v7 cache: allow cache calls to be
optimized) got dropped when the merge conflicts for moving the contents
of the files in commit 753790e713d (ARM: move cache/processor/fault
glue to separate include files) was fixed up in merge bd1274dc005
(Merge branch 'v6v7' into devel).

This puts the change back.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

[media] dma-mapping: fix dma_common_get_sgtable() conditional compilation

dma_common_get_sgtable() function doesn't depend on
ARCH_HAS_DMA_DECLARE_COHERENT_MEMORY, so it must not be compiled
conditionally.

Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>

ARM: dma-mapping: Add arm_iommu_detach_device()

A counter part of arm_iommu_attach_device().

Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

arm: dma mapping: export arm iommu functions

This patch adds EXPORT_SYMBOL_GPL calls to the three arm iommu
functions - arm_iommu_create_mapping, arm_iommu_free_mapping
and arm_iommu_attach_device. These three functions are arm specific
wrapper functions for creating/freeing/using an iommu mapping and
they are called by various drivers. If any of these drivers need
to be built as dynamic modules, these functions need to be exported.

Changelog v2: using EXPORT_SYMBOL_GPL as suggested by Marek.

Signed-off-by: Prathyush K <prathyush.k@samsung.com>
[m.szyprowski: extended with recently introduced
 EXPORT_SYMBOL_GPL(arm_iommu_detach_device)]
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: add coherent iommu dma ops

Remove arch_is_coherent() from iommu dma ops and implement separate
coherent ops functions.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: kill off arch_is_coherent

With ixp2xxx removed, there are no platforms that define arch_is_coherent,
so the last occurrences of arch_is_coherent can be removed. Any new
platform with coherent i/o should use coherent dma mapping functions.

Change-Id: I688c6340134562bd8ef43ec99f599aba357f0c04
Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: Remove unsed var at arm_coherent_iommu_unmap_page

Signed-off-by: Hiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: mm: Remove unused arm_vmregion priv field

Commit e9da6e9905e639b0f842a244bc770b48ad0523e9 ("ARM: dma-mapping:
remove custom consistent dma region") removed the last users of the
field. Remove it.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: add more sanity checks in arm_dma_mmap()

Add some sanity checks and forbid mmaping of buffers into vma areas larger
than allocated dma buffer.

Change-Id: Ie774225f5a030324febef1c7a22ff0ac70575bdb
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: dma-mapping: add support for DMA_ATTR_FORCE_CONTIGUOUS attribute

This patch adds support for DMA_ATTR_FORCE_CONTIGUOUS attribute for
dma_alloc_attrs() in IOMMU-aware implementation. For allocating physically
contiguous buffers Contiguous Memory Allocator is used.

Change-Id: Id217168bbadb80c1ccececaae30a6781655dc08d
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>

ARM: ensure vm_struct has its phys_addr member filled in

This allows /proc/vmallocinfo to show the physical address for
ioremap mappings.

Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ion: invalidate buffers before zero'ing

Currently, we invalidate the cache for buffers being zero'd *after* we
zero them. However, this is racey since a dirty cache line from an
earlier mapping could be written out to memory between the zero'ing and
the invalidation. Fix this by invalidating the cache *before* zero'ing
the buffer.

Before this change, I was seeing corruption with:

    $ cat <<EOF | ./memory_prof -e -i -
    simple_alloc,1,ION_SYSTEM_HEAP_ID,ION_FLAG_CACHED,0x100000,1MB
    simple_basic_sanity,1
    simple_profile,1
    simple_free,1
    EOF

After this change, no corruption.

CRs-Fixed: 585524
Change-Id: I1f2603604efc9be4ebf9112b06728e0aeca60ea0
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>