dm thin metadata: pass correct space map to dm_sm_root_size

Fix a harmless typo.

The root is a chunk of data that gets written to the superblock.  This
data is used to recreate the space map when opening a metadata area.
We have two space maps; one tracking space on the metadata device and
one of the data device.  Both of these use the same format for their
root, so this typo was harmless.

Signed-off-by: Joe Thornber <ejt@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Signed-off-by: Alasdair G Kergon <agk@redhat.com>
dm raid: handle failed devices during start up

The dm-raid code currently fails to create a RAID array if any of the
superblocks cannot be read.  This was an oversight as there is already
code to handle this case if the values ('- -') were provided for the
failed array position.

With this patch, if a superblock cannot be read, the array position's
fields are initialized as though '- -' was set in the table.  That is,
the device is failed and the position should not be used, but if there
is sufficient redundancy, the array should still be activated.

Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>
Signed-off-by: Alasdair G Kergon <agk@redhat.com>
dm: reject trailing characters in sccanf input

Device mapper uses sscanf to convert arguments to numbers. The problem is that
the way we use it ignores additional unmatched characters in the scanned string.

For example, this `if (sscanf(string, "%d", &number) == 1)' will match a number,
but also it will match number with some garbage appended, like "123abc".

As a result, device mapper accepts garbage after some numbers. For example
the command `dmsetup create vg1-new --table "0 16384 linear 254:1bla 34816bla"'
will pass without an error.

This patch fixes all sscanf uses in device mapper. It appends "%c" with
a pointer to a dummy character variable to every sscanf statement.

The construct `if (sscanf(string, "%d%c", &number, &dummy) == 1)' succeeds
only if string is a null-terminated number (optionally preceded by some
whitespace characters). If there is some character appended after the number,
sscanf matches "%c", writes the character to the dummy variable and returns 2.
We check the return value for 1 and consequently reject numbers with some
garbage appended.

Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Acked-by: Mike Snitzer <snitzer@redhat.com>
Signed-off-by: Alasdair G Kergon <agk@redhat.com>
dm thin: commit outstanding data every second

Commit unwritten data every second to prevent too much building up.

Released blocks don't become available until after the next commit
(for crash resilience).  Prior to this patch commits were only
triggered by a message to the target or a REQ_{FLUSH,FUA} bio.  This
allowed far too big a position to build up.

The interval is hard-coded to 1 second.  This is a sensible setting.
I'm not making this user configurable, since there isn't much to be
gained by tweaking this - and a lot lost by setting it far too high.

Signed-off-by: Joe Thornber <ejt@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Signed-off-by: Alasdair G Kergon <agk@redhat.com>
dm persistent data: remove space map ref_count entries if redundant

Save space by removing entries from the space map ref_count tree if
they're no longer needed.

Ref counts are stored in two places: a bitmap if the ref_count is
below 3, or a btree of uint32_t if 3 or above.

When a ref_count that was above 3 drops below we can remove it from
the tree and save some metadata space.  This removal was commented out
before because I was unsure why this was causing under-populated btree
nodes.  Earlier patches have fixed this issue.

Signed-off-by: Joe Thornber <ejt@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Signed-off-by: Alasdair G Kergon <agk@redhat.com>
dm thin: relax hard limit on the maximum size of a metadata device

The thin metadata format can only make use of a device that is <=
THIN_METADATA_MAX_SECTORS (currently 15.9375 GB).  Therefore, there is no
practical benefit to using a larger device.

However, it may be that other factors impose a certain granularity for
the space that is allocated to a device (E.g. lvm2 can impose a coarse
granularity through the use of large, >= 1 GB, physical extents).

Rather than reject a larger metadata device, during thin-pool device
construction, switch to allowing it but issue a warning if a device
larger than THIN_METADATA_MAX_SECTORS_WARNING (16 GB) is
provided.  Any space over 15.9375 GB will not be used.

Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Signed-off-by: Alasdair G Kergon <agk@redhat.com>
dm thin: support read only external snapshot origins

Support the use of an external _read only_ device as an origin for a thin
device.

Any read to an unprovisioned area of the thin device will be passed
through to the origin.  Writes trigger allocation of new blocks as
usual.

One possible use case for this would be VM hosts that want to run
guests on thinly-provisioned volumes but have the base image on another
device (possibly shared between many VMs).

Signed-off-by: Joe Thornber <ejt@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Signed-off-by: Alasdair G Kergon <agk@redhat.com>
dm thin: use dm_target_offset

Use dm_target_offset wrapper instead of referencing the awkward ti->begin
explicitly.

Signed-off-by: Alasdair G Kergon <agk@redhat.com>
dm thin: prepare to support discard

This patch contains the ground work needed for dm-thin to support discard.

  - Adds endio function that replaces shared_read_endio.

  - Introduce an explicit 'quiesced' flag into the new_mapping structure.
    Before, this was implicitly indicated by m->list being empty.

  - The map_info->ptr remains constant for the duration of a bio's trip
    through the thin target.  Make it easier to reason about it.

Signed-off-by: Joe Thornber <ejt@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Signed-off-by: Alasdair G Kergon <agk@redhat.com>
dm thin: support discards

Support discards in the thin target.

On discard the corresponding mapping(s) are removed from the thin
device.  If the associated block(s) are no longer shared the discard
is passed to the underlying device.

All bios other than discards now have an associated deferred_entry
that is saved to the 'all_io_entry' in endio_hook.  When non-discard
IO completes and associated mappings are quiesced any discards that
were deferred, via ds_add_work() in process_discard(), will be queued
for processing by the worker thread.

Signed-off-by: Joe Thornber <ejt@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Signed-off-by: Alasdair G Kergon <agk@redhat.com>

drivers/md/dm-thin.c |  173 ++++++++++++++++++++++++++++++++++++++++++++++----
 drivers/md/dm-thin.c |  172 ++++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 158 insertions(+), 14 deletions(-)
dm thin: add pool target flags to control discard

Add dm thin target arguments to control discard support.

ignore_discard: Disables discard support

no_discard_passdown: Don't pass discards down to the underlying data
device, but just remove the mapping within the thin provisioning target.

Signed-off-by: Joe Thornber <ejt@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Signed-off-by: Alasdair G Kergon <agk@redhat.com>
dm bufio: prefetch

This patch introduces a new function dm_bufio_prefetch. It prefetches
the specified range of blocks into dm-bufio cache without waiting
for i/o completion.

Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: Alasdair G Kergon <agk@redhat.com>
dm: add verity target

This device-mapper target creates a read-only device that transparently
validates the data on one underlying device against a pre-generated tree
of cryptographic checksums stored on a second device.

Two checksum device formats are supported: version 0 which is already
shipping in Chromium OS and version 1 which incorporates some
improvements.

Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: Mandeep Singh Baines <msb@chromium.org>
Signed-off-by: Will Drewry <wad@chromium.org>
Signed-off-by: Elly Jones <ellyjones@chromium.org>
Cc: Milan Broz <mbroz@redhat.com>
Cc: Olof Johansson <olofj@chromium.org>
Cc: Steffen Klassert <steffen.klassert@secunet.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Alasdair G Kergon <agk@redhat.com>
x86/apic/amd: Be more verbose about LVT offset assignments

Add information about LVT offset assignments to better debug firmware
bugs related to this. See following examples.

 # dmesg | grep -i 'offset\|ibs'
 LVT offset 0 assigned for vector 0xf9
 [Firmware Bug]: cpu 0, try to use APIC500 (LVT offset 0) for vector 0x10400, but the register is already in use for vector 0xf9 on another cpu
 [Firmware Bug]: cpu 0, IBS interrupt offset 0 not available (MSRC001103A=0x0000000000000100)
 Failed to setup IBS, -22

In this case the BIOS assigns both offsets for MCE (0xf9) and IBS
(0x400) vectors to offset 0, which is why the second APIC setup (IBS)
failed.

With correct setup you get:

 # dmesg | grep -i 'offset\|ibs'
 LVT offset 0 assigned for vector 0xf9
 LVT offset 1 assigned for vector 0x400
 IBS: LVT offset 1 assigned
 perf: AMD IBS detected (0x00000007)
 oprofile: AMD IBS detected (0x00000007)

Note: The vector includes also the message type to handle also NMIs
(0x400). In the firmware bug message the format is the same as of the
APIC500 register and includes the mask bit (bit 16) in addition.

Signed-off-by: Robert Richter <robert.richter@amd.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Merge tag 'drivers2' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc

Pull "ARM: More SoC driver updates" from Olof Johansson:
 "This branch contains a handful of driver updates, mostly to the
  LPC32xx platform but also for Samsung EXYNOS and Davinci.

  It had a few context conflicts against patches already merged through
  fixes-non-critical.  We should have resolved this early during the
  development cycle by pulling them in as a dependency, instead I did it
  after the fact this time."

* tag 'drivers2' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc:
  gpio/samsung: use ioremap() for EXYNOS4 GPIOlib
  gpio/samsung: add support GPIOlib for EXYNOS5250
  ARM: EXYNOS: add support GPIO for EXYNOS5250
  ARM: LPC32xx: Ethernet support
  ARM: LPC32xx: USB Support
  ARM: davinci: dm644x evm: add support for VPBE display
  ARM: davinci: dm644x: add support for v4l2 video display
  ARM: EXYNOS: Hook up JPEG PD to generic PD infrastructure
  ARM: EXYNOS: Hook up G2D PD to generic PD infrastructure
  arm: lpc32xx: phy3250: add rtc & touch device
  ARM: LPC32xx: clock.c: Clock registration fixes
  ARM: LPC32xx: clock.c: jiffies wrapping
  ARM: LPC32xx: clock.c: Missing header file
  ARM: LPC32XX: Remove broken non-static declaration
  ARM: LPC32xx: clock.c: Fix mutex lock issues
  ARM: LPC32xx: clock.c: warning fix
  ARM: LPC32xx: Added lpc32xx_defconfig
Merge tag 'soc2' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc

Pull "ARM: More SoC support updates" from Olof Johansson:
 "This branch contains a handful of updates of SoC base code that had
  dependencies on other external trees that have now been merged:

   * Support for the new EXYNOS5250 SoC from Samsung
   * SMP and power domain support for Tegra3 from NVIDIA
   * ux500 updates for exporting SoC information through sysfs"

Fix up trivial merge conflicts as per Olof.

* tag 'soc2' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc: (30 commits)
  ARM: mach-shmobile: ap4evb: Reserve DMA memory for the frame buffer
  ARM: EXYNOS: Fix compilation error with mach-exynos4-dt board
  ARM: dts: add initial dts file for EXYNOS5250, SMDK5250
  ARM: EXYNOS: add support device tree enabled board file for EXYNOS5
  ARM: EXYNOS: add support ARCH_EXYNOS5 for EXYNOS5 SoCs
  ARM: EXYNOS: add support get_core_count() for EXYNOS5250
  ARM: EXYNOS: support EINT for EXYNOS4 and EXYNOS5
  ARM: EXYNOS: add interrupt definitions for EXYNOS5250
  ARM: EXYNOS: add support for EXYNOS5250 SoC
  ARM: EXYNOS: add support uart for EXYNOS4 and EXYNOS5
  ARM: EXYNOS: add initial setup-i2c0 for EXYNOS5
  ARM: EXYNOS: add clock part for EXYNOS5250 SoC
  ARM: EXYNOS: use exynos_init_uarts() instead of exynos4_init_uarts()
  ARM: EXYNOS: to declare static for mach-exynos/common.c
  ARM: EXYNOS: Add clkdev lookup entry for lcd clock
  ARM: dt: Explicitly configure all serial ports on Tegra Cardhu
  ARM: tegra: support for secondary cores on Tegra30
  ARM: tegra: support for Tegra30 CPU powerdomains
  ARM: tegra: add support for Tegra30 powerdomains
  ARM: tegra: export tegra_powergate_is_powered()
  ...
Merge tag 'dt2' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc

Pull "ARM: More device tree support updates" from Olof Johansson:
 "This branch contains a number of updates for device tree support on
  several ARM platforms, in particular:

   * AT91 continues the device tree conversion adding support for a
     number of on-chip drivers and other functionality
   * ux500 adds probing of some of the core SoC blocks through device
     tree
   * Initial device tree support for ST SPEAr600 platforms
   * kirkwood continues the conversion to device-tree probing"

Manually merge arch/arm/mach-ux500/Kconfig due to MACH_U8500 rename, and
drivers/usb/gadget/at91_udc.c due to header file include cleanups.

Also do an "evil merge" for the MACH_U8500 config option rename that the
affected RMI4 touchscreen driver in staging.  It's called MACH_MOP500
now, and it was missed during previous merges.

* tag 'dt2' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc: (48 commits)
  ARM: SPEAr600: Add device-tree support to SPEAr600 boards
  ARM: ux500: Provide local timer support for Device Tree
  ARM: ux500: Enable PL022 SSP Controller in Device Tree
  ARM: ux500: Enable PL310 Level 2 Cache Controller in Device Tree
  ARM: ux500: Enable PL011 AMBA UART Controller for Device Tree
  ARM: ux500: Enable Cortex-A9 GIC (Generic Interrupt Controller) in Device Tree
  ARM: ux500: db8500: list most devices in the snowball device tree
  ARM: ux500: split dts file for snowball into generic part
  ARM: ux500: combine the board init functions for DT boot
  ARM: ux500: Initial Device Tree support for Snowball
  ARM: ux500: CONFIG: Enable Device Tree support for future endeavours
  ARM: kirkwood: use devicetree for rtc-mv
  ARM: kirkwood: rtc-mv devicetree bindings
  ARM: kirkwood: fdt: define uart[01] as disabled, enable uart0
  ARM: kirkwood: fdt: facilitate new boards during fdt migration
  ARM: kirkwood: fdt: absorb kirkwood_init()
  ARM: kirkwood: fdt: use mrvl ticker symbol
  ARM: orion: wdt: use resource vice direct access
  ARM: Kirkwood: Remove tclk from kirkwood_asoc_platform_data.
  ARM: orion: spi: remove enable_clock_fix which is not used
  ...
Merge tag 'common-clk-api' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc

Pull "drivers/clk: common clock framework" from Olof Johansson:
 "This branch contains patches from Mike Turquette adding a common clock
  framework to be shared across platforms.  This is part of the work
  towards building a common zImage for several ARM platforms."

* tag 'common-clk-api' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc:
  clk: make CONFIG_COMMON_CLK invisible
  clk: basic clock hardware types
  clk: introduce the common clock framework
  Documentation: common clk API
Merge branch 'drm-fixes' of git://people.freedesktop.org/~airlied/linux

Pull drm fixes from Dave Airlie:
 "gma500 build fix + some regression fixes for nouveau/radeon"

* 'drm-fixes' of git://people.freedesktop.org/~airlied/linux:
  drm/radeon: Only warn if the intra-domain offset actually exceeds the limit.
  drm/radeon/kms: add htile support to the cs checker v3
  drm/radeon/kms/atom: force bpc to 8 for now
  drm/nouveau/i2c: fix thinko/regression on really old chipsets
  drm/nouveau: default to 8bpc for non-LVDS panels if EDID isn't useful
  drm/nouveau: fix thinko causing init to fail on cards without accel
  gma500: medfield: fix build without CONFIG_BACKLIGHT_CLASS_DEVICE
Merge tag 'dm-3.4-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/agk/linux-dm

Pull device-mapper changes for 3.4 from Alasdair Kergon:

 - Update thin provisioning to support read-only external snapshot
   origins and discards.
 - A new target, dm verity, for device content validation.
 - Mark dm uevent and dm raid as no-longer-experimental.
 - Miscellaneous other fixes and clean-ups.

* tag 'dm-3.4-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/agk/linux-dm: (27 commits)
  dm: add verity target
  dm bufio: prefetch
  dm thin: add pool target flags to control discard
  dm thin: support discards
  dm thin: prepare to support discard
  dm thin: use dm_target_offset
  dm thin: support read only external snapshot origins
  dm thin: relax hard limit on the maximum size of a metadata device
  dm persistent data: remove space map ref_count entries if redundant
  dm thin: commit outstanding data every second
  dm: reject trailing characters in sccanf input
  dm raid: handle failed devices during start up
  dm thin metadata: pass correct space map to dm_sm_root_size
  dm persistent data: remove redundant value_size arg from value_ptr
  dm mpath: detect invalid map_context
  dm: clear bi_end_io on remapping failure
  dm table: simplify call to free_devices
  dm thin: correct comments
  dm raid: no longer experimental
  dm uevent: no longer experimental
  ...