Python-lang gdb script to extract x86_64 guest vmcore from qemu coredump

When qemu dies unexpectedly, for example in response to an explicit
abort() call, or (more importantly) when an external signal is delivered
to it that results in a coredump, sometimes it is useful to extract the
guest vmcore from the qemu process' memory image. The guest vmcore might
help understand an emulation problem in qemu, or help debug the guest.

This script reimplements (and cuts many features of) the
qmp_dump_guest_memory() command in gdb/Python,

  https://sourceware.org/gdb/current/onlinedocs/gdb/Python-API.html

working off the saved memory image of the qemu process. The docstring in
the patch (serving as gdb help text) describes the limitations relative to
the QMP command.

Dependencies of qmp_dump_guest_memory() have been reimplemented as needed.
I sought to follow the general structure, sticking to original function
names where possible. However, keeping it simple prevailed in some places.

The patch has been tested with a 4 VCPU, 768 MB, RHEL-6.4
(2.6.32-358.el6.x86_64) guest:

- The script printed

> guest RAM blocks:
> target_start     target_end       host_addr        message count
> ---------------- ---------------- ---------------- ------- -----
> 0000000000000000 00000000000a0000 00007f95d0000000 added       1
> 00000000000a0000 00000000000b0000 00007f960ac00000 added       2
> 00000000000c0000 00000000000ca000 00007f95d00c0000 added       3
> 00000000000ca000 00000000000cd000 00007f95d00ca000 joined      3
> 00000000000cd000 00000000000d0000 00007f95d00cd000 joined      3
> 00000000000d0000 00000000000f0000 00007f95d00d0000 joined      3
> 00000000000f0000 0000000000100000 00007f95d00f0000 joined      3
> 0000000000100000 0000000030000000 00007f95d0100000 joined      3
> 00000000fc000000 00000000fc800000 00007f960ac00000 added       4
> 00000000fffe0000 0000000100000000 00007f9618800000 added       5
> dumping range at 00007f95d0000000 for length 00000000000a0000
> dumping range at 00007f960ac00000 for length 0000000000010000
> dumping range at 00007f95d00c0000 for length 000000002ff40000
> dumping range at 00007f960ac00000 for length 0000000000800000
> dumping range at 00007f9618800000 for length 0000000000020000

- The vmcore was checked with "readelf", comparing the results against a
  vmcore written by qmp_dump_guest_memory():

> --- theirs      2013-09-12 17:38:59.797289404 +0200
> +++ mine        2013-09-12 17:39:03.820289404 +0200
> @@ -27,16 +27,16 @@
>    Type           Offset             VirtAddr           PhysAddr
>                   FileSiz            MemSiz              Flags  Align
>    NOTE           0x0000000000000190 0x0000000000000000 0x0000000000000000
> -                 0x0000000000000ca0 0x0000000000000ca0         0
> -  LOAD           0x0000000000000e30 0x0000000000000000 0x0000000000000000
> +                 0x000000000000001c 0x000000000000001c         0
> +  LOAD           0x00000000000001ac 0x0000000000000000 0x0000000000000000
>                   0x00000000000a0000 0x00000000000a0000         0
> -  LOAD           0x00000000000a0e30 0x0000000000000000 0x00000000000a0000
> +  LOAD           0x00000000000a01ac 0x0000000000000000 0x00000000000a0000
>                   0x0000000000010000 0x0000000000010000         0
> -  LOAD           0x00000000000b0e30 0x0000000000000000 0x00000000000c0000
> +  LOAD           0x00000000000b01ac 0x0000000000000000 0x00000000000c0000
>                   0x000000002ff40000 0x000000002ff40000         0
> -  LOAD           0x000000002fff0e30 0x0000000000000000 0x00000000fc000000
> +  LOAD           0x000000002fff01ac 0x0000000000000000 0x00000000fc000000
>                   0x0000000000800000 0x0000000000800000         0
> -  LOAD           0x00000000307f0e30 0x0000000000000000 0x00000000fffe0000
> +  LOAD           0x00000000307f01ac 0x0000000000000000 0x00000000fffe0000
>                   0x0000000000020000 0x0000000000020000         0
>
>  There is no dynamic section in this file.
> @@ -47,13 +47,6 @@
>
>  No version information found in this file.
>
> -Notes at offset 0x00000190 with length 0x00000ca0:
> +Notes at offset 0x00000190 with length 0x0000001c:
>    Owner                Data size       Description
> -  CORE         0x00000150      NT_PRSTATUS (prstatus structure)
> -  CORE         0x00000150      NT_PRSTATUS (prstatus structure)
> -  CORE         0x00000150      NT_PRSTATUS (prstatus structure)
> -  CORE         0x00000150      NT_PRSTATUS (prstatus structure)
> -  QEMU         0x000001b0      Unknown note type: (0x00000000)
> -  QEMU         0x000001b0      Unknown note type: (0x00000000)
> -  QEMU         0x000001b0      Unknown note type: (0x00000000)
> -  QEMU         0x000001b0      Unknown note type: (0x00000000)
> +  NONE         0x00000005      Unknown note type: (0x00000000)

- The vmcore was checked with "crash" too, again comparing the results
  against a vmcore written by qmp_dump_guest_memory():

> --- guest.vmcore.log2   2013-09-12 17:52:27.074289201 +0200
> +++ example.dump.log2   2013-09-12 17:52:15.904289203 +0200
> @@ -22,11 +22,11 @@
>  This GDB was configured as "x86_64-unknown-linux-gnu"...
>
>       KERNEL: /usr/lib/debug/lib/modules/2.6.32-358.el6.x86_64/vmlinux
> -    DUMPFILE: /home/lacos/tmp/guest.vmcore
> +    DUMPFILE: /home/lacos/tmp/example.dump
>          CPUS: 4
> -        DATE: Thu Sep 12 17:16:11 2013
> -      UPTIME: 00:01:09
> -LOAD AVERAGE: 0.07, 0.03, 0.00
> +        DATE: Thu Sep 12 17:17:41 2013
> +      UPTIME: 00:00:38
> +LOAD AVERAGE: 0.18, 0.05, 0.01
>         TASKS: 130
>      NODENAME: localhost.localdomain
>       RELEASE: 2.6.32-358.el6.x86_64
> @@ -38,12 +38,12 @@
>       COMMAND: "swapper"
>          TASK: ffffffff81a8d020  (1 of 4)  [THREAD_INFO: ffffffff81a00000]
>           CPU: 0
> -       STATE: TASK_RUNNING (PANIC)
> +       STATE: TASK_RUNNING (ACTIVE)
> +     WARNING: panic task not found
>
>  crash> bt
>  PID: 0      TASK: ffffffff81a8d020  CPU: 0   COMMAND: "swapper"
> - #0 [ffffffff81a01ed0] default_idle at ffffffff8101495d
> - #1 [ffffffff81a01ef0] cpu_idle at ffffffff81009fc6
> + #0 [ffffffff81a01ef0] cpu_idle at ffffffff81009fc6
>  crash> task ffffffff81a8d020
>  PID: 0      TASK: ffffffff81a8d020  CPU: 0   COMMAND: "swapper"
>  struct task_struct {
> @@ -75,7 +75,7 @@
>        prev = 0xffffffff81a8d080
>      },
>      on_rq = 0,
> -    exec_start = 8618466836,
> +    exec_start = 7469214014,
>      sum_exec_runtime = 0,
>      vruntime = 0,
>      prev_sum_exec_runtime = 0,
> @@ -149,7 +149,7 @@
>    },
>    tasks = {
>      next = 0xffff88002d621948,
> -    prev = 0xffff880029618f28
> +    prev = 0xffff880023b74488
>    },
>    pushable_tasks = {
>      prio = 140,
> @@ -165,7 +165,7 @@
>      }
>    },
>    mm = 0x0,
> -  active_mm = 0xffff88002929b780,
> +  active_mm = 0xffff8800297eb980,
>    exit_state = 0,
>    exit_code = 0,
>    exit_signal = 0,
> @@ -177,7 +177,7 @@
>    sched_reset_on_fork = 0,
>    pid = 0,
>    tgid = 0,
> -  stack_canary = 2483693585637059287,
> +  stack_canary = 7266362296181431986,
>    real_parent = 0xffffffff81a8d020,
>    parent = 0xffffffff81a8d020,
>    children = {
> @@ -224,14 +224,14 @@
>    set_child_tid = 0x0,
>    clear_child_tid = 0x0,
>    utime = 0,
> -  stime = 3,
> +  stime = 2,
>    utimescaled = 0,
> -  stimescaled = 3,
> +  stimescaled = 2,
>    gtime = 0,
>    prev_utime = 0,
>    prev_stime = 0,
>    nvcsw = 0,
> -  nivcsw = 1000,
> +  nivcsw = 1764,
>    start_time = {
>      tv_sec = 0,
>      tv_nsec = 0

- <name_dropping>I asked for Dave Anderson's help with verifying the
  extracted vmcore, and his comments make me think I should post
  this.</name_dropping>

Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Add DSDT node for AppleSMC

AppleSMC (-device isa-applesmc) is required to boot OS X guests.
OS X expects a SMC node to be present in the ACPI DSDT. This patch
adds a SMC node to the DSDT, and dynamically patches the return value
of SMC._STA to either 0x0B if the chip is present, or otherwise to 0x00,
before booting the guest.

Signed-off-by: Gabriel Somlo <somlo@cmu.edu>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
ACPI: Fix AppleSMC _STA size

Minimize the storage used for AppleSMC's _STA (8bit), relying on ASL
to implicitly convert it to the officially specified 32bit value.

Signed-off-by: Gabriel Somlo <somlo@cmu.edu>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
virtio: Fix return value for dummy function vhost_net_virtqueue_pending

cgcc complains that -ENOSYS is not a good value for 'bool'.

A dummy virtio will never have pending queue entries, so let us return
false.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
acpi unit-test: add test files

Added unit-test's expected aml files to be compared
with the actual ACPI tables.

Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
configure: added acpi unit-test files

Ensure configure will set-up links for the files
if the build is created in other directory.

Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
acpi unit-test: compare DSDT and SSDT tables against expected values

This test will run only if iasl is installed on the host machine.
The test plan:
 1. Dumps the ACPI tables as AML on the disk.
 2. Runs iasl to disassembly the tables into ASL files.
 3. Runs iasl to disassembly the offline AML files into ASL files.
 4. Compares the ASL files.

The test runs for both default machine and q35.
In case the test fails, it can be easily tweaked to
show the differences between the ASL files and
understand the issue.

Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
configure: add CONFIG_IASL to config-host.h

Acpi unit-tests will extract iasl executable
from CONFIG_IASL define.

Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
acpi unit-test: extract iasl executable from configuration

The test checked if iasl is installed by running "iasl"
and checking the error output.
It is better to use the iasl executable as appears
in configuration.

Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
acpi unit-test: added script to rebuild the expected aml files

Acpi unit-test will fail every time the acpi tables change.
This script rebuild the expected aml files, so the test
will pass. It also validates the modifications.

Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
acpi unit-test: hook to rebuild expected aml files

When running the test with TEST_ACPI_REBUILD_AML=y environment
variable, the test will rebuild and validate the expected aml
files.

Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
tests: fix acpi to work on bigendian host

Double endianness convertion make this test failing on POWERPC machine
running in big-endian.

This fixes the test to success on big-endian host.

Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
acpi unit-test: renamed ssdt_tables to tables

Just a refactoring, ssdt_tables name was confusing as
it included other tables as well.

Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
acpi unit-test: resolved iasl crash

It seems that iasl has an issue when disassembles
some ACPI tables using the command line:
iasl -e DSDT -e SSDT -d HPET

Modified the iasl command line to "iasl -d HPET"
until the problem is solved. The command line
remained the same for DSDT and SSDT tables.

Reported-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
acpi unit-test: do not fail on asl mismatch

The asl comparison will break every time the ACPI
tables are updated. This may break the git bisect.
Instead of failing print a warning on stderr
including the retained asl files, so they can be
compared offline.

Signed-off-by: Marcel Apfelbaum <marcel.a@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
pc: make: fix dependencies: rebuild when included file is changed

some *.dsl files include another *.dsl files but there weren't
any dependicies and when included file changed target table wasn't
rebuild. Fix this by using the same auto dependency generation
as for C files.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
pci: add pci_for_each_bus_depth_first

Useful for ACPI hotplug.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
pcihp: generalization of piix4 acpi

Add ACPI based PCI hotplug library with bridge hotplug
support.
Design
   - each bus gets assigned "bsel" property.
   - ACPI code writes this number
     to a new BNUM register, then uses existing
     UP/DOWN registers to probe slot status;
     to eject, write number to BNUM register,
     then slot into existing EJ.

The interface is actually backwards-compatible with
existing PIIX4 ACPI (though not migration compatible).

This is split out from PIIX4 codebase so we can
reuse it for Q35 as well.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
piix4: add acpi pci hotplug support

Add support for acpi pci hotplug using the
new infrastructure.
PIIX4 legacy interface is maintained as is for
machine types 1.7 and older.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
acpi-build: enable hotplug for PCI bridges

This enables support for device hotplug behind
pci bridges. Bridge devices themselves need
to be pre-configured on qemu command line.

Design:
    - at machine init time, assign "bsel" property to bridges with
      hotplug support
    - dynamically (At ACPI table read) generate ACPI code to handle
      hotplug events for each bridge with "bsel" property

Note: ACPI doesn't support adding or removing bridges by hotplug.
We detect and prevent removal of bridges by hotplug,
unless they were added by hotplug previously
(and so, are not described by ACPI).

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>