scripts: Python3 support for scripts/silentant.py

Add explicit conversion form bytes array to string which returned by
Popen.communicate().

Use Python3 style print function.

Signed-off-by: Zifei Tong <zifeitong@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tests: test various buffer sizes in misc-bdev-write test

Xen block driver now supports new featute called indirect descriptors.
This feature allows to put more data into each ring cell but it activates
for "long" reads and writes only - longer than 11 pages.

With this patch test by default runs 2 scenarious:
  * 1 page buffers
  * 32 pages buffers

Also introduced command line parameter to specify
size of buffers explicitly.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
tests: introduce write/read/verify test

Test misc-bdev-rw introduced.

The test writes buffers of various lengths to block device,
reads data back and verifies data read is the same as data written.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
blkfront: introduce functions for device features detection

This is a refactoring commit that isolated some xenstore access
logic to make it reusable

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
blkfront: isolate command data grant references aquisition

This is a refactoring commit to simplify future indirect descriptors code.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
blkfront: isolate commands allocation

This is a refactoring commit to simplify future indirect descriptors code.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
blkfront: introduce indirect descriptors support

Indirect descriptors for Xen PV disks is a newly
introduced protocol feature that increases
block devices performance in case of long transfers.

Detailed description:
http://blog.xen.org/index.php/2013/08/07/indirect-descriptors-for-xen-pv-disks/

Measurement results (RAM-backed storage):

Xen w/o indirect descriptors:

1046.251 Mb/s
1097.931 Mb/s
1149.672 Mb/s
1137.696 Mb/s
1135.481 Mb/s
1249.163 Mb/s
1115.417 Mb/s
1118.063 Mb/s
Wrote 11344.750 MB in 10.00 s = 1134.165 Mb/s

Xen w/ indirect descriptors:

bdev-write test offset limit: 250000000 byte(s)
1234.715 Mb/s
1360.013 Mb/s
1323.663 Mb/s
1336.916 Mb/s
1342.617 Mb/s
1332.882 Mb/s
1302.094 Mb/s
Wrote 13233.250 MB in 10.04 s = 1318.639 Mb/s

KVM:

bdev-write test offset limit: 250000000 byte(s)
674.729 Mb/s
698.736 Mb/s
627.677 Mb/s
630.209 Mb/s
742.977 Mb/s
759.205 Mb/s
681.476 Mb/s
655.717 Mb/s
713.231 Mb/s
688.478 Mb/s
Wrote 6872.750 MB in 10.00 s = 687.124 Mb/s

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
support vga cursor

Add cursor update callback to libtsm, update cursor position register on VGA device using the callback.
Fixes #220.

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
scripts: Add script for building a Capstan image

This adds a script for creating Capstan compatible images that can be
uploaded to a S3 bucket for distribution.

To use it to build a base image, for example, type:

  $ ./scripts/build-capstan-img cloudius/osv-base empty "OSv base image"

which builds a Capstan image named "cloudius/osv-base" with "make
image=empty" and places QEMU and VirtualBox images under
"build/capstan/cloudius/osv-base" together with an index file.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
scripts: Add script for building OSv release

This adds a script for building a Capstan repository for an OSv release
with the following images:

  - OSv core (no API, no CLI)
  - Default (with API, CLI)
  - Cassandra virtual appliance (with API)
  - Tomcat virtual appliance (with API)
  - Iperf virtual appliance (with API)

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
scripts: Add script for uploading OSv release

This adds a script for uploading OSv release to Capstan S3 bucket using
"s3cmd sync".

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Update osv-apps submodule

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mgmt: update to latest

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
gdb: add function to check waiters

Useful to figure out which thread is waiting on what

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
trace: fix use of undefined symbol

You'll run into this when you have nested trace samples
with undefined backtrace elements.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
x64: fix early halt()

When halt() is called very early, before smp_launch(), it crashes when
calling crash_other_processors() because the other processors' IDT was
not yet set up. For example, in loader.cc's prepare_commands() we call
abort() when we failed to parse the command line, and this caused a
crash reported in issue #252.

With this patch, crash_other_processors does nothing when other processors
have not yet been set up. This is normally the case before smp_launch(),
but note that on single-vcpu VM, it will remain the case throughout the run.

Fixes #252.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
build-osv-release: OSv memcached server

This adds our own memcached server to an OSv release that is pushed to
Capstan S3 repository.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
build-osv-release: OpenJDK/OSv base image

Add a OpenJDK/OSv base image for developers who want to use Capstan to
package and run their Java applications.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
gdb: update flag translation

We have added a new mmap_small flag, so translation need to be updated.
's' is taken, so let's go for 'm'.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
balloon: fix move-to-itself bug

While working in the reclaim policy, I came across an unhandled balloon case
that seem to happen in some certain low memory conditions. The balloon will be
moved to the same address it was before. While this seems weird at first, it is
perfectly plausible if we think that we can have a space before the balloon almost
a page in size. If the JVM is only shifting the elements slightly, they will end up
at the same ballooned region.

The problem with that is that evacuate() is called first - the vma is still in place,
so it will be found, and it will call delete. When the jvm fault handler executes,
it will also call delete and we will have a double free.

This patch avoids the delete, but the problem is actually more intricate than
that: if we just follow the normal path, we will remove the newly created
address from the candidates list right after we insert it (because the
addresses are the same). Also, if the address is *not* the same, but falls in
the same region (this is theoretical), evacuate will be called and split the region
in two. Our code to handle finish_move would have to be significantly complicated
to deal with this fact. This patch also handles this theoretical case by making
sure that the whole region is always disposed at once (by finishing the old balloon
manually before evacuate is called)

Why I am using a new mmu flag: when we search for the old vma, we need to
identify which type of vma we are dealing with. If it is an anonymous or
ballooned area. I use an extra flag to avoid a type field or anything like
that.

Couldn't we just take the vma out of the list before we delete it? In this case
the region would be empty, and we could assume that was a ballooned region.
That would also be a solution, but because we are copying the flag from the
previous vma, we would have to fallback to a default flag set. The solution
of marking the JVM vma and leaving it in place allows the flag information to
be carried on.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>