run.py: Do not run Xen guests with user mode networking implicitely

Xen guests do not support user mode networking.
Fall back to qemu when hypervisor detection used and no external
networking configured.

This logic is required for build to support Xen hosts properly.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
loader.py: make 'osv info threads' display the thread on which joiner waits

Saves a bit of manual work.

Example:

  gdb$ osv info threads
  0 (0x1de9e08) cpu0x0 waiting    sched::thread::join() at core/sched.cc (SNIP)
	joining on 0xffffc0003e41e010

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Skip lo0 when initializing interfaces in loader.cc

Attempt to reassign IP for lo0 fails and leaves it without an address.

This breaks tomcat:

  SEVERE: StandardServer.await: create[localhost:8005]:
  java.net.BindException: Address not available

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Fix lack of locking in libc which was causing memory corruption

In libc.h there was an optimization which disabled locking when there
is only one thread. It was using 'threads_minus_1' field from struct
__libc. In OSv this field was always 0, so there was no locking.

It was causing memory corruption when running apache benchmark against
tomcat in OSv on SMP. This patch makes them go away.

The fix is to always lock as in OSv we always have more than one
thread.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
elf: Ignore PT_PAX_FLAGS

Fixed the following abort when build was running mkzfs.py.

Done!
eth0: ethernet address: 52:54:0:12:34:56
[I/20 dhcp]: Waiting for IP...
[I/26 dhcp]: Server acknowledged IP for interface eth0
[I/26 dhcp]: Configuring eth0: ip 192.168.122.15 subnet mask 255.255.255.0
gateway 192.168.122.1
Aborted

Signed-off-by: David Leifker <dlei...@gmail.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
run.py: Make Xen treat OSv image as QCOW2 file

Xen requires explicit image format specification.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
build: option to choose OSv image format via make command line

Default image format stays QCOW2 as before.  May be changed via img_format
variable:

  make img_fomat=raw all

This option is required by release-ec2 script as EC2 does not support QCOW2.
Also will be useful for other hypervisors like VMWARE.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
release-ec2: force build to create raw OSv image

EC2 does not support QCOW2. Force build to create RAW image.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
ip: fix IP defragmentation mechanism for big packets

Current limit of fragments number for IP packet is 16.
This is not enough for packets bigger than 24K on standard MTU.
This patch increases this number up to theoretical maximum.

The problem found during UDP RX performance testing - throughput
dropped to 0 for 32K UDP packets.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
ovs: fix build error. (bsd/porting/networking.cc)

do not need initialize variable. (or use memset(3) for zero clear.)
Because set of values by ioctl(SIOCGIFFLAGS)

```
(snip)
  CC bsd/sys/kern/sys_socket.o
  CC bsd/sys/kern/subr_disk.o
  CC bsd/porting/route.o
  CXX bsd/porting/networking.o
../../bsd/porting/networking.cc: In function ‘int osv::ifup(std::string)’:
 ../../bsd/porting/networking.cc:99:30: error: missing braces around
initializer for ‘char [16]’ [-Werror=missing-braces]
cc1plus: all warnings being treated as errors
make[1]: *** [bsd/porting/networking.o] Error 1
make[1]: Leaving directory
`/home/kouki-o/work/kaishuu0123-osv/build/release'
make: *** [all] Error 2
(snip)
```

Signed-off-by: Kouki Ooyatsu <kaishuu0123@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Fix make flag in readme for debug mode

The -j must have been included by mistake, otherwise make uses unlimited
jobs.

Signed-off-by: David Leifker <dlei...@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
bsd: Fix formatting in porting/network.cc

Use four spaces, not tabs for indentation.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
build: fix .tls_template_size calculation

For an unknown reason, the current calculation of .tls_template_size
yields 0x10 instead of the correct value.  This results in part of the
initial tls block being freed by arch::setup(), and subsequent
corruption.

Fix by switching to the ld SIZEOF() operator.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
release-ec2: fix AMI replication code

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
x64: Fix stack alignment in fault handlers

Make sure stack pointer is 16-byte aligned in fault handler as required
by x86-64 ABI. This is needed for the page fault handler to be able to
use stack for FPU state save/restore.

Spotted by Nadav.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mmu: VMA permission flags

Add permission flags to VMAs. They will be used by mprotect() and the
page fault handler.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mmu: Anonymous memory demand paging

Switch to demand paging for anonymous virtual memory.

I used SPECjvm2008 to verify performance impact. The numbers are mostly
the same with few exceptions, most visible in the 'serial' benchmark.
However, there's quite a lot of variance between SPECjvm2008 runs so I
wouldn't read too much into them.

As we need the demand paging mechanism and the performance numbers
suggest that the implementation is reasonable, I'd merge the patch as-is
and see optimize it later.

  Before:

    Running specJVM2008 benchmarks on an OSV guest.
    Score on compiler.compiler: 331.23 ops/m
    Score on compiler.sunflow: 131.87 ops/m
    Score on compress: 118.33 ops/m
    Score on crypto.aes: 41.34 ops/m
    Score on crypto.rsa: 204.12 ops/m
    Score on crypto.signverify: 196.49 ops/m
    Score on derby: 170.12 ops/m
    Score on mpegaudio: 70.37 ops/m
    Score on scimark.fft.large: 36.68 ops/m
    Score on scimark.lu.large: 13.43 ops/m
    Score on scimark.sor.large: 22.29 ops/m
    Score on scimark.sparse.large: 29.35 ops/m
    Score on scimark.fft.small: 195.19 ops/m
    Score on scimark.lu.small: 233.95 ops/m
    Score on scimark.sor.small: 90.86 ops/m
    Score on scimark.sparse.small: 64.11 ops/m
    Score on scimark.monte_carlo: 145.44 ops/m
    Score on serial: 94.95 ops/m
    Score on sunflow: 73.24 ops/m
    Score on xml.transform: 207.82 ops/m
    Score on xml.validation: 343.59 ops/m

  After:

    Score on compiler.compiler: 346.78 ops/m
    Score on compiler.sunflow: 132.58 ops/m
    Score on compress: 116.05 ops/m
    Score on crypto.aes: 40.26 ops/m
    Score on crypto.rsa: 206.67 ops/m
    Score on crypto.signverify: 194.47 ops/m
    Score on derby: 175.22 ops/m
    Score on mpegaudio: 76.18 ops/m
    Score on scimark.fft.large: 34.34 ops/m
    Score on scimark.lu.large: 15.00 ops/m
    Score on scimark.sor.large: 24.80 ops/m
    Score on scimark.sparse.large: 33.10 ops/m
    Score on scimark.fft.small: 168.67 ops/m
    Score on scimark.lu.small: 236.14 ops/m
    Score on scimark.sor.small: 110.77 ops/m
    Score on scimark.sparse.small: 121.29 ops/m
    Score on scimark.monte_carlo: 146.03 ops/m
    Score on serial: 87.03 ops/m
    Score on sunflow: 77.33 ops/m
    Score on xml.transform: 205.73 ops/m
    Score on xml.validation: 351.97 ops/m

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mmu: MAP_POPULATE support for anon mmap()

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tests: Anonymous mmap microbenchmark

This adds a simple mmap and write microbenchmark that can be run on both
OSv and Linux.  The same test is run for both demand paged and
pre-populated memory region using MAP_POPULATE flag with mmap().

The numbers on 2-core Sandy Bridge i7 are as follows in an OSv guest:

       time (seconds)
   MiB demand populate
     1 0.008  0.002
     2 0.001  0.001
     4 0.002  0.002
     8 0.004  0.004
    16 0.010  0.008
    32 0.018  0.017
    64 0.036  0.034
   128 0.074  0.068
   256 0.145  0.135
   512 0.406  0.271
  1024 0.792  0.541

For reference, running the same test on Linux host gives:

       time (seconds)
   MiB demand populate
     1 0.003  0.003
     2 0.006  0.005
     4 0.012  0.011
     8 0.023  0.023
    16 0.046  0.045
    32 0.091  0.085
    64 0.175  0.170
   128 0.350  0.340
   256 0.694  0.681
   512 1.384  1.361
  1024 2.888  2.732

The numbers suggest that there's room for improvement in OSv's demand
paging code but the overall performance is reasonable for now.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tests: mincore() tests for demand paging

As suggested by Nadav, add tests for mincore() interraction with demand
paging.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>