vfs: Fix duplicate in-memory vnodes

Currently, namei() does vget() unconditionally if no dentry is found.
This is wrong because the path can be a hard link that points to a vnode
that's already in memory.

To fix the problem:

  - Use inode number as part of the hash in vget()

  - Use vn_lookup() in vget() to make sure we have one vnode in memory
    per inode number.

  - Push the vget() calls down to individual filesystems and make
    VOP_LOOKUP return an vnode

  - Drop lock in vn_lookup() and assert that vnode_lock is held.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Add tests into tst-fs-link.so to check vnode duplicity

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
xen: int vs long issues - OSv side

It seems that we also had problems with our own code for int vs long
issues. I am really surprised that the C++ compiler didn't throw any
warnings for this since all word sizes are quite explicit. In any case,
this seems to be the missing piece for xen booting with many CPUs.

It boots fine now with up to 32 CPUs. After that, other problems start
to appear.

Fixes #113

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
sched: initialize clock later

Right now we are taking a clock measure very early for cpu initialization.
That forces an unnecessary dependency between sched and clock initializations.

Since that lock is used to determine for how long the cpu has been running, we
can initialize the runtime later, when we init the idle thread. Nothing should
be running before it. After doing this, we can move the sched initialization
a bit earlier.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
xen: disable pvclock for more than 32 CPUs

Xen's shared info contains hardcoded space for only 32 CPUs.  Because we use
those structure to derive timing information, we would be basically accessing
random memory after that. This is very hard to test and trigger, so what I'd
did to demonstrate I was right (although that wasn't really needed, math could
be used for that...) was to print the first timing information a cpu would
produce. I could verify that the timing on CPUs > 32 was behind in time than
the time produced in CPUs < 32.

It is possible to move the vcpu area to a different location, but this is a
relatively new feature of the Xen Hypervisor: Amazon won't support it. So
we need a disable path anyway. I will open up an issue for somebody to implement
that support eventually.

Another user of the vcpu structure is interrupts. But for interrupts the story
is easier, since we can select which CPUs we can take interrupts at, and only
take them in the first 32 CPUs. In any case, we're taking them all in CPU0 now,
so already under control

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
sched: Use an integer for thread ids

Linux uses a 32-bit integer for pid_t, so let's do it as well. This is because
there are function in which we have to return our id back to the application.
One application is gettid, that we already have in the tree.

Theoretically, we could come up with a mapping between our 64-bit id and the
Linux one, but since we have to maintain the mapping anyway, we might as well
just use the Linux pids as our default IDs. The max size for that is 32-bit. It
is not enough if we're just allocating pids by bumping the counter, but again,
since we will have to maintain the bitmaps, 32-bit will allow us as much as 4
billion PIDs.

avi: remove unneeded #include

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
sched: standardize call to _cleanup

set_cleanup is quite a complicated piece of code. It is very easy to get it to
race with other thread destruction sites, which was made abundantly clear when
we tried to implement pthread detach.

This patch tries to make it easier, by restricting how and when set_cleanup can
be called. The trick here is that currently, a thread may or may not have a
cleanup function, and through a call to set_cleanup, our decision to cleanup
may change.

From this point on, set_cleanup will only tell us *how* to cleanup. If and
when, is a decision that we will make ourselves. For instance, if a thread
is block-local, the destructor will be called by the end of the block. In
that case, the _cleanup function will be there anyhow: we'll just not call
it.

We're setting here a default cleanup function for all created threads, that
just deletes the current thread object. Anything coming from pthread will try
to override it by also deleting the pthread object. And again, it is important
to node that they will set up those cleanup function unconditionally.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
sched: implement pthread_detach

I needed to call detach in a test code of mine, and this is isn't implemented.
The code I wrote to use it may or may not stay in the end, but nevertheless,
let's implement it.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
tests: add test for thread completion

That test goes together with thread detach, but I am also calling joins
to make sure we're not breaking them. It is unfortunate that this is quite
non-deterministic and we can't really surely test for failure. But on the
flip side, it did help me catch a couple of bugs in my implementation. So
it will eventually explode somewhere if a bug appears.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
virtio: Do not scale desc_needed by _num / 2

There is no reason we should do the scale.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Dor Laor <dor@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
virtio: Fix vring::use_indirect

When the _avail_count is less than 1/3 of the ring, we start using
indirect descriptor.

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Dor Laor <dor@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
build: use -Og for mode=debug, if available

gcc recommends -Og for debugging; follow its advice.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Don't assert() in tests/tst-readdir.cc

Before this patch, tst-readdir.cc assert()ed on every test, meaning that
a failure will cause a crash. Change this to use a report() function,
which counts failures instead of immediately crashing on the first one.

This patch doesn't change anything in what this test actually tests for.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
libc: Fix remove() return value

The remove() function is part of the ISO C 1989 standard, and used, for
example, to implement Java's File.delete(). It's supposed to remove a
file, regardless of whether unlink() or rmdir() is needed to remove it.

Our implementation (from Musl's) assumed that unlink() on a directory fails
with EISDIR, and only on that case it tried rmdir(). However, returning
EISDIR on unlink() is a Linux extension, which (deliberately) goes against
the Posix standard - which specified EPERM should be returned in that case.
Our ZFS implementation of unlink, following Solaris and FreeBSD (and not
Linux), returns EPERM in that case.

This meant that remove() used to fail deleting empty directories, and
Java code (like the SpecJVM2008 "derby" benchmark) using it to recursively
delete a directory, left behind undeleted empty directories.

So this patch fixes remove() to try rmdir() if unlink() returned either
the Linux-specific EISDIR, or the Posix-standard EPERM. It also adds
to the readdir test another test which verifies that remove() can delete
all files in a directory - both regular files and empty directories.

Fixes #112.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>