kernel/x86[_64]: remove get_optimized_functions from cpu modules

The possibility to specify custom memcpy and memset implementations
in cpu modules is currently unused and there is generally no point
in such feature.

There are only 2 x86 vendors that really matter and there isn't
very big difference in performance of the generic optmized versions
of these funcions across different models. Even if we wanted different
versions of memset and memcpy depending on the processor model or
features much better solution would be to use STT_GNU_IFUNC and save
one indirect call.

Long story short, we don't really benefit in any way from
get_optimized_functions and the feature it implements and it only adds
unnecessary complexity to the code.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
kernel/x86_64: remove memset and memcpy from commpage

There is absolutely no reason for these functions to be in commpage,
they don't do anything that involves the kernel in any way.

Additionaly, this patch rewrites memset and memcpy to C++, current
implementation is quite simple (though it may perform surprisingly
well when dealing with large buffers on cpus with ermsb). Better
versions are coming soon.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
kernel/x86_64: make sure stack is properly aligned in syscalls

Just following the path of least resistance and adding andq $~15, %rsp
where appropriate. That should also make things harder to break
when changing the amount of stuff placed on stack before calling the
actual syscall routine.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
boot/x86_64: enable sse early

Enable SSE as a part of the "preparation of the environment to run any
C or C++ code" in the entry points of stage2 bootloader.

SSE2 is going to be used by memset() and memcpy().

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
kernel/x86_64: save fpu state at interrupts

The kernel is allowed to use fpu anywhere so we must make sure that
user state is not clobbered by saving fpu state at interrupt entry.
There is no need to do that in case of system calls since all fpu
data registers are caller saved.

We do not need, though, to save the whole fpu state at task swich
(again, thanks to calling convention). Only status and control
registers are preserved. This patch actually adds xmm0-15 register
to clobber list of task swich code, but the only reason of that is
to make sure that nothing bad happens inside the function that
executes that task swich. Inspection of the generated code shows
that no xmm registers are actually saved.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
kernel/x86_64: clear xmm0-15 registers on syscall exit

As Alex pointed out we can leak possibly sensitive data in xmm registers
when returning from the kernel. To prevent that xmm0-15 are zeroed
before sysret or iret. The cost is negligible.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
libroot/x86_64: new memset implementation

This patch introduces new memset() implementation that improves the
performance when the buffer is small. It was written for processors that
support ERMSB, but performs reasonably well on older CPUs as well.

The following benchmarks were done on Haswell i7 running Debian Jessie
with Linux 3.16.1. In each iteration 64MB buffer was memset()ed, the
parameter "size" is the size of the buffer passed in a single call (i.e.
for "size: 2" memset() was called ~32 million times to memset the whole
64MB).

f - original implementation, g - new implementation, all buffers 16 byte
aligned

set, size:        8, f:    66885 µs, g:    17768 µs, ∆:   73.44%
set, size:       32, f:    17123 µs, g:     9163 µs, ∆:   46.49%
set, size:      128, f:     6677 µs, g:     6919 µs, ∆:   -3.62%
set, size:      512, f:    11656 µs, g:     7715 µs, ∆:   33.81%
set, size:     1024, f:     9156 µs, g:     7359 µs, ∆:   19.63%
set, size:     4096, f:     4936 µs, g:     5159 µs, ∆:   -4.52%

f - glibc 2.19 implementation, g - new implementation, all buffers 16 byte
aligned

set, size:        8, f:    19631 µs, g:    17828 µs, ∆:    9.18%
set, size:       32, f:     8545 µs, g:     9047 µs, ∆:   -5.87%
set, size:      128, f:     8304 µs, g:     6874 µs, ∆:   17.22%
set, size:      512, f:     7373 µs, g:     7486 µs, ∆:   -1.53%
set, size:     1024, f:     9007 µs, g:     7344 µs, ∆:   18.46%
set, size:     4096, f:     8169 µs, g:     5146 µs, ∆:   37.01%

Apparently, glibc uses SSE even for large buffers and therefore does not
takes advantage of ERMSB:

set, size:    16384, f:     7007 µs, g:     3223 µs, ∆:   54.00%
set, size:    32768, f:     6979 µs, g:     2930 µs, ∆:   58.02%
set, size:    65536, f:     6907 µs, g:     2826 µs, ∆:   59.08%
set, size:   131072, f:     6919 µs, g:     2752 µs, ∆:   60.23%

The new implementation handles unaligned buffers quite well:

f - glibc 2.19 implementation, g - new implementation, all buffers unaligned

set, size:       16, f:    10045 µs, g:    10498 µs, ∆:   -4.51%
set, size:       32, f:     8590 µs, g:     9358 µs, ∆:   -8.94%
set, size:       64, f:     8618 µs, g:     8585 µs, ∆:    0.38%
set, size:      128, f:     8393 µs, g:     6893 µs, ∆:   17.87%
set, size:      256, f:     8042 µs, g:     7621 µs, ∆:    5.24%
set, size:      512, f:     9661 µs, g:     7738 µs, ∆:   19.90%

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
libroot/x86_64: new memcpy implementation

This patch introduces new memcpy() implementation that improves the
performance when the buffer is small. It was written for processors that
support ERMSB, but performs reasonably well on older CPUs as well.

The following benchmarks were done on Haswell i7 running Debian Jessie
with Linux 3.16.1. In each iteration 64MB buffer was copied, the
parameter "size" is the size of the buffer passed in a single call (i.e.
for "size: 2" memcpy() was called ~32 million times to copy the whole
64MB).

f - original implementation, g - new implementation, all buffers 16 byte
aligned

cpy, size:        8, f:    79971 µs, g:    20419 µs, ∆:   74.47%
cpy, size:       32, f:    42068 µs, g:    12159 µs, ∆:   71.10%
cpy, size:      128, f:    13408 µs, g:    10359 µs, ∆:   22.74%
cpy, size:      512, f:    10634 µs, g:    10433 µs, ∆:    1.89%
cpy, size:     1024, f:    10474 µs, g:    10536 µs, ∆:   -0.59%
cpy, size:     4096, f:     9419 µs, g:     8630 µs, ∆:    8.38%

f - glibc 2.19 implementation, g - new implementation, all buffers 16 byte
aligned

cpy, size:        8, f:    26299 µs, g:    20919 µs, ∆:   20.46%
cpy, size:       32, f:    11146 µs, g:    12159 µs, ∆:   -9.09%
cpy, size:      128, f:    10778 µs, g:    10354 µs, ∆:    3.93%
cpy, size:      512, f:    12291 µs, g:    10426 µs, ∆:   15.17%
cpy, size:     1024, f:    13923 µs, g:    10571 µs, ∆:   24.08%
cpy, size:     4096, f:    11770 µs, g:     8671 µs, ∆:   26.33%

f - glibc 2.19 implementation, g - new implementation, all buffers unaligned

cpy, size:       16, f:    13376 µs, g:    13009 µs, ∆:    2.74%
cpy, size:       32, f:    11130 µs, g:    12171 µs, ∆:   -9.35%
cpy, size:       64, f:    11017 µs, g:    11231 µs, ∆:   -1.94%
cpy, size:      128, f:    10884 µs, g:    10407 µs, ∆:    4.38%
cpy, size:      256, f:    10826 µs, g:    10106 µs, ∆:    6.65%
cpy, size:      512, f:    12354 µs, g:    10396 µs, ∆:   15.85%

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
Merge branch 'memcpy-v3'

This is a major rework of how Haiku implements memset() and memcpy() on
x64. These functions are now removed from the commpage and reimplemented
in C++ using sse2 where it proved to be beneficial. That required some
serious changes in handling fpu state. Now, full fpu state is saved
at each interrupt, but not on task switch.

Some numbers: results of building targets: kernel libroot.so runtime_loader
and HaikuDepot on Intel i7 4770 with 16GB of memory.

real
user
sys

before:
1m54.367
7m40.617
0m58.641

1m33.922
8m12.362
1m0.852

1m32.922
8m10.509
1m1.006

1m31.947
8m12.596
1m1.353

after:
1m50.959
7m43.118
0m58.923

1m30.644
8m6.385
1m0.584

1m31.549
8m7.976
0m59.792

1m31.546
8m6.733
1m0.242