Revert "sched: reduce the overhead of obtain factor"

This reverts commit ef3e5dee50096940e0a89e78afcf6aa6bc30ebfd.
sched: smart wake-affine foundation

wake-affine stuff is always trying to pull wakee close to waker, by theory,
this will bring benefit if waker's cpu cached hot data for wakee, or the
extreme ping-pong case.

And testing show it could benefit hackbench 15% at most.

However, the whole stuff is somewhat blindly and time-consuming, some
workload therefore suffer.

And testing show it could damage pgbench 50% at most.

Thus, wake-affine stuff should be more smart, and realise when to stop
it's thankless effort.

This patch introduced 'nr_wakee_switch', which will be increased each
time the task switch it's wakee.

So a high 'nr_wakee_switch' means the task has more than one wakee, and
bigger the number, higher the wakeup frequency.

Now when making the decision on whether to pull or not, pay attention on
the wakee with a high 'nr_wakee_switch', pull such task may benefit wakee,
but also imply that waker will face cruel competition later, it could be
very cruel or very fast depends on the story behind 'nr_wakee_switch',
whatever, waker therefore suffer.

Furthermore, if waker also has a high 'nr_wakee_switch', imply that multiple
tasks rely on it, then waker's higher latency will damage all of them, pull
wakee seems to be a bad deal.

Thus, when 'waker->nr_wakee_switch / wakee->nr_wakee_switch' become higher
and higher, the deal seems to be worse and worse.

The patch therefore help wake-affine stuff to stop it's work when:

	wakee->nr_wakee_switch > factor &&
	waker->nr_wakee_switch > (factor * wakee->nr_wakee_switch)
The factor here is the node-size of current-cpu, so bigger node will lead
to more pull since the trial become more severe.

After applied the patch, pgbench show 40% improvement at most.

Test:
	Tested with 12 cpu X86 server and tip 3.10.0-rc7.

	pgbench		    base	smart

	| db_size | clients |  tps  |	|  tps  |
	+---------+---------+-------+   +-------+
	| 22 MB   |       1 | 10598 |   | 10796 |
	| 22 MB   |       2 | 21257 |   | 21336 |
	| 22 MB   |       4 | 41386 |   | 41622 |
	| 22 MB   |       8 | 51253 |   | 57932 |
	| 22 MB   |      12 | 48570 |   | 54000 |
	| 22 MB   |      16 | 46748 |   | 55982 | +19.75%
	| 22 MB   |      24 | 44346 |   | 55847 | +25.93%
	| 22 MB   |      32 | 43460 |   | 54614 | +25.66%
	| 7484 MB |       1 |  8951 |   |  9193 |
	| 7484 MB |       2 | 19233 |   | 19240 |
	| 7484 MB |       4 | 37239 |   | 37302 |
	| 7484 MB |       8 | 46087 |   | 50018 |
	| 7484 MB |      12 | 42054 |   | 48763 |
	| 7484 MB |      16 | 40765 |   | 51633 | +26.66%
	| 7484 MB |      24 | 37651 |   | 52377 | +39.11%
	| 7484 MB |      32 | 37056 |   | 51108 | +37.92%
	| 15 GB   |       1 |  8845 |   |  9104 |
	| 15 GB   |       2 | 19094 |   | 19162 |
	| 15 GB   |       4 | 36979 |   | 36983 |
	| 15 GB   |       8 | 46087 |   | 49977 |
	| 15 GB   |      12 | 41901 |   | 48591 |
	| 15 GB   |      16 | 40147 |   | 50651 | +26.16%
	| 15 GB   |      24 | 37250 |   | 52365 | +40.58%
	| 15 GB   |      32 | 36470 |   | 50015 | +37.14%
CC: Ingo Molnar <mingo@kernel.org>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Mike Galbraith <efault@gmx.de>
Signed-off-by: Michael Wang <wangyun@linux.vnet.ibm.com>
sched: reduce the overhead of obtain factor

Smart wake-affine is using node-size as the factor, but the overhead of
mask operation is high.

Thus, this patch introduce the 'sd_llc_size', which will record the highest
cache-share domain size, and make it to be the new factor, in order to
reduce the overhead and make more reasonable.

And we suppose it will benefit a lot when facing a huge platform.

Test:
	Tested with 12 cpu X86 server and tip 3.10.0-rc7.

	pgbench		    base	smart + optimization

	| db_size | clients |  tps  |	|  tps  |
	+---------+---------+-------+   +-------+
	| 22 MB   |       1 | 10598 |   | 10781 |
	| 22 MB   |       2 | 21257 |   | 21328 |
	| 22 MB   |       4 | 41386 |   | 41622 |
	| 22 MB   |       8 | 51253 |   | 60351 |
	| 22 MB   |      12 | 48570 |   | 54255 |
	| 22 MB   |      16 | 46748 |   | 55534 | +18.79%
	| 22 MB   |      24 | 44346 |   | 55976 | +26.23%
	| 22 MB   |      32 | 43460 |   | 55279 | +27.20%
	| 7484 MB |       1 |  8951 |   |  9054 |
	| 7484 MB |       2 | 19233 |   | 19252 |
	| 7484 MB |       4 | 37239 |   | 37354 |
	| 7484 MB |       8 | 46087 |   | 51218 |
	| 7484 MB |      12 | 42054 |   | 49510 |
	| 7484 MB |      16 | 40765 |   | 52151 | +27.93%
	| 7484 MB |      24 | 37651 |   | 52720 | +40.02%
	| 7484 MB |      32 | 37056 |   | 51094 | +37.88%
	| 15 GB   |       1 |  8845 |   |  9139 |
	| 15 GB   |       2 | 19094 |   | 19379 |
	| 15 GB   |       4 | 36979 |   | 37077 |
	| 15 GB   |       8 | 46087 |   | 50490 |
	| 15 GB   |      12 | 41901 |   | 48235 |
	| 15 GB   |      16 | 40147 |   | 51878 | +29.22%
	| 15 GB   |      24 | 37250 |   | 52676 | +41.41%
	| 15 GB   |      32 | 36470 |   | 50198 | +37.64%
CC: Ingo Molnar <mingo@kernel.org>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Mike Galbraith <efault@gmx.de>
Signed-off-by: Michael Wang <wangyun@linux.vnet.ibm.com>
mm, vmalloc: use well-defined find_last_bit() func

Our intention in here is to find last_bit within the region to flush.
There is well-defined function, find_last_bit() for this purpose and
it's performance may be slightly better than current implementation.
So change it.

Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
mm/vmalloc: use wrapper function get_vm_area_size to caculate size of vm area

Use wrapper function get_vm_area_size to calculate size of vm area.

Signed-off-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
selinux: fix problems in netnode when BUG() is compiled out

When the BUG() macro is disabled at compile time it can cause some
problems in the SELinux netnode code: invalid return codes and
uninitialized variables.  This patch fixes this by making sure we take
some corrective action after the BUG() macro.

Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Paul Moore <pmoore@redhat.com>
Signed-off-by: Eric Paris <eparis@redhat.com>

Signed-off-by: Amperific <amperific@gmail.com>
Revert "selinux: fix problems in netnode when BUG() is compiled out"

This reverts commit aa34f62011b0697c8df70046c13d0e2a811a9476.
SELinux: Reduce overhead of mls_level_isvalid() function call

While running the high_systime workload of the AIM7 benchmark on
a 2-socket 12-core Westmere x86-64 machine running 3.10-rc4 kernel
(with HT on), it was found that a pretty sizable amount of time was
spent in the SELinux code. Below was the perf trace of the "perf
record -a -s" of a test run at 1500 users:

  5.04%            ls  [kernel.kallsyms]     [k] ebitmap_get_bit
  1.96%            ls  [kernel.kallsyms]     [k] mls_level_isvalid
  1.95%            ls  [kernel.kallsyms]     [k] find_next_bit

The ebitmap_get_bit() was the hottest function in the perf-report
output.  Both the ebitmap_get_bit() and find_next_bit() functions
were, in fact, called by mls_level_isvalid(). As a result, the
mls_level_isvalid() call consumed 8.95% of the total CPU time of
all the 24 virtual CPUs which is quite a lot. The majority of the
mls_level_isvalid() function invocations come from the socket creation
system call.

Looking at the mls_level_isvalid() function, it is checking to see
if all the bits set in one of the ebitmap structure are also set in
another one as well as the highest set bit is no bigger than the one
specified by the given policydb data structure. It is doing it in
a bit-by-bit manner. So if the ebitmap structure has many bits set,
the iteration loop will be done many times.

The current code can be rewritten to use a similar algorithm as the
ebitmap_contains() function with an additional check for the
highest set bit. The ebitmap_contains() function was extended to
cover an optional additional check for the highest set bit, and the
mls_level_isvalid() function was modified to call ebitmap_contains().

With that change, the perf trace showed that the used CPU time drop
down to just 0.08% (ebitmap_contains + mls_level_isvalid) of the
total which is about 100X less than before.

  0.07%            ls  [kernel.kallsyms]     [k] ebitmap_contains
  0.05%            ls  [kernel.kallsyms]     [k] ebitmap_get_bit
  0.01%            ls  [kernel.kallsyms]     [k] mls_level_isvalid
  0.01%            ls  [kernel.kallsyms]     [k] find_next_bit

The remaining ebitmap_get_bit() and find_next_bit() functions calls
are made by other kernel routines as the new mls_level_isvalid()
function will not call them anymore.

This patch also improves the high_systime AIM7 benchmark result,
though the improvement is not as impressive as is suggested by the
reduction in CPU time spent in the ebitmap functions. The table below
shows the performance change on the 2-socket x86-64 system (with HT
on) mentioned above.

+--------------+---------------+----------------+-----------------+
|   Workload   | mean % change | mean % change  | mean % change   |
|              | 10-100 users  | 200-1000 users | 1100-2000 users |
+--------------+---------------+----------------+-----------------+
| high_systime |     +0.1%     |     +0.9%      |     +2.6%       |
+--------------+---------------+----------------+-----------------+

Signed-off-by: Waiman Long <Waiman.Long@hp.com>
Acked-by:  Stephen Smalley <sds@tycho.nsa.gov>
Signed-off-by: Paul Moore <pmoore@redhat.com>

--
v4->v5:
  - Fix scripts/checkpatch.pl warning.

v3->v4:
  - Merge the 2 separate while loops in ebitmap_contains() into
    a single one.

v2->v3:
  - Remove unused local variables i, node from mls_level_isvalid().

v1->v2:
 - Move the new ebitmap comparison logic from mls_level_isvalid()
   into the ebitmap_contains() helper function.
 - Rerun perf and performance tests on the latest v3.10-rc4 kernel.

Signed-off-by: Amperific <amperific@gmail.com>
SELinux: Enable setting security contexts on rootfs inodes.

rootfs (ramfs) can support setting of security contexts
by userspace due to the vfs fallback behavior of calling
the security module to set the in-core inode state
for security.* attributes when the filesystem does not
provide an xattr handler.  No xattr handler required
as the inodes are pinned in memory and have no backing
store.

This is useful in allowing early userspace to label individual
files within a rootfs while still providing a policy-defined
default via genfs.

Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
Signed-off-by: Paul Moore <pmoore@redhat.com>

Signed-off-by: Amperific <amperific@gmail.com>
staging: zsmalloc: access page->private by using page_private macro

Signed-off-by: Sunghan Suh <sunghan.suh@samsung.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Signed-off-by: Amperific <amperific@gmail.com>
SELinux: fix selinuxfs policy file on big endian systems

The /sys/fs/selinux/policy file is not valid on big endian systems like
ppc64 or s390.  Let's see why:

static int hashtab_cnt(void *key, void *data, void *ptr)
{
	int *cnt = ptr;
	*cnt = *cnt + 1;

	return 0;
}

static int range_write(struct policydb *p, void *fp)
{
	size_t nel;
[...]
	/* count the number of entries in the hashtab */
	nel = 0;
	rc = hashtab_map(p->range_tr, hashtab_cnt, &nel);
	if (rc)
		return rc;
	buf[0] = cpu_to_le32(nel);
	rc = put_entry(buf, sizeof(u32), 1, fp);

So size_t is 64 bits.  But then we pass a pointer to it as we do to
hashtab_cnt.  hashtab_cnt thinks it is a 32 bit int and only deals with
the first 4 bytes.  On x86_64 which is little endian, those first 4
bytes and the least significant, so this works out fine.  On ppc64/s390
those first 4 bytes of memory are the high order bits.  So at the end of
the call to hashtab_map nel has a HUGE number.  But the least
significant 32 bits are all 0's.

We then pass that 64 bit number to cpu_to_le32() which happily truncates
it to a 32 bit number and does endian swapping.  But the low 32 bits are
all 0's.  So no matter how many entries are in the hashtab, big endian
systems always say there are 0 entries because I screwed up the
counting.

The fix is easy.  Use a 32 bit int, as the hashtab_cnt expects, for nel.

Signed-off-by: Eric Paris <eparis@redhat.com>
Signed-off-by: Paul Moore <pmoore@redhat.com>

Signed-off-by: Amperific <amperific@gmail.com>
SELinux: remove crazy contortions around proc

We check if the fsname is proc and if so set the proc superblock security
struct flag.  We then check if the flag is set and use the string 'proc'
for the fsname instead of just using the fsname.  What's the point?  It's
always proc...  Get rid of the useless conditional.

Signed-off-by: Eric Paris <eparis@redhat.com>

Signed-off-by: Amperific <amperific@gmail.com>
SELinux: use define for number of bits in the mnt flags mask

We had this random hard coded value of '8' in the code (I put it there)
for the number of bits to check for mount options.  This is stupid.  Instead
use the #define we already have which tells us the number of mount
options.

Signed-off-by: Eric Paris <eparis@redhat.com>

Signed-off-by: Amperific <amperific@gmail.com>
SELinux: rename SE_SBLABELSUPP to SBLABEL_MNT

Just a flag rename as we prepare to make it not so special.

Signed-off-by: Eric Paris <eparis@redhat.com>

Signed-off-by: Amperific <amperific@gmail.com>
SELinux: do all flags twiddling in one place

Currently we set the initialize and seclabel flag in one place.  Do some
unrelated printk then we unset the seclabel flag.  Eww.  Instead do the flag
twiddling in one place in the code not seperated by unrelated printk.  Also
don't set and unset the seclabel flag.  Only set it if we need to.

Signed-off-by: Eric Paris <eparis@redhat.com>

Signed-off-by: Amperific <amperific@gmail.com>
SELinux: renumber the superblock options

Just to make it clear that we have mount time options and flags,
separate them.  Since I decided to move the non-mount options above
above 0x10, we need a short instead of a char.  (x86 padding says
this takes up no additional space as we have a 3byte whole in the
structure)

Signed-off-by: Eric Paris <eparis@redhat.com>

Signed-off-by: Amperific <amperific@gmail.com>
SELinux: change sbsec->behavior to short

We only have 6 options, so char is good enough, but use a short as that
packs nicely.  This shrinks the superblock_security_struct just a little
bit.

Signed-off-by: Eric Paris <eparis@redhat.com>

Signed-off-by: Amperific <amperific@gmail.com>
SELinux: do not handle seclabel as a special flag

Instead of having special code around the 'non-mount' seclabel mount option
just handle it like the mount options.

Signed-off-by: Eric Paris <eparis@redhat.com>

Signed-off-by: Amperific <amperific@gmail.com>
SELinux: pass a superblock to security_fs_use

Rather than passing pointers to memory locations, strings, and other
stuff just give up on the separation and give security_fs_use the
superblock.  It just makes the code easier to read (even if not easier to
reuse on some other OS)

Signed-off-by: Eric Paris <eparis@redhat.com>

Signed-off-by: Amperific <amperific@gmail.com>
SELinux: use a helper function to determine seclabel

Use a helper to determine if a superblock should have the seclabel flag
rather than doing it in the function.  I'm going to use this in the
security server as well.

Signed-off-by: Eric Paris <eparis@redhat.com>

Signed-off-by: Amperific <amperific@gmail.com>