tdb: Fix a tdb corruption

tdb_purge_dead can change the next pointer of "rec" if we purge the record
right behind the current record to be deleted. Just overwrite the magic,
not the whole record with stale data.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
tdb: Don't purge records to a blocked freelist

If the freelist is heavily contended, we should avoid accessing it

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
tdb: Do a best fit search for dead records

Hash chains are (or can be made) short enough that a full search for the
best-fitting dead record is feasible. The freelist can become much longer,
there we don't do the full search but accept records which are too large.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
tdb: Move adding tailer space to tdb_find_dead

This aligns the tdb_find_dead API with the tdb_allocate API and thus makes it a
bit easier to understand, at least for me.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
tdb: Add "last_ptr" to tdb_find_dead

Will be used soon to unlink a dead record from a chain

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
tdb: Make "tdb_find_dead" internally public

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
tdb: Make "tdb_purge_dead" internally public

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
tdb: Reduce freelist contention

In a metadata-intensive benchmark we have seen the locking.tdb freelist to be
one of the central contention points. This patch removes most of the contention
on the freelist. Ages ago we already reduced freelist contention by using the
even much older DEAD records: If TDB_VOLATILE is set, don't directly put
deleted records on the freelist, but just mark a few of them just as DEAD. The
next new record can them re-use that space without consulting the freelist.

This patch builds upon the DEAD records: If we need space and the freelist is
busy, instead of doing a blocking wait on the freelist, start looking into
other chains for DEAD records and steal them from there. This way every hash
chain becomes a small freelist. Just wander around the hash chains as long as
the freelist is still busy.

With this patch and the tdb mutex patch (following hopefully some time soon)
you can see a heavily busy clustered smbd run without locking.tdb futex
syscalls.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
tdb: change version to 1.2.13.

* internal code cleanups
* always open internal TDBs with incompatible hash
* avoid reallocations in locking code
* systematize output format in tdbtool dump
* reduce freelist contention when allocating new records
  - try to find dead records also in other chains
  - don't do blocking locks on the freelist

Signed-off-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Michael Adam <obnox@samba.org>
Autobuild-Date(master): Tue Mar 18 15:42:48 CET 2014 on sn-devel-104
s3: smbd: Fileserving share access checks.

Git commit 86d1e1db8e2747e30c89627cda123fde1e84f579
fixed share_access not being reset between users,
by changing make_connection_snum() to call a common
function check_user_share_access() in the same way
that change_to_user() (which can be called on any
incoming packet) does.

Unfortunately that bugfix was incorrect and
broke "force user" and "force group" as it
called check_user_share_access() inside
make_connection_snum() using the conn->session_info
pointer instead of the vuser->session_info pointer.

conn->session_info represents the token to use
when actually accessing the file system, and so
is modified by force user and force group.

conn->session_info represents the "pristine"
token of the user logging in, and is never modified
by force user and force group.

Samba 3.6.x checked the share access based on
the "pristine" token of the user logging in,
not the token modified by force user and force group.
This change restores the expected behavior.

Fixes bug #9878 - force user does not work as expected

https://bugzilla.samba.org/show_bug.cgi?id=9878

Signed-off-by: Jeremy Allison <jra@samba.org>
Tested-by: Gerhard Wiesinger <lists@wiesinger.com>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Andreas Schneider <asn@cryptomilk.org>
Autobuild-Date(master): Tue Mar 18 19:19:31 CET 2014 on sn-devel-104
s4-torture: add test_EnumPrinterDrivers_buffers function.

This will allow to test the enumdriver call with pre-allocated buffer.

Guenther

Signed-off-by: Günther Deschner <gd@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
s4-torture: add some tests for pre-allocated buffers in enumprinterdrivers call.

Guenther

Signed-off-by: Günther Deschner <gd@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
s3-rpc_server: Fix handling of fragmented rpc requests.

We need to call pipe_init_outgoing_data() as the first thing in
process_complete_pdu(). Otherwise the caller may use uninitialized
memory and tries to write a response into the socket.

The problem happens only if a real socket is used, which means
in all cases for master and only with external rpc daemons in v4-0
and v4-1.

The problem looks like this in the logs.

    [2014/03/20 14:49:35.531663, 10, pid=7309, effective(0, 0), real(0, 0), class=rpc_srv] ../source3/rpc_server/srv_pipe.c:1627(process_complete_pdu)
      Processing packet type 0
    [2014/03/20 14:49:35.531695, 10, pid=7309, effective(0, 0), real(0, 0), class=rpc_srv] ../source3/rpc_server/srv_pipe.c:1472(dcesrv_auth_request)
      Checking request auth.
    [2014/03/20 14:49:35.531738, 10, pid=7309, effective(0, 0), real(0, 0)] ../source3/rpc_server/rpc_server.c:521(named_pipe_packet_process)
      Sending 1 fragments in a total of 0 bytes
    [2014/03/20 14:49:35.531769, 10, pid=7309, effective(0, 0), real(0, 0)] ../source3/rpc_server/rpc_server.c:526(named_pipe_packet_process)
      Sending PDU number: 0, PDU Length: 4294967228
    [2014/03/20 14:49:35.531801,  2, pid=7309, effective(0, 0), real(0, 0)] ../source3/rpc_server/rpc_server.c:565(named_pipe_packet_done)
      Writev failed!
    [2014/03/20 14:49:35.531845,  2, pid=7309, effective(0, 0), real(0, 0)] ../source3/rpc_server/rpc_server.c:595(named_pipe_packet_done)
      Fatal error(Message too long). Terminating client(127.0.0.1) connection!

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10481

Pair-Programmed-With: Stefan Metzmacher <metze@samba.org>
Signed-off-by: Andreas Schneider <asn@cryptomilk.org>
Reviewed-by: Guenther Deschner <gd@samba.org

Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Thu Mar 20 18:30:17 CET 2014 on sn-devel-104
dbcheck: Ensure dbcheck can operate with --attrs set

This also includes a test to ensure we do not regress on this point.

Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
build: Exclude source4/selftest/provisions/release-4-1-0rc3 from the tarball

Change-Id: Id4ddaabb91363174d2fbef09e823f53b13912a51
Signed-off-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Fri Mar 21 10:06:04 CET 2014 on sn-devel-104
s3: smbd: Ensure we always go via getgroups_unix_user() when creating an NT token.

This has to be done in every code path that creates
an NT token, as remote users may have been added to
the local /etc/group database. Tokens created merely
from the info3 structs (via the DC or via the krb5 PAC)
won't have these local groups.

https://bugzilla.samba.org/show_bug.cgi?id=10508

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Simo Sorce <idra@samba.org>
dfs_server: randomize the server redirect set

comply with [MS-DFSC] section 3.2.1.1

Signed-off-by: Arvid Requate <requate@univention.de>
Reviewed-by: Andrew Bartlett <abartlet@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>
smbd: Explicitly pass "file_id" to rename_open_files

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
smbd: Explicitly pass "file_id" to schedule_defer_open

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
smbd: Use fsp->file_id in open_file_ntcreate

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>