lib/util: use proper include for struct stat

use <sys/stat.h> instead of forward declaring it
This fixes the build on AIX

Bug: https://bugzilla.samba.org/show_bug.cgi?id=10276

Signed-off-by: Christian Ambach <ambi@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Christian Ambach <ambi@samba.org>
Autobuild-Date(master): Thu Nov 21 22:12:45 CET 2013 on sn-devel-104
(cherry picked from commit 4ea92841b72a7b701a7c0fadf63430a7d0a3fac8)
smbd: Fix regression for the dropbox case.

We need to allow to save a file to a directory with perm -wx.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10297

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>
(cherry picked from commit 5b49fe24c906cbae12beff7a1b45de6809258cab)
smbd: change flag name from UCF_CREATING_FILE to UCF_PREP_CREATEFILE

In preparation to using it for all open calls.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10297

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
(cherry picked from commit 874318a97868e08837a1febb1be8e8a167b5ae0f)
smbd: Always use UCF_PREP_CREATEFILE for filename_convert calls to resolve a path for open.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10297

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Mon Dec  9 21:02:21 CET 2013 on sn-devel-104

(cherry picked from commit f98d10af2a05f0261611f4cabdfe274cd9fe91c0)
s3:smb2_server: fix drain_socket error handling

smbd_smb2_request_error_ex() should return NTSTATUS and the caller
will terminate the connection.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
(cherry picked from commit 9393e28df59954414313bfae70ffb796d3e332fe)
s3:smb2_server: for performance reasons we use tevent_fd and readv/writev directly

Going via tevent_req_create/talloc_free at multiple layer costs
too much cpu cycles per request.

I tested downloading a 16GB (sparse) file with smbclient -b1 -mNT1,
and -mSMB2_02. Using smb2 max read = 64512, which means smb1 and smb2
will use the same read size.

I build with -O3 -g and compared the results with valgrind --tool=callgrind.

With -mNT1 the server uses about 2.000.000.000 cpu cycles.

This patch reduces the userspace cpu cycles for -mSMB2_02
from about ~ 8.000.000.000 down to ~ 4.000.000.000.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
(cherry picked from commit 4244a2686cddcdc754c284df884ae497afa4053a)
s3:smb2_server: use tevent_req_notify_callback() in smbd_smb2_request_pending_queue()

If the request is already done we can avoid one iteration
of tevent_loop_once(), which means we avoids one
talloc_stackframe_pool/talloc_free pair.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
(cherry picked from commit acfd4b068a5b99ac1d3fe716afff34cb7d2a0147)
s3:smb2_server: allocate smbd_smb2_request on talloc_tos()

This matches the behavior for smb1 requests
and avoids an additional malloc() per request.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
(cherry picked from commit 9d33a3f3e814e2924a423496ccc133c6c73fcd12)
s3:smb2_server: generate a header blob for the sendfile path

We need to pass the NBT header, SMB2 header and SMB2 Read header
as header blob to SMB_VFS_SENDFILE(). This allows the usage
of MSG_SEND or other tricks to avoid multiple TCP packets
on the wire.

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>
(cherry picked from commit 36efaac2597d2d36826c02f23be15e7323b09784)
s3:smb2_server: avoid calling set_current_user_info() for each request

Signed-off-by: Stefan Metzmacher <metze@samba.org>
Reviewed-by: David Disseldorp <ddiss@samba.org>

Autobuild-User(master): Stefan Metzmacher <metze@samba.org>
Autobuild-Date(master): Wed Nov 27 16:31:44 CET 2013 on sn-devel-104
(cherry picked from commit 3cc0651d9feda00b6a04f84b76744b2acc3a0446)

The last 6 patches address bug #10298 - smb2_server processing overhead.
s3-aio: Use correct locking context for SMB2

The synchronous SMB2 reads and writes use open_persistent_id. The AIO
codepathes have to use the same, otherwise a write will conflict with a
lock on the same open file.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit dfef0701c398982226dde8a8e15ff97bba0fef53)
s3: Return correct error code from SMB2 AIO read failure

This is similar to commit 27e20d5d60ea8aa526bcb7c2dfc18dd2de0bb97b which
fixed the same case for SMB2 writes: When sending the AIO read fails,
return the real error instead of mapping it to NT_STATUS_FILE_CLOSED.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit eadb2a54d1733a482999eb770182156dad1e184d)
selftest: Introduce share for testing AIO

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit 63727c15450e1db2be49ade758c369aa4599657a)
selftest: Run smb2.lock tests also against AIO share

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>
(cherry picked from commit d551d5256f9b1ca57b8018d816ea665c9b847ced)
selftest: Remove samba3.smb2.lock.*.rw-exclusive from flapping file

This test demonstrates a problem with byte range locks and AIO.

Signed-off-by: Christof Schmitt <cs@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Fri Dec  6 05:19:37 CET 2013 on sn-devel-104
(cherry picked from commit 8c3bf7b84950fbb0305bcccd49ecfc202e08901a)

The last 5 patches address bug #10310 - Fix AIO with SMB2 and locks.
smbd: Fix a panic when a smb2 brlock times out

Found by Peter Somogyi.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Jeremy Allison <jra@samba.org>

Autobuild-User(master): Jeremy Allison <jra@samba.org>
Autobuild-Date(master): Thu Dec  5 21:21:35 CET 2013 on sn-devel-104

Fix bug #10311 - SMB2 server can panic when a smb2 brlock times out.
s3-winbindd: Fix DEBUG statement in winbind_msg_offline().

Guenther

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10285

Reviewed-by: Günther Deschner <gd@samba.org>
Reviewed-by: Andreas Schneider <asn@samba.org>

Autobuild-User(master): Günther Deschner <gd@samba.org>
Autobuild-Date(master): Mon Dec  9 17:28:37 CET 2013 on sn-devel-104

Autobuild-User(v4-1-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-1-test): Tue Dec 10 15:58:13 CET 2013 on sn-devel-104
smbd - allow updates on directory write times on open handles.

If we set a non-null 'old timestamp' in the share mode database
when creating a directory handle, this prevents mtime (write time)
updates from being seen by clients, as we will always return the
timestamp stored in the database whilst the handle is open.

For files this is ok, as we update the stored timestamp
ourselves when we write to the handle. For directories
we should just rely on the mtime value from the underlying
filesystem.

Torture test to follow.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=9870

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>
smbtorture: New torture test for bug #9870.

Not fetching the latest modification time on a folder if we have read locks on it.

Prove we should just rely on the mtime value from the underlying
filesystem, even with an open handle.

  BUG: https://bugzilla.samba.org/show_bug.cgi?id=9870

Signed-off-by: Jeremy Allison <jra@samba.org>
Reviewed-by: Volker Lendecke <vl@samba.org>

Autobuild-User(master): Volker Lendecke <vl@samba.org>
Autobuild-Date(master): Thu Dec  5 10:05:06 CET 2013 on sn-devel-104

BUG: https://bugzilla.samba.org/show_bug.cgi?id=9870
Not fetching the latest modification time on a folder if we have read locks on
it.

Autobuild-User(v4-1-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-1-test): Tue Dec 10 20:19:08 CET 2013 on sn-devel-104
s3-lib: Fix %G substitution for domain users in smbd

BUG: https://bugzilla.samba.org/show_bug.cgi?id=10286
Signed-off-by: Andreas Schneider <asn@samba.org>
Reviewed-by: Christian Ambach <ambi@samba.org>

Autobuild-User(master): Christian Ambach <ambi@samba.org>
Autobuild-Date(master): Tue Dec 10 16:39:43 CET 2013 on sn-devel-104

(cherry picked from commit 8eef4ab79ec5fb7e96ad2f2ad6c9bf30db13a50d)
Signed-off-by: Andreas Schneider <asn@samba.org>

Autobuild-User(v4-1-test): Karolin Seeger <kseeger@samba.org>
Autobuild-Date(v4-1-test): Sat Dec 14 23:47:20 CET 2013 on sn-devel-104