kernel-3.15-truncate_inode_pages_final

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
kernel-3.15-remount-sync

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
kernel-3.15-rename2

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
kernel-3.15-map_pages

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
Merge branch 'master' into user

* hirofumi:
  kernel-3.15-map_pages
  kernel-3.15-rename2
  kernel-3.15-remount-sync
  kernel-3.15-truncate_inode_pages_final
  tux3: Add tux3 to fs makefile and config
  tux3: Track api change to __generic_file_aio_write
  tux3: Fix missing i_ctime update for xattr
  tux3: Fix timestamp update for special files
user-3.15-truncate_inode_pages_final

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
user-3.15-err-bool

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
Add writeback super operation (draft)

Hi,

This is the first of four core changes we would like for Tux3. We
start with a hard one and suggest a simple solution.

Traditionally, core kernel tells each mounted filesystems which
dirty pages of which inodes should be flushed to disk, but
unfortunately, is blissfully ignorant of filesystem-specific
ordering constraints. This scheme was really invented for Ext2 and
has not evolved much recently. Tux3, with its strong ordering and
optimized delta commit, cannot tolerate random flushing and
therefore takes responsibility for flush ordering itself. On the
other hand, Tux3 has no good way to know when is the right time to
flush, but core is good at that. This proposed API harmonizes those
two competencies so that Tux3 and core each take care of what they
are good at, and not more.

The API extension consists of a new writeback hook and two helpers
to be used within the hook. The hook sits about halfway down the
fs-writeback stack, just after core has determined that some dirty
inodes should be flushed to disk and just before it starts thinking
about which inodes those should be. At that point, core calls Tux3
instead of continuing on down the usual do_writepages path. Tux3
responds by staging a new delta commit, using the new helpers to
tell core which inodes were flushed versus being left dirty in
cache. This is pretty much the same behavior as the traditional
writeout path, but less convoluted, probably more efficient, and
certainly easier to analyze.

The new writeback hook looks like:

    progress = sb->s_op->writeback(sb, &writeback_control, &nr_pages);

This should be self-explanatory: nr_pages and progress have the
semantics of existing usage in fs-writeback; writeback_control is
ignored by Tux3, but that is only because Tux3 always flushes
everything and does not require hints for now. We can safely assume
that &wbc or equivalent is desirable here. An obvious wart is the
overlap between "progress" and "nr_pages", but fs-writeback thinks
that way, so it would not make much sense to improve this interface
without also improving the larger fs-writeback scheme.

Tux3 necessarily keeps its own dirty inode list, which is an area
of overlap with fs-writeback. In a perfect world, there would be
just one dirty inode list per superblock, on which both fs-writeback
and Tux3 would operate. That would be a deeper core change than
seems appropriate right now.

Potential races are a big issue with this API, which is no surprise.
The fs-writeback scheme requires keeping several kinds of object in
sync: tux3 dirty inode lists, fs-writeback dirty inode lists and
inode dirty state. The new helpers inode_writeback_done(inode) and
inode_writeback_touch(inode) take care of that while hiding
internal details of the fs-writeback implementation.

Tux3 calls inode_writeback_done when it has flushed an inode and
marked it clean, or calls inode_writeback_touch if it intends to
retain a dirty inode in cache. These have simple implementations.
The former just removes a clean inode from any fs-writeback list.
The latter updates the inode's dirty timestamp so that fs-writeback
does not keep trying flush it. Both these things could be done more
efficiently by re-engineering fs-writeback, but we prefer to work
with the existing scheme for now.

Hirofumi's masterful hack nicely avoided racy removal of inodes from
the writeback list by taking an internal fs-writeback lock inside
filesystem code. The new helper requires dropping i_lock inside
filesystem code and retaking it in the helper, so inode redirty can
race with writeback list removal. This does not seem to present a
problem because filesystem code is able to enforce strict
alternation of cleaning and calling the helper. As an offsetting
advantage, writeback lock contention is reduced.

Compared to Hirofumi's hack, the cost of this interface is one
additional spinlock per inode_writeback_done, which is tiny
compared to the convoluted code path that is avoided.

Regards,

Daniel

commit c3f83a3bf3ad9b6ef3c326ee5169f9da2e8214c6
Author: Daniel Phillips <daniel@tux3.org>
Date:   Mon May 26 23:01:33 2014 -0700

Add a "writeback" super operation to be called in the
form:

        progress = sb->s_op->writeback(sb, &wbc, &pages);

The filesystem is expected to flush some inodes to disk
and return progress of at least 1, or if no inodes are
flushed, return progress of zero. The filesystem should
try to flush at least the number of pages specified in
*pages, or if that is not possible, return approximately
the number of pages not flushed into *pages.

Within the ->writeback callback, the filesystem should
call inode_writeback_done(inode) for each inode flushed
(and therefore set clean) or inode_writeback_touch(inode)
for any inode that will be retained dirty in cache.

Signed-off-by: Daniel Phillips
Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
Merge branch 'master' into user

* hirofumi:
  tux3: Remove writeback hack using new writeback
  Add writeback super operation (draft)
tux3: Userland build fix for writeback hook

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
tux3: Remove writeback hack using new writeback

Instead of re-implementing part of fs/fs-writeback.c, use a proposed
net ->writeback super operation to drive delta writeback. For each
inode that is cleaned, call inode_writeback_done(inode). For each
inode that will be kept dirty in cache, call inode_writeback_touch
so that the inode appears young to fs-writeback and does not trigger
repeated ->writeback flushes.

Signed-off-by: Daniel Phillips
Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
core-mmap-page-cow

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
mmap-core-change

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
mmap-core-change2

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
core-page_cow_file

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
mmap-page_cow_file

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
core-truncate_page

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
mmap-truncate_page

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
Merge branch 'master' into user

* hirofumi:
  mmap-truncate_page
  core-truncate_page
  mmap-page_cow_file
  core-page_cow_file
  mmap-core-change2
  mmap-core-change
  core-mmap-page-cow
user-truncate_page

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>