Repository: ansible/ansible

Issue:
Add support for f2fs to filesystems
<!---
Verify first that your issue/request is not already reported on GitHub.
THIS FORM WILL BE READ BY A MACHINE, COMPLETE ALL SECTIONS AS DESCRIBED.
Also test if the latest release, and devel branch are affected too.
ALWAYS add information AFTER (OUTSIDE) these html comments.
Otherwise it may end up being automatically closed by our bot. -->

##### ISSUE TYPE
 - Feature Idea

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->
filesystem

##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
ansible 2.5.3
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/langhorst/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]
```

##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->
no output


##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->
linux

##### SUMMARY
<!--- Explain the problem briefly -->
cannot make an f2fs filesystem from ansible
```
failed: [seq-flash] (item={u'name': u'md0', u'level': 0, u'devices': [u'/dev/nvme0n1p5', u'/dev/nvme1n1p1'], u'state': u'present', u'filesystem': u'f2fs', u'mountpoint': u'/seq_data'}) => {"changed": false, "item": {"devices": ["/dev/nvme0n1p5", "/dev/nvme1n1p1"], "filesystem": "f2fs", "level": 0, "mountpoint": "/seq_data", "name": "md0", "state": "present"}, "msg": "value of fstype must be one of: ext4, ext3, ext2, btrfs, vfat, ext4dev, reiserfs, lvm, xfs, got: f2fs"}
```
##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
  - include_role:
      name: mrlesmithjr.mdadm
    vars:
      mdadm_arrays:
       - name: md0
         devices: [/dev/nvme0n1p5, /dev/nvme1n1p1]
         filesystem: f2fs
         level: 0
         mountpoint: /seq_data
         state: present
```




Discussion:
Files identified in the description:
* [lib/ansible/modules/system/filesystem.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/system/filesystem.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @abulimov @pilou-
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/modules/system/filesystem.py b/lib/ansible/modules/system/filesystem.py
--- a/lib/ansible/modules/system/filesystem.py
+++ b/lib/ansible/modules/system/filesystem.py
@@ -22,7 +22,7 @@
 version_added: "1.2"
 options:
   fstype:
-    choices: [ btrfs, ext2, ext3, ext4, ext4dev, lvm, reiserfs, xfs, vfat, ocfs2 ]
+    choices: [ btrfs, ext2, ext3, ext4, ext4dev, f2fs, lvm, ocfs2, reiserfs, xfs, vfat ]
     description:
     - Filesystem type to be created.
     - reiserfs support was added in 2.2.
@@ -30,6 +30,7 @@
     - since 2.5, I(dev) can be an image file.
     - vfat support was added in 2.5
     - ocfs2 support was added in 2.6
+    - f2fs support was added in 2.7
     required: yes
     aliases: [type]
   dev:
@@ -45,7 +46,7 @@
   resizefs:
     description:
     - If C(yes), if the block device and filesytem size differ, grow the filesystem into the space.
-    - Supported for C(ext2), C(ext3), C(ext4), C(ext4dev), C(lvm), C(xfs) and C(vfat) filesystems.
+    - Supported for C(ext2), C(ext3), C(ext4), C(ext4dev), C(f2fs), C(lvm), C(xfs) and C(vfat) filesystems.
     - XFS Will only grow if mounted.
     - vFAT will likely fail if fatresize < 1.04.
     type: bool
@@ -109,6 +110,8 @@ class Filesystem(object):
     MKFS = None
     MKFS_FORCE_FLAGS = ''
 
+    LANG_ENV = {'LANG': 'C', 'LC_ALL': 'C', 'LC_MESSAGES': 'C'}
+
     def __init__(self, module):
         self.module = module
 
@@ -160,7 +163,7 @@ class Ext(Filesystem):
     def get_fs_size(self, dev):
         cmd = self.module.get_bin_path('tune2fs', required=True)
         # Get Block count and Block size
-        _, size, _ = self.module.run_command([cmd, '-l', str(dev)], check_rc=True)
+        _, size, _ = self.module.run_command([cmd, '-l', str(dev)], check_rc=True, environ_update=self.LANG_ENV)
         for line in size.splitlines():
             if 'Block count:' in line:
                 block_count = int(line.split(':')[1].strip())
@@ -188,7 +191,7 @@ class XFS(Filesystem):
 
     def get_fs_size(self, dev):
         cmd = self.module.get_bin_path('xfs_growfs', required=True)
-        _, size, _ = self.module.run_command([cmd, '-n', str(dev)], check_rc=True)
+        _, size, _ = self.module.run_command([cmd, '-n', str(dev)], check_rc=True, environ_update=self.LANG_ENV)
         for line in size.splitlines():
             col = line.split('=')
             if col[0].strip() == 'data':
@@ -233,6 +236,49 @@ class Ocfs2(Filesystem):
     MKFS_FORCE_FLAGS = '-Fx'
 
 
+class F2fs(Filesystem):
+    MKFS = 'mkfs.f2fs'
+    GROW = 'resize.f2fs'
+
+    @property
+    def MKFS_FORCE_FLAGS(self):
+        mkfs = self.module.get_bin_path(self.MKFS, required=True)
+        cmd = "%s %s" % (mkfs, os.devnull)
+        _, out, _ = self.module.run_command(cmd, check_rc=False, environ_update=self.LANG_ENV)
+        # Looking for "	F2FS-tools: mkfs.f2fs Ver: 1.10.0 (2018-01-30)"
+        # mkfs.f2fs displays version since v1.2.0
+        match = re.search(r"F2FS-tools: mkfs.f2fs Ver: ([0-9.]+) \(", out)
+        if match is not None:
+            # Since 1.9.0, mkfs.f2fs check overwrite before make filesystem
+            # before that version -f switch wasn't used
+            if LooseVersion(match.group(1)) >= LooseVersion('1.9.0'):
+                return '-f'
+
+        return ''
+
+    def get_fs_size(self, dev):
+        cmd = self.module.get_bin_path('dump.f2fs', required=True)
+        # Get sector count and sector size
+        _, dump, _ = self.module.run_command([cmd, str(dev)], check_rc=True, environ_update=self.LANG_ENV)
+        sector_size = None
+        sector_count = None
+        for line in dump.splitlines():
+            if 'Info: sector size = ' in line:
+                # expected: 'Info: sector size = 512'
+                sector_size = int(line.split()[4])
+            elif 'Info: total FS sectors = ' in line:
+                # expected: 'Info: total FS sectors = 102400 (50 MB)'
+                sector_count = int(line.split()[5])
+
+            if None not in (sector_size, sector_count):
+                break
+        else:
+            self.module.warn("Unable to process dump.f2fs output '%s'", '\n'.join(dump))
+            self.module.fail_json(msg="Unable to process dump.f2fs output for %s" % dev)
+
+        return sector_size * sector_count
+
+
 class VFAT(Filesystem):
     if get_platform() == 'FreeBSD':
         MKFS = "newfs_msdos"
@@ -242,7 +288,7 @@ class VFAT(Filesystem):
 
     def get_fs_size(self, dev):
         cmd = self.module.get_bin_path(self.GROW, required=True)
-        _, output, _ = self.module.run_command([cmd, '--info', str(dev)], check_rc=True)
+        _, output, _ = self.module.run_command([cmd, '--info', str(dev)], check_rc=True, environ_update=self.LANG_ENV)
         for line in output.splitlines()[1:]:
             param, value = line.split(':', 1)
             if param.strip() == 'Size':
@@ -271,6 +317,7 @@ def get_fs_size(self, dev):
     'ext3': Ext3,
     'ext4': Ext4,
     'ext4dev': Ext4,
+    'f2fs': F2fs,
     'reiserfs': Reiserfs,
     'xfs': XFS,
     'btrfs': Btrfs,


Test patch:
diff --git a/test/integration/targets/filesystem/defaults/main.yml b/test/integration/targets/filesystem/defaults/main.yml
--- a/test/integration/targets/filesystem/defaults/main.yml
+++ b/test/integration/targets/filesystem/defaults/main.yml
@@ -5,6 +5,11 @@ tested_filesystems:
   # Other minimal sizes:
   # - XFS: 20Mo
   # - Btrfs: 100Mo (50Mo when "--metadata single" is used)
+  # - f2fs:
+  #     - 1.2.0 requires at leat 116Mo
+  #     - 1.7.0 requires at least 30Mo
+  #     - 1.10.0 requires at least 38Mo
+  #     - resizefs asserts when initial fs is smaller than 60Mo and seems to require 1.10.0
   ext4: {fssize: 10, grow: True}
   ext4dev: {fssize: 10, grow: True}
   ext3: {fssize: 10, grow: True}
@@ -13,4 +18,5 @@ tested_filesystems:
   btrfs: {fssize: 100, grow: False}  # grow not implemented
   vfat: {fssize: 20, grow: True}
   ocfs2: {fssize: '{{ ocfs2_fssize }}', grow: False}  # grow not implemented
+  f2fs: {fssize: '{{ f2fs_fssize|default(60) }}', grow: 'f2fs_version is version("1.10.0", ">=")'}
   # untested: lvm, requires a block device
diff --git a/test/integration/targets/filesystem/tasks/main.yml b/test/integration/targets/filesystem/tasks/main.yml
--- a/test/integration/targets/filesystem/tasks/main.yml
+++ b/test/integration/targets/filesystem/tasks/main.yml
@@ -22,8 +22,14 @@
   when:
     - 'not (item.key == "btrfs" and ansible_system == "FreeBSD")'
     - 'not (item.key == "ocfs2" and ansible_os_family != "Debian")'
-    # On Ubuntu trusty, blkid is unable to identify filesystem smaller than 256Mo, see:
+    # On Ubuntu trusty, blkid (2.20.1) is unable to identify filesystem smaller than 256Mo, see:
     # https://www.kernel.org/pub/linux/utils/util-linux/v2.21/v2.21-ChangeLog
     # https://anonscm.debian.org/cgit/collab-maint/pkg-util-linux.git/commit/?id=04f7020eadf31efc731558df92daa0a1c336c46c
     - 'not (item.key == "btrfs" and (ansible_distribution == "Ubuntu" and ansible_distribution_release == "trusty"))'
+    - 'not (item.key == "f2fs" and ansible_system == "FreeBSD")'
+    # f2fs-tools package not available with RHEL/CentOS
+    - 'not (item.key == "f2fs" and ansible_distribution in ["CentOS", "RedHat"])'
+    # On Ubuntu trusty, blkid (2.20.1) is unable to identify F2FS filesystem. blkid handles F2FS since v2.23, see:
+    # https://mirrors.edge.kernel.org/pub/linux/utils/util-linux/v2.23/v2.23-ReleaseNotes
+    - 'not (item.key == "f2fs" and ansible_distribution == "Ubuntu" and ansible_distribution_version is version("14.04", "<="))'
   loop: "{{ lookup('dict', tested_filesystems) }}"
diff --git a/test/integration/targets/filesystem/tasks/setup.yml b/test/integration/targets/filesystem/tasks/setup.yml
--- a/test/integration/targets/filesystem/tasks/setup.yml
+++ b/test/integration/targets/filesystem/tasks/setup.yml
@@ -35,6 +35,24 @@
       name: ocfs2-tools
       state: present
     when: ansible_os_family == 'Debian'
+
+  - when:
+      - ansible_os_family != 'RedHat' or ansible_distribution == 'Fedora'
+      - ansible_distribution != 'Ubuntu' or ansible_distribution_version is version('16.04', '>=')
+    block:
+      - name: install f2fs
+        package:
+          name: f2fs-tools
+          state: present
+
+      - name: fetch f2fs version
+        command: mkfs.f2fs /dev/null
+        ignore_errors: yes
+        register: mkfs_f2fs
+
+      - set_fact:
+          f2fs_version: '{{ mkfs_f2fs.stdout | regex_search("F2FS-tools: mkfs.f2fs Ver:.*") | regex_replace("F2FS-tools: mkfs.f2fs Ver: ([0-9.]+) .*", "\1") }}'
+
   when: ansible_system == 'Linux'
 
 - block:
diff --git a/test/integration/targets/filesystem/vars/Ubuntu-14.04.yml b/test/integration/targets/filesystem/vars/Ubuntu-14.04.yml
--- a/test/integration/targets/filesystem/vars/Ubuntu-14.04.yml
+++ b/test/integration/targets/filesystem/vars/Ubuntu-14.04.yml
@@ -1,2 +1,3 @@
 ---
 ocfs2_fssize: 108
+f2fs_fssize: 116