Repository: intel/dffml

Issue:
source: ini: Create source for parsing .ini files
The goal of this issue is to show how to create a source derived from `FileSource` and `MemorySource` which saves and loads data from `.ini` files.

To do this we will create a new source, `INISource` (the call to `entrypoint` will be `@entrypoint("ini")`).

The source will be similar in structure to the `IDX1` and `IDX3` sources. Only we'll be implementing the `dump_fd` method in this one.

The [`configparser`](https://docs.python.org/3/library/configparser.html) library will be helpful here, since it parses `.ini` files.

Since we're working with file objects in `load_fd` and `dump_fd`, rather than filenames, we'll want to use the [`read_file`](https://docs.python.org/3/library/configparser.html#configparser.ConfigParser.read_file) method within `load_fd` .

In `dump_fd`, we'll likely want to use [`read_dict`](https://docs.python.org/3/library/configparser.html#configparser.ConfigParser.read_dict) to create a `Configparser`. Both `load_fd` and `dump_fd` should have local variables which are an instance of `Configparser`. We don't want to make it class local because if features in an individual record are deleted then they would still show up in the output.

You'll want to make sure to call `record.features()` and using that as the data to save for the record. Also assume when loading that you're loading the feature data. Each `[section]` should be used as a record. With the name of the section being the `key` for that record.

Make sure to write lots of comments, since this will be used as the example for another tutorial on Sources (see issue https://github.com/intel/dffml/issues/282).

Here is the `idx1` source as an example.

https://github.com/intel/dffml/blob/155c114da35b80d5a17c96eece5c45dc8a6322c5/dffml/source/idx1.py#L6-L56

Comments:

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 5a76e35f2..05e232829 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Doctestable examples to `operation/mapping.py`
 - load function in high_level API
 - Doctestable examples to `db` operations.
+- Source for parsing `.ini` file formats
 ### Changed
 - `Edit on Github` button now hidden for plugins.
 - Doctests now run via unittests
diff --git a/dffml/source/ini.py b/dffml/source/ini.py
new file mode 100644
index 000000000..9a7ddf0dd
--- /dev/null
+++ b/dffml/source/ini.py
@@ -0,0 +1,66 @@
+from configparser import ConfigParser
+
+from ..base import config
+from ..record import Record
+from .file import FileSource
+from .memory import MemorySource
+from ..util.data import parser_helper
+from ..util.entrypoint import entrypoint
+
+
+@config
+class INISourceConfig:
+    filename: str
+    readwrite: bool = False
+    allowempty: bool = False
+
+
+@entrypoint("ini")
+class INISource(FileSource, MemorySource):
+    """
+    Source to read files in .ini format.
+    """
+
+    CONFIG = INISourceConfig
+
+    async def load_fd(self, ifile):
+        # Creating an instance of configparser
+        parser = ConfigParser()
+        # Read from a file object
+        parser.read_file(ifile)
+        # Get all the sections present in the file
+        sections = parser.sections()
+
+        self.mem = {}
+
+        # Go over each section
+        for section in sections:
+            # Get data under each section as a dict
+            temp_dict = {}
+            for k, v in parser.items(section):
+                temp_dict[k] = parser_helper(v)
+            # Each section used as a record
+            self.mem[str(section)] = Record(
+                str(section), data={"features": temp_dict},
+            )
+
+        self.logger.debug("%r loaded %d sections", self, len(self.mem))
+
+    async def dump_fd(self, fd):
+        # Create an instance of configparser
+        parser = ConfigParser()
+
+        # Go over each section and record in mem
+        for section, record in self.mem.items():
+            # Get each section data as a dict
+            section_data = record.features()
+            if section not in parser.keys():
+                # If section does not exist add new section
+                parser.add_section(section)
+            # Set section data
+            parser[section] = section_data
+
+        # Write to the fileobject
+        parser.write(fd)
+
+        self.logger.debug("%r saved %d sections", self, len(self.mem))
diff --git a/setup.py b/setup.py
index d6b610728..0ec52f2a1 100644
--- a/setup.py
+++ b/setup.py
@@ -77,6 +77,7 @@ setup(
             "idx1 = dffml.source.idx1:IDX1Source",
             "idx3 = dffml.source.idx3:IDX3Source",
             "db = dffml.source.db:DbSource",
+            "ini = dffml.source.ini:INISource",
         ],
         "dffml.port": ["json = dffml.port.json:JSON"],
         "dffml.service.cli": ["dev = dffml.service.dev:Develop"],


Test patch:
diff --git a/tests/source/test_ini.py b/tests/source/test_ini.py
new file mode 100644
index 000000000..524e3aa1f
--- /dev/null
+++ b/tests/source/test_ini.py
@@ -0,0 +1,30 @@
+import os
+from tempfile import TemporaryDirectory
+
+from dffml.record import Record
+from dffml.source.ini import INISource
+from dffml.high_level import load, save
+from dffml.util.asynctestcase import AsyncTestCase
+
+
+class TestINISource(AsyncTestCase):
+    async def test_ini(self):
+        with TemporaryDirectory() as testdir:
+            self.testfile = os.path.join(testdir, "testfile.ini")
+            # Create a source
+            source = INISource(
+                filename=self.testfile, allowempty=True, readwrite=True,
+            )
+            # Save some data in the source
+            await save(
+                source,
+                Record("section1", data={"features": {"A": 1, "B": 2,}}),
+                Record("section2", data={"features": {"C": 3, "D": 4,}}),
+            )
+            # Load all the records
+            records = [record async for record in load(source)]
+
+            self.assertIsInstance(records, list)
+            self.assertEqual(len(records), 2)
+            self.assertDictEqual(records[0].features(), {"a": 1, "b": 2})
+            self.assertDictEqual(records[1].features(), {"c": 3, "d": 4})