Repository: google/dfindexeddb

Issue:
Add CLI support to parse files as blink values
Add a CLI command to parse large IndexedDB values that are stored external to the leveldb on the filesystem.
 
## Expected Behavior

```
dfindexeddb blink -s <someblobfile>
```

These should be defined by the ExternalObjectEntry with mime type `application/vnd.blink-idb-value-wrapper`

ref: https://www.iana.org/assignments/media-types/application/vnd.blink-idb-value-wrapper

Discussion:

Patch:
diff --git a/README.md b/README.md
index f386786..2bd5ad0 100644
--- a/README.md
+++ b/README.md
@@ -75,69 +75,42 @@ options:
   -h, --help    show this help message and exit
 ```
 
-To parse IndexedDB records from an sqlite file for Safari, use the following
-command:
+#### Examples:
 
-```
-dfindexeddb db --format safari [-o {json,jsonl,repr}] -h
-usage: dfindexeddb db [-h] -s SOURCE --format {chromium,chrome,safari} [--use_manifest] [-o {json,jsonl,repr}]
+To parse IndexedDB records from an sqlite file for Safari and output the
+results as JSON-L, use the following command:
 
-options:
-  -h, --help            show this help message and exit
-  -s SOURCE, --source SOURCE
-                        The source IndexedDB folder (for chrome/chromium) or file (for safari).
-  --format {chromium,chrome,safari}
-                        The type of IndexedDB to parse.
-  --use_manifest        Use manifest file to determine active/deleted records.
-  -o {json,jsonl,repr}, --output {json,jsonl,repr}
-                        Output format. Default is json
+```
+dfindexeddb db -s SOURCE --format safari -o jsonl
 ```
 
-To parse IndexedDB records from a LevelDB folder for chrome/chromium, use the following command:
+To parse IndexedDB records from a LevelDB folder for Chrome/Chromium, using the
+manifest file to determine recovered records and output as JSON, use the
+following command:
 
 ```
-dfindexeddb db --format chrome [--use_manifest] [-o {json,jsonl,repr}] -h
-usage: dfindexeddb db [-h] -s SOURCE --format {chromium,chrome,safari} [--use_manifest] [-o {json,jsonl,repr}]
-
-options:
-  -h, --help            show this help message and exit
-  -s SOURCE, --source SOURCE
-                        The source IndexedDB folder (for chrome/chromium) or file (for safari).
-  --format {chromium,chrome,safari}
-                        The type of IndexedDB to parse.
-  --use_manifest        Use manifest file to determine active/deleted records.
-  -o {json,jsonl,repr}, --output {json,jsonl,repr}
-                        Output format. Default is json
+dfindexeddb db -s SOURCE --format chrome --use_manifest
 ```
 
-To parse IndexedDB records from a LevelDB ldb (.ldb) file, use the following
-command:
+To parse IndexedDB records from a LevelDB ldb (.ldb) file and output the
+results as JSON-L, use the following command:
 
 ```
-dfindexeddb ldb -s SOURCE [-o {json,jsonl,repr}] -h
-usage: dfindexeddb ldb [-h] -s SOURCE [-o {json,jsonl,repr}]
-
-options:
-  -h, --help            show this help message and exit
-  -s SOURCE, --source SOURCE
-                        The source .ldb file.
-  -o {json,jsonl,repr}, --output {json,jsonl,repr}
-                        Output format. Default is json
+dfindexeddb ldb -s SOURCE -o jsonl
 ```
 
-To parse IndexedDB records from a LevelDB log (.log) file, use the following
-command:
+To parse IndexedDB records from a LevelDB log (.log) file and output the
+results as the Python printable representation, use the following command:
 
 ```
-dfindexeddb log -s SOURCE [-o {json,jsonl,repr}] -h
-usage: dfindexeddb log [-h] -s SOURCE [-o {json,jsonl,repr}]
+dfindexeddb log -s SOURCE -o repr
+```
 
-options:
-  -h, --help            show this help message and exit
-  -s SOURCE, --source SOURCE
-                        The source .log file.
-  -o {json,jsonl,repr}, --output {json,jsonl,repr}
-                        Output format. Default is json
+To parse a file as a Chrome/Chromium IndexedDB blink value and output the
+results as JSON:
+
+```
+dfindexeddb blink -s SOURCE
 ```
 
 ### LevelDB
@@ -159,52 +132,32 @@ options:
   -h, --help            show this help message and exit
 ```
 
+#### Examples
+
 To parse records from a LevelDB folder, use the following command:
 
 ```
-dfindexeddb db -s SOURCE [-o {json,jsonl,repr}] -h
-usage: dfindexeddb db [-h] -s SOURCE [--use_manifest] [-o {json,jsonl,repr}]
-
-options:
-  -h, --help            show this help message and exit
-  -s SOURCE, --source SOURCE
-                        The source leveldb folder
-  --use_manifest        Use manifest file to determine active/recovered records.
-  -o {json,jsonl,repr}, --output {json,jsonl,repr}
-                        Output format. Default is json
+dfindexeddb db -s SOURCE
 ```
 
-To parse records from a LevelDB log (.log) file, use the following command:
+To parse blocks / physical records/ write batches / internal key records from a
+LevelDB log (.log) file, use the following command, specifying the type (block,
+physical_records, etc) via the `-t` option.  By default, internal key records are parsed:
 
 ```
-$ dfleveldb log  -s SOURCE [-o {json,jsonl,repr}] [-t {blocks,physical_records,write_batches,parsed_internal_key}]
-
-options:
-  -h, --help            show this help message and exit
-  -s SOURCE, --source SOURCE
-                        The source leveldb file
-  -o {json,jsonl,repr}, --output {json,jsonl,repr}
-                        Output format. Default is json
-  -t {blocks,physical_records,write_batches,parsed_internal_key}, --structure_type {blocks,physical_records,write_batches,parsed_internal_key}
-                        Parses the specified structure. Default is parsed_internal_key.
+$ dfleveldb log  -s SOURCE [-t {blocks,physical_records,write_batches,parsed_internal_key}]
 ```
 
-To parse records from a LevelDB table (.ldb) file, use the following command:
+To parse blocks / records from a LevelDB table (.ldb) file, use the following
+command, specifying the type (blocks, records) via the `-t` option.  By
+default, records are parsed:
 
 ```
-$ dfleveldb ldb -s SOURCE [-o {json,jsonl,repr}] [-t {blocks,records}]
-
-options:
-  -h, --help            show this help message and exit
-  -s SOURCE, --source SOURCE
-                        The source leveldb file
-  -o {json,jsonl,repr}, --output {json,jsonl,repr}
-                        Output format. Default is json
-  -t {blocks,records}, --structure_type {blocks,records}
-                        Parses the specified structure. Default is records.
+$ dfleveldb ldb -s SOURCE [-t {blocks,records}]
 ```
 
-To parse version edit records from a Descriptor (MANIFEST) file:
+To parse version edit records from a Descriptor (MANIFEST) file, use the
+following command:
 
 ```
 $ dfleveldb descriptor -s SOURCE [-o {json,jsonl,repr}] [-t {blocks,physical_records,versionedit} | -v]
diff --git a/dfindexeddb/indexeddb/chromium/record.py b/dfindexeddb/indexeddb/chromium/record.py
index 1a1b7ee..9ade013 100644
--- a/dfindexeddb/indexeddb/chromium/record.py
+++ b/dfindexeddb/indexeddb/chromium/record.py
@@ -17,7 +17,11 @@ from __future__ import annotations
 from dataclasses import dataclass, field
 from datetime import datetime
 import io
-from typing import Any, BinaryIO, Optional, Tuple, Type, TypeVar, Union
+import pathlib
+import sys
+import traceback
+from typing import Any, BinaryIO, Generator, Optional, Tuple, Type, TypeVar, \
+    Union
 
 from dfindexeddb import errors
 from dfindexeddb.indexeddb.chromium import blink
@@ -456,7 +460,7 @@ class MaxDatabaseIdKey(BaseIndexedDBKey):
       cls, decoder: utils.LevelDBDecoder, key_prefix: KeyPrefix,
       base_offset: int = 0
   ) -> MaxDatabaseIdKey:
-    """Decodes the maximum databse key."""
+    """Decodes the maximum database key."""
     offset, key_type = decoder.DecodeUint8()
     if key_type != definitions.GlobalMetadataKeyType.MAX_DATABASE_ID:
       raise errors.ParserError('Not a MaxDatabaseIdKey')
@@ -1331,12 +1335,14 @@ class IndexedDBRecord:
   """An IndexedDB Record.
 
   Attributes:
+    path: the source file path
     offset: the offset of the record.
     key: the key of the record.
     value: the value of the record.
     sequence_number: if available, the sequence number of the record.
     type: the type of the record.
-    level: the leveldb level, None indicates the record came from a log file.
+    level: the leveldb level, if applicable, None can indicate the record
+        originated from a log file or the level could not be determined.
     recovered: True if the record is a recovered record.
   """
   path: str
@@ -1350,7 +1356,8 @@ class IndexedDBRecord:
 
   @classmethod
   def FromLevelDBRecord(
-      cls, db_record: record.LevelDBRecord
+      cls,
+      db_record: record.LevelDBRecord
   ) -> IndexedDBRecord:
     """Returns an IndexedDBRecord from a ParsedInternalKey."""
     idb_key = IndexedDbKey.FromBytes(
@@ -1366,3 +1373,74 @@ class IndexedDBRecord:
         type=db_record.record.record_type,
         level=db_record.level,
         recovered=db_record.recovered)
+
+  @classmethod
+  def FromFile(
+      cls,
+      file_path: pathlib.Path
+  ) -> Generator[IndexedDBRecord, None, None]:
+    """Yields IndexedDBRecords from a file."""
+    for db_record in record.LevelDBRecord.FromFile(file_path):
+      try:
+        yield cls.FromLevelDBRecord(db_record)
+      except(
+          errors.ParserError,
+          errors.DecoderError,
+          NotImplementedError) as err:
+        print((
+            'Error parsing Indexeddb record: '
+            f'{err} at offset {db_record.record.offset} in '
+            f'{db_record.path}'),
+            file=sys.stderr)
+        print(f'Traceback: {traceback.format_exc()}', file=sys.stderr)
+
+
+class FolderReader:
+  """A IndexedDB folder reader for Chrome/Chromium.
+
+  Attributes:
+    foldername (str): the source LevelDB folder.
+  """
+
+  def __init__(self, foldername: pathlib.Path):
+    """Initializes the FileReader.
+
+    Args:
+      foldername: the source IndexedDB folder.
+
+    Raises:
+      ValueError: if foldername is None or not a directory.
+    """
+    if not foldername or not foldername.is_dir():
+      raise ValueError(f'{foldername} is None or not a directory')
+    self.foldername = foldername
+
+  def GetRecords(
+      self,
+      use_manifest: bool = False
+  ) -> Generator[IndexedDBRecord, None, None]:
+    """Yield LevelDBRecords.
+
+    Args:
+      use_manifest: True to use the current manifest in the folder as a means to
+          find the active file set.
+
+    Yields:
+      IndexedDBRecord.
+    """
+    leveldb_folder_reader = record.FolderReader(self.foldername)
+    for leveldb_record in leveldb_folder_reader.GetRecords(
+        use_manifest=use_manifest):
+      try:
+        yield IndexedDBRecord.FromLevelDBRecord(
+            leveldb_record)
+      except(
+          errors.ParserError,
+          errors.DecoderError,
+          NotImplementedError) as err:
+        print((
+            'Error parsing Indexeddb record: '
+            f'{err} at offset {leveldb_record.record.offset} in '
+            f'{leveldb_record.path}'),
+            file=sys.stderr)
+        print(f'Traceback: {traceback.format_exc()}', file=sys.stderr)
diff --git a/dfindexeddb/indexeddb/cli.py b/dfindexeddb/indexeddb/cli.py
index e6d1083..d0abdb9 100644
--- a/dfindexeddb/indexeddb/cli.py
+++ b/dfindexeddb/indexeddb/cli.py
@@ -19,16 +19,14 @@ import enum
 from datetime import datetime
 import json
 import pathlib
-import sys
-import traceback
 
-from dfindexeddb import errors
 from dfindexeddb import version
-from dfindexeddb.leveldb import record as leveldb_record
+from dfindexeddb.indexeddb.chromium import blink
 from dfindexeddb.indexeddb.chromium import record as chromium_record
 from dfindexeddb.indexeddb.chromium import v8
 from dfindexeddb.indexeddb.safari import record as safari_record
 
+
 _VALID_PRINTABLE_CHARACTERS = (
     ' abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789' +
     '!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~.')
@@ -73,84 +71,35 @@ def _Output(structure, output):
     print(structure)
 
 
+def BlinkCommand(args):
+  """The CLI for processing a file as a blink value."""
+  with open(args.source, 'rb') as fd:
+    buffer = fd.read()
+    blink_value = blink.V8ScriptValueDecoder.FromBytes(buffer)
+    _Output(blink_value, output=args.output)
+
+
 def DbCommand(args):
   """The CLI for processing a directory as IndexedDB."""
   if args.format in ('chrome', 'chromium'):
-    if args.use_manifest:
-      for db_record in leveldb_record.LevelDBRecord.FromManifest(args.source):
-        record = db_record.record
-        try:
-          idb_record = chromium_record.IndexedDBRecord.FromLevelDBRecord(
-              db_record)
-        except(
-            errors.ParserError,
-            errors.DecoderError,
-            NotImplementedError) as err:
-          print((
-              f'Error parsing Indexeddb record {record.__class__.__name__}: '
-              f'{err} at offset {record.offset} in {db_record.path}'),
-              file=sys.stderr)
-          print(f'Traceback: {traceback.format_exc()}', file=sys.stderr)
-          continue
-        _Output(idb_record, output=args.output)
-    else:
-      for db_record in leveldb_record.LevelDBRecord.FromDir(args.source):
-        record = db_record.record
-        try:
-          idb_record = chromium_record.IndexedDBRecord.FromLevelDBRecord(
-              db_record)
-        except(
-            errors.ParserError,
-            errors.DecoderError,
-            NotImplementedError) as err:
-          print((
-              f'Error parsing Indexeddb record {record.__class__.__name__}: '
-              f'{err} at offset {record.offset} in {db_record.path}'),
-              file=sys.stderr)
-          print(f'Traceback: {traceback.format_exc()}', file=sys.stderr)
-          continue
-        _Output(idb_record, output=args.output)
+    for db_record in chromium_record.FolderReader(
+        args.source).GetRecords(use_manifest=args.use_manifest):
+      _Output(db_record, output=args.output)
   elif args.format == 'safari':
-    for db_record in safari_record.Reader(args.source).Records():
+    for db_record in safari_record.FileReader(args.source).Records():
       _Output(db_record, output=args.output)
 
 
 def LdbCommand(args):
   """The CLI for processing a LevelDB table (.ldb) file as IndexedDB."""
-  for db_record in leveldb_record.LevelDBRecord.FromFile(args.source):
-    record = db_record.record
-    try:
-      idb_record = chromium_record.IndexedDBRecord.FromLevelDBRecord(
-          db_record)
-    except(
-        errors.ParserError,
-        errors.DecoderError,
-        NotImplementedError) as err:
-      print(
-          (f'Error parsing Indexeddb record {record.__class__.__name__}: {err} '
-           f'at offset {record.offset} in {db_record.path}'), file=sys.stderr)
-      print(f'Traceback: {traceback.format_exc()}', file=sys.stderr)
-      continue
-    _Output(idb_record, output=args.output)
+  for db_record in chromium_record.IndexedDBRecord.FromFile(args.source):
+    _Output(db_record, output=args.output)
 
 
 def LogCommand(args):
   """The CLI for processing a LevelDB log file as IndexedDB."""
-  for db_record in leveldb_record.LevelDBRecord.FromFile(args.source):
-    record = db_record.record
-    try:
-      idb_record = chromium_record.IndexedDBRecord.FromLevelDBRecord(
-          db_record)
-    except(
-        errors.ParserError,
-        errors.DecoderError,
-        NotImplementedError) as err:
-      print(
-          (f'Error parsing Indexeddb record {record.__class__.__name__}: {err} '
-           f'at offset {record.offset} in {db_record.path}'), file=sys.stderr)
-      print(f'Traceback: {traceback.format_exc()}', file=sys.stderr)
-      continue
-    _Output(idb_record, output=args.output)
+  for db_record in chromium_record.IndexedDBRecord.FromFile(args.source):
+    _Output(db_record, output=args.output)
 
 
 def App():
@@ -162,6 +111,25 @@ def App():
 
   subparsers = parser.add_subparsers()
 
+  parser_blink = subparsers.add_parser(
+      'blink', help='Parse a file as a blink value.')
+  parser_blink.add_argument(
+      '-s', '--source',
+      required=True,
+      type=pathlib.Path,
+      help=(
+        'The source file.'))
+  parser_blink.add_argument(
+      '-o',
+      '--output',
+      choices=[
+          'json',
+          'jsonl',
+          'repr'],
+      default='json',
+      help='Output format.  Default is json')
+  parser_blink.set_defaults(func=BlinkCommand)
+
   parser_db = subparsers.add_parser(
       'db', help='Parse a directory as IndexedDB.')
   parser_db.add_argument(
diff --git a/dfindexeddb/indexeddb/safari/record.py b/dfindexeddb/indexeddb/safari/record.py
index 379d2c8..1efee3b 100644
--- a/dfindexeddb/indexeddb/safari/record.py
+++ b/dfindexeddb/indexeddb/safari/record.py
@@ -62,7 +62,7 @@ class IndexedDBRecord:
   record_id: int
 
 
-class Reader:
+class FileReader:
   """A reader for Safari IndexedDB sqlite3 files.
 
   Attributes:
diff --git a/dfindexeddb/leveldb/cli.py b/dfindexeddb/leveldb/cli.py
index 894ceea..93382e8 100644
--- a/dfindexeddb/leveldb/cli.py
+++ b/dfindexeddb/leveldb/cli.py
@@ -66,12 +66,9 @@ def _Output(structure, output):
 
 def DbCommand(args):
   """The CLI for processing leveldb folders."""
-  if args.use_manifest:
-    for rec in record.LevelDBRecord.FromManifest(args.source):
-      _Output(rec, output=args.output)
-  else:
-    for rec in record.LevelDBRecord.FromDir(args.source):
-      _Output(rec, output=args.output)
+  for leveldb_record in record.FolderReader(
+      args.source).GetRecords(use_manifest=args.use_manifest):
+    _Output(leveldb_record, output=args.output)
 
 
 def LdbCommand(args):
@@ -257,8 +254,7 @@ def App():
       '-v',
       '--version_history',
       action='store_true',
-      help='Parses the leveldb version history.'
-  )
+      help='Parses the leveldb version history.')
   parser_descriptor.set_defaults(func=DescriptorCommand)
 
   args = parser.parse_args()
diff --git a/dfindexeddb/leveldb/record.py b/dfindexeddb/leveldb/record.py
index 9a5b9a3..769bde3 100644
--- a/dfindexeddb/leveldb/record.py
+++ b/dfindexeddb/leveldb/record.py
@@ -18,7 +18,7 @@ import dataclasses
 import pathlib
 import re
 import sys
-from typing import Any, Generator, Optional, Union
+from typing import Generator, Optional, Union
 
 from dfindexeddb import errors
 from dfindexeddb.leveldb import definitions
@@ -37,7 +37,8 @@ class LevelDBRecord:
   Attributes:
     path: the file path where the record was parsed from.
     record: the leveldb record.
-    level: the leveldb level, None indicates the record came from a log file.
+    level: the leveldb level, None indicates the record came from a log file or
+        a file not part of the active file set (determined by a MANIFEST file).
     recovered: True if the record is a recovered record.
   """
   path: str
@@ -51,7 +52,7 @@ class LevelDBRecord:
   def FromFile(
       cls,
       file_path: pathlib.Path
-  ) -> Generator[LevelDBRecord, Any, Any]:
+  ) -> Generator[LevelDBRecord, None, None]:
     """Yields leveldb records from the given path.
 
     Yields:
@@ -74,45 +75,47 @@ class LevelDBRecord:
     else:
       print(f'Unsupported file type {file_path.as_posix()}', file=sys.stderr)
 
-  @classmethod
-  def FromDir(
-      cls,
-      path: pathlib.Path
-  ) -> Generator[LevelDBRecord, Any, Any]:
-    """Yields LevelDBRecords from the given directory.
+
+class FolderReader:
+  """A LevelDB folder reader.
+
+  Attributes:
+    foldername (str): the source LevelDB folder.
+  """
+
+  def __init__(self, foldername: pathlib.Path):
+    """Initializes the FolderReader.
 
     Args:
-      path: the file path.
+      foldername: the source LevelDB folder.
 
-    Yields:
-      LevelDBRecords
+    Raises:
+      ValueError: if foldername is None or not a directory.
     """
-    if not path or not path.is_dir():
-      raise ValueError(f'{path} is not a directory')
-    for file_path in path.iterdir():
-      yield from cls.FromFile(file_path=file_path)
+    if not foldername or not foldername.is_dir():
+      raise ValueError(f'{foldername} is None or not a directory')
+    self.foldername = foldername
 
-  @classmethod
-  def FromManifest(
-      cls,
-      path: pathlib.Path
-  ) -> Generator[LevelDBRecord, Any, Any]:
-    """Yields LevelDBRecords from the given directory using the manifest.
+  def LogFiles(self) -> Generator[pathlib.Path, None, None]:
+    """Returns the log filenames."""
+    yield from self.foldername.glob('*.log')
 
-    Args:
-      path: the file path.
+  def LdbFiles(self) -> Generator[pathlib.Path, None, None]:
+    """Returns the ldb filenames."""
+    yield from self.foldername.glob('*.ldb')
 
-    Yields:
-      LevelDBRecords
+  def Manifest(self) -> Generator[pathlib.Path, None, None]:
+    """Returns the Manifest filenames."""
+    yield from self.foldername.glob('MANIFEST-*')
+
+  def GetCurrentManifestPath(self) -> pathlib.Path:
+    """Returns the path of the current manifest file.
 
     Raises:
-      ParserError: if the CURRENT or MANIFEST-* file does not exist.
-      ValueError: if path is not a directory.
+      ParserError: when the CURRENT file does not exist/contain the expected
+          content or when the expected MANIFEST file does not exist.
     """
-    if not path or not path.is_dir():
-      raise ValueError(f'{path} is not a directory')
-
-    current_path = path / 'CURRENT'
+    current_path = self.foldername / 'CURRENT'
     if not current_path.exists():
       raise errors.ParserError(f'{current_path!s} does not exist.')
 
@@ -122,34 +125,113 @@ class LevelDBRecord:
       raise errors.ParserError(
           f'{current_path!s} does not contain the expected content')
 
-    manifest_path = path / current_manifest
+    manifest_path = self.foldername / current_manifest
     if not manifest_path.exists():
       raise errors.ParserError(f'{manifest_path!s} does not exist.')
+    return manifest_path
 
+  def GetLatestVersion(self) -> descriptor.LevelDBVersion:
+    """Returns the latest LevelDBVersion.
+
+    Raises:
+      ParserError: when the leveldb version could not be parsed.
+    """
+    current_manifest_path = self.GetCurrentManifestPath()
     latest_version = descriptor.FileReader(
-        str(manifest_path)).GetLatestVersion()
+        str(current_manifest_path)).GetLatestVersion()
     if not latest_version:
       raise errors.ParserError(
-          f'Could not parse a leveldb version from {manifest_path!s}')
+          f'Could not parse a leveldb version from {current_manifest_path!s}')
+    return latest_version
+
+  def _GetRecordsByFile(
+      self, filename: pathlib.Path) -> Generator[LevelDBRecord, None, None]:
+    """Yields the LevelDBRecords from a file.
+
+    Non-log/ldb files are ignored.
+
+    Args:
+      filename: the source LevelDB file.
+
+    Yields:
+      LevelDBRecords
+    """
+    if filename.name.endswith('.log'):
+      yield from self._GetLogRecords(filename)
+    elif filename.name.endswith('.ldb'):
+      yield from self._GetLdbRecords(filename)
+    elif filename.name.startswith('MANIFEST'):
+      print(f'Ignoring descriptor file {filename.as_posix()}', file=sys.stderr)
+    elif filename.name in ('LOCK', 'CURRENT', 'LOG', 'LOG.old'):
+      print(f'Ignoring {filename.as_posix()}', file=sys.stderr)
+    else:
+      print(f'Unsupported file type {filename.as_posix()}', file=sys.stderr)
+
+  def _GetLogRecords(
+      self,
+      filename: pathlib.Path
+  ) -> Generator[LevelDBRecord, None, None]:
+    """Yields the LevelDBRecords from a log file.
 
-    # read log records
+    Args:
+      filename: the source LevelDB file.
+
+    Yields:
+      LevelDBRecords
+    """
+    for record in log.FileReader(filename.as_posix()).GetParsedInternalKeys():
+      yield LevelDBRecord(path=filename.as_posix(), record=record)
+
+  def _GetLdbRecords(
+      self,
+      filename: pathlib.Path
+  ) -> Generator[LevelDBRecord, None, None]:
+    """Yields the LevelDBRecords from a log file.
+
+    Args:
+      filename: the source LevelDB file.
+
+    Yields:
+      LevelDBRecords
+    """
+    for record in ldb.FileReader(filename.as_posix()).GetKeyValueRecords():
+      yield LevelDBRecord(path=filename.as_posix(), record=record)
+
+  def _RecordsByManifest(self) -> Generator[LevelDBRecord, None, None]:
+    """Yields LevelDBRecords using active files determined by the MANIFEST file.
+
+    Yields:
+      LevelDBRecords.
+    """
+    latest_version = self.GetLatestVersion()
+
+    processed_files = set()
+
+    # read and cache the log records
     log_records = []
     if latest_version.current_log:
-      current_log = path / latest_version.current_log
-      if current_log.exists():
-        for log_record in cls.FromFile(file_path=current_log):
-          log_records.append(log_record)
+      current_log_filename = self.foldername / latest_version.current_log
+      if current_log_filename.exists():
+        log_records = list(self._GetLogRecords(filename=current_log_filename))
+        processed_files.add(current_log_filename)
     else:
       print('No current log file.', file=sys.stderr)
 
-    # read records from the "young" or 0-level
+    # read and cache the records from the "young" or 0-level
     young_records = []
     for active_file in latest_version.active_files.get(0, {}).keys():
-      current_young = path / active_file
-      if current_young.exists():
-        for young_record in cls.FromFile(current_young):
-          young_records.append(young_record)
+      current_young_filename = self.foldername / active_file
+      if current_young_filename.exists():
+        young_records = list(self._GetLdbRecords(current_young_filename))
+        processed_files.add(current_young_filename)
+      else:
+        print(
+            f'Could not find {current_young_filename} for level 0.',
+            file=sys.stderr)
 
+    # sort the log records by the leveldb sequence number in reverse
+    # order and update the recovered attribute based on the highest sequence
+    # number for a key.
     active_records = {}
     for record in sorted(
         log_records,
@@ -161,6 +243,10 @@ class LevelDBRecord:
       else:
         record.recovered = True
 
+    # sort the young records by the leveldb sequence number in reverse
+    # order and update:
+    # * the recovered attribute based on the highest sequence number for a key
+    # * the level attribute to 0
     for record in sorted(
         young_records,
         key=lambda record: record.record.sequence_number,
@@ -177,14 +263,55 @@ class LevelDBRecord:
         key=lambda record: record.record.sequence_number,
         reverse=False)
 
+    # read records from the active files in each level (except the 0 level)
+    # and update the recovered and level attribute.
     if latest_version.active_files.keys():
       for level in range(1, max(latest_version.active_files.keys()) + 1):
         for filename in latest_version.active_files.get(level, []):
-          current_filename = path / filename
-          for record in cls.FromFile(file_path=current_filename):
-            if record.record.key in active_records:
-              record.recovered = True
-            else:
-              record.recovered = False
-            record.level = level
-            yield record
+          current_filename = self.foldername / filename
+          if current_filename.exists():
+            processed_files.add(current_filename)
+            for record in self._GetLdbRecords(filename=current_filename):
+              record.recovered = record.record.key in active_records
+              record.level = level
+              yield record
+          else:
+            print(
+                f'Could not find {current_filename} for level {level}.',
+                file=sys.stderr)
+
+    # as a final step, parse any other log/ldb files which we will consider
+    # any records as recovered since they are not listed in the the active file
+    # set.
+    for log_file in self.LogFiles():
+      if log_file in processed_files:
+        continue
+      for record in self._GetLogRecords(filename=log_file):
+        record.recovered = True
+        yield record
+
+    for ldb_file in self.LdbFiles():
+      if ldb_file in processed_files:
+        continue
+      for record in self._GetLdbRecords(filename=ldb_file):
+        record.recovered = True
+        yield record
+
+  def GetRecords(
+      self,
+      use_manifest: bool = False
+  ) -> Generator[LevelDBRecord, None, None]:
+    """Yield LevelDBRecords.
+
+    Args:
+      use_manifest: True to use the current manifest in the folder as a means to
+          find the active file set.
+
+    Yields:
+      LevelDBRecords.
+    """
+    if use_manifest:
+      yield from self._RecordsByManifest()
+    else:
+      for filename in self.foldername.iterdir():
+        yield from LevelDBRecord.FromFile(filename)
diff --git a/dfindexeddb/version.py b/dfindexeddb/version.py
index 7cba406..438eb44 100644
--- a/dfindexeddb/version.py
+++ b/dfindexeddb/version.py
@@ -15,7 +15,7 @@
 """Version information for dfIndexeddb."""
 
 
-__version__ = "20240417"
+__version__ = "20240501"
 
 
 def GetVersion():
diff --git a/pyproject.toml b/pyproject.toml
index 16ab1fd..2978aa5 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "dfindexeddb"
-version = "20240417"
+version = "20240501"
 requires-python = ">=3.8"
 description = "dfindexeddb is an experimental Python tool for performing digital forensic analysis of IndexedDB and leveldb files."
 license = {file = "LICENSE"}


Test patch:
diff --git a/tests/dfindexeddb/indexeddb/safari/record.py b/tests/dfindexeddb/indexeddb/safari/record.py
index 1a6b6e3..2a780d9 100644
--- a/tests/dfindexeddb/indexeddb/safari/record.py
+++ b/tests/dfindexeddb/indexeddb/safari/record.py
@@ -25,7 +25,7 @@ class SafariIndexedDBTest(unittest.TestCase):
   """Unit tests for Safari IndexedDB encoded sqlite3 databases."""
 
   def setUp(self):
-    self.db = record.Reader(
+    self.db = record.FileReader(
         './test_data/indexeddb/safari/17.3.1/IndexedDB.sqlite3')
 
   def test_nonexistent_record(self):