Repository: cobbler/cobbler

Issue:
Deleting a system (or any object) may not persist across cobbler restarts
Problem is seen in 

```
Cobbler 3.1.2
  source: ?, ?
  build time: Sun May 31 02:32:34 2020
```

This bug was actually fixed via commit 63d8312bac0bc4a1567d7e511beddfa89f4e1562 in December 2020.  File modules/serializers/file.py functions serialize_item() and serialize_delete() were appending the string ".json" to the filename twice.    Rather than trying to delete file "object.json", serialize_delete() would try to delete "object.json.json" which didn't exist, and so the code did nothing.  After a cobbler restart, the "object.json" file would be seen and the object re-appeared in cobbler.   In this case the object was created before the last cobbler upgrade, so it has the "object.json" filename format, it didn't have the ".json.json" suffix.  Objects created after the last cobbler upgrade all have the ".json.json" suffix, there is no "object.json" file.

The duplication was removed, but this leaves existing sites with a mess in their collections subdirectories.  Depending upon when they created objects relative to when they last upgraded cobbler, they can have files with name "object.json", "object.json.json", or both files can exist simultaneously.

Seems like there should be some code added to detect this situation, and rename the "object.json.json" files back to their appropriate "object.json" names.



Discussion:
Okay I agree that we need code to fix that automatically. Or at least document this more extensively.
Related PR is https://github.com/cobbler/cobbler/pull/2451
@krehm If you update to 3.2.0, does that error persist? Or is that something you can't do?

Resolving patch:
diff --git a/cobbler/modules/serializers/file.py b/cobbler/modules/serializers/file.py
--- a/cobbler/modules/serializers/file.py
+++ b/cobbler/modules/serializers/file.py
@@ -33,20 +33,35 @@
 libpath = "/var/lib/cobbler/collections"
 
 
-def register():
+def register() -> str:
     """
     The mandatory Cobbler module registration hook.
     """
     return "serializer"
 
 
-def what():
+def what() -> str:
     """
     Module identification function
     """
     return "serializer/file"
 
 
+def __find_double_json_files(filename: str):
+    """
+    Finds a file with duplicate .json ending and renames it.
+    :param filename: Filename to be checked
+    :raises FileExistsError: If both JSON files exist
+    """
+
+    if not os.path.isfile(filename):
+        if os.path.isfile(filename + ".json"):
+            os.rename(filename + ".json", filename)
+    else:
+        if os.path.isfile(filename + ".json"):
+            raise FileExistsError("Both JSON files (%s) exist!" % filename)
+
+
 def serialize_item(collection, item):
     """
     Save a collection item to file system
@@ -60,6 +75,7 @@ def serialize_item(collection, item):
 
     collection_types = collection.collection_types()
     filename = os.path.join(libpath, collection_types, item.name + ".json")
+    __find_double_json_files(filename)
 
     _dict = item.to_dict()
 
@@ -86,6 +102,7 @@ def serialize_delete(collection, item):
 
     collection_types = collection.collection_types()
     filename = os.path.join(libpath, collection_types, item.name + ".json")
+    __find_double_json_files(filename)
 
     if os.path.exists(filename):
         os.remove(filename)
@@ -128,26 +145,6 @@ def deserialize_raw(collection_types):
         return results
 
 
-def filter_upgrade_duplicates(file_list):
-    """
-    In a set of files, some ending with .json, some not, return the list of files with the .json ones taking priority
-    over the ones that are not.
-
-    :param file_list: The list of files to remove duplicates from.
-    :return: The filtered list of files. Normally this should only return ``.json``-Files.
-    """
-    bases = {}
-    for f in file_list:
-        basekey = f.replace(".json", "")
-        if f.endswith(".json"):
-            bases[basekey] = f
-        else:
-            lookup = bases.get(basekey, "")
-            if not lookup.endswith(".json"):
-                bases[basekey] = f
-    return list(bases.values())
-
-
 def deserialize(collection, topological=True):
     """
     Load a collection from file system.


Test patch: