Repository: KSP-CKAN/NetKAN-Infra

Issue:
[Bug] Bot only crawling about 30 mods at the moment
## Problem

1. Go to http://status.ksp-ckan.space/
2. Sort by Last Checked
3. Scroll down to where the times jump from short to long; currently it's here:
  ![image](https://user-images.githubusercontent.com/1559108/67504543-309dad80-f64f-11e9-99f1-ae192ea2275e.png)

The mods with recent times only number about 30. The rest aren't being checked.
Are the remaining mods not being scheduled? Is the Inflator stopping mid-pass?

### Clues

- Started happening after #48 was merged
- The inflation webhooks are working
- Inflator hasn't been changed
- Indexer hasn't been changed
- Scheduler had a parameter and a local variable renamed, which should have had no effect, but who knows
- The most recently checked mod is pretty consistently `ContractConfigurator-KFiles`

Discussion:

Patch:
diff --git a/netkan/netkan/metadata.py b/netkan/netkan/metadata.py
index 0f8a6ca..f598a46 100644
--- a/netkan/netkan/metadata.py
+++ b/netkan/netkan/metadata.py
@@ -16,8 +16,11 @@ class Netkan:
             self.contents = contents
         self._raw = json.loads(self.contents)
         # Extract kref_src + kref_id from the kref
-        self.kref_src, self.kref_id = self.KREF_PATTERN.match(
-            self.kref).groups()
+        if self.has_kref:
+            self.kref_src, self.kref_id = self.KREF_PATTERN.match(self.kref).groups()
+        else:
+            self.kref_src = None
+            self.kref_id = None
 
     def __getattr__(self, name):
         # Return kref host, ie `self.on_spacedock`. Current krefs include
diff --git a/netkan/netkan/scheduler.py b/netkan/netkan/scheduler.py
index 3269241..44860ab 100644
--- a/netkan/netkan/scheduler.py
+++ b/netkan/netkan/scheduler.py
@@ -22,7 +22,7 @@ class NetkanScheduler:
 
     def netkans(self):
         # This can easily be recursive with '**/*.netkan', however
-        # implemeneting like for like initially.
+        # implementing like for like initially.
         return (Netkan(f) for f in self.path.glob('*.netkan'))
 
     def sqs_batch_entries(self, batch_size=10):


Test patch:
diff --git a/netkan/tests/metadata.py b/netkan/tests/metadata.py
index b0042c6..45fc597 100644
--- a/netkan/tests/metadata.py
+++ b/netkan/tests/metadata.py
@@ -86,6 +86,27 @@ class TestNetKANNetkan(TestNetKAN):
         self.assertFalse(self.netkan.hook_only())
 
 
+class TestNetKANNoKref(TestNetKAN):
+
+    def setUp(self):
+        self.netkan = Netkan(contents="""{
+            "spec_version": "v1.4",
+            "identifier":   "LightsOut-Fwiffo",
+            "license":      "MIT",
+            "version":      "v0.1.5.1",
+            "download":     "https://cdn.rawgit.com/rkagerer/KSP-Fwiffo-Repository/master/Mods/LightsOut-Fwiffo-v0.1.5.1.zip"
+        }""")
+
+    def test_kref(self):
+        self.assertFalse(self.netkan.has_kref)
+
+    def test_kref_src(self):
+        self.assertEqual(self.netkan.kref_src, None)
+
+    def test_kref_id(self):
+        self.assertEqual(self.netkan.kref_id, None)
+
+
 class TestNetKANVref(TestNetKAN):
 
     def setUp(self):
@@ -101,7 +122,7 @@ class TestNetKANVref(TestNetKAN):
 class TestCkanSimple(unittest.TestCase):
 
     def setUp(self):
-        self.ckan = Ckan(contents = """{
+        self.ckan = Ckan(contents="""{
             "spec_version": "v1.4",
             "identifier":   "AwesomeMod",
             "version":      "1.0.0",
@@ -135,7 +156,7 @@ class TestCkanSimple(unittest.TestCase):
 class TestCkanComplex(unittest.TestCase):
 
     def setUp(self):
-        self.ckan = Ckan(contents = """{
+        self.ckan = Ckan(contents="""{
             "spec_version": "v1.4",
             "identifier":   "AwesomeMod",
             "version":      "1.0.0",