Repository: AlexsLemonade/refinebio

Issue:
Find a way to assign volume IDs to instances without EBS
### Context

https://github.com/AlexsLemonade/refinebio/issues/2254

### Problem or idea

We stopped creating EBS volumes now that we do all of the processing on the SSDs, but now we can only run one instance at once because instances are always assigned volume index 0.

### Solution or next step

Brainstorm ways to assign volume indices to instances. Maybe we can query nomad for active instances?


Discussion:
Querying nomad might create race conditions if multiple instances are starting up simultaneously.

What if we [tag](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html#Using_Tags_CLI) each instance with an index and then have them query for it in their startup scripts?
Can the spot request tag instances with a unique index?
Ohhh right.... hmm, there might be more elegant solutions, but what if we just.... made 1GB EBS volumes for it?

Resolving patch:
diff --git a/common/data_refinery_common/migrations/0059_auto_20200803_1719.py b/common/data_refinery_common/migrations/0059_auto_20200803_1719.py
new file mode 100644
--- /dev/null
+++ b/common/data_refinery_common/migrations/0059_auto_20200803_1719.py
@@ -0,0 +1,23 @@
+# Generated by Django 2.2.13 on 2020-08-03 17:19
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("data_refinery_common", "0058_auto_20200625_1402"),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name="downloaderjob",
+            name="volume_index",
+            field=models.CharField(max_length=25, null=True),
+        ),
+        migrations.AlterField(
+            model_name="processorjob",
+            name="volume_index",
+            field=models.CharField(max_length=25, null=True),
+        ),
+    ]
diff --git a/common/data_refinery_common/models/jobs/downloader_job.py b/common/data_refinery_common/models/jobs/downloader_job.py
--- a/common/data_refinery_common/models/jobs/downloader_job.py
+++ b/common/data_refinery_common/models/jobs/downloader_job.py
@@ -54,7 +54,11 @@ class Meta:
 
     # Resources
     ram_amount = models.IntegerField(default=1024)
-    volume_index = models.CharField(max_length=3, null=True)
+
+    # The volume index is the instance id of an AWS EC2 machine. It looks like
+    # these are 19 characters, but just to be safe we'll make the max length a
+    # bit higher
+    volume_index = models.CharField(max_length=25, null=True)
 
     # This field represents how many times this job has been
     # retried. It starts at 0 and each time the job has to be retried
diff --git a/common/data_refinery_common/models/jobs/processor_job.py b/common/data_refinery_common/models/jobs/processor_job.py
--- a/common/data_refinery_common/models/jobs/processor_job.py
+++ b/common/data_refinery_common/models/jobs/processor_job.py
@@ -52,7 +52,11 @@ class Meta:
 
     # Resources
     ram_amount = models.IntegerField(default=2048)
-    volume_index = models.CharField(max_length=3, null=True)
+
+    # The volume index is the instance id of an AWS EC2 machine. It looks like
+    # these are 19 characters, but just to be safe we'll make the max length a
+    # bit higher
+    volume_index = models.CharField(max_length=25, null=True)
 
     # Tracking
     start_time = models.DateTimeField(null=True)


Test patch: