Repository: learningequality/kolibri

Issue:
Content download request progress tracking
Depends on: https://github.com/learningequality/kolibri/pull/10365

## Summary
With https://github.com/learningequality/kolibri/pull/10365, we implemented the automatic import of content assigned in lessons and exams. The automatic import also processes user-initiated downloads in addition to sync-initiated downloads. For user-initiated downloads, the user can see the state the current state of the download request. It would be helpful to surface the actual progress percentage of the download request, because users with low-bandwidth connections, or downloads of large files, could take a while.

## Deliverables
### The mixin class `JobProgressMixin`
- It should be refactored such that the mixin, and its existing usage with the resource import classes, can be used to track progress elsewhere than a job

### The function [`process_download_request`](https://github.com/learningequality/kolibri/blob/c588e4b5d73ae0342d3ff56dd2f4f2bf400e698b/kolibri/core/content/utils/content_request.py#L405)
- It should leverage the above updates to the `JobProgressMixin`, inherited by `RemoteChannelResourceImportManager`, to attach a method that will update the `ContentDownloadRequest` with the progress of the download
- The progress should either be tracked in a new `ContentRequest` field or the `metadata` field

### The component `MyDownloads`
- It should be updated to render a determinate loader with the percentage progress of the download

## References
https://learningequality.slack.com/archives/CB37UM23A/p1679665808045409

### Setting up manual testing scenario
Be sure to use separate Kolibri home directories for each instance by using the environment variable `KOLIBRI_HOME`

#### For Instance A
1) Provision a new full-facility Kolibri instance through the setup wizard: Group learning > Full device > Create new learning facility. Note
2) Complete the setup
2) Navigate to Device > Channels
3) Import some content, which will be used for creating a lesson
2) Navigate to Facility > Users
2) Add a new Learner, remember their password
3) Navigate to Coach > Plan
4) Create a new Lesson and assign it to the Learner
5) Add some resources to the lesson
6) Ensure the lesson is visible to the learner / assignees

#### For Instance B
1) Provision a new learn-only Kolibri instance, running on a different port than Instance A: Group learning > Learn-only device > Create new learning facility > Import one or more existing user accounts from an existing facility
2) Select Instance A from the 'Select network address' modal
3) Input the Learner credentials you created on Instance A
4) Complete the setup
5) The initial import of the learner should trigger the automatic content import of assigned materials in the lesson

#### Further testing
To continue testing the automatic content import you may:
- Continue creating new lessons or quizzes, or marking them as invisible for the learners for removal testing, in Instance A and let them auto sync to Instance B while they're both running, or
- Delete the Kolibri home for Instance B and reprovision it allowing the initial import to be tested


Discussion:
Hi @bjester, can I help with this task?
@Jaspreet-singh-1032 Yes, I followed up in our slack channel. You may skip the updates to the component `MyDownloads`, since that frontend component likely won't be updated yet to actually receive this information from the backend
Hello @bjester, can you please tell me how can I trigger and test this process?
@Jaspreet-singh-1032 I added some manually testing instructions to the issue
Thanks!
Hi @bjester, I could not get time to work on it last week. But now I have resumed work on this one. I have set up two instances of Kolibri locally for admin and learner. And will let you know about the further implementation.
Hi @Jaspreet-singh-1032, could you share a progress update?
Yes @bjester, can we pass the `download_request` from [here](https://github.com/learningequality/kolibri/blob/develop/kolibri/core/content/utils/content_request.py#L481) like this

![image](https://github.com/learningequality/kolibri/assets/69707565/54a6619f-01c3-4c11-b395-761481a7a868)

And assign `download_request` to the object instance and then reuse the current methods of `JobProgressMixin` just by adding 
 a condition to check if the object has the attribute `download_request` and then update its progress.
Something like this

![image](https://github.com/learningequality/kolibri/assets/69707565/4d8ffc3d-e9dc-4bc2-b6e7-b9544e1c5b7f)

Let me know what do you think about this.
@Jaspreet-singh-1032 Yes that should work fine. Since this is a very specific use case, I think it's generally wise to think about how this could be integrated without directly coupling the two, but the changes are few so we could revisit it another time.

It could make sense to add a `@classmethod` that acts as a factory and builds the `RemoteChannelResourceImportManager` instance from the `ContentDownloadRequest`, similar to `.from_manifest`
Thanks. Sure, I will add a classmethod in `ContentDownloadRequest` to build `RemoteChannelResourceImportManager` and I think I will also have to add two new fields in `ContentDownloadRequest` model for saving the `progress` and the `total_progress`.
What is `total_progress` used for? Since a download request maps to a content node, I would think `total_progress` would be simply 100%
I thought of adding it because this is how currently `JobProgressMixin` is tracking progress. What I think is `total_progress` is the number of files and `progress` is currently processed data and the percentage can be calculated using both of them. Let me know if it is not like this.
I think having a new `total_progress` field stored in the database is unnecessary
Let me see if we can make it work only with one progress field.
@Jaspreet-singh-1032 When do you expect a PR to be opened?
Actually, I was currently working on it. I know it gets late but I will be opening a PR in the upcoming days maybe before this weekend.
Thank you @Jaspreet-singh-1032 

Resolving patch:
diff --git a/kolibri/core/content/models.py b/kolibri/core/content/models.py
--- a/kolibri/core/content/models.py
+++ b/kolibri/core/content/models.py
@@ -466,6 +466,20 @@ def build_for_user(cls, user):
             status=ContentRequestStatus.Pending,
         )
 
+    def update_progress(self, progress, total_progress):
+        self.metadata = self.metadata or {}
+        self.metadata["progress"] = progress
+        self.metadata["total_progress"] = total_progress
+        self.save()
+
+    @property
+    def progress(self):
+        return self.metadata.get("progress", 0) if self.metadata else 0
+
+    @property
+    def total_progress(self):
+        return self.metadata.get("total_progress", 0) if self.metadata else 0
+
 
 class ContentRequestManager(models.Manager):
     request_type = None
diff --git a/kolibri/core/content/utils/content_request.py b/kolibri/core/content/utils/content_request.py
--- a/kolibri/core/content/utils/content_request.py
+++ b/kolibri/core/content/utils/content_request.py
@@ -21,7 +21,7 @@
 from kolibri.core.content.utils.assignment import DeletedAssignment
 from kolibri.core.content.utils.channel_import import import_channel_from_data
 from kolibri.core.content.utils.resource_import import (
-    RemoteChannelResourceImportManager,
+    ContentDownloadRequestResourceImportManager,
 )
 from kolibri.core.device.models import DeviceStatus
 from kolibri.core.device.models import LearnerDeviceStatus
@@ -474,11 +474,12 @@ def process_download_request(download_request):
         channel_id = ContentNode.objects.get(
             pk=download_request.contentnode_id
         ).channel_id
-        import_manager = RemoteChannelResourceImportManager(
+        import_manager = ContentDownloadRequestResourceImportManager(
             channel_id,
             peer_id=peer.id,
             baseurl=peer.base_url,
             node_ids=[download_request.contentnode_id],
+            download_request=download_request,
         )
         import_manager.run()
 
diff --git a/kolibri/core/content/utils/resource_import.py b/kolibri/core/content/utils/resource_import.py
--- a/kolibri/core/content/utils/resource_import.py
+++ b/kolibri/core/content/utils/resource_import.py
@@ -518,3 +518,46 @@ def get_import_data(self):
             renderable_only=self.renderable_only,
             drive_id=self.drive_id,
         )
+
+
+class ContentDownloadRequestResourceImportManager(RemoteChannelResourceImportManager):
+    def __init__(
+        self,
+        channel_id,
+        peer_id=None,
+        baseurl=None,
+        node_ids=None,
+        exclude_node_ids=None,
+        renderable_only=True,
+        fail_on_error=False,
+        content_dir=None,
+        download_request=None,
+        timeout=transfer.Transfer.DEFAULT_TIMEOUT,
+    ):
+        super(ContentDownloadRequestResourceImportManager, self).__init__(
+            channel_id,
+            peer_id=peer_id,
+            baseurl=baseurl,
+            node_ids=node_ids,
+            exclude_node_ids=exclude_node_ids,
+            renderable_only=renderable_only,
+            fail_on_error=fail_on_error,
+            content_dir=content_dir,
+            timeout=timeout,
+        )
+        self.download_request = download_request
+
+    def start_progress(self, total=100):
+        super(ContentDownloadRequestResourceImportManager, self).start_progress(total)
+        if self.download_request:
+            self.download_request.update_progress(0, total)
+
+    def update_progress(self, increment=1, message="", extra_data=None):
+        super(ContentDownloadRequestResourceImportManager, self).update_progress(
+            increment, message, extra_data
+        )
+        if self.download_request:
+            self.download_request.update_progress(
+                self.download_request.progress + increment,
+                self.download_request.total_progress,
+            )


Test patch: