Repository: oppia/oppia

Issue:
[BUG]: Topics and Skills dashboard not working on test server.
### Describe the bug

On trying to access the topics and skills dashboard on the test server and encountered some issues.
Because of these issues there was no data rendering in the dashboard.


This behaviour is observed only on test server and not on prod.

### URL of the page where the issue is observed.

https://www.oppiatestserver.org/topics-and-skills-dashboard

### Steps To Reproduce

Visit 
https://www.oppiatestserver.org/topics-and-skills-dashboard

### Expected Behavior

There shouldn't be any console errors and the dashboard should show all the data correctly.

### Screenshots/Videos

![Screenshot from 2023-10-10 11-02-12](https://github.com/oppia/oppia/assets/91470808/0e1b7262-d6b3-49a1-ac73-34cefee4549d)


### What device are you using?

Desktop

### Operating System

Linux

### What browsers are you seeing the problem on?

Chrome

### Browser version

_No response_

### Additional context

Backend stacktrace

Traceback (most recent call last):
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/webapp2.py", line 604, in dispatch
    return method(*args, **kwargs)
  File "/workspace/core/controllers/acl_decorators.py", line 3610, in test_can_access_topics_and_skills_dashboard
    return handler(self, **kwargs)
  File "/workspace/core/controllers/topics_and_skills_dashboard.py", line 115, in get
    topic_services.get_chapter_counts_in_topic_summaries(
  File "/workspace/core/domain/topic_services.py", line 1733, in get_chapter_counts_in_topic_summaries
    stories = [story_id_mapping[story_reference.story_id] for
  File "/workspace/core/domain/topic_services.py", line 1733, in <listcomp>
    stories = [story_id_mapping[story_reference.story_id] for
KeyError: 'sprepA0Id6oc' 



### Tips for developers

Before tackling the bug, please use `git bisect` (see https://git-scm.com/docs/git-bisect) to investigate which PR caused it (you only need to go back as far as commit https://github.com/oppia/oppia/commits/9a334e9). If you find the PR, leave a comment on this issue pointing to it, or just say that it happened before commit 9a334e9 if you could reproduce it there. This will help us fix the issue by reverting the faulty PR.

Also, if this is your first issue, please make sure to follow https://github.com/oppia/oppia/wiki/Contributing-code-to-Oppia#choosing-a-good-first-issue and https://github.com/oppia/oppia/wiki/Contributing-code-to-Oppia#setting-things-up before claiming it. Thanks!



Discussion:
can you assign me this issue ?
@kevintab95 -- assigning this per discussion with Kshitij in email, please reassign if needed. Thanks!
It looks like this issue is occurring because of corrupted data on the test server. Per looking at the logs, this line throws a KeyError because the story_id is invalid -- the story it refers to does not exist.

https://github.com/oppia/oppia/blob/0e13814a67ae59d52acc31a8694abcedde8937fa/core/domain/topic_services.py#L1733-L1734

There is no way to fix this via the UI currently. An admin utility or a beam job might be required to fix this for the affected topic.

I would suggest updating the code such that this scenario logs a backend error but the dashboard is not completely broken as a result of that.
Hi @prafulbbandre, any updates?

Just a suggestion, I think doing this might be sufficient for now -- given that this is blocking the release.

Change this:
```
        stories = [story_id_mapping[story_reference.story_id] for
            story_reference in topic.canonical_story_references]
```

To this:
```
        stories = []
        for story_reference in topic.canonical_story_references:
            if story_reference.story_id in story_id_mapping:
                stories.append(story_id_mapping[story_reference.story_id])
```

But I would suggest creating a new task to clean up topics on the test server with invalid story references.

Resolving patch:
diff --git a/core/domain/topic_services.py b/core/domain/topic_services.py
--- a/core/domain/topic_services.py
+++ b/core/domain/topic_services.py
@@ -1730,8 +1730,14 @@ def get_chapter_counts_in_topic_summaries(
         overdue_chapters_count = 0
         total_chapter_counts = []
         published_chapter_counts = []
-        stories = [story_id_mapping[story_reference.story_id] for
-            story_reference in topic.canonical_story_references]
+        stories = []
+        for story_reference in topic.canonical_story_references:
+            if story_reference.story_id in story_id_mapping:
+                stories.append(story_id_mapping[story_reference.story_id])
+            else:
+                logging.error(
+                    'Topic %s has an invalid story reference ID %s' % (
+                        topic.id, story_reference.story_id))
         for story in stories:
             nodes = story.story_contents.nodes
             total_chapters_count = len(nodes)


Test patch:
diff --git a/core/domain/topic_services_test.py b/core/domain/topic_services_test.py
--- a/core/domain/topic_services_test.py
+++ b/core/domain/topic_services_test.py
@@ -18,6 +18,7 @@
 
 from __future__ import annotations
 
+import logging
 import os
 
 from core import feconf
@@ -2376,6 +2377,69 @@ def mock_get_current_time_in_millisecs() -> int:
                 chapter_counts[self.TOPIC_ID].
                     published_chapter_counts_for_each_story, [0, 0, 1])
 
+    def test_get_chapter_counts_in_topic_summaries_logs_error(self) -> None:
+        topic_model = topic_models.TopicModel.get_by_id(self.TOPIC_ID)
+        invalid_story_reference_dict = {
+            'story_id': 'invalid_story_id',
+            'story_is_published': False
+        }
+        topic_model.canonical_story_references = [invalid_story_reference_dict]
+        topic_model.commit(self.user_id, 'reference invalid story id', [])
+
+        def mock_get_current_time_in_millisecs() -> int:
+            return 1690555400000
+
+        with self.swap(
+            utils, 'get_current_time_in_millisecs',
+            mock_get_current_time_in_millisecs):
+            with self.capture_logging(min_level=logging.ERROR) as logs:
+                topic_summary = (
+                    topic_fetchers.get_topic_summary_by_id(
+                    self.TOPIC_ID).to_dict())
+                frontend_topic_summary: (
+                    topic_domain.FrontendTopicSummaryDict
+                ) = {
+                    'id': topic_summary['id'],
+                    'name': topic_summary['name'],
+                    'url_fragment': topic_summary['url_fragment'],
+                    'language_code': topic_summary['language_code'],
+                    'description': topic_summary['description'],
+                    'version': topic_summary['version'],
+                    'canonical_story_count': topic_summary[
+                        'canonical_story_count'],
+                    'additional_story_count': topic_summary[
+                        'canonical_story_count'],
+                    'uncategorized_skill_count': topic_summary[
+                        'additional_story_count'],
+                    'subtopic_count': topic_summary['subtopic_count'],
+                    'total_skill_count': topic_summary['total_skill_count'],
+                    'total_published_node_count': topic_summary[
+                        'total_published_node_count'],
+                    'thumbnail_filename': topic_summary['thumbnail_filename'],
+                    'thumbnail_bg_color': topic_summary['thumbnail_bg_color'],
+                    'topic_model_created_on': topic_summary[
+                        'topic_model_created_on'],
+                    'topic_model_last_updated': topic_summary[
+                        'topic_model_last_updated'],
+                    'is_published': True,
+                    'can_edit_topic': True,
+                    'classroom': None,
+                    'total_upcoming_chapters_count': 0,
+                    'total_overdue_chapters_count': 0,
+                    'total_chapter_counts_for_each_story': [],
+                    'published_chapter_counts_for_each_story': []
+                }
+                topic_services.get_chapter_counts_in_topic_summaries(
+                    [frontend_topic_summary]
+                )
+                self.assertEqual(
+                    logs,
+                    [
+                        'Topic %s has an invalid story reference ID '
+                        'invalid_story_id' % self.TOPIC_ID
+                    ]
+                )
+
 
 # TODO(#7009): Remove this mock class and the SubtopicMigrationTests class
 # once the actual functions for subtopic migrations are implemented.