Repository: wordpress-mobile/WordPress-Android (language: kotlin)

Issue:
Excessive Media Requests Delay Critical Post Operations
### Expected behavior
- The app only generates one request to the media endpoint for each featured image of the posts to be fetched.
- Post management operations aren't locked due to fetching featured images of posts.

### Actual behavior
When opening the app for the first time and navigating a site with a long list of posts with featured images, the app generates significantly more requests to the media endpoint than the number of posts. For instance, 40 posts with featured images resulted in 177 requests. This issue is exacerbated by slow network connections, leading to even more requests.

During this period, critical post operations like saving, publishing, and deleting are locked until all the featured image requests are completed. This results in a situation where post management is blocked for several minutes, severely impacting the user experience.

### Steps to reproduce the behavior

**Preparation:** 
- Create multiple posts (20 or more) and attach a featured image. This can be done quickly by using the [WPCOM console](https://developer.wordpress.com/docs/api/console/) and the endpoint rest/v1.1/sites/${siteId}/posts/new.

#### Extra requests to the media endpoint
1. On the device/emulator, clear storage of the app, or delete and reinstall the app.
2. Open the app and navigate to the site.
3. Connect the device/emulator to the Charles proxy.
4. Open the app and go to the Post screen.
5. Observe all extra requests to the media endpoint in the Charles proxy.

#### Post manage operations locked
1. On the device/emulator, clear storage of the app, or delete and reinstall the app.
2. Open the app and navigate to the site.
3. Connect the device/emulator to the Charles proxy.
4. Slow down the network connection. This can be achieved by enabling throttling in the Charles proxy.
5. Open the app and go to the Post screen.
6. Edit a post and save it.
7. Observe that the saving operation takes a significant delay.

##### Tested on Samsung Galaxy S20 FE 5G, Android 13, JPAndroid / WPAndroid 25.0-rc-1



Discussion hints:
Thanks for reporting! 👍 <!-- generated_by_dangermattic -->

After investigating the issue, I noticed the following:

- The request to the media endpoint is related to the featured image of posts shown in the post list ([reference](https://github.com/wordpress-mobile/WordPress-Android/blob/f8db3db3ee1f2ce13c8da6344aaa2353a2bd6131/WordPress/src/main/java/org/wordpress/android/ui/posts/PostListFeaturedImageTracker.kt#L27-L47)).
- The function that fetches the featured image (`getFeaturedImageUrl`) dispatches a request if the media item is not cached. However, it doesn't check if it was requested before. This results in extra requests for the same item as outlined in the issue.
- Going up in the stack trace, I see that all calls are triggered by `PagedListPositionalDataSource.loadInitial` ([reference](https://github.com/wordpress-mobile/WordPress-FluxC-Android/blob/7d5b075778e2c2eb7195e296ed6b8c3df8817356/fluxc/src/main/java/org/wordpress/android/fluxc/model/list/PagedListFactory.kt#L40)). However, I saw that this function is called multiple times with the same data set. I don't have the context to understand why this is happening, but I presumed that it has to do with the paging logic of the Android library `androidx.paging`.

_Stack trace of a call to `getFeaturedImageUrl`:_
<img width="1299" alt="Screenshot 2024-05-29 at 12 00 47" src="https://github.com/wordpress-mobile/WordPress-Android/assets/14905380/9c33263c-56d7-4605-9c48-aeb5b384a25e">

Pull request: Post List: Fix excessive media requests
Fixes #20905 

This PR fixes the excessive calls to fetch media when loading the post list by introducing and managing ongoing fetch requests in `PostListFeaturedImageTracker`. These changes aim to prevent redundant fetch requests.

Changes include
- In `PostListFeaturedImageTracker` Introduced `ongoingRequests`, to keep track of ongoing image fetch requests and prevent redundant requests.
- Updated the `getFeaturedImageUrl()` to check and update the `ongoingRequests` set appropriately.
- Updated `PostListEventListener.onMediaChanged()` to invoke `invalidateFeaturedMedia` on both success and error. This ensures that the `featuredImageMap` and `ongoingRequests` are properly updated whether the media fetch was successful or not.
-----

## To Test:
**Steps to reproduce the behavior**
**Preparation:**
1. Create multiple posts (20 or more) and attach a featured image. This can be done quickly by using the [WPCOM console](https://developer.wordpress.com/docs/api/console/) and the endpoint rest/v1.1/sites/${siteId}/posts/new.

**Existing: Extra requests to the media endpoint for the same featuredImageId**
1. Using "trunk" branch
2. On the device/emulator, **clear storage of the app**, or delete and reinstall the app.
3. Open the app and navigate to the site (the same site that you created the 20+ posts with featured images)
4. Connect the device/emulator to the Charles proxy (or see below for alternative)
5. Open the app and go to the Post List screen.
6. Observe all extra requests to the media endpoint in the Charles proxy.

**Fix: Extra requests to the media endpoint are eliminated**
1. Using the app from this PR
2. On the device/emulator, **clear storage of the app**, or delete and reinstall the app.
3. Open the app and navigate to the site (the same site that you created the 20+ posts with featured images)
4. Connect the device/emulator to the Charles proxy (or see below for alternative)
5. Open the app and go to the Post list screen.
6. ✅ Verify all the extra requests for the same media id have been eliminated.

<details>
<summary>Alternative to using Charles</summary>

- Checkout the branch you will test (trunk for pre-existing test or this branch for the fix test
- Checkout FluxC (trunk)
- Use local FluxC to build WP
- Update `GSONRequest.parseNetworkRequest` to log info

```
@Override
    protected Response<T> parseNetworkResponse(NetworkResponse response) {
        logGsonRequest();
        try {
            String json = new String(response.data, HttpHeaderParser.parseCharset(response.headers));
            T res;
            if (mClass == null) {
                res = mGson.fromJson(json, mType);
            } else {
                res = mGson.fromJson(json, mClass);
            }
            return Response.success(res, createCacheEntry(response));
        } catch (UnsupportedEncodingException | JsonSyntaxException e) {
            logRequestPath();
            return Response.error(new ParseError(e));
        }
    }

    private void logGsonRequest() {
        String path = getPath();
        String url = getUrl();

        if (path != null && (path.contains("media"))) {
            if (url != null) {
                Log.i("GsonRequest", "***=> Request URL is: " + url);
            }
        }
    }

```
</details>

-----

## Regression Notes

1. Potential unintended areas of impact
Multiple media requests continue to be excessive

2. What I did to test those areas of impact (or what existing automated tests I relied on)
Manual and unit testing

3. What automated tests I added (or what prevented me from doing so)
Existing tests

-----

## PR Submission Checklist:
- [X] I have completed the Regression Notes.
- [X] I have considered adding accessibility improvements for my changes.
- [X] I have considered if this change warrants user-facing release notes and have added them to `RELEASE-NOTES.txt` if necessary.

-----

## Testing Checklist (strike-out the not-applying and unnecessary ones): N/A


Test command: ./gradlew :WordPress:testWordpressVanillaDebugUnitTest --tests "org.wordpress.android.ui.posts.PostListFeaturedImageTrackerTest"

Fix patch:
diff --git a/WordPress/src/main/java/org/wordpress/android/ui/posts/PostListEventListener.kt b/WordPress/src/main/java/org/wordpress/android/ui/posts/PostListEventListener.kt
index 16d7e3067a53..85e775d67c66 100644
--- a/WordPress/src/main/java/org/wordpress/android/ui/posts/PostListEventListener.kt
+++ b/WordPress/src/main/java/org/wordpress/android/ui/posts/PostListEventListener.kt
@@ -174,8 +174,8 @@ class PostListEventListener(
     @Suppress("unused", "SpreadOperator")
     @Subscribe(threadMode = BACKGROUND)
     fun onMediaChanged(event: OnMediaChanged) {
+        featuredMediaChanged(*event.mediaList.map { it.mediaId }.toLongArray())
         if (!event.isError) {
-            featuredMediaChanged(*event.mediaList.map { it.mediaId }.toLongArray())
             uploadStatusChanged(*event.mediaList.map { it.localPostId }.toIntArray())
         }
     }
diff --git a/WordPress/src/main/java/org/wordpress/android/ui/posts/PostListFeaturedImageTracker.kt b/WordPress/src/main/java/org/wordpress/android/ui/posts/PostListFeaturedImageTracker.kt
index d94dd6e0a582..32a9a63b2518 100644
--- a/WordPress/src/main/java/org/wordpress/android/ui/posts/PostListFeaturedImageTracker.kt
+++ b/WordPress/src/main/java/org/wordpress/android/ui/posts/PostListFeaturedImageTracker.kt
@@ -1,6 +1,7 @@
 package org.wordpress.android.ui.posts
 
 import android.annotation.SuppressLint
+import androidx.annotation.VisibleForTesting
 import org.wordpress.android.fluxc.Dispatcher
 import org.wordpress.android.fluxc.generated.MediaActionBuilder
 import org.wordpress.android.fluxc.model.MediaModel
@@ -22,13 +23,27 @@ class PostListFeaturedImageTracker(private val dispatcher: Dispatcher, private v
     https://github.com/wordpress-mobile/WordPress-Android/issues/11487
      */
     @SuppressLint("UseSparseArrays")
-    private val featuredImageMap = HashMap<Long, String>()
+    @VisibleForTesting(otherwise = VisibleForTesting.PRIVATE)
+    internal val featuredImageMap = HashMap<Long, String>()
+
+    @VisibleForTesting(otherwise = VisibleForTesting.PRIVATE)
+    internal val ongoingRequests = HashSet<Long>()
 
     fun getFeaturedImageUrl(site: SiteModel, featuredImageId: Long): String? {
         if (featuredImageId == 0L) {
             return null
         }
-        featuredImageMap[featuredImageId]?.let { return it }
+
+        featuredImageMap[featuredImageId]?.let {
+            return it
+        }
+
+        // Check if a request for this image is already ongoing
+        if (ongoingRequests.contains(featuredImageId)) {
+            // If the request is ongoing, just return. The callback will be invoked upon completion.
+            return null
+        }
+
         mediaStore.getSiteMediaWithId(site, featuredImageId)?.let { media ->
             // This should be a pretty rare case, but some media seems to be missing url
             return if (media.url.isNotBlank()) {
@@ -36,7 +51,11 @@ class PostListFeaturedImageTracker(private val dispatcher: Dispatcher, private v
                 media.url
             } else null
         }
+
         // Media is not in the Store, we need to download it
+        // Mark the request as ongoing
+        ongoingRequests.add(featuredImageId)
+
         val mediaToDownload = MediaModel(
             site.id,
             featuredImageId
@@ -47,6 +66,9 @@ class PostListFeaturedImageTracker(private val dispatcher: Dispatcher, private v
     }
 
     fun invalidateFeaturedMedia(featuredImageIds: List<Long>) {
-        featuredImageIds.forEach { featuredImageMap.remove(it) }
+        featuredImageIds.forEach {
+            featuredImageMap.remove(it)
+            ongoingRequests.remove(it)
+        }
     }
 }


Test patch:
diff --git a/WordPress/src/test/java/org/wordpress/android/ui/posts/PostListFeaturedImageTrackerTest.kt b/WordPress/src/test/java/org/wordpress/android/ui/posts/PostListFeaturedImageTrackerTest.kt
new file mode 100644
index 000000000000..a9616c3fd5b1
--- /dev/null
+++ b/WordPress/src/test/java/org/wordpress/android/ui/posts/PostListFeaturedImageTrackerTest.kt
@@ -0,0 +1,111 @@
+package org.wordpress.android.ui.posts
+
+import kotlinx.coroutines.ExperimentalCoroutinesApi
+import org.junit.Assert.assertEquals
+import org.junit.Assert.assertNull
+import org.junit.Before
+import org.mockito.kotlin.any
+import org.mockito.kotlin.mock
+import org.mockito.kotlin.never
+import org.mockito.kotlin.verify
+import org.mockito.kotlin.whenever
+import org.wordpress.android.BaseUnitTest
+import org.wordpress.android.fluxc.Dispatcher
+import org.wordpress.android.fluxc.model.MediaModel
+import org.wordpress.android.fluxc.model.SiteModel
+import org.wordpress.android.fluxc.store.MediaStore
+import kotlin.test.Test
+
+@Suppress("UNCHECKED_CAST")
+@ExperimentalCoroutinesApi
+class PostListFeaturedImageTrackerTest : BaseUnitTest() {
+    private val dispatcher: Dispatcher = mock()
+    private val mediaStore: MediaStore = mock()
+
+    private lateinit var tracker: PostListFeaturedImageTracker
+
+    private val site = SiteModel().apply { id = 123 }
+
+    @Before
+    fun setup() {
+        tracker = PostListFeaturedImageTracker(dispatcher, mediaStore)
+    }
+
+    @Test
+    fun `given id exists in map, when getFeaturedImageUrl invoked, then return url`() {
+        val imageId = 123L
+        val imageUrl = "https://example.com/image.jpg"
+        tracker.featuredImageMap[imageId] = imageUrl
+
+        val result = tracker.getFeaturedImageUrl(site, imageId)
+
+        assertEquals(imageUrl, result)
+    }
+
+    @Test
+    fun `given id is 0, when getFeaturedImageUrl invoked, then return null`() {
+        val result = tracker.getFeaturedImageUrl(site, 0L)
+
+        assertNull(result)
+    }
+
+    @Test
+    fun `given id not in map and exists in store, when invoked, then return url from media store`() {
+        val imageId = 456L
+        val imageUrl = "https://example.com/image.jpg"
+        val mediaModel = MediaModel(site.id, imageId).apply {
+            url = imageUrl
+        }
+
+        whenever(mediaStore.getSiteMediaWithId(site, imageId)).thenReturn(mediaModel)
+
+        val result = tracker.getFeaturedImageUrl(site, imageId)
+
+        assertEquals(imageUrl, result)
+        assertEquals(imageUrl, tracker.featuredImageMap[imageId])
+    }
+
+    @Test
+    fun `given id not in map or store, when invoked, then return null and dispatch fetch request`() {
+        val imageId = 123L
+
+        whenever(mediaStore.getSiteMediaWithId(site, imageId)).thenReturn(null)
+
+        val result = tracker.getFeaturedImageUrl(site, imageId)
+
+        assertNull(result)
+        verify(dispatcher).dispatch(any())
+        assert(tracker.ongoingRequests.contains(imageId))
+    }
+
+    @Test
+    fun `given request ongoing for id, when invoked, should return null`() {
+        val imageId = 123L
+
+        tracker.ongoingRequests.add(imageId)
+
+        val result = tracker.getFeaturedImageUrl(site, imageId)
+
+        assertNull(result)
+        verify(mediaStore, never()).getSiteMediaWithId(site, imageId)
+        verify(dispatcher, never()).dispatch(any())
+    }
+
+    @Test
+    fun `given id in map and ongoingRequests, when invalidate, then remove id from map and ongoingRequests`() {
+        val imageId1 = 123L
+        val imageId2 = 456L
+
+        tracker.featuredImageMap[imageId1] = "https://example.com/image1.jpg"
+        tracker.featuredImageMap[imageId2] = "https://example.com/image2.jpg"
+        tracker.ongoingRequests.add(imageId1)
+        tracker.ongoingRequests.add(imageId2)
+
+        tracker.invalidateFeaturedMedia(listOf(imageId1, imageId2))
+
+        assertNull(tracker.featuredImageMap[imageId1])
+        assertNull(tracker.featuredImageMap[imageId2])
+        assert(!tracker.ongoingRequests.contains(imageId1))
+        assert(!tracker.ongoingRequests.contains(imageId2))
+    }
+}