Repository: zulip/zulip-flutter (language: dart)

Issue:
Loading indicator slow to go away when returning to app, though in fact up to date
We added a loading indicator in #852 that appears when even queue polling fails with any error.

However, it seems that the loading indicator can stay on even the event queue is doing fine, and does not go away until you get a new event.

I don't have a consistent way to reproduce this, but what I did was basically:

1. Leave the app on and lock the phone
2. After a while, unlock the phone
3. Observe if the loading indicator stays there longer than usual
4. Try to do something (like starring a message), and the loading indicator should go away promptly

My guess is that when the event queue goes stale, `isLoading` gets set to true. When we do get the new event queue, we do not set `isLoading` back to false.


Discussion hints:
Thanks for the report!

I don't think I've seen this symptom myself yet. It sounds reminiscent of this other event-queue-related symptom I saw once back in July:
* #809

but I haven't seen that one since then either.

I think this won't be actionable until we hear more reports of it happening, so I'll push it out to the post-launch milestone. If we do see or hear of it again, this thread will help us aggregate those reports.

I happened to see this symptom yesterday during development, and thanks to seeing the debug log I think I have a diagnosis. This was on my Pixel 8 running Android 15, while connected by USB to my desktop for debugging.

What happens is:
* The device goes to sleep, perhaps while the app was in the foreground.
* The app actually keeps running — one can tell because it keeps printing log messages to the console.
* But it isn't able to make network requests.
  * The symptom of this is that the requests fail with SocketException, generally with DNS errors that it can't find `chat.zulip.org` (or whatever the host is).
    * We'd previously seen that those errors are common; that's why they're the one type of transient failure we silenced from reporting to the user, in #868.
  * In retrospect this makes sense as a compromise for the OS to choose: let the foreground app keep running, at least for a little while, in case the user wakes the device right back up again; but disable the network, because the wifi radio and especially the cell radio are expensive for the battery.
* So our event-poll loop keeps getting failures, and backs off farther and farther. So far, so good.
* Now the user wakes the device, and the app is able to use the network again. Two things go wrong that both contribute to this symptom.
* First, we may be in the middle of a long backoff. So it could be up to 10 seconds (typically more like 3 seconds) before we make the next request. During the remaining backoff time, the loading indicator will continue; and we won't learn of new events, even events that already happened while we were offline.
  * This first issue, I was basically already aware of. It'd be good to fix but isn't the main cause of the symptom this thread is about.
* Then, once the current backoff expires and we do make a new request… _that request may take up to a minute to return_, even if the network and server are functioning ideally. Until the request completes, we keep the loading indicator going.
  * This is the new point which I hadn't thought of before I saw it happening yesterday.
  * Specifically this happens if there are no events waiting for us, and no new events show up. After all, our event-polling is a long-poll — the point of which is that the server may sit on the request a long time, so that when it does have something to say it can send a response immediately.
    * The only reason this ends even at a minute is because the server sends a "heartbeat" event just for the sake of making a response, as a workaround for some network intermediaries breaking connections that sit silently for longer than that.
  * That's why starring a message stops the loading indicator: it generates an event, causing the server to send a response.
  * While this issue is active, the app's data is actually up to date — if there were news for us, the server would have responded with it already. So the only problem is that we're showing the loading indicator, making it look out of date. In that sense it's milder than the first issue above.
  * On the other hand, this one can last much longer — it'll routinely be almost one minute, if the realm isn't busy. So in that sense it's more severe.

So, now that we have a diagnosis, moving this issue earlier. I think it's still a post-launch issue, though — I wouldn't hold up shipping the app for it.

This behavior continues to bug me sometimes, even though I know exactly what's happening.

It also came up today from a user trying the beta: https://github.com/zulip/zulip-flutter/issues/1525#issuecomment-2920818667
I just discovered the `dont_block` param of `GET /events`: https://zulip.com/api/get-events#parameter-dont_block

From reading its doc, I think the fix is to pass `true` for that, in the first poll request after a failure? (Currently on mobile; I can explain more later if needed.)
Interesting! Yeah, that might be all that's needed.

Pull request: store: Pass `dont_block` in first poll attempt after a failure
Fixes #979.

Discussion:
  https://github.com/zulip/zulip-flutter/issues/979#issuecomment-2993296322

------

Tested manually, with and without this change:

1. Log out from CZO (too active; events coming in frequently); connect to a not-very-active org

2. Make an artificial network error in the `getEvents` binding:

```diff
diff --git lib/api/route/events.dart lib/api/route/events.dart
index bd14521c7..da7bf4fea 100644
--- lib/api/route/events.dart
+++ lib/api/route/events.dart
@@ -1,6 +1,7 @@
 import 'package:json_annotation/json_annotation.dart';
 
 import '../core.dart';
+import '../exception.dart';
 import '../model/events.dart';
 import '../model/initial_snapshot.dart';
 
@@ -27,6 +28,7 @@ Future<InitialSnapshot> registerQueue(ApiConnection connection) {
 Future<GetEventsResult> getEvents(ApiConnection connection, {
   required String queueId, int? lastEventId, bool? dontBlock,
 }) {
+  throw NetworkException(routeName: 'foo', message: 'foo', cause: 'foo');
   return connection.get('getEvents', GetEventsResult.fromJson, 'events', {
     'queue_id': RawParameter(queueId),
     if (lastEventId != null) 'last_event_id': lastEventId,
```

3. Watch the logs to see retries going by
4. Comment out the `throw NetworkException` line and do a hot-reload
5. Watch the logs for `dont_block` in the `GET /events`, and watch the app to see if the loading indicator goes away quickly
   - Without the change, I don't see `dont_block`, and the loading indicator doesn't go away until the next event comes in
   - With the change, I see `dont_block`, and the loading indicator goes away quickly

Test command: flutter test test/model/store_test.dart test/widgets/app_bar_test.dart --reporter expanded 2>&1

Fix patch:
diff --git a/lib/model/store.dart b/lib/model/store.dart
index 13a999aa37..9973cbc33e 100644
--- a/lib/model/store.dart
+++ b/lib/model/store.dart
@@ -582,12 +582,12 @@ class PerAccountStore extends PerAccountStoreBase with
     _updateMachine = value;
   }
 
-  bool get isLoading => _isLoading;
-  bool _isLoading = false;
+  bool get isRecoveringEventStream => _isRecoveringEventStream;
+  bool _isRecoveringEventStream = false;
   @visibleForTesting
-  set isLoading(bool value) {
-    if (_isLoading == value) return;
-    _isLoading = value;
+  set isRecoveringEventStream(bool value) {
+    if (_isRecoveringEventStream == value) return;
+    _isRecoveringEventStream = value;
     notifyListeners();
   }
 
@@ -1437,7 +1437,13 @@ class UpdateMachine {
         final GetEventsResult result;
         try {
           result = await getEvents(store.connection,
-            queueId: store.queueId, lastEventId: lastEventId);
+            queueId: store.queueId,
+            lastEventId: lastEventId,
+            // If the UI shows we're busy getting event-polling to work again,
+            // ask the server to tell us immediately that it's working again,
+            // rather than waiting for an event, which could take up to a minute
+            // in the case of a heartbeat event. See #979.
+            dontBlock: store.isRecoveringEventStream ? true : null);
           if (_disposed) return;
         } catch (e, stackTrace) {
           if (_disposed) return;
@@ -1504,14 +1510,14 @@ class UpdateMachine {
     // if we stayed at the max backoff interval; partway toward what would
     // happen if we weren't backing off at all.
     //
-    // But at least for [getEvents] requests, as here, it should be OK,
-    // because this is a long-poll.  That means a typical successful request
-    // takes a long time to come back; in fact longer than our max backoff
-    // duration (which is 10 seconds).  So if we're getting a mix of successes
-    // and failures, the successes themselves should space out the requests.
+    // Successful retries won't actually space out the requests, because retries
+    // are done with the `dont_block` param, asking the server to respond
+    // immediately instead of waiting through the long-poll period.
+    // (See comments on that code for why this behavior is helpful.)
+    // If server logs show pressure from too many requests, we can investigate.
     _pollBackoffMachine = null;
 
-    store.isLoading = false;
+    store.isRecoveringEventStream = false;
     _accumulatedTransientFailureCount = 0;
     reportErrorToUserBriefly(null);
   }
@@ -1530,7 +1536,7 @@ class UpdateMachine {
   ///  * [_handlePollError], which handles errors from the rest of [poll]
   ///    and errors this method rethrows.
   Future<void> _handlePollRequestError(Object error, StackTrace stackTrace) async {
-    store.isLoading = true;
+    store.isRecoveringEventStream = true;
 
     if (error is! ApiRequestException) {
       // Some unexpected error, outside even making the HTTP request.
@@ -1588,7 +1594,7 @@ class UpdateMachine {
     // or an unexpected exception representing a bug in our code or the server.
     // Either way, the show must go on.  So reload server data from scratch.
 
-    store.isLoading = true;
+    store.isRecoveringEventStream = true;
 
     bool isUnexpected;
     // TODO(#1054): handle auth failure
diff --git a/lib/widgets/app_bar.dart b/lib/widgets/app_bar.dart
index f548557681..77f77ba7e2 100644
--- a/lib/widgets/app_bar.dart
+++ b/lib/widgets/app_bar.dart
@@ -79,7 +79,7 @@ class _ZulipAppBarBottom extends StatelessWidget implements PreferredSizeWidget
   @override
   Widget build(BuildContext context) {
     final store = PerAccountStoreWidget.of(context);
-    if (!store.isLoading) return const SizedBox.shrink();
+    if (!store.isRecoveringEventStream) return const SizedBox.shrink();
     return LinearProgressIndicator(minHeight: 4.0, backgroundColor: backgroundColor);
   }
 }


Test patch:
diff --git a/test/model/store_checks.dart b/test/model/store_checks.dart
index 93e24dffdd..c157ac2190 100644
--- a/test/model/store_checks.dart
+++ b/test/model/store_checks.dart
@@ -45,7 +45,7 @@ extension GlobalStoreChecks on Subject<GlobalStore> {
 
 extension PerAccountStoreChecks on Subject<PerAccountStore> {
   Subject<ApiConnection> get connection => has((x) => x.connection, 'connection');
-  Subject<bool> get isLoading => has((x) => x.isLoading, 'isLoading');
+  Subject<bool> get isRecoveringEventStream => has((x) => x.isRecoveringEventStream, 'isRecoveringEventStream');
   Subject<Uri> get realmUrl => has((x) => x.realmUrl, 'realmUrl');
   Subject<String> get zulipVersion => has((x) => x.zulipVersion, 'zulipVersion');
   Subject<bool> get realmMandatoryTopics => has((x) => x.realmMandatoryTopics, 'realmMandatoryTopics');
diff --git a/test/model/store_test.dart b/test/model/store_test.dart
index 3736064f84..5ce42626cf 100644
--- a/test/model/store_test.dart
+++ b/test/model/store_test.dart
@@ -782,13 +782,14 @@ void main() {
       updateMachine.poll();
     }
 
-    void checkLastRequest({required int lastEventId}) {
+    void checkLastRequest({required int lastEventId, bool expectDontBlock = false}) {
       check(connection.takeRequests()).single.isA<http.Request>()
         ..method.equals('GET')
         ..url.path.equals('/api/v1/events')
         ..url.queryParameters.deepEquals({
           'queue_id': store.queueId,
           'last_event_id': lastEventId.toString(),
+          if (expectDontBlock) 'dont_block': 'true',
         });
     }
 
@@ -841,7 +842,7 @@ void main() {
         await prepareError();
         updateMachine.debugAdvanceLoop();
         async.elapse(Duration.zero);
-        check(store).isLoading.isTrue();
+        check(store).isRecoveringEventStream.isTrue();
 
         if (expectBackoff) {
           // The reload doesn't happen immediately; there's a timer.
@@ -853,7 +854,7 @@ void main() {
         // The global store has a new store.
         check(globalStore.perAccountSync(store.accountId)).not((it) => it.identicalTo(store));
         updateFromGlobalStore();
-        check(store).isLoading.isFalse();
+        check(store).isRecoveringEventStream.isFalse();
 
         // The new UpdateMachine updates the new store.
         updateMachine.debugPauseLoop();
@@ -878,8 +879,8 @@ void main() {
         prepareError();
         updateMachine.debugAdvanceLoop();
         async.elapse(Duration.zero);
-        checkLastRequest(lastEventId: 1);
-        check(store).isLoading.isTrue();
+        checkLastRequest(lastEventId: 1, expectDontBlock: false);
+        check(store).isRecoveringEventStream.isTrue();
 
         // Polling doesn't resume immediately; there's a timer.
         check(async.pendingTimers).length.equals(1);
@@ -893,9 +894,9 @@ void main() {
           HeartbeatEvent(id: 2),
         ], queueId: null).toJson());
         async.flushTimers();
-        checkLastRequest(lastEventId: 1);
+        checkLastRequest(lastEventId: 1, expectDontBlock: true);
         check(updateMachine.lastEventId).equals(2);
-        check(store).isLoading.isFalse();
+        check(store).isRecoveringEventStream.isFalse();
       });
     }
 
@@ -1032,11 +1033,13 @@ void main() {
         await preparePoll(lastEventId: 1);
       }
 
-      void pollAndFail(FakeAsync async, {bool shouldCheckRequest = true}) {
+      void pollAndFail(FakeAsync async, {bool shouldCheckRequest = true, bool expectDontBlock = false}) {
         updateMachine.debugAdvanceLoop();
         async.elapse(Duration.zero);
-        if (shouldCheckRequest) checkLastRequest(lastEventId: 1);
-        check(store).isLoading.isTrue();
+        if (shouldCheckRequest) {
+          checkLastRequest(lastEventId: 1, expectDontBlock: expectDontBlock);
+        }
+        check(store).isRecoveringEventStream.isTrue();
       }
 
       Subject<String> checkReported(void Function() prepareError) {
@@ -1054,9 +1057,11 @@ void main() {
         return awaitFakeAsync((async) async {
           await prepare();
 
+          bool expectDontBlock = false;
           for (int i = 0; i < UpdateMachine.transientFailureCountNotifyThreshold; i++) {
             prepareError();
-            pollAndFail(async);
+            pollAndFail(async, expectDontBlock: expectDontBlock);
+            expectDontBlock = true;
             check(takeLastReportedError()).isNull();
             async.flushTimers();
             if (!identical(store, globalStore.perAccountSync(store.accountId))) {
@@ -1064,11 +1069,14 @@ void main() {
               updateFromGlobalStore();
               updateMachine.debugPauseLoop();
               updateMachine.poll();
+              // Loading indicator is cleared on successful /register;
+              // we don't need dont_block for the new queue's first poll.
+              expectDontBlock = false;
             }
           }
 
           prepareError();
-          pollAndFail(async);
+          pollAndFail(async, expectDontBlock: expectDontBlock);
           return check(takeLastReportedError()).isNotNull();
         });
       }
@@ -1077,9 +1085,11 @@ void main() {
         return awaitFakeAsync((async) async {
           await prepare();
 
+          bool expectDontBlock = false;
           for (int i = 0; i < UpdateMachine.transientFailureCountNotifyThreshold; i++) {
             prepareError();
-            pollAndFail(async);
+            pollAndFail(async, expectDontBlock: expectDontBlock);
+            expectDontBlock = true;
             check(takeLastReportedError()).isNull();
             async.flushTimers();
             if (!identical(store, globalStore.perAccountSync(store.accountId))) {
@@ -1087,11 +1097,14 @@ void main() {
               updateFromGlobalStore();
               updateMachine.debugPauseLoop();
               updateMachine.poll();
+              // Loading indicator is cleared on successful /register;
+              // we don't need dont_block for the new queue's first poll.
+              expectDontBlock = false;
             }
           }
 
           prepareError();
-          pollAndFail(async);
+          pollAndFail(async, expectDontBlock: expectDontBlock);
           // Still no error reported, even after the same number of iterations
           // where other errors get reported (as [checkLateReported] checks).
           check(takeLastReportedError()).isNull();
@@ -1186,7 +1199,7 @@ void main() {
       globalStore.clearCachedApiConnections();
       updateMachine.debugAdvanceLoop();
       async.elapse(Duration.zero); // the bad-event-queue error arrives
-      check(store).isLoading.isTrue();
+      check(store).isRecoveringEventStream.isTrue();
     }
 
     test('user logged out before new store is loaded', () => awaitFakeAsync((async) async {
diff --git a/test/widgets/app_bar_test.dart b/test/widgets/app_bar_test.dart
index 099471d4f7..f4178b455a 100644
--- a/test/widgets/app_bar_test.dart
+++ b/test/widgets/app_bar_test.dart
@@ -31,7 +31,7 @@ void main() {
     await tester.pumpAndSettle();
     final rectBefore = tester.getRect(find.byType(ZulipAppBar));
     check(finder.evaluate()).isEmpty();
-    store.isLoading = true;
+    store.isRecoveringEventStream = true;
 
     await tester.pump();
     check(tester.getRect(find.byType(ZulipAppBar))).equals(rectBefore);