Repository: RocketMap/RocketMap

Issue:
TTH Search Never Finishes
In January, PR #2416 changed the constraint for recording spawn timers in the RM DB. Since then, the TTH calculations on the Status page were not accurate. If you look, there may be a few TTHs that are yet to be found. This is caused by a spawn that has 'latest_seen' = '3600' and 'earliest_unseen' = '0' instead of being equal to each other.

The calculations done in schedulers.py would count that as a missing TTH because it is looking for spawns that have 'latest_seen' = 'earliest_unseen' but it shows in the DB as an hour long spawn ('kind' = 'ssss'). The log file notes them as 0 minute spawns "[   search-overseer][    schedulers][    INFO] Spawnpoints with a 0m range to find TTH: xx". 

Also noticed that the the spawnpointdetectiondata table does not record a tth_secs vaule for these spawnpoints. I looked through most of mine and saw that they stopped checking for a tth after 5 scans. I have reset the missed_count before noticing this so I am not sure if that is related. There is a mismatch between the last scan time in spawnpoints and spawnpointdetectiondata. So it probably stopped checking for a TTH once it marked 'latest_seen' = '3600' and 'earliest_unseen' = '0'.

Pokemon do show in these spots on the map every hour (I haven't verified if they are there in person yet). The spawnpoint does not note "Spawn times not yet determined. Current guess...". I once saw two pokemon on the same spawnpoint meaning these are still missing TTH.

![image](https://user-images.githubusercontent.com/16466177/36912871-26802142-1e16-11e8-9955-de691e0545e3.png)

![image](https://user-images.githubusercontent.com/16466177/36912603-3149a1da-1e15-11e8-970d-9f250ae14d81.png)

![image](https://user-images.githubusercontent.com/16466177/36913358-b06dbd3c-1e17-11e8-9093-bf651adbaef1.png)

![image](https://user-images.githubusercontent.com/16466177/36913651-a4966d5a-1e18-11e8-91c9-d95065982464.png)

![image](https://user-images.githubusercontent.com/16466177/36912549-0a69e5fc-1e15-11e8-9613-8625cc495cd1.png)

I would like to hear if other people noticed this behavior also.

## Expected Behavior
The status page should show 100% TTH found once the TTH search is completed. The spawnpoint data for 'latest_seen' and 'earliest_unseen' should be equal once a TTH is found.

## Current Behavior
The status page should show < 100% TTH found because of the mismatching 'latest_seen' and 'earliest_unseen' variables. RM no long checks for a TTH on these spawns once 'latest_seen' = '3600' and 'earliest_unseen' = '0'.

## Possible Solution
Not sure of the best way to correct this. The issue could be that the tth_secs vaule is not being recorded before the spawn is marked with 'kind'='ssss'. Or that the 'kind' should be 'hhhs' since the log notes this as a spawn with less than 1 minute.

## Steps to Reproduce (for bugs)
1. Rescan an area that used to suffer from the u_constraint error.
2. Wait to finish the initial and TTH scans (TTH may show a few missing and never complete).
3. Query the spawnpoints within that area for 'latest_seen' = '3600' and 'earliest_unseen' = '0'.
4. Query the spawnpointdetectdata table for those spawn IDs to see the last time to see if it stopped checking for a TTH.

## Context
Spawns still show in those areas but I am not sure if their timers are accurate. I rarely have people report that the timers are off but there was one case that a spawn was on my map with 20 minutes left and it had already despawned. I did not check into this since it was only one time but it could have been on one of these spots.

My current scanners do not schedule TTH searches anymore so these spots are just stuck in their current state.

This affects 46 out 58,000 spawnpoints (0.1%) in my database to give you an idea of how rare these outcomes are.

## Your Environment
RM pull to PR #2505 (latest)
MariaDB-10.2.11
Schema version 28
HiediSQL 9.4.0.5215
Ubuntu 17.04


Discussion:
> Spawns still show in those areas but I am not sure if their timers are accurate. I rarely have people report that the timers are off but there was one case that a spawn was on my map with 20 minutes left and it had already despawned.

Despawns/respawns/spawn changes (even attribute changes) are known to happen on weather-related changes. Since RM doesn't currently rescan scanned Pokémon (there was no need to before weather changes, and it would be a massive increase in amount of requests/hashing required), there will be inconsistencies between the Pokémon Go app and the scanned map.

> The calculations done in schedulers.py would count that as a missing TTH because it is looking for spawns that have 'latest_seen' = 'earliest_unseen'

https://github.com/RocketMap/RocketMap/blob/develop/pogom/models.py#L1290

```
def tth_found(sp):
    latest_seen = (sp['latest_seen'] % 3600)
    earliest_unseen = (sp['earliest_unseen'] % 3600)
    return latest_seen - earliest_unseen == 0
```

In #2416, all references were updated to include `% 3600` ([modulo](https://en.wikipedia.org/wiki/Modulo_operation)), which makes the distance between `a % 3600` and `b % 3600` equal to `0` for the numbers `0` and `3600`.

Are you sure your reasoning was correct?
You are right, the math in models.py works out to be 0 when the 'latest_seen' = '3600' and 'earliest_unseen' = '0'. That is probably why they are not being scanned anymore.

In schedulers.py, it process would set 'tth_found' to '1' if the two values were the same. Since they are different, it is set to '0'. So the status page shows a missing TTH.
https://github.com/RocketMap/RocketMap/blob/develop/pogom/schedulers.py#L774

I am not that great with python so let me know if that doesn't sound right. 

I changed one of the 3600s to a different number (based on the time seen in spawnpointdetectiondata) and it was rescanned. That happened because of the code you pointed out, I made the mod value not equal to 0 anymore. But once that one had a new 'latest_seen' and 'earliest_unseen' value saved, the status page showed one less TTH to find and the spawnpointdetectiondata recorded a tth_secs value for that spawnpoint. 

That value probably wasn't accurate because of my manual manipulation but it was worth a test.
@Kneckter You're correct, line 774 of schedulers.py doesn't make use of `tth_found(sp)` for some reason. Legacy code. This will cause the issue you're describing. I'll push a fix.

Resolving patch:
diff --git a/pogom/models.py b/pogom/models.py
--- a/pogom/models.py
+++ b/pogom/models.py
@@ -1285,11 +1285,11 @@ def get_spawnpoints(swLat, swLng, neLat, neLng, timestamp=0,
 
         return list(spawnpoints.values())
 
-    # Confirm if tth has been found.
+    # Confirm if TTH has been found.
     @staticmethod
     def tth_found(sp):
-        # Fully indentified if no '?' in links and
-        # latest_seen == earliest_unseen.
+        # Fully identified if no '?' in links and
+        # latest_seen % 3600 == earliest_unseen % 3600.
         # Warning: python uses modulo as the least residue, not as
         # remainder, so we don't apply it to the result.
         latest_seen = (sp['latest_seen'] % 3600)
diff --git a/pogom/schedulers.py b/pogom/schedulers.py
--- a/pogom/schedulers.py
+++ b/pogom/schedulers.py
@@ -758,6 +758,7 @@ def schedule(self):
                     len(filter(lambda e: e['kind'] == 'spawn', Scanned_list))
                 spawns_missed = len(
                     filter(lambda e: e['kind'] == 'spawn', Missed_list))
+
                 band_percent = self.band_status()
                 kinds = {}
                 tth_ranges = {}
@@ -767,12 +768,14 @@ def schedule(self):
                 spawns_reached = 100.0
                 spawnpoints = SpawnPoint.select_in_hex_by_cellids(
                     self.scans.keys(), self.location_change_date)
+
                 for sp in spawnpoints:
                     if sp['missed_count'] > 5:
                         continue
+
                     self.active_sp += 1
-                    self.tth_found += (sp['earliest_unseen'] ==
-                                       sp['latest_seen'])
+                    self.tth_found += SpawnPoint.tth_found(sp)
+
                     kind = sp['kind']
                     kinds[kind] = kinds.get(kind, 0) + 1
                     tth_range = str(int(round(
@@ -782,6 +785,7 @@ def schedule(self):
 
                 tth_ranges['0'] = tth_ranges.get('0', 0) - self.tth_found
                 len_spawnpoints = len(spawnpoints) + (not len(spawnpoints))
+
                 log.info('Total Spawn Points found in hex: %d',
                          len(spawnpoints))
                 log.info('Inactive Spawn Points found in hex: %d or %.1f%%',
@@ -791,26 +795,33 @@ def schedule(self):
                 log.info('Active Spawn Points found in hex: %d or %.1f%%',
                          self.active_sp,
                          self.active_sp * 100.0 / len_spawnpoints)
+
                 self.active_sp += self.active_sp == 0
+
                 for k in sorted(kinds.keys()):
                     log.info('%s kind spawns: %d or %.1f%%', k,
                              kinds[k], kinds[k] * 100.0 / self.active_sp)
+
                 log.info('Spawns with found TTH: %d or %.1f%% [%d missing]',
                          self.tth_found,
                          self.tth_found * 100.0 / self.active_sp,
                          self.active_sp - self.tth_found)
+
                 for k in sorted(tth_ranges.keys(), key=int):
                     log.info('Spawnpoints with a %sm range to find TTH: %d', k,
                              tth_ranges[k])
+
                 log.info('Over last %d minutes: %d new bands, %d Pokemon ' +
                          'found', self.minutes, bands_timed, spawns_all)
                 log.info('Of the %d total spawns, %d were targeted, and %d ' +
                          'found scanning for others', spawns_all, spawns_timed,
                          spawns_all - spawns_timed)
+
                 scan_total = spawns_timed + bands_timed
                 spm = scan_total / self.minutes
                 seconds_per_scan = self.minutes * 60 * \
                     self.args.workers / scan_total if scan_total else 0
+
                 log.info('%d scans over %d minutes, %d scans per minute, %d ' +
                          'secs per scan per worker', scan_total, self.minutes,
                          spm, seconds_per_scan)
@@ -838,26 +849,34 @@ def schedule(self):
                     sum = spawns_missed + self.spawns_found
                     found_percent = (
                         self.spawns_found * 100.0 / sum if sum else 0)
+
                     log.info('%d spawns scanned and %d spawns were not ' +
                              'there when expected for %.1f%%',
                              self.spawns_found, spawns_missed, found_percent)
+
                     self.spawn_percent.append(round(found_percent, 1))
+
                     if self.spawns_missed_delay:
                         log.warning('Missed spawn IDs with times after spawn:')
                         log.warning(self.spawns_missed_delay)
+
                     log.info('History: %s', str(
                         self.spawn_percent).strip('[]'))
 
                 sum = self.scans_done + len(self.scans_missed_list)
                 good_percent = self.scans_done * 100.0 / sum if sum else 0
+
                 log.info(
                     '%d scans successful and %d scans missed for %.1f%% found',
                     self.scans_done, len(self.scans_missed_list), good_percent)
+
                 self.scan_percent.append(round(good_percent, 1))
+
                 if self.scans_missed_list:
                     log.warning('Missed scans: %s', Counter(
                         self.scans_missed_list).most_common(3))
                     log.info('History: %s', str(self.scan_percent).strip('[]'))
+
                 self.status_message = ('Initial scan: {:.2f}%, TTH found: ' +
                                        '{:.2f}% [{} missing], ').format(
                     band_percent, self.tth_found * 100.0 / self.active_sp,
@@ -871,8 +890,7 @@ def schedule(self):
 
             except Exception as e:
                 log.error(
-                    'Performance statistics had an Exception: {}'.format(
-                        repr(e)))
+                    'Performance statistics had an exception: %s.', e)
                 traceback.print_exc(file=sys.stdout)
 
     # Find the best item to scan next


Test patch: