Repository: RocketMap/RocketMap

Issue:
Statistics Page Rework & Fixes
<!--- Provide a general summary of your changes in the Title above -->

## Description
<!--- Describe your changes in detail -->
This PR does 3 things.

1) Changed the /stats page to match the style of /status so that things are more consistent across the site.
2) Significantly speeds up the load time of the stats page
3) Fixes the scroll issue on safari (and other browsers?) after changing duration

## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here. -->
Fixes/Address issues:

#1825 #1415 #1192 

## How Has This Been Tested?
<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, and the tests you ran to -->
<!--- see how your change affects other areas of the code, etc. -->
On my local machine and phone with safari & chrome

## Screenshots (if appropriate):

![newlook](https://cloud.githubusercontent.com/assets/3094101/26031125/4624da3e-38ab-11e7-9541-49d7c7df7165.jpg)

![speed increase](https://cloud.githubusercontent.com/assets/3094101/26031131/8ecb7bd0-38ab-11e7-94b7-6f6de488e12c.jpg)

## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Checklist:
<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
<!--  NOTE: In order to check code style locally and avoid having your build rejected by Travis, -->
<!--  run the following commands before you commit: `flake8 .` and `npm run lint`. Fix any -->
<!--  issues they point out. Note also that flake's NOQA is disabled on Travis. -->
- [x] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.

Can't scroll stats on multiple browsers after changing "duration"
## Expected Behavior

To always be able to scroll the full stats page when using Safari.
## Current Behavior

After changing "Duration", the stats page will be stuck on the top when trying to scroll.
## Steps to Reproduce (for bugs)
1. Launch pokemongo-map on Safari
2. Click on stats
3. Click on full stats
4. Try to scroll the page -- it works.
5. Click on Options
6. Change "Duration" from "Last Day" to "Last 6 Hours"
7. Try again to scroll the page -- it keeps immediately resetting to the top. <<<
## Your Environment
- Latest revision on develop (e92fa83), but it has been happening for a few days.
- Testing on OS X 10.11.6,  Safari 9.3
- Also an issue on latest iOS Safari.

This does not seem to affect Chrome (Version 53.0.2785.89 (64-bit))



Discussion:

Occasionaly happens on Firefox 48.0.2 too. Doesn't happen with Microsoft Edge.

I can confirm tested on OSX FF 48.0.1 and Linux FF 45.3.0.

Happening for me with Internet Explorer 11.

My testing:

All on windows 10.
No issues with Chrome
No issues with Edge
Easily replicated with IE11

It seems to me that changing any option in the full stats page prevents scrolling on Safari 10 on macOS Sierra. Not just the duration option. Testing in Chrome, the issue did not present itself.

same issue with iOS 10 Chrome and Safari


Resolving patch:
diff --git a/pogom/app.py b/pogom/app.py
--- a/pogom/app.py
+++ b/pogom/app.py
@@ -11,8 +11,6 @@
 from datetime import datetime
 from s2sphere import LatLng
 from pogom.utils import get_args
-from datetime import timedelta
-from collections import OrderedDict
 from bisect import bisect_left
 
 from .models import (Pokemon, Gym, Pokestop, ScannedLocation,
@@ -359,28 +357,20 @@ def raw_data(self):
                         swLat, swLng, neLat, neLng, oSwLat=oSwLat,
                         oSwLng=oSwLng, oNeLat=oNeLat, oNeLng=oNeLng)
 
-        selected_duration = None
-
-        # for stats and changed nest points etc, limit pokemon queried.
-        for duration in (
-                self.get_valid_stat_input()["duration"]["items"].values()):
-            if duration["selected"] == "SELECTED":
-                selected_duration = duration["value"]
-                break
-
         if request.args.get('seen', 'false') == 'true':
-            d['seen'] = Pokemon.get_seen(selected_duration)
+            d['seen'] = Pokemon.get_seen(int(request.args.get('duration')))
 
         if request.args.get('appearances', 'false') == 'true':
             d['appearances'] = Pokemon.get_appearances(
-                request.args.get('pokemonid'), selected_duration)
+                request.args.get('pokemonid'),
+                int(request.args.get('duration')))
 
         if request.args.get('appearancesDetails', 'false') == 'true':
             d['appearancesTimes'] = (
                 Pokemon.get_appearances_times_by_spawnpoint(
                     request.args.get('pokemonid'),
                     request.args.get('spawnpoint_id'),
-                    selected_duration))
+                    int(request.args.get('duration'))))
 
         if request.args.get('spawnpoints', 'false') == 'true':
             if lastspawns != 'true':
@@ -489,89 +479,6 @@ def list_pokemon(self):
                                show=visibility_flags
                                )
 
-    def get_valid_stat_input(self):
-        duration = request.args.get("duration", type=str)
-        sort = request.args.get("sort", type=str)
-        order = request.args.get("order", type=str)
-        valid_durations = OrderedDict()
-        valid_durations["1h"] = {
-            "display": "Last Hour",
-            "value": timedelta(hours=1),
-            "selected": ("SELECTED" if duration == "1h" else "")}
-        valid_durations["3h"] = {
-            "display": "Last 3 Hours",
-            "value": timedelta(hours=3),
-            "selected": ("SELECTED" if duration == "3h" else "")}
-        valid_durations["6h"] = {
-            "display": "Last 6 Hours",
-            "value": timedelta(hours=6),
-            "selected": ("SELECTED" if duration == "6h" else "")}
-        valid_durations["12h"] = {
-            "display": "Last 12 Hours",
-            "value": timedelta(hours=12),
-            "selected": ("SELECTED" if duration == "12h" else "")}
-        valid_durations["1d"] = {
-            "display": "Last Day",
-            "value": timedelta(days=1),
-            "selected": ("SELECTED" if duration == "1d" else "")}
-        valid_durations["7d"] = {
-            "display": "Last 7 Days",
-            "value": timedelta(days=7),
-            "selected": ("SELECTED" if duration == "7d" else "")}
-        valid_durations["14d"] = {
-            "display": "Last 14 Days",
-            "value": timedelta(days=14),
-            "selected": ("SELECTED" if duration == "14d" else "")}
-        valid_durations["1m"] = {
-            "display": "Last Month",
-            "value": timedelta(days=365 / 12),
-            "selected": ("SELECTED" if duration == "1m" else "")}
-        valid_durations["3m"] = {
-            "display": "Last 3 Months",
-            "value": timedelta(days=3 * 365 / 12),
-            "selected": ("SELECTED" if duration == "3m" else "")}
-        valid_durations["6m"] = {
-            "display": "Last 6 Months",
-            "value": timedelta(days=6 * 365 / 12),
-            "selected": ("SELECTED" if duration == "6m" else "")}
-        valid_durations["1y"] = {
-            "display": "Last Year",
-            "value": timedelta(days=365),
-            "selected": ("SELECTED" if duration == "1y" else "")}
-        valid_durations["all"] = {
-            "display": "Map Lifetime",
-            "value": 0,
-            "selected": ("SELECTED" if duration == "all" else "")}
-        if duration not in valid_durations:
-            valid_durations["1d"]["selected"] = "SELECTED"
-        valid_sort = OrderedDict()
-        valid_sort["count"] = {
-            "display": "Count",
-            "selected": ("SELECTED" if sort == "count" else "")}
-        valid_sort["id"] = {
-            "display": "Pokedex Number",
-            "selected": ("SELECTED" if sort == "id" else "")}
-        valid_sort["name"] = {
-            "display": "Pokemon Name",
-            "selected": ("SELECTED" if sort == "name" else "")}
-        if sort not in valid_sort:
-            valid_sort["count"]["selected"] = "SELECTED"
-        valid_order = OrderedDict()
-        valid_order["asc"] = {
-            "display": "Ascending",
-            "selected": ("SELECTED" if order == "asc" else "")}
-        valid_order["desc"] = {
-            "display": "Descending",
-            "selected": ("SELECTED" if order == "desc" else "")}
-        if order not in valid_order:
-            valid_order["desc"]["selected"] = "SELECTED"
-        valid_input = OrderedDict()
-        valid_input["duration"] = {
-            "display": "Duration", "items": valid_durations}
-        valid_input["sort"] = {"display": "Sort", "items": valid_sort}
-        valid_input["order"] = {"display": "Order", "items": valid_order}
-        return valid_input
-
     def get_stats(self):
         args = get_args()
         visibility_flags = {
@@ -583,7 +490,6 @@ def get_stats(self):
                                lat=self.current_location[0],
                                lng=self.current_location[1],
                                gmaps_key=args.gmaps_key,
-                               valid_input=self.get_valid_stat_input(),
                                show=visibility_flags
                                )
 
diff --git a/pogom/models.py b/pogom/models.py
--- a/pogom/models.py
+++ b/pogom/models.py
@@ -245,16 +245,21 @@ def get_active_by_id(ids, swLat, swLng, neLat, neLng):
     @cached(cache)
     def get_seen(timediff):
         if timediff:
-            timediff = datetime.utcnow() - timediff
+            timediff = datetime.utcnow() - timedelta(hours=timediff)
+
+        # Note: pokemon_id+0 forces SQL to ignore the pokemon_id index
+        # and should use the disappear_time index and hopefully
+        # improve performance
         pokemon_count_query = (Pokemon
-                               .select(Pokemon.pokemon_id,
-                                       fn.COUNT(Pokemon.pokemon_id).alias(
+                               .select((Pokemon.pokemon_id+0).alias(
+                                           'pokemon_id'),
+                                       fn.COUNT((Pokemon.pokemon_id+0)).alias(
                                            'count'),
                                        fn.MAX(Pokemon.disappear_time).alias(
                                            'lastappeared')
                                        )
                                .where(Pokemon.disappear_time > timediff)
-                               .group_by(Pokemon.pokemon_id)
+                               .group_by((Pokemon.pokemon_id+0))
                                .alias('counttable')
                                )
         query = (Pokemon
@@ -296,7 +301,7 @@ def get_appearances(pokemon_id, timediff):
         :return: list of Pokemon appearances over a selected period
         '''
         if timediff:
-            timediff = datetime.utcnow() - timediff
+            timediff = datetime.utcnow() - timedelta(hours=timediff)
         query = (Pokemon
                  .select(Pokemon.latitude, Pokemon.longitude,
                          Pokemon.pokemon_id,
@@ -315,6 +320,7 @@ def get_appearances(pokemon_id, timediff):
     @staticmethod
     def get_appearances_times_by_spawnpoint(pokemon_id, spawnpoint_id,
                                             timediff):
+
         '''
         :param pokemon_id: id of Pokemon that we need appearances times for.
         :param spawnpoint_id: spawnpoint id we need appearances times for.
@@ -322,7 +328,7 @@ def get_appearances_times_by_spawnpoint(pokemon_id, spawnpoint_id,
         :return: list of time appearances over a selected period.
         '''
         if timediff:
-            timediff = datetime.utcnow() - timediff
+            timediff = datetime.utcnow() - timedelta(hours=timediff)
         query = (Pokemon
                  .select(Pokemon.disappear_time)
                  .where((Pokemon.pokemon_id == pokemon_id) &


Test patch: