Repository: cmu-delphi/covidcast-indicators
Issue: doctor-visits reporting fewer megafips than usual

**Actual Behavior:**

[Ryan identified a problem](https://delphi-org.slack.com/archives/C0130CSQRN3/p1605131448128700)(slack thread) with doctor-visits where fewer megafips regions than usual are being reported. The effect recedes after a few days, so megafips _are_ being produced, just fewer than (notably) before the geo refactor.

**Expected behavior**

Number of megafips produced by doctor-visits in `fbd87a2f497` should be roughly equal to that produced by doctor-visits in `47016ab6e`. Hopefully the effect is duplicated in the synthetic data file, otherwise this will be way harder to debug.

**Context**

Since doctor-visits receives FIPS codes from the source, and not ZIP codes, it's clear that this problem is not due to the geo utility splitting ZIPs that overlap multiple counties. There is [a note in the pull request](https://github.com/cmu-delphi/covidcast-indicators/pull/401#discussion_r513817620) about a threshold that seems to control the number of megafips being reported; that threshold may just need adjusting.


Changed files:
doctor_visits/delphi_doctor_visits/geo_maps.py

diff --git a/doctor_visits/delphi_doctor_visits/geo_maps.py b/doctor_visits/delphi_doctor_visits/geo_maps.py
index 5d881e2a..3adccb69 100644
--- a/doctor_visits/delphi_doctor_visits/geo_maps.py
+++ b/doctor_visits/delphi_doctor_visits/geo_maps.py
@@ -123,53 +123,20 @@ class GeoMaps:
         return data.groupby("hrr"), "hrr"
 
     def county_to_megacounty(self, data, threshold_visits, threshold_len):
-        """A megacounty for a given day is all of the counties in a certain state who have:
-                 1) Denominator sum over <threshold_len> days below <threshold_visits>, or
-                 2) 0 denominator the last <min_recent_obs> days (not relevant for code
-                    because 0 denominator is not present)
+        """Convert to megacounty and groupby FIPS using GeoMapper package.
 
         Args:
             data: dataframe aggregated to the daily-county resolution (all 7 cols expected)
+            threshold_visits: count threshold to determine when to convert to megacounty.
+            threshold_len: number of days to use when thresholding.
 
         Returns: tuple of dataframe at the daily-state resolution, and geo_id column name
         """
-
-        dates = np.unique(data["ServiceDate"])
-        fipss = np.unique(data["PatCountyFIPS"])
-
-        # get denominator by day and location for all possible date-fips pairs
-        # this fills in 0 if unobserved
-        denom_dayloc = np.zeros((len(dates), len(fipss)))
-        by_fips = data.groupby("PatCountyFIPS")
-        for j, fips in enumerate(fipss):
-            denom_dayloc[:, j] = DoctorVisitsSensor.fill_dates(
-                by_fips.get_group(fips).set_index("ServiceDate"), dates
-            )["Denominator"].values
-
-        # get rolling sum across <threshold_len> days
-        num_recent_visits = np.concatenate(
-            (np.zeros((threshold_len, len(fipss))), np.cumsum(denom_dayloc, axis=0)),
-            axis=0,
-        )
-        num_recent_visits = (
-            num_recent_visits[threshold_len:] - num_recent_visits[:-threshold_len]
-        )
-        recent_visits_df = pd.DataFrame(
-            [
-                (dates[x[0]], fipss[x[1]], val)
-                for x, val in np.ndenumerate(num_recent_visits)
-            ],
-            columns=["ServiceDate", "PatCountyFIPS", "RecentVisits"],
-        )
-        data = data.merge(
-            recent_visits_df, how="left", on=["ServiceDate", "PatCountyFIPS"]
-        )
-
         data = self.gmpr.fips_to_megacounty(data,
                                             threshold_visits,
                                             threshold_len,
                                             fips_col="PatCountyFIPS",
-                                            thr_col="RecentVisits",
+                                            thr_col="Denominator",
                                             date_col="ServiceDate")
         data.rename({"megafips": "PatCountyFIPS"}, axis=1, inplace=True)
         return data.groupby("PatCountyFIPS"), "PatCountyFIPS"