Repository: mie-lab/trackintel
Pull request #218 (2021-05-19 22:32:32)

ENH: Location identifier OSNA method.
Added a second method to the location_identifier module.
OSNA uses weekday data divided in three time frames ["rest", "work", "leisure"]. Finds most popular home location for time frames "rest" and "leisure" and most popular "work" location for "work" time frame.

codecov[bot]: # [Codecov](https://codecov.io/gh/mie-lab/trackintel/pull/218?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab) Report
> Merging [#218](https://codecov.io/gh/mie-lab/trackintel/pull/218?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab) (c361fc3) into [master](https://codecov.io/gh/mie-lab/trackintel/commit/76bdb34fd7c0ba29ccd40cef0b0991d351066f19?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab) (76bdb34) will **decrease** coverage by `4.03%`.
> The diff coverage is `100.00%`.

> :exclamation: Current head c361fc3 differs from pull request most recent head 2bc16ae. Consider uploading reports for the commit 2bc16ae to get more accurate results
[![Impacted file tree graph](https://codecov.io/gh/mie-lab/trackintel/pull/218/graphs/tree.svg?width=650&height=150&src=pr&token=ITDZEEE1MO&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab)](https://codecov.io/gh/mie-lab/trackintel/pull/218?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab)

```diff
@@            Coverage Diff             @@
##           master     #218      +/-   ##
==========================================
- Coverage   94.89%   90.86%   -4.04%     
==========================================
  Files          37       37              
  Lines        1548     1576      +28     
  Branches      259      263       +4     
==========================================
- Hits         1469     1432      -37     
- Misses         41      106      +65     
  Partials       38       38              
```


| [Impacted Files](https://codecov.io/gh/mie-lab/trackintel/pull/218?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab) | Coverage Δ | |
|---|---|---|
| [trackintel/analysis/location\_identification.py](https://codecov.io/gh/mie-lab/trackintel/pull/218/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab#diff-dHJhY2tpbnRlbC9hbmFseXNpcy9sb2NhdGlvbl9pZGVudGlmaWNhdGlvbi5weQ==) | `98.93% <100.00%> (+0.45%)` | :arrow_up: |
| [trackintel/io/postgis.py](https://codecov.io/gh/mie-lab/trackintel/pull/218/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab#diff-dHJhY2tpbnRlbC9pby9wb3N0Z2lzLnB5) | `17.80% <0.00%> (-82.20%)` | :arrow_down: |
| [trackintel/model/trips.py](https://codecov.io/gh/mie-lab/trackintel/pull/218/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab#diff-dHJhY2tpbnRlbC9tb2RlbC90cmlwcy5weQ==) | `95.23% <0.00%> (-4.77%)` | :arrow_down: |
| [trackintel/model/locations.py](https://codecov.io/gh/mie-lab/trackintel/pull/218/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab#diff-dHJhY2tpbnRlbC9tb2RlbC9sb2NhdGlvbnMucHk=) | `96.00% <0.00%> (-4.00%)` | :arrow_down: |
| [trackintel/model/positionfixes.py](https://codecov.io/gh/mie-lab/trackintel/pull/218/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab#diff-dHJhY2tpbnRlbC9tb2RlbC9wb3NpdGlvbmZpeGVzLnB5) | `97.05% <0.00%> (-2.95%)` | :arrow_down: |
| [trackintel/model/triplegs.py](https://codecov.io/gh/mie-lab/trackintel/pull/218/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab#diff-dHJhY2tpbnRlbC9tb2RlbC90cmlwbGVncy5weQ==) | `97.22% <0.00%> (-2.78%)` | :arrow_down: |
| [trackintel/model/staypoints.py](https://codecov.io/gh/mie-lab/trackintel/pull/218/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab#diff-dHJhY2tpbnRlbC9tb2RlbC9zdGF5cG9pbnRzLnB5) | `97.22% <0.00%> (-2.78%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/mie-lab/trackintel/pull/218?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/mie-lab/trackintel/pull/218?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab). Last update [76bdb34...2bc16ae](https://codecov.io/gh/mie-lab/trackintel/pull/218?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab).

hong2223: Sorry for taking a long time. Thanks for the nice methods and fixes. I think we can now merge this.

Patch:
diff --git a/trackintel/analysis/__init__.py b/trackintel/analysis/__init__.py
index b1f1727..987be42 100644
--- a/trackintel/analysis/__init__.py
+++ b/trackintel/analysis/__init__.py
@@ -8,7 +8,7 @@ from .modal_split import calculate_modal_split
 
 from .location_identification import location_identifier
 from .location_identification import pre_filter_locations
-from .location_identification import freq_method
+from .location_identification import freq_method, osna_method
 
 __all__ = [
     "temporal_tracking_quality",
@@ -19,4 +19,5 @@ __all__ = [
     "location_identifier",
     "pre_filter_locations",
     "freq_method",
+    "osna_method",
 ]
diff --git a/trackintel/analysis/location_identification.py b/trackintel/analysis/location_identification.py
index 1022df0..32f3603 100644
--- a/trackintel/analysis/location_identification.py
+++ b/trackintel/analysis/location_identification.py
@@ -1,3 +1,4 @@
+import warnings
 import numpy as np
 import pandas as pd
 import trackintel as ti
@@ -11,11 +12,13 @@ def location_identifier(spts, method="FREQ", pre_filter=True, **pre_filter_kwarg
     spts : Geodataframe (as trackintel staypoints)
         Staypoints with column "location_id".
 
-    method : {'FREQ'}, default "FREQ"
+    method : {'FREQ', 'OSNA'}, default "FREQ"
         Choose which method to use.
 
-        - FREQ: "Generate a activity label per user by assigning the most visited location the label "home"
+        - FREQ: Generate an activity label per user by assigning the most visited location the label "home"
           and the second most visited location the label "work". The remaining locations get no label.
+        - OSNA: Use weekdays data divided in three time frames ["rest", "work", "leisure"]. Finds most popular
+        home location for timeframes "rest" and "leisure" and most popular "work" location for "work" timeframe.
 
     pre_filter : bool, default True
         Prefiltering the staypoints to exclude locations with not enough data.
@@ -64,6 +67,8 @@ def location_identifier(spts, method="FREQ", pre_filter=True, **pre_filter_kwarg
 
     if method == "FREQ":
         method_val = freq_method(spts[f], "home", "work")
+    elif method == "OSNA":
+        method_val = osna_method(spts[f])
     else:
         raise ValueError(f"Method {method} does not exist.")
 
@@ -207,7 +212,7 @@ def _freq_transform(group, *labels):
     Returns
     -------
     pd.Series
-        dtype : `object`
+        dtype : object
     """
     group_agg = group.groupby("location_id").agg({"duration": sum})
     group_agg["activity_label"] = _freq_assign(group_agg["duration"], *labels)
@@ -233,3 +238,97 @@ def _freq_assign(duration, *labels):
     label_array = np.full(len(duration), fill_value=None)
     label_array[kth] = labels
     return label_array
+
+
+def osna_method(spts):
+    """Find "home" location for timeframes "rest" and "leisure" and "work" location for "work" timeframe.
+
+    Use weekdays data divided in three time frames ["rest", "work", "leisure"] to generate location labels.
+    "rest" + "leisure" locations are weighted together. The location with the longest duration is assigned "home" label.
+    The longest "work" location is assigned "work" label.
+
+    Parameters
+    ----------
+    spts : GeoDataFrame (as trackintel staypoints)
+        Staypoints with the column "location_id"
+
+    Returns
+    -------
+    GeoDataFrame (as trackintel staypoints)
+        with additional column "activity_label"
+
+    Note
+    ----
+    The method is adapted from [1].
+    When "home" and "work" label overlap, the method selects the "work" location by the 2nd highest score.
+    The original algorithm count the distinct hours at a location as the home location is derived from
+    geo-tagged tweets. We directly sum the time spent at a location.
+
+    References
+    ----------
+    [1] Efstathiades, Hariton, Demetris Antoniades, George Pallis, and Marios Dikaiakos. 2015.
+    ‘Identification of Key Locations Based on Online Social Network Activity’.
+    In https://doi.org/10.1145/2808797.2808877.
+
+    Examples
+    --------
+    >>> staypoints = ti.analysis.osna_method(staypoints)
+
+    """
+    spts_in = spts  # no copy --> used to join back later.
+    spts = spts_in.copy()
+    spts["duration"] = spts["finished_at"] - spts["started_at"]
+    spts["mean_time"] = spts["started_at"] + spts["duration"] / 2
+
+    spts["label"] = spts["mean_time"].apply(_osna_label_timeframes)
+    spts.loc[spts["label"] == "rest", "duration"] *= 0.739  # weight given in paper
+    spts.loc[spts["label"] == "leisure", "duration"] *= 0.358  # weight given in paper
+
+    groups_map = {
+        "rest": "home",
+        "leisure": "home",
+        "work": "work",
+    }  # weekends aren't included in analysis!
+    # groupby user, location and label.
+    groups = ["user_id", "location_id", spts["label"].map(groups_map)]
+
+    spts_agg = spts.groupby(groups)["duration"].sum()
+    if spts_agg.empty:
+        warnings.warn("Got empty table in the osna method, check if the dates lie in weekends.")
+        spts_in["activity_label"] = pd.NA
+        return spts_in
+
+    # create a pivot table -> labels "home" and "work" as columns. ("user_id", "location_id" still in index.)
+    spts_pivot = spts_agg.unstack()
+    # get index of maximum for columns "work" and "home"
+    spts_idxmax = spts_pivot.groupby(["user_id"]).idxmax()
+    # first assign "home" label
+    spts_pivot.loc[spts_idxmax["home"], "activity_label"] = "home"
+
+    # The "home" label could overlap with the "work" label
+    # we set the rows where "home" is maximum to zero (pd.NaT) and recalculate index of work maximum.
+    redo_work = spts_idxmax[spts_idxmax["home"] == spts_idxmax["work"]]
+    spts_pivot.loc[redo_work["work"], "work"] = pd.NaT
+    spts_idxmax_work = spts_pivot.groupby(["user_id"])["work"].idxmax()
+    spts_pivot.loc[spts_idxmax_work, "activity_label"] = "work"
+
+    # now join it back together
+    sel = spts_in.columns != "activity_label"  # no overlap with older "activity_label"
+    return pd.merge(
+        spts_in.loc[:, sel],
+        spts_pivot["activity_label"],
+        how="left",
+        left_on=["user_id", "location_id"],
+        right_index=True,
+    )
+
+
+def _osna_label_timeframes(dt, weekend=[5, 6], start_rest=2, start_work=8, start_leisure=19):
+    """Help function to assign "weekend", "rest", "work", "leisure"."""
+    if dt.weekday() in weekend:
+        return "weekend"
+    if start_rest <= dt.hour < start_work:
+        return "rest"
+    if start_work <= dt.hour < start_leisure:
+        return "work"
+    return "leisure"


Test patch:
diff --git a/tests/analysis/test_location_identification.py b/tests/analysis/test_location_identification.py
index 7d47d72..6b31b34 100644
--- a/tests/analysis/test_location_identification.py
+++ b/tests/analysis/test_location_identification.py
@@ -1,19 +1,20 @@
 import datetime
 
 import geopandas as gpd
-from geopandas.testing import assert_geodataframe_equal
 import numpy as np
 import pandas as pd
 import pytest
-from shapely.geometry import Point
-
 import trackintel as ti
+from geopandas.testing import assert_geodataframe_equal
+from shapely.geometry import Point
 from trackintel.analysis.location_identification import (
-    pre_filter_locations,
+    _freq_assign,
     _freq_transform,
     freq_method,
-    _freq_assign,
     location_identifier,
+    pre_filter_locations,
+    osna_method,
+    _osna_label_timeframes,
 )
 
 
@@ -230,3 +231,166 @@ class TestLocation_Identifier:
         li = location_identifier(example_freq, method="FREQ", pre_filter=False)
         fr = freq_method(example_freq)
         assert_geodataframe_equal(li, fr)
+
+    def test_osna_method(self, example_osna):
+        """Test if function calls osna method correctly."""
+        li = location_identifier(example_osna, method="OSNA", pre_filter=False)
+        osna = osna_method(example_osna)
+        assert_geodataframe_equal(li, osna)
+
+
+@pytest.fixture
+def example_osna():
+    """Generate example staypoints with 3 location for 1 user within 3 different timeframes."""
+    weekday = "2021-05-19 "
+    t_rest = pd.Timestamp(weekday + "07:00:00", tz="utc")
+    t_work = pd.Timestamp(weekday + "18:00:00", tz="utc")
+    t_leisure = pd.Timestamp(weekday + "01:00:00", tz="utc")
+    h = pd.Timedelta("1h")
+
+    list_dict = [
+        {"user_id": 0, "location_id": 0, "started_at": t_rest},
+        {"user_id": 0, "location_id": 0, "started_at": t_leisure},
+        {"user_id": 0, "location_id": 0, "started_at": t_work},
+        {"user_id": 0, "location_id": 1, "started_at": t_rest},
+        {"user_id": 0, "location_id": 1, "started_at": t_work},
+        {"user_id": 0, "location_id": 1, "started_at": t_work},
+        {"user_id": 0, "location_id": 2, "started_at": t_leisure},
+    ]
+
+    p = Point(8.0, 47.0)  # geometry isn't used
+    for d in list_dict:
+        d["finished_at"] = d["started_at"] + h
+        d["geom"] = p
+
+    spts = gpd.GeoDataFrame(data=list_dict, geometry="geom", crs="EPSG:4326")
+    spts.index.name = "id"
+    assert "location_id" in spts.columns
+    assert spts.as_staypoints
+    return spts
+
+
+class TestOsna_Method:
+    """Test for the osna_method() function."""
+
+    def test_default(self, example_osna):
+        """Test with no changes to test data."""
+        osna = osna_method(example_osna)
+        example_osna.loc[example_osna["location_id"] == 0, "activity_label"] = "home"
+        example_osna.loc[example_osna["location_id"] == 1, "activity_label"] = "work"
+        assert_geodataframe_equal(example_osna, osna)
+
+    def test_overlap(self, example_osna):
+        """Test if overlap of home and work location the 2nd location is taken as work location."""
+        # add 2 work times to location 0,
+        # location 0 would be the most stayed location for both home and work
+        t = pd.Timestamp("2021-05-19 12:00:00", tz="utc")
+        h = pd.to_timedelta("1h")
+        p = Point(0.0, 0.0)
+        list_dict = [
+            {"user_id": 0, "location_id": 0, "started_at": t, "finished_at": t + h, "geom": p},
+            {"user_id": 0, "location_id": 0, "started_at": t, "finished_at": t + h, "geom": p},
+        ]
+        spts = gpd.GeoDataFrame(data=list_dict, geometry="geom", crs="EPSG:4326")
+        spts.index.name = "id"
+        spts = example_osna.append(spts)
+
+        result = osna_method(spts).iloc[:-2]
+        example_osna.loc[example_osna["location_id"] == 0, "activity_label"] = "home"
+        example_osna.loc[example_osna["location_id"] == 1, "activity_label"] = "work"
+        assert_geodataframe_equal(result, example_osna)
+
+    def test_only_weekends(self, example_osna):
+        """Test if an "empty df" warning rises if only weekends are included."""
+        weekend = "2021-05-22"  # a saturday
+
+        def _insert_weekend(dt, day=weekend):
+            """Take datetime and return new datetime with same time but new day."""
+            time = dt.time().strftime("%H:%M:%S")
+            new_dt = " ".join((day, time))
+            return pd.Timestamp(new_dt, tz=dt.tz)
+
+        # replace all days with weekends --> no label in data.
+        example_osna["started_at"] = example_osna["started_at"].apply(_insert_weekend)
+        example_osna["finished_at"] = example_osna["finished_at"].apply(_insert_weekend)
+
+        # check if warning is raised if all points are excluded (weekend)
+        with pytest.warns(UserWarning):
+            result = osna_method(example_osna)
+
+        # activity_label column is all pd.NA
+        example_osna["activity_label"] = pd.NA
+        assert_geodataframe_equal(result, example_osna)
+
+    def test_two_users(self, example_osna):
+        """Test if two users are handled correctly."""
+        two_user = example_osna.append(example_osna)
+        two_user.iloc[len(example_osna) :, 0] = 1  # second user gets id 1
+        result = osna_method(two_user)
+        two_user.loc[two_user["location_id"] == 0, "activity_label"] = "home"
+        two_user.loc[two_user["location_id"] == 1, "activity_label"] = "work"
+        assert_geodataframe_equal(result, two_user)
+
+    def test_leisure_weighting(self):
+        """Test if leisure has the weight given in the paper."""
+        weight_rest = 0.739
+        weight_leis = 0.358
+        ratio = weight_rest / weight_leis
+        ratio += 0.01  # tip the scale in favour of leisure
+        weekday = "2021-05-19 "
+        t_rest = pd.Timestamp(weekday + "07:00:00", tz="utc")
+        t_work = pd.Timestamp(weekday + "18:00:00", tz="utc")
+        t_leis = pd.Timestamp(weekday + "01:00:00", tz="utc")
+        h = pd.Timedelta("1h")
+
+        list_dict = [
+            {"user_id": 0, "location_id": 0, "started_at": t_rest, "finished_at": t_rest + h},
+            {"user_id": 0, "location_id": 1, "started_at": t_leis, "finished_at": t_leis + ratio * h},
+            {"user_id": 0, "location_id": 2, "started_at": t_work, "finished_at": t_work + h},
+        ]
+        p = Point(8.0, 47.0)  # geometry isn't used
+        for d in list_dict:
+            d["geom"] = p
+        spts = gpd.GeoDataFrame(data=list_dict, geometry="geom", crs="EPSG:4326")
+        spts.index.name = "id"
+        result = osna_method(spts)
+        spts.loc[spts["location_id"] == 1, "activity_label"] = "home"
+        spts.loc[spts["location_id"] == 2, "activity_label"] = "work"
+        assert_geodataframe_equal(spts, result)
+
+    def test_prior_activity_label(self, example_osna):
+        """Test that prior activity_label column does not corrupt output."""
+        example_osna["activity_label"] = np.arange(len(example_osna))
+        result = osna_method(example_osna)
+        del example_osna["activity_label"]
+        example_osna.loc[example_osna["location_id"] == 0, "activity_label"] = "home"
+        example_osna.loc[example_osna["location_id"] == 1, "activity_label"] = "work"
+        assert_geodataframe_equal(example_osna, result)
+
+
+class Test_osna_label_timeframes:
+    """Test for the _osna_label_timeframes() function."""
+
+    def test_weekend(self):
+        """Test if weekend only depends on day and not time."""
+        t1 = pd.Timestamp("2021-05-22 01:00:00")
+        t2 = pd.Timestamp("2021-05-22 07:00:00")
+        t3 = pd.Timestamp("2021-05-22 08:00:00")
+        t4 = pd.Timestamp("2021-05-22 20:00:00")
+        assert _osna_label_timeframes(t1) == "weekend"
+        assert _osna_label_timeframes(t2) == "weekend"
+        assert _osna_label_timeframes(t3) == "weekend"
+        assert _osna_label_timeframes(t4) == "weekend"
+
+    def test_weekday(self):
+        """Test the different labels on a weekday."""
+        t1 = pd.Timestamp("2021-05-20 01:00:00")
+        t2 = pd.Timestamp("2021-05-20 02:00:00")
+        t3 = pd.Timestamp("2021-05-20 08:00:00")
+        t4 = pd.Timestamp("2021-05-20 19:00:00")
+        t5 = pd.Timestamp("2021-05-20 18:59:59")
+        assert _osna_label_timeframes(t1) == "leisure"
+        assert _osna_label_timeframes(t2) == "rest"
+        assert _osna_label_timeframes(t3) == "work"
+        assert _osna_label_timeframes(t4) == "leisure"
+        assert _osna_label_timeframes(t5) == "work"