Repository: mie-lab/trackintel

Issue:
Examples for functions in geogr.distances
For #86 we should provide examples for all external visible functions.
There are missing examples for
- `calculate_distance_matrix`
- `meters_to_decimal_degrees`

they can be found [here](https://github.com/mie-lab/trackintel/blob/b89f0c8c35cc1adcc858a8bdc60a32d32b50a2b9/trackintel/geogr/distances.py#L15)

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/mie-lab/trackintel/pull/345?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab) Report
> Merging [#345](https://codecov.io/gh/mie-lab/trackintel/pull/345?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab) (d947d42) into [master](https://codecov.io/gh/mie-lab/trackintel/commit/5cd2b9a492984969dd6c8d8a6ca6dbd4b31e45ad?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab) (5cd2b9a) will **decrease** coverage by `2.23%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/mie-lab/trackintel/pull/345/graphs/tree.svg?width=650&height=150&src=pr&token=ITDZEEE1MO&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab)](https://codecov.io/gh/mie-lab/trackintel/pull/345?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab)

```diff
@@            Coverage Diff             @@
##           master     #345      +/-   ##
==========================================
- Coverage   96.41%   94.17%   -2.24%     
==========================================
  Files          40       40              
  Lines        1924     1924              
  Branches      313      313              
==========================================
- Hits         1855     1812      -43     
- Misses         31       75      +44     
+ Partials       38       37       -1     
```


| [Impacted Files](https://codecov.io/gh/mie-lab/trackintel/pull/345?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab) | Coverage Δ | |
|---|---|---|
| [trackintel/geogr/distances.py](https://codecov.io/gh/mie-lab/trackintel/pull/345/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab#diff-dHJhY2tpbnRlbC9nZW9nci9kaXN0YW5jZXMucHk=) | `93.20% <ø> (ø)` | |
| [trackintel/io/postgis.py](https://codecov.io/gh/mie-lab/trackintel/pull/345/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab#diff-dHJhY2tpbnRlbC9pby9wb3N0Z2lzLnB5) | `49.36% <0.00%> (-48.11%)` | :arrow_down: |
| [trackintel/model/locations.py](https://codecov.io/gh/mie-lab/trackintel/pull/345/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab#diff-dHJhY2tpbnRlbC9tb2RlbC9sb2NhdGlvbnMucHk=) | `96.87% <0.00%> (-3.13%)` | :arrow_down: |
| [trackintel/model/trips.py](https://codecov.io/gh/mie-lab/trackintel/pull/345/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab#diff-dHJhY2tpbnRlbC9tb2RlbC90cmlwcy5weQ==) | `97.22% <0.00%> (-2.78%)` | :arrow_down: |
| [trackintel/model/positionfixes.py](https://codecov.io/gh/mie-lab/trackintel/pull/345/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab#diff-dHJhY2tpbnRlbC9tb2RlbC9wb3NpdGlvbmZpeGVzLnB5) | `97.82% <0.00%> (-2.18%)` | :arrow_down: |
| [trackintel/model/staypoints.py](https://codecov.io/gh/mie-lab/trackintel/pull/345/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab#diff-dHJhY2tpbnRlbC9tb2RlbC9zdGF5cG9pbnRzLnB5) | `98.14% <0.00%> (-1.86%)` | :arrow_down: |
| [trackintel/model/triplegs.py](https://codecov.io/gh/mie-lab/trackintel/pull/345/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab#diff-dHJhY2tpbnRlbC9tb2RlbC90cmlwbGVncy5weQ==) | `98.27% <0.00%> (-1.73%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/mie-lab/trackintel/pull/345?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/mie-lab/trackintel/pull/345?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab). Last update [5cd2b9a...d947d42](https://codecov.io/gh/mie-lab/trackintel/pull/345?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=mie-lab).


Patch:
diff --git a/docs/modules/preprocessing.rst b/docs/modules/preprocessing.rst
index cbf76da..707b33c 100644
--- a/docs/modules/preprocessing.rst
+++ b/docs/modules/preprocessing.rst
@@ -31,6 +31,11 @@ frequently visits and/or infer if they correspond to activities.
 
 .. autofunction:: trackintel.preprocessing.staypoints.generate_locations
 
+Due to tracking artifacts, it can occur that one activity is split into several staypoints. 
+We can aggregate the staypoints horizontally that are close in time and at the same location.
+
+.. autofunction:: trackintel.preprocessing.staypoints.merge_staypoints
+
 Triplegs
 ========
 
diff --git a/trackintel/geogr/distances.py b/trackintel/geogr/distances.py
index 58ee163..2c428a7 100644
--- a/trackintel/geogr/distances.py
+++ b/trackintel/geogr/distances.py
@@ -55,6 +55,10 @@ def calculate_distance_matrix(X, Y=None, dist_metric="haversine", n_jobs=0, **kw
     D: np.array
         matrix of shape (len(X), len(X)) or of shape (len(X), len(Y)) if Y is provided.
 
+    Examples
+    --------
+    >>> calculate_distance_matrix(staypoints, dist_metric="haversine")
+    >>> calculate_distance_matrix(triplegs_1, triplegs_2, dist_metric="dtw")
     """
     geom_type = X.geometry.iat[0].geom_type
     if Y is None:
@@ -179,6 +183,10 @@ def meters_to_decimal_degrees(meters, latitude):
     -------
     float
         An approximation of a distance (given in meters) in degrees.
+
+    Examples
+    --------
+    >>> meters_to_decimal_degrees(500.0, 47.410)
     """
     return meters / (111.32 * 1000.0 * cos(latitude * (pi / 180.0)))
 
@@ -205,7 +213,6 @@ def check_gdf_planar(gdf, transform=False):
     gdf : GeoDataFrame
         if transform is True, return the re-projected gdf.
 
-
     Examples
     --------
     >>> from trackintel.geogr.distances import check_gdf_planar
diff --git a/trackintel/io/file.py b/trackintel/io/file.py
index f444527..879b322 100644
--- a/trackintel/io/file.py
+++ b/trackintel/io/file.py
@@ -1,15 +1,11 @@
 import warnings
-from geopandas.geodataframe import GeoDataFrame
 
-import numpy as np
-import dateutil
-import dateutil.parser
 import geopandas as gpd
+import numpy as np
 import pandas as pd
 import pytz
-import warnings
+from geopandas.geodataframe import GeoDataFrame
 from shapely import wkt
-from shapely import geometry
 from shapely.geometry import Point
 
 
@@ -24,6 +20,9 @@ def read_positionfixes_csv(*args, columns=None, tz=None, index_col=object(), crs
 
     Parameters
     ----------
+    args
+        Arguments as passed to pd.read_csv().
+
     columns : dict, optional
         The column names to rename in the format {'old_name':'trackintel_standard_name'}.
         The required columns for this function include: "user_id", "tracked_at", "latitude"
@@ -41,6 +40,9 @@ def read_positionfixes_csv(*args, columns=None, tz=None, index_col=object(), crs
         by pyproj.CRS.from_user_input(), such as an authority string
         (eg 'EPSG:4326') or a WKT string.
 
+    kwargs
+        Additional keyword arguments passed to pd.read_csv().
+
     Returns
     -------
     pfs : GeoDataFrame (as trackintel positionfixes)
@@ -116,10 +118,20 @@ def write_positionfixes_csv(positionfixes, filename, *args, **kwargs):
     filename : str
         The file to write to.
 
+    args
+        Additional arguments passed to pd.DataFrame.to_csv().
+
+    kwargs
+        Additional keyword arguments passed to pd.DataFrame.to_csv().
+
     Notes
     -----
     "longitude" and "latitude" is extracted from the geometry column and the orignal
     geometry column is dropped.
+
+    Examples
+    ---------
+    >>> ps.as_positionfixes.to_csv("export_pfs.csv")
     """
     gdf = positionfixes.copy()
     gdf["longitude"] = positionfixes.geometry.apply(lambda p: p.coords[0][0])
@@ -139,6 +151,9 @@ def read_triplegs_csv(*args, columns=None, tz=None, index_col=object(), crs=None
 
     Parameters
     ----------
+    args
+        Arguments as passed to pd.read_csv().
+
     columns : dict, optional
         The column names to rename in the format {'old_name':'trackintel_standard_name'}.
         The required columns for this function include: "user_id", "started_at", "finished_at"
@@ -156,6 +171,9 @@ def read_triplegs_csv(*args, columns=None, tz=None, index_col=object(), crs=None
         by pyproj.CRS.from_user_input(), such as an authority string
         (eg “EPSG:4326”) or a WKT string.
 
+    kwargs
+        Additional keyword arguments passed to pd.read_csv().
+
     Returns
     -------
     tpls : GeoDataFrame (as trackintel triplegs)
@@ -209,7 +227,7 @@ def write_triplegs_csv(triplegs, filename, *args, **kwargs):
     """
     Write triplegs to csv file.
 
-    Wraps the pandas to_csv function, but transforms the geom into WKT
+    Wraps the pandas to_csv function, but transforms the geometry into WKT
     before writing.
 
     Parameters
@@ -219,6 +237,16 @@ def write_triplegs_csv(triplegs, filename, *args, **kwargs):
 
     filename : str
         The file to write to.
+
+    args
+        Additional arguments passed to pd.DataFrame.to_csv().
+
+    kwargs
+        Additional keyword arguments passed to pd.DataFrame.to_csv().
+
+    Examples
+    --------
+    >>> tpls.as_triplegs.to_csv("export_tpls.csv")
     """
     geo_col_name = triplegs.geometry.name
     df = pd.DataFrame(triplegs, copy=True)
@@ -237,6 +265,9 @@ def read_staypoints_csv(*args, columns=None, tz=None, index_col=object(), crs=No
 
     Parameters
     ----------
+    args
+        Arguments as passed to pd.read_csv().
+
     columns : dict, optional
         The column names to rename in the format {'old_name':'trackintel_standard_name'}.
         The required columns for this function include: "user_id", "started_at", "finished_at"
@@ -254,6 +285,9 @@ def read_staypoints_csv(*args, columns=None, tz=None, index_col=object(), crs=No
         by pyproj.CRS.from_user_input(), such as an authority string
         (eg “EPSG:4326”) or a WKT string.
 
+    kwargs
+        Additional keyword arguments passed to pd.read_csv().
+
     Returns
     -------
     sp : GeoDataFrame (as trackintel staypoints)
@@ -307,7 +341,7 @@ def write_staypoints_csv(staypoints, filename, *args, **kwargs):
     """
     Write staypoints to csv file.
 
-    Wraps the pandas to_csv function, but transforms the geom into WKT
+    Wraps the pandas to_csv function, but transforms the geometry into WKT
     before writing.
 
     Parameters
@@ -317,6 +351,16 @@ def write_staypoints_csv(staypoints, filename, *args, **kwargs):
 
     filename : str
         The file to write to.
+
+    args
+        Additional arguments passed to pd.DataFrame.to_csv().
+
+    kwargs
+        Additional keyword arguments passed to pd.DataFrame.to_csv().
+
+    Examples
+    --------
+    >>> tpls.as_triplegs.to_csv("export_tpls.csv")
     """
     geo_col_name = staypoints.geometry.name
     df = pd.DataFrame(staypoints, copy=True)
@@ -335,6 +379,9 @@ def read_locations_csv(*args, columns=None, index_col=object(), crs=None, **kwar
 
     Parameters
     ----------
+    args
+        Arguments as passed to pd.read_csv().
+
     columns : dict, optional
         The column names to rename in the format {'old_name':'trackintel_standard_name'}.
         The required columns for this function include: "user_id" and "center".
@@ -348,6 +395,9 @@ def read_locations_csv(*args, columns=None, index_col=object(), crs=None, **kwar
         by pyproj.CRS.from_user_input(), such as an authority string
         (eg “EPSG:4326”) or a WKT string.
 
+    kwargs
+        Additional keyword arguments passed to pd.read_csv().
+
     Returns
     -------
     locs : GeoDataFrame (as trackintel locations)
@@ -404,6 +454,16 @@ def write_locations_csv(locations, filename, *args, **kwargs):
 
     filename : str
         The file to write to.
+
+    args
+        Additional arguments passed to pd.DataFrame.to_csv().
+
+    kwargs
+        Additional keyword arguments passed to pd.DataFrame.to_csv().
+
+    Examples
+    --------
+    >>> locs.as_locations.to_csv("export_locs.csv")
     """
     df = pd.DataFrame(locations, copy=True)
     df["center"] = locations["center"].apply(wkt.dumps)
@@ -422,6 +482,9 @@ def read_trips_csv(*args, columns=None, tz=None, index_col=object(), **kwargs):
 
     Parameters
     ----------
+    args
+        Arguments as passed to pd.read_csv().
+
     columns : dict, optional
         The column names to rename in the format {'old_name':'trackintel_standard_name'}.
         The required columns for this function include: "user_id", "started_at",
@@ -435,6 +498,9 @@ def read_trips_csv(*args, columns=None, tz=None, index_col=object(), **kwargs):
         column name to be used as index. If None the default index is assumed
         as unique identifier.
 
+    kwargs
+        Additional keyword arguments passed to pd.read_csv().
+
     Returns
     -------
     trips : (Geo)DataFrame (as trackintel trips)
@@ -494,6 +560,7 @@ def write_trips_csv(trips, filename, *args, **kwargs):
     Write trips to csv file.
 
     Wraps the pandas to_csv function.
+    Geometry get transformed to WKT before writing.
 
     Parameters
     ----------
@@ -502,6 +569,16 @@ def write_trips_csv(trips, filename, *args, **kwargs):
 
     filename : str
         The file to write to.
+
+    args
+        Additional arguments passed to pd.DataFrame.to_csv().
+
+    kwargs
+        Additional keyword arguments passed to pd.DataFrame.to_csv().
+
+    Examples
+    --------
+    >>> trips.as_trips.to_csv("export_trips.csv")
     """
     df = trips.copy()
     if isinstance(df, GeoDataFrame):
@@ -520,16 +597,26 @@ def read_tours_csv(*args, columns=None, index_col=object(), tz=None, **kwargs):
 
     Parameters
     ----------
+    args
+        Arguments as passed to pd.read_csv().
+
     columns : dict, optional
         The column names to rename in the format {'old_name':'trackintel_standard_name'}.
 
     tz : str, optional
         pytz compatible timezone string. If None UTC is assumed.
 
+    kwargs
+        Additional keyword arguments passed to pd.read_csv().
+
     Returns
     -------
     tours : DataFrame (as trackintel tours)
         A DataFrame containing the tours.
+
+    Examples
+    --------
+    >>> trackintel.read_tours_csv('data.csv', columns={'uuid':'user_id'})
     """
     columns = {} if columns is None else columns
 
@@ -571,6 +658,16 @@ def write_tours_csv(tours, filename, *args, **kwargs):
 
     filename : str
         The file to write to.
+
+    args
+        Additional arguments passed to pd.DataFrame.to_csv().
+
+    kwargs
+        Additional keyword arguments passed to pd.DataFrame.to_csv().
+
+    Examples
+    --------
+    >>> tours.as_tours.to_csv("export_tours.csv")
     """
     tours.to_csv(filename, index=True, *args, **kwargs)
 
diff --git a/trackintel/preprocessing/staypoints.py b/trackintel/preprocessing/staypoints.py
index 679c565..ed8837e 100644
--- a/trackintel/preprocessing/staypoints.py
+++ b/trackintel/preprocessing/staypoints.py
@@ -233,7 +233,8 @@ def merge_staypoints(staypoints, triplegs, max_time_gap="10min", agg={}):
     Parameters
     ----------
     staypoints : GeoDataFrame (as trackintel staypoints)
-        The staypoints have to follow the standard definition for staypoints DataFrames.
+        The staypoints must contain a column `location_id` (see `generate_locations` function) and have to follow the
+        standard trackintel definition for staypoints DataFrames.
 
     triplegs: GeoDataFrame (as trackintel triplegs)
         The triplegs have to follow the standard definition for triplegs DataFrames.
@@ -277,6 +278,7 @@ def merge_staypoints(staypoints, triplegs, max_time_gap="10min", agg={}):
     # otherwise check if it's a Timedelta already, and raise error if not
     elif not isinstance(max_time_gap, pd.Timedelta):
         raise TypeError("Parameter max_time_gap must be either of type String or pd.Timedelta!")
+    assert "location_id" in staypoints.columns, "Staypoints must contain column location_id"
 
     sp_merge = staypoints.copy()
     index_name = staypoints.index.name
@@ -285,6 +287,9 @@ def merge_staypoints(staypoints, triplegs, max_time_gap="10min", agg={}):
     tpls_merge = triplegs.copy()
     tpls_merge["type"] = "tripleg"
     sp_merge["type"] = "staypoint"
+    # convert datatypes in order to preserve the datatypes (especially ints) despite of NaNs during concat
+    sp_merge = sp_merge.convert_dtypes()
+
     # a joined dataframe sp_tpls is constructed to add the columns 'type' and 'next_type' to the 'sp_merge' table
     # concat and sort by time
     sp_tpls = pd.concat([sp_merge, tpls_merge]).sort_values(by=["user_id", "started_at"])
@@ -327,7 +332,13 @@ def merge_staypoints(staypoints, triplegs, max_time_gap="10min", agg={}):
         cond_old = cond.copy()
 
     # Staypoint-required columnsare aggregated in the following manner:
-    agg_dict = {index_name: "first", "user_id": "first", "started_at": "first", "finished_at": "last"}
+    agg_dict = {
+        index_name: "first",
+        "user_id": "first",
+        "started_at": "first",
+        "finished_at": "last",
+        "location_id": "first",
+    }
     # User-defined further aggregation
     agg_dict.update(agg)
 


Test patch:
diff --git a/tests/preprocessing/test_staypoints.py b/tests/preprocessing/test_staypoints.py
index 79a8745..10c4425 100644
--- a/tests/preprocessing/test_staypoints.py
+++ b/tests/preprocessing/test_staypoints.py
@@ -399,7 +399,8 @@ class TestMergeStaypoints:
         """Test staypoint merging."""
         sp, tpls = example_staypoints_merge
         # first test with empty tpls
-        merged_sp = sp.as_staypoints.merge_staypoints(tpls, agg={"geom": "first", "location_id": "first"})
+        merged_sp = sp.as_staypoints.merge_staypoints(tpls, agg={"geom": "first"})
+        merged_sp = merged_sp.reindex(columns=sp.columns)
         assert len(merged_sp) == len(sp) - 3
         # some staypoints stay the same (not merged)
         assert (merged_sp.loc[1] == sp.loc[1]).all()
@@ -458,3 +459,11 @@ class TestMergeStaypoints:
         # the geom should correspond to the first one
         assert sp.loc[7, "geom"] == merged_sp.loc[7, "geom"]
         assert sp.loc[2, "geom"] == merged_sp.loc[2, "geom"]
+
+    def test_merge_staypoints_error(self, example_staypoints_merge):
+        sp, tpls = example_staypoints_merge
+        sp.drop(columns=["location_id"], inplace=True)
+        with pytest.raises(AssertionError) as excinfo:
+            _ = sp.as_staypoints.merge_staypoints(tpls)
+
+        assert "Staypoints must contain column location_id" in str(excinfo.value)