Repository: M3Works/metloom (language: python)

Issue:
Snotel functionality



Discussion / hints:

Fix patch:
diff --git a/.travis.yml b/.travis.yml
deleted file mode 100644
index aaba487..0000000
--- a/.travis.yml
+++ /dev/null
@@ -1,15 +0,0 @@
-# Config file for automatic testing at travis-ci.com
-
-language: python
-python:
-  - 3.8
-  - 3.7
-  - 3.6
-
-# Command to install dependencies, e.g. pip install -r requirements.txt --use-mirrors
-install: pip install -U tox-travis
-
-# Command to run tests, e.g. python setup.py test
-script: tox
-
-
diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
index 8db90e1..4925257 100644
--- a/CONTRIBUTING.rst
+++ b/CONTRIBUTING.rst
@@ -15,7 +15,7 @@ Types of Contributions
 Report Bugs
 ~~~~~~~~~~~
 
-Report bugs at https://github.com/micah-prime/dataloom/issues.
+Report bugs at https://github.com/M3Works/dataloom/issues.
 
 If you are reporting a bug, please include:
 
@@ -45,7 +45,7 @@ articles, and such.
 Submit Feedback
 ~~~~~~~~~~~~~~~
 
-The best way to send feedback is to file an issue at https://github.com/micah-prime/dataloom/issues.
+The best way to send feedback is to file an issue at https://github.com/M3Works/dataloom/issues.
 
 If you are proposing a feature:
 
@@ -103,7 +103,7 @@ Before you submit a pull request, check that it meets these guidelines:
    your new functionality into a function with a docstring, and add the
    feature to the list in README.rst.
 3. The pull request should work for Python 3.5, 3.6, 3.7 and 3.8, and for PyPy. Check
-   https://travis-ci.com/micah-prime/dataloom/pull_requests
+   https://travis-ci.com/M3Works/dataloom/pull_requests
    and make sure that the tests pass for all supported Python versions.
 
 Tips
diff --git a/README.rst b/README.rst
index b865de2..c76e5f6 100644
--- a/README.rst
+++ b/README.rst
@@ -6,27 +6,106 @@ dataloom
 .. image:: https://img.shields.io/pypi/v/dataloom.svg
         :target: https://pypi.python.org/pypi/dataloom
 
-.. image:: https://img.shields.io/travis/micah-prime/dataloom.svg
-        :target: https://travis-ci.com/micah-prime/dataloom
+.. image:: https://img.shields.io/github/workflow/status/M3Works/dataloom/dataloom
+    :target: https://github.com/M3Works/dataloom/actions/workflows/testing.yml
+    :alt: Testing Status
 
 .. image:: https://readthedocs.org/projects/dataloom/badge/?version=latest
         :target: https://dataloom.readthedocs.io/en/latest/?version=latest
         :alt: Documentation Status
+.. image:: https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/micah-prime/04da387b53bdb4a3aa31253789550a9f/raw/dataloom__heads_main.json
+        :target: https://github.com/M3Works/dataloom
+        :alt: Code Coverage
 
 
+Location Oriented Observed Meteorology
 
+dataloom is a python library created with the goal of consistent, simple sampling of
+meteorology and snow related point measurments from a variety of datasources accross the
+Western US.
 
-Location Oriented Observed Meteorology
+Warning - This software is provided as is (see the license), so use at your own risk.
+This is an opensource package with the goal of making data wrangling easier. We make
+no guarantees about the quality or accuracy of the data and any interpretation of the meaning
+of the data is up to you.
 
 
 * Free software: BSD license
-* Documentation: https://dataloom.readthedocs.io.
 
 
 Features
 --------
 
-* TODO
+* Sampling of daily, hourly, and snow course data
+* Searching for stations from a datasource within a shapefile
+* Current data sources:
+    * `CDEC <https://cdec.water.ca.gov/>`_
+    * `SNOTEL <https://www.nrcs.usda.gov/wps/portal/wcc/home/dataAccessHelp/webService/webServiceReference/>`_
+
+Requirements
+-------
+python >= 3.7
+
+Install
+-------
+.. code-block:: bash
+
+    python3 -m pip install dataloom
+
+
+Local install for dev
+-------
+The recommendation is to use virtualenv, but other local python
+environment isolation tools will work (pipenv, conda)
+.. code-block:: bash
+    python3 -m pip install -r requirements_dev
+    python3 setup.py install
+
+Testing
+-------
+
+.. code-block:: bash
+
+    pytest
+
+If contributing to the codebase, code coverage should not decrease
+from the contributions. Make sure to check code coverage before
+opening a pull request.
+
+.. code-block:: bash
+
+    pytest --cov=dataloom
+
+Documentation
+-------
+readthedocs coming soon
+
+https://dataloom.readthedocs.io.
+
+Extending classes for your own variables
+-------
+Not all of the available variables for each datasource are implemented
+within this package. It is easy to extend the classes to add more variables
+
+.. code-block:: python
+
+    from datetime import datetime
+    from dataloom.variables import CDECStationVariables:
+    from dataloom.pointdata import CDECPointData
+
+
+    class MyVariables(CDEcStationVariables):
+        DEWPT = SensorDescription("36", "Dew Point", "TEMPERATURE, DEW POINT")
+
+
+    class MyCDECPointData(CDECPointData):
+        ALLOWED_VARIABLES = MyVariables
+
+
+    MyCDECPointData("TNY", "Tenaya Lake").get_daily_data(
+        datetime(2020, 1, 3), datetime(2020, 1, 7), [MyVariables.DEWPT]
+    )
+
 
 Credits
 -------
diff --git a/dataloom/__init__.py b/dataloom/__init__.py
index 8513c81..4db57fe 100644
--- a/dataloom/__init__.py
+++ b/dataloom/__init__.py
@@ -1,5 +1,5 @@
 """Top-level package for dataloom."""
 
 __author__ = """M3Works"""
-__email__ = 'm3worksllc@gmail.com'
-__version__ = '0.1.0'
+__email__ = "m3worksllc@gmail.com"
+__version__ = "0.1.0"
diff --git a/dataloom/cli.py b/dataloom/cli.py
deleted file mode 100644
index c9d2fe5..0000000
--- a/dataloom/cli.py
+++ /dev/null
@@ -1,19 +0,0 @@
-"""Console script for dataloom."""
-import argparse
-import sys
-
-
-def main():
-    """Console script for dataloom."""
-    parser = argparse.ArgumentParser()
-    parser.add_argument('_', nargs='*')
-    args = parser.parse_args()
-
-    print("Arguments: " + str(args._))
-    print("Replace this message by putting your code into "
-          "dataloom.cli.main")
-    return 0
-
-
-if __name__ == "__main__":
-    sys.exit(main())  # pragma: no cover
diff --git a/dataloom/dataframe_utils.py b/dataloom/dataframe_utils.py
new file mode 100644
index 0000000..61dea90
--- /dev/null
+++ b/dataloom/dataframe_utils.py
@@ -0,0 +1,54 @@
+from logging import getLogger
+from typing import Optional
+
+import pandas as pd
+
+LOG = getLogger("dataloom.dataframe_utils")
+
+
+def join_df(
+    df: Optional[pd.DataFrame], new_df: Optional[pd.DataFrame], how="left", on=None
+):
+    """
+    join two dataframes handling None
+    Args:
+        df: optional dataframe
+        new_df: optional dataframe
+        how: method for merging
+        on: optional kwarg for DataFrame.join
+
+    Returns:
+        The joined dataframes. This method prefers values from the first if
+        columns are overlapping and renames the overlapping values from
+        the `new_df` to <column>_unused
+    """
+    if df is None:
+        result_df = new_df
+    elif new_df is None:
+        result_df = df
+    else:
+        try:
+            result_df = df.join(new_df, how=how, on=on, rsuffix="_unused")
+        except Exception as e:
+            LOG.error("failed joining dataframes.")
+            raise e
+
+    return result_df
+
+
+def append_df(df: Optional[pd.DataFrame], new_df: Optional[pd.DataFrame]):
+    """
+    append 2 dfs handling Nones
+    Args:
+        df: optional dataframe
+        new_df: optional dataframe
+    Returns:
+        dataframe or None
+    """
+    if df is None:
+        result_df = new_df
+    elif new_df is None:
+        result_df = df
+    else:
+        result_df = df.append(new_df)
+    return result_df
diff --git a/dataloom/pointdata/__init__.py b/dataloom/pointdata/__init__.py
new file mode 100644
index 0000000..242189d
--- /dev/null
+++ b/dataloom/pointdata/__init__.py
@@ -0,0 +1,5 @@
+from .base import PointData, PointDataCollection
+from .cdec import CDECPointData
+from .snotel import SnotelPointData
+
+__all__ = ["PointData", "PointDataCollection", "CDECPointData", "SnotelPointData"]
diff --git a/dataloom/pointdata/base.py b/dataloom/pointdata/base.py
new file mode 100644
index 0000000..c05d7fd
--- /dev/null
+++ b/dataloom/pointdata/base.py
@@ -0,0 +1,242 @@
+from datetime import datetime
+from typing import List
+
+import geopandas as gpd
+import pandas as pd
+import logging
+
+from ..variables import VariableBase, SensorDescription
+
+LOG = logging.getLogger("dataloom.pointdata.base")
+
+
+class PointDataCollection:
+    """
+    Iterator class for a collection of PointData objects.
+    This allows conversion to a GeoDataFrame
+    """
+
+    def __init__(self, points: List[object] = None):
+        """
+
+        Args:
+            points: List of point data objects
+        """
+        self.points = points or []
+        self._index = 0
+
+    def add_point(self, point):
+        """
+        Append point to collection of PointData objects
+
+        Args:
+            point: PointData object
+        """
+        self.points.append(point)
+
+    def to_dataframe(self):
+        """
+        Returns:
+            GeoDataFrame of points. Columns are ['name', 'id', 'geometry']
+        """
+        names = []
+        ids = []
+        meta = []
+        for point in self.points:
+            names += [point.name]
+            ids += [point.id]
+            meta += [point.metadata]
+
+        obj = {"name": names, "id": ids}
+        return gpd.GeoDataFrame.from_dict(obj, geometry=meta)
+
+    def __len__(self):
+        return len(self.points)
+
+    def __iter__(self):
+        for item in self.points:
+            yield item
+
+
+class PointData(object):
+    ALLOWED_VARIABLES = VariableBase
+    ITERATOR_CLASS = PointDataCollection
+
+    def __init__(self, station_id, name, metadata=None):
+        """
+
+        Args:
+            station_id: code used within datasource API to access station
+            name: station name. This will be used in the GeoDataFrames
+            metadata: Optional shapely point. This will bypass the
+                _get_metadata method if provided
+        """
+        self.id = station_id
+        self.name = name
+        self._metadata = metadata
+        self.desired_tzinfo = "UTC"
+
+    def get_daily_data(
+        self,
+        start_date: datetime,
+        end_date: datetime,
+        variables: List[SensorDescription],
+    ):
+        """
+        Get daily measurement data
+        Args:
+            start_date: datetime object for start of data collection period
+            end_date: datetime object for end of data collection period
+            variables: List of dataloom.variables.SensorDescription object
+                from self.ALLOWED_VARIABLES
+        Returns:
+            GeoDataFrame of data. The dataframe should be indexed on
+            ['datetime', 'site'] and have columns
+            ['geometry', 'site', 'measurementDate']. Additionally, for each
+            variables, it should have column f'{variable.name}' and
+            f'{variable.name}_UNITS'
+            See CDECPointData._get_data for example implementation and
+            TestCDECStation.tny_daily_expected for example dataframe.
+            Datetimes should be in UTC
+        """
+        raise NotImplementedError("get_daily_data is not implemented")
+
+    def get_hourly_data(
+        self,
+        start_date: datetime,
+        end_date: datetime,
+        variables: List[SensorDescription],
+    ):
+        """
+        Get hourly measurement data
+        Args:
+            start_date: datetime object for start of data collection period
+            end_date: datetime object for end of data collection period
+            variables: List of dataloom.variables.SensorDescription object
+                from self.ALLOWED_VARIABLES
+        Returns:
+            GeoDataFrame of data. The dataframe should be indexed on
+            ['datetime', 'site'] and have columns
+            ['geometry', 'site', 'measurementDate']. Additionally, for each
+            variables, it should have column f'{variable.name}' and
+            f'{variable.name}_UNITS'
+            See CDECPointData._get_data for example implementation and
+            TestCDECStation.tny_daily_expected for example dataframe.
+            Datetimes should be in UTC
+        """
+        raise NotImplementedError("get_hourly_data is not implemented")
+
+    def get_snow_course_data(
+        self,
+        start_date: datetime,
+        end_date: datetime,
+        variables: List[SensorDescription],
+    ):
+        """
+        Get snow course data
+        Args:
+            start_date: datetime object for start of data collection period
+            end_date: datetime object for end of data collection period
+            variables: List of dataloom.variables.SensorDescription object
+                from self.ALLOWED_VARIABLES
+        Returns:
+            GeoDataFrame of data. The dataframe should be indexed on
+            ['datetime', 'site'] and have columns
+            ['geometry', 'site', 'measurementDate']. Additionally, for each
+            variables, it should have column f'{variable.name}' and
+            f'{variable.name}_UNITS'
+            See CDECPointData._get_data for example implementation and
+            TestCDECStation.tny_daily_expected for example dataframe.
+            Datetimes should be in UTC
+        """
+        raise NotImplementedError("get_snow_course_data is not implemented")
+
+    def _get_metadata(self):
+        """
+        Method to get a shapely Point object to describe the
+        Returns:
+            shapely.point.Point object in Longitude, Latitude
+        """
+        raise NotImplementedError("_get_metadata is not implemented")
+
+    def _handle_df_tz(self, val):
+        """
+        Covert one entry from a df from cls.TZINFO to UTC
+        """
+        if pd.isna(val):
+            return val
+        else:
+            local = val.tz_localize(self.tzinfo)
+            return local.tz_convert(self.desired_tzinfo)
+
+    @property
+    def tzinfo(self):
+        """
+        tzinfo that pandas can use for tz_localize
+        """
+        return self._tzinfo
+
+    @property
+    def metadata(self):
+        """
+        metadata property
+        Returns:
+            shapely.point.Point object in Longitude, Latitude with z in ft
+        """
+        if self._metadata is None:
+            self._metadata = self._get_metadata()
+        return self._metadata
+
+    def points_from_geometry(
+        self,
+        geometry: gpd.GeoDataFrame,
+        variables: List[SensorDescription],
+        snow_courses=False,
+    ):
+        """
+        Find a collection of points with measurements for certain variables
+        contained within a shapefile. Any point in the shapefile with
+        measurements for any of the variables should be included
+        Args:
+            geometry: GeoDataFrame for shapefile from gpd.read_file
+            variables: List of SensorDescription
+            snow_courses: boolean for including only snowcourse data or no
+                snowcourse data
+        Returns:
+            PointDataCollection
+        """
+        raise NotImplementedError("points_from_geometry not implemented")
+
+    def validate_sensor_df(self, gdf: gpd.GeoDataFrame):
+        """
+        Validate that the GeoDataFrame returned is formatted correctly.
+        The goal of this method is to ensure base classes are returning a
+        consistent format of dataframe
+        """
+        expected_columns = ["measurementDate", "geometry"]
+        expected_indexes = ["datetime", "site"]
+        assert isinstance(gdf, gpd.GeoDataFrame)
+        columns = gdf.columns
+        index_names = gdf.index.names
+        # check for required indexes
+        for ei in expected_indexes:
+            assert ei in index_names
+        # check for expected columns
+        for column in expected_columns:
+            assert column in columns
+        remaining_columns = [c for c in columns if c not in expected_columns]
+        # make sure all variables have a units column as well
+        for rc in remaining_columns:
+            if "_units" not in rc:
+                assert f"{rc}_units" in remaining_columns
+
+    def __repr__(self):
+        return f"{self.__class__.__name__}({self.id!r}, {self.name!r})"
+
+    def __str__(self):
+        return f"{self.name} ({self.id})"
+
+    def __eq__(self, other):
+        if not isinstance(other, self.__class__):
+            return False
+        return self.id == other.id and self.name == other.name
diff --git a/dataloom/pointdata/cdec.py b/dataloom/pointdata/cdec.py
new file mode 100644
index 0000000..ea89852
--- /dev/null
+++ b/dataloom/pointdata/cdec.py
@@ -0,0 +1,342 @@
+from datetime import datetime
+from typing import List
+
+import geopandas as gpd
+import numpy as np
+import pandas as pd
+import pytz
+import requests
+import logging
+
+from .base import PointData
+from ..variables import CdecStationVariables, SensorDescription
+from ..dataframe_utils import join_df
+
+LOG = logging.getLogger("dataloom.pointdata.cdec")
+
+
+class CDECPointData(PointData):
+    """
+    Implement PointData methods for CDEC data source
+    API documentation here https://cdec.water.ca.gov/dynamicapp/
+    """
+
+    ALLOWED_VARIABLES = CdecStationVariables
+    CDEC_URL = "http://cdec.water.ca.gov/dynamicapp/req/JSONDataServlet"
+    META_URL = "http://cdec.water.ca.gov/cdecstation2/CDecServlet/" "getStationInfo"
+
+    def __init__(self, station_id, name, metadata=None):
+        """
+        See docstring for PointData.__init__
+        """
+        super(CDECPointData, self).__init__(station_id, name, metadata=metadata)
+        self._raw_metadata = None
+        self._tzinfo = pytz.timezone("US/Pacific")
+
+    def _get_all_metadata(self):
+        """
+        Get all the raw metadata for a station. This is a list of sensor
+        descriptions for the station
+        Returns:
+            A list of dictionaries describing the sensors at a station
+        """
+        if self._raw_metadata is None:
+            resp = requests.get(self.META_URL, params={"stationID": self.id})
+            resp.raise_for_status()
+            self._raw_metadata = resp.json()["STATION"]
+        return self._raw_metadata
+
+    def is_only_snow_course(self):
+        """
+        Determine if a station only has snow course measurements
+        """
+        data = self._get_all_metadata()
+        manual_check = [d["DUR_CODE"] == "M" for d in data if d["SENS_GRP"] == "snow"]
+        result = False
+        if len(manual_check) > 0 and all(manual_check):
+            result = True
+        if result and not self.is_only_monthly():
+            # This would happen if all snow sensors have code "M"
+            # but there are other hourly or daily sensors
+            raise Exception(
+                f"We have not accounted for this scenario. Please talk to "
+                f"a Micah about how {self.id} violates their assumptions."
+            )
+        return result
+
+    def is_partly_snow_course(self):
+        """
+        Determine if any of the snow sensors at a station are on a monthly
+        interval
+        Assumption: Monthly snow sensor measurements are snow courses
+        """
+        data = self._get_all_metadata()
+        return any([d["DUR_CODE"] == "M" for d in data if d["SENS_GRP"] == "snow"])
+
+    def is_only_monthly(self):
+        """
+        determine if all sensors for a station are on a monthly interval
+        """
+        data = self._get_all_metadata()
+        manual_check = [d["DUR_CODE"] == "M" for d in data]
+        if len(manual_check) > 0 and all(manual_check):
+            return True
+        return False
+
+    def _get_metadata(self):
+        """
+        See docstring for PointData._get_metadata
+        """
+        data = self._get_all_metadata()
+        # TODO: Should this be sensor specific?
+        metadata_by_name = {d["SENS_LONG_NAME"]: d for d in data}
+        # default to the first sensor
+        chosen_sensor_data = data[0]
+        # try to replace it with a desired sensor
+        # TODO: Change this
+        for choice in [
+            "SNOW, WATER CONTENT",
+            "SNOW DEPTH",
+            "PRECIPITATION, ACCUMULATED",
+        ]:
+            if metadata_by_name.get(choice) is not None:
+                chosen_sensor_data = metadata_by_name[choice]
+                break
+
+        return gpd.points_from_xy(
+            [chosen_sensor_data["LONGITUDE"]],
+            [chosen_sensor_data["LATITUDE"]],
+            z=[chosen_sensor_data["ELEVATION"]],
+        )[0]
+
+    def _data_request(self, params):
+        """
+        Make get request to CDEC and return JSON
+        Args:
+            params: dictionary of request parameters
+        Returns:
+            dictionary of response values
+        """
+        resp = requests.get(self.CDEC_URL, params=params)
+        resp.raise_for_status()
+        return resp.json()
+
+    def _sensor_response_to_df(self, response_data, sensor, final_columns):
+        """
+        Convert the response data from the API to a GeoDataFrame
+        Format and map columns in the dataframe
+        Args:
+            response_data: JSON list response from CDEC API
+            sensor: SensorDescription obj
+            final_columns: List of columns used for filtering
+        Returns:
+            GeoDataFrame
+        """
+        sensor_df = gpd.GeoDataFrame.from_dict(
+            response_data,
+            geometry=[self.metadata] * len(response_data),
+        )
+        sensor_df.replace(-9999.0, np.nan, inplace=True)
+        # this mapping is important. Sometimes obsDate is null
+        sensor_df.rename(
+            columns={
+                "date": "datetime",
+                "obsDate": "measurementDate",
+                "value": sensor.name,
+                "units": f"{sensor.name}_units",
+                "stationId": "site",
+            },
+            inplace=True,
+        )
+        final_columns += [sensor.name, f"{sensor.name}_units"]
+        sensor_df["datetime"] = pd.to_datetime(sensor_df["datetime"])
+        sensor_df["measurementDate"] = pd.to_datetime(sensor_df["measurementDate"])
+        sensor_df["datetime"] = sensor_df["datetime"].apply(self._handle_df_tz)
+        sensor_df["measurementDate"] = sensor_df["measurementDate"].apply(
+            self._handle_df_tz
+        )
+        # set index so joinng works
+        sensor_df.set_index("datetime", inplace=True)
+        sensor_df = sensor_df.filter(final_columns)
+        return sensor_df
+
+    def _get_data(
+        self,
+        start_date: datetime,
+        end_date: datetime,
+        variables: List[SensorDescription],
+        duration: str,
+    ):
+        """
+        Args:
+            start_date: datetime object for start of data collection period
+            end_date: datetime object for end of data collection period
+            variables: List of dataloom.variables.SensorDescription object
+                from self.ALLOWED_VARIABLES
+            duration: CDEC duration code ['M', 'H', 'D']
+        Returns:
+            GeoDataFrame of data, indexed on datetime, site
+        """
+        params = {
+            "Stations": self.id,
+            "dur_code": duration,
+            "Start": start_date.isoformat(),
+            "End": end_date.isoformat(),
+        }
+        df = None
+        final_columns = ["geometry", "site", "measurementDate"]
+        for sensor in variables:
+            params["SensorNums"] = sensor.code
+            response_data = self._data_request(params)
+            if response_data:
+                sensor_df = self._sensor_response_to_df(
+                    response_data, sensor, final_columns
+                )
+                df = join_df(df, sensor_df)
+
+        if df is not None and len(df.index) > 0:
+            df.reset_index(inplace=True)
+            df.set_index(keys=["datetime", "site"], inplace=True)
+            df.index.set_names(["datetime", "site"], inplace=True)
+        self.validate_sensor_df(df)
+        return df
+
+    def get_daily_data(
+        self,
+        start_date: datetime,
+        end_date: datetime,
+        variables: List[SensorDescription],
+    ):
+        """
+        See docstring for PointData.get_daily_data
+        Example query:
+        https://cdec.water.ca.gov/dynamicapp/req/JSONDataServlet?
+        Stations=TNY&SensorNums=3&dur_code=D&Start=2021-05-16&End=2021-05-16
+        """
+        return self._get_data(start_date, end_date, variables, "D")
+
+    def get_hourly_data(
+        self,
+        start_date: datetime,
+        end_date: datetime,
+        variables: List[SensorDescription],
+    ):
+        """
+        See docstring for PointData.get_hourly_data
+        """
+        return self._get_data(start_date, end_date, variables, "H")
+
+    def get_snow_course_data(
+        self,
+        start_date: datetime,
+        end_date: datetime,
+        variables: List[SensorDescription],
+    ):
+        """
+        See docstring for PointData.get_snow_course_data
+        """
+        if not self.is_partly_snow_course():
+            raise ValueError(f"{self.id} is not a snow course")
+        return self._get_data(start_date, end_date, variables, "M")
+
+    @staticmethod
+    def _station_sensor_search(
+        bounds, sensor: SensorDescription, dur=None, collect=None
+    ):
+        """
+        Station search form https://cdec.water.ca.gov/dynamicapp/staSearch?
+        Search for stations using the CDEC station search utility
+        Args:
+            bounds: dictionary of Longitude and Latitidue bounds with keys
+                minx, maxx, miny, maxy
+            sensor: SensorDescription object
+            dur: optional CDEC duration code ['M', 'H', 'D']
+            collect: optional CDEC collection type string i.e. 'MANUAL+ENTRY'
+        Returns:
+            Pandas Dataframe of table result or None if no table found
+
+        """
+        # TODO: do we want this buffer?
+        buffer = 0.00
+        dur_str = f"&dur_chk=on&dur={dur}" if dur else "&dur="
+        collect_str = (
+            f"&collect_chk=on&collect={collect}"
+            if collect
+            else "&collect=NONE+SPECIFIED"
+        )
+        url = (
+            f"https://cdec.water.ca.gov/dynamicapp/staSearch?sta="
+            f"&sensor_chk=on&sensor={sensor.code}"
+            f"{collect_str}"
+            f"{dur_str}"
+            f"&active_chk=on&active=Y"
+            f"&loc_chk=on"
+            f"&lon1={bounds['minx']-buffer}&lon2={bounds['maxx']+buffer}"
+            f"&lat1={bounds['miny']-buffer}&lat2={bounds['maxy']+buffer}"
+            f"&elev1=-5&elev2=99000&nearby=&basin=NONE+SPECIFIED"
+            f"&hydro=NONE+SPECIFIED&county=NONE+SPECIFIED&agency_num=160"
+            f"&display=sta"
+        )
+        try:
+            return pd.read_html(url)[0]
+        except ValueError:
+            LOG.error(f"No tables for {url}")
+            return None
+
+    @classmethod
+    def points_from_geometry(
+        cls,
+        geometry: gpd.GeoDataFrame,
+        variables: List[SensorDescription],
+        snow_courses=False,
+    ):
+        """
+        See docstring for PointData.points_from_geometry
+        """
+        # Assume station search result is in 4326
+        projected_geom = geometry.to_crs(4326)
+        bounds = projected_geom.bounds.iloc[0]
+        search_df = None
+        station_search_kwargs = {}
+        # Filter to manual, monthly measurements if looking for snow courses
+        if snow_courses:
+            station_search_kwargs["dur"] = "M"
+            station_search_kwargs["collect"] = "MANUAL+ENTRY"
+        for variable in variables:
+            result_df = cls._station_sensor_search(
+                bounds, variable, **station_search_kwargs
+            )
+            if result_df is not None:
+                result_df["index_id"] = result_df["ID"]
+                result_df.set_index("index_id")
+                search_df = join_df(search_df, result_df, how="outer")
+        # return empty collection if we didn't find any points
+        if search_df is None:
+            return cls.ITERATOR_CLASS([])
+        gdf = gpd.GeoDataFrame(
+            search_df,
+            geometry=gpd.points_from_xy(
+                search_df["Longitude"],
+                search_df["Latitude"],
+                z=search_df["ElevationFeet"],
+            ),
+        )
+        # filter to points within shapefile
+        # TODO: do we want to make this optional?
+        filtered_gdf = gdf[gdf.within(projected_geom.iloc[0]["geometry"])]
+
+        points = [
+            cls(row[0], row[1], metadata=row[2])
+            for row in zip(
+                filtered_gdf["ID"],
+                filtered_gdf["Station Name"],
+                filtered_gdf["geometry"],
+            )
+        ]
+        # filter to snow courses or not snowcourses depending on desired result
+        if snow_courses:
+            return cls.ITERATOR_CLASS([p for p in points if p.is_partly_snow_course()])
+        else:
+            return cls.ITERATOR_CLASS(
+                [p for p in points if not p.is_only_snow_course()]
+            )
diff --git a/dataloom/pointdata/snotel.py b/dataloom/pointdata/snotel.py
new file mode 100644
index 0000000..3ae565a
--- /dev/null
+++ b/dataloom/pointdata/snotel.py
@@ -0,0 +1,293 @@
+from datetime import datetime, timezone, timedelta
+from typing import List
+
+import logging
+import geopandas as gpd
+import pandas as pd
+from climata import snotel
+from climata.base import FilterOpt, DateOpt, as_list
+from wq.io.exceptions import NoData
+
+from .base import PointData
+from ..variables import SnotelVariables, VariableBase, SensorDescription
+from ..dataframe_utils import join_df
+
+
+LOG = logging.getLogger("dataloom.pointdata.snotel")
+
+
+class LoomStationIO(snotel.StationIO):
+    """
+    Extend climata.snotel.StationIO to allow filtering based on network
+    and longitude
+    """
+
+    max_longitude = FilterOpt(url_param="maxLongitude")
+    min_longitude = FilterOpt(url_param="minLongitude")
+    networks = FilterOpt(url_param="networkCds")
+
+
+class MonthlyDataIO(snotel.SnotelIO):
+    """
+    Extend climata.snotel.SnotelIO for sampling SEMIMONTHLY data.
+    This allows sampling of snowcourses
+    """
+
+    data_function = "getData"
+
+    # Applicable WebserviceLoader default options
+    station = FilterOpt(required=True, url_param="stationTriplets")
+    parameter = FilterOpt(required=True, url_param="elementCd")
+    start_date = DateOpt(required=True, url_param="beginDate")
+    end_date = DateOpt(required=True, url_param="endDate")
+
+    default_params = {
+        "ordinal": 1,
+        "duration": "SEMIMONTHLY",
+        "getFlags": "true",
+        "alwaysReturnDailyFeb29": "false",
+    }
+
+    def parse(self):
+        data = self.data[0]
+        if not data or "values" not in data:
+            raise NoData
+        dates = as_list(data["collectionDates"])
+        vals = as_list(data["values"])
+        flags = as_list(data["flags"])
+
+        self.data = [
+            {
+                "date": datetime.strptime(date_val, "%Y-%m-%d").date(),
+                "value": val,
+                "flag": flag,
+            }
+            for date_val, val, flag in zip(dates, vals, flags)
+            if date_val is not None
+        ]
+
+
+class StationMonthlyDataIO(snotel.StationDataIO):
+    """
+    Extension of climata.snotel.StationDataIO to allow snowcourse sampling
+    """
+
+    inner_io_class = MonthlyDataIO
+    duration = "SEMIMONTHLY"
+
+
+class SnotelPointData(PointData):
+    """
+    Implement PointData methods for SNOTEL data source
+    API documentation here:
+    https://www.nrcs.usda.gov/wps/portal/wcc/home/dataAccessHelp/webService/webServiceReference/
+    Website has variable and network codes
+    Possible testing resource:
+    https://wcc.sc.egov.usda.gov/awdbWebService/webservice/testwebservice.jsf?webserviceName=/awdbWebService
+    """
+
+    ALLOWED_VARIABLES = SnotelVariables
+
+    def __init__(self, station_id, name, metadata=None):
+        """
+        See docstring for PointData.__init__
+        """
+        super(SnotelPointData, self).__init__(station_id, name, metadata=metadata)
+        self._raw_metadata = None
+        self._tzinfo = None
+
+    def _snotel_response_to_df(
+        self,
+        response: snotel.StationDataIO,
+        variables: List[VariableBase],
+        date_key="date",
+    ):
+        """
+        Convert the response from climata.snotel classes into
+        Args:
+            response: climata.snotel data response
+            variables: list of variables to extract
+            date_key: which key the response uses to store date(time)s
+        """
+        df = None
+        # TODO: possible DRY oportunity here too
+        final_columns = ["geometry", "site", "measurementDate"]
+        for param in response:
+            for variable in variables:
+                if param.elementcd == variable.code:
+                    data = param.data
+                    transformed = [
+                        {
+                            "datetime": getattr(v, date_key),
+                            "site": self.id,
+                            "measurementDate": getattr(v, date_key),
+                            variable.name: v.value,
+                            f"{variable.name}_units": param.storedunitcd,
+                        }
+                        for v in data
+                    ]
+                    final_columns += [variable.name, f"{variable.name}_units"]
+                    sensor_df = gpd.GeoDataFrame.from_dict(
+                        transformed, geometry=[self.metadata] * len(transformed)
+                    )
+                    # TODO: possibly an opportunity for DRY here (see CDEC)
+                    sensor_df["datetime"] = pd.to_datetime(sensor_df["datetime"])
+                    sensor_df["measurementDate"] = pd.to_datetime(
+                        sensor_df["measurementDate"]
+                    )
+                    sensor_df["datetime"] = sensor_df["datetime"].apply(
+                        self._handle_df_tz
+                    )
+                    sensor_df["measurementDate"] = sensor_df["measurementDate"].apply(
+                        self._handle_df_tz
+                    )
+                    # set index so joining works
+                    sensor_df.set_index("datetime", inplace=True)
+                    sensor_df = sensor_df.filter(final_columns)
+                    df = join_df(df, sensor_df)
+
+        if df is not None and len(df.index) > 0:
+            df.reset_index(inplace=True)
+            df.set_index(keys=["datetime", "site"], inplace=True)
+            df.index.set_names(["datetime", "site"], inplace=True)
+        self.validate_sensor_df(df)
+        return df
+
+    def get_daily_data(
+        self,
+        start_date: datetime,
+        end_date: datetime,
+        variables: List[SensorDescription],
+    ):
+        """
+        See docstring for PointData.get_daily_data
+        """
+        params = snotel.StationDailyDataIO(
+            station=self.id,
+            start_date=start_date.date().isoformat(),
+            end_date=end_date.date().isoformat(),
+        )
+        return self._snotel_response_to_df(params, variables)
+
+    def get_hourly_data(
+        self,
+        start_date: datetime,
+        end_date: datetime,
+        variables: List[SensorDescription],
+    ):
+        """
+        See docstring for PointData.get_hourly_data
+        """
+        params = snotel.StationHourlyDataIO(
+            station=self.id,
+            start_date=start_date.date().isoformat(),
+            end_date=end_date.date().isoformat(),
+        )
+        return self._snotel_response_to_df(params, variables, date_key="datetime")
+
+    def get_snow_course_data(
+        self,
+        start_date: datetime,
+        end_date: datetime,
+        variables: List[SensorDescription],
+    ):
+        """
+        See docstring for PointData.get_snow_course_data
+        """
+        params = StationMonthlyDataIO(
+            station=self.id,
+            start_date=start_date.date().isoformat(),
+            end_date=end_date.date().isoformat(),
+        )
+        return self._snotel_response_to_df(params, variables, date_key="date")
+
+    def _get_all_metadata(self):
+        """
+        Set _raw_metadata once using climata.snotel
+        """
+        if self._raw_metadata is None:
+            self._raw_metadata = snotel.StationMetaIO(station=self.id)
+        return self._raw_metadata
+
+    def _get_metadata(self):
+        """
+        See docstring for PointData._get_metadata
+        """
+        all_metadata = self._get_all_metadata()
+        data = all_metadata[0]
+        return gpd.points_from_xy(
+            [data.longitude], [data.latitude], z=[data.elevation]
+        )[0]
+
+    def _get_tzinfo(self):
+        """
+        Return timezone info that pandas can use from the raw_metadata
+        """
+        all_metadata = self._get_all_metadata()
+        metadata = all_metadata[0]
+        # Snow courses might not have a timezone attached
+        if hasattr(metadata, "stationdatatimezone"):
+            tz_hours = metadata.stationdatatimezone
+        else:
+            LOG.error(f"Could not find timezone info for {self.id} ({self.name})")
+            tz_hours = 0
+        return timezone(timedelta(hours=tz_hours))
+
+    @property
+    def tzinfo(self):
+        if self._tzinfo is None:
+            self._tzinfo = self._get_tzinfo()
+        return self._tzinfo
+
+    @classmethod
+    def points_from_geometry(
+        cls,
+        geometry: gpd.GeoDataFrame,
+        variables: List[SensorDescription],
+        snow_courses=False,
+    ):
+        """
+        See docstring for PointData.points_from_geometry
+        """
+        projected_geom = geometry.to_crs(4326)
+        bounds = projected_geom.bounds.iloc[0]
+        # TODO: network may need to change to get streamflow
+        network = "SNOW" if snow_courses else "SNTL"
+        df = None
+        for variable in variables:
+            # this search is default AND on all parameters
+            # so search for each variable seperately
+            response = LoomStationIO(
+                max_latitude=bounds["maxy"],
+                min_latitude=bounds["miny"],
+                max_longitude=bounds["maxx"],
+                min_longitude=bounds["minx"],
+                networks=network,  # only snotel, not
+                parameter=variable.code,
+                # ordinals=  # TODO: what are ordinals?
+            )
+            if len(response.data) > 0:
+                result_df = pd.DataFrame.from_records(response.data)
+                result_df["index_id"] = result_df["stationTriplet"]
+                df = join_df(df, result_df, how="outer")
+        # TODO: possible DRYing out here
+
+        if df is None:
+            return cls.ITERATOR_CLASS([])
+
+        gdf = gpd.GeoDataFrame(
+            df,
+            geometry=gpd.points_from_xy(
+                df["longitude"], df["latitude"], z=df["elevation"]
+            ),
+        )
+        filtered_gdf = gdf[gdf.within(projected_geom.iloc[0]["geometry"])]
+        points = [
+            cls(row[0], row[1], metadata=row[2])
+            for row in zip(
+                filtered_gdf["stationTriplet"],
+                filtered_gdf["name"],
+                filtered_gdf["geometry"],
+            )
+        ]
+        return cls.ITERATOR_CLASS(points)
diff --git a/dataloom/variables.py b/dataloom/variables.py
new file mode 100644
index 0000000..926c1ba
--- /dev/null
+++ b/dataloom/variables.py
@@ -0,0 +1,90 @@
+from dataclasses import dataclass
+
+
+@dataclass
+class SensorDescription:
+    """
+    data class for describing a snow sensor
+    """
+
+    code: str = "-1"  # code used within the applicable API
+    name: str = "basename"  # desired name for the sensor
+    description: str = None  # description of the sensor
+    accumulated: bool = False  # whether or not the data is accumulated
+
+
+class VariableBase:
+    """
+    Base class to store all variables for a specific datasource. Each
+    datasource should implement the class. The goal is that the variables
+    are synonymous across implementations.(i.e. PRECIPITATION should have the
+    same meaning in each implementation).
+    Variables in this base class should ideally be implemented by all classes
+    and cannot be directly used from the base class.
+    """
+
+    PRECIPITATION = SensorDescription()
+    SWE = SensorDescription()
+    SNOWDEPTH = SensorDescription()
+
+    @staticmethod
+    def _validate_sensor(sensor: SensorDescription):
+        """
+        Validate that a sensor is not using the default values since they
+        are meaningless
+        """
+        default = SensorDescription()
+        if sensor.name == default.name and sensor.code == default.code:
+            raise ValueError(f"{sensor.name} is the default implementation")
+
+    @classmethod
+    def from_code(cls, code):
+        """
+        Get the correct sensor description from the code
+        """
+        for k, v in cls.__dict__.items():
+            if isinstance(v, SensorDescription) and v.code == str(code):
+                cls._validate_sensor(v)
+                return v
+        raise ValueError(f"Could not find sensor for code {code}")
+
+
+class CdecStationVariables(VariableBase):
+    """
+    Available sensors from CDEC.
+    Exhaustive list:
+    http://cdec4gov.water.ca.gov/reportapp/javareports?name=SensList
+    """
+
+    PRECIPITATIONACCUM = SensorDescription(
+        "2", "ACCUMULATED PRECIPITATION", "PRECIPITATION, ACCUMULATED", True
+    )
+    PRECIPITATION = SensorDescription(
+        "45", "PRECIPITATION", "PRECIPITATION, INCREMENTAL", False
+    )
+    SNOWDEPTH = SensorDescription("18", "SNOWDEPTH", "SNOW DEPTH")
+    SWE = SensorDescription("3", "SWE", "SNOW, WATER CONTENT", True)
+    TEMP = SensorDescription("4", "AIR TEMP", "TEMPERATURE, AIR")
+    TEMPAVG = SensorDescription("30", "AVG AIR TEMP", "TEMPERATURE, AIR AVERAGE")
+    TEMPMIN = SensorDescription("32", "MIN AIR TEMP", "TEMPERATURE, AIR MINIMUM")
+    TEMPMAX = SensorDescription("31", "MAX AIR TEMP", "TEMPERATURE, AIR MAXIMUM")
+
+
+class SnotelVariables(VariableBase):
+    """
+    Available sensors from SNOTEL
+    """
+
+    SNOWDEPTH = SensorDescription("SNWD", "SNOWDEPTH")
+    SWE = SensorDescription("WTEQ", "SWE")
+    TEMP = SensorDescription("TOBS", "AIR TEMP")
+    TEMPAVG = SensorDescription("TAVG", "AVG AIR TEMP",
+                                "AIR TEMPERATURE AVERAGE")
+    TEMPMIN = SensorDescription("TMIN", "MIN AIR TEMP",
+                                "AIR TEMPERATURE MINIMUM")
+    TEMPMAX = SensorDescription("TMAX", "MAX AIR TEMP",
+                                "AIR TEMPERATURE MAXIMUM")
+    PRECIPITATION = SensorDescription("PRCPSA", "PRECIPITATON",
+                                      "PRECIPITATION INCREMENT SNOW-ADJUSTED")
+    PRECIPITATIONACCUM = SensorDescription("PREC", "ACCUMULATED PRECIPITATION",
+                                           "PRECIPITATION ACCUMULATION")
diff --git a/docs/installation.rst b/docs/installation.rst
index 32f7646..b5be08d 100644
--- a/docs/installation.rst
+++ b/docs/installation.rst
@@ -32,13 +32,13 @@ You can either clone the public repository:
 
 .. code-block:: console
 
-    $ git clone git://github.com/micah-prime/dataloom
+    $ git clone git://github.com/M3Works/dataloom
 
 Or download the `tarball`_:
 
 .. code-block:: console
 
-    $ curl -OJL https://github.com/micah-prime/dataloom/tarball/master
+    $ curl -OJL https://github.com/M3Works/dataloom/tarball/master
 
 Once you have a copy of the source, you can install it with:
 
@@ -47,5 +47,5 @@ Once you have a copy of the source, you can install it with:
     $ python setup.py install
 
 
-.. _Github repo: https://github.com/micah-prime/dataloom
-.. _tarball: https://github.com/micah-prime/dataloom/tarball/master
+.. _Github repo: https://github.com/M3Works/dataloom
+.. _tarball: https://github.com/M3Works/dataloom/tarball/master
diff --git a/requirements_dev.txt b/requirements_dev.txt
index ebb3f1d..7baa24c 100644
--- a/requirements_dev.txt
+++ b/requirements_dev.txt
@@ -1,12 +1,13 @@
-pip==19.2.3
+pip==21.2.4
 bump2version==0.5.11
 wheel==0.33.6
 watchdog==0.9.0
 flake8==3.7.8
 tox==3.14.0
-coverage==4.5.4
+coverage==5.5
 Sphinx==1.8.5
 twine==1.14.0
 
 pytest==6.2.4
 black==21.7b0
+pytest-cov==2.12.1
diff --git a/setup.py b/setup.py
index 6c25193..b52c05b 100644
--- a/setup.py
+++ b/setup.py
@@ -10,28 +10,32 @@
 with open('HISTORY.rst') as history_file:
     history = history_file.read()
 
-requirements = [ ]
+requirements = [
+    'geopandas>=0.9.0,<1.0.0',
+    'lxml==4.6.3',
+    'requests>2.0.0,<3.0.0',
+    'climata==0.5.0'
+]
 
 test_requirements = ['pytest>=3', ]
 
 setup(
     author="M3Works",
     author_email='m3worksllc@gmail.com',
-    python_requires='>=3.6',
+    python_requires='>=3.7',
     classifiers=[
         'Development Status :: 2 - Pre-Alpha',
         'Intended Audience :: Developers',
         'License :: OSI Approved :: BSD License',
         'Natural Language :: English',
         'Programming Language :: Python :: 3',
-        'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
         'Programming Language :: Python :: 3.8',
+        'Programming Language :: Python :: 3.9',
     ],
-    description="Location Oriented Observed Meteorology",
+    description="Location Oriented Observed Meteorology (LOOM)",
     entry_points={
         'console_scripts': [
-            'dataloom=dataloom.cli:main',
         ],
     },
     install_requires=requirements,
@@ -43,7 +47,7 @@
     packages=find_packages(include=['dataloom', 'dataloom.*']),
     test_suite='tests',
     tests_require=test_requirements,
-    url='https://github.com/micah-prime/dataloom',
+    url='https://github.com/M3Works/dataloom',
     version='0.1.0',
     zip_safe=False,
 )
diff --git a/tox.ini b/tox.ini
index e62c3cd..5b627c1 100644
--- a/tox.ini
+++ b/tox.ini
@@ -3,9 +3,9 @@ envlist = py36, py37, py38, flake8
 
 [travis]
 python =
+    3.9: py39
     3.8: py38
     3.7: py37
-    3.6: py36
 
 [testenv:flake8]
 basepython = python


Test patch:
diff --git a/.github/workflows/testing.yml b/.github/workflows/testing.yml
new file mode 100644
index 0000000..1b2d7d5
--- /dev/null
+++ b/.github/workflows/testing.yml
@@ -0,0 +1,85 @@
+# Configuration for unit testing
+# Built from example here https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
+name: Testing and Coverage
+
+on:
+  pull_request:
+    branches: [ main, integration ]
+    paths-ignore:
+        - '**.md'
+        - '**.rst'
+
+jobs:
+    flake8:
+        runs-on: ubuntu-latest
+        steps:
+            -   uses: actions/checkout@v2
+
+            -   name: Set up Python
+                uses: actions/setup-python@v2
+                with:
+                    python-version: '3.7'
+
+            -   name: Install dependencies
+                run: |
+                    python3 -m pip install --upgrade pip
+                    python3 -m pip install flake8
+            -   name: Lint with flake8
+                run: |
+                    # stop the build if there are Python syntax errors or undefined names
+                    flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
+                    # use flake8 to check formatting, based on example here https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html
+                    flake8 . --count --select=C,E,F,W,B,B950 --max-complexity=10 --max-line-length=88 --statistics --extend-ignore=E203
+    test:
+        needs: [flake8]
+        runs-on: ubuntu-latest
+        strategy:
+          matrix:
+            python-version: [3.7, 3.8, 3.9]
+
+        steps:
+          - uses: actions/checkout@v2
+          - name: Set up Python ${{ matrix.python-version }}
+            uses: actions/setup-python@v2
+            with:
+              python-version: ${{ matrix.python-version }}
+          - name: Install dependencies
+            run: |
+                python3 -m pip install -r requirements_dev.txt
+                python3 -m pip install .
+          - name: Test with pytest
+            run: |
+              pytest -s
+          - if: ${{ matrix.python-version == '3.9' }}
+            name: Extract coverage
+            run: |
+                # Run coverage save the results
+                python3 -m pytest --cov=dataloom --cov-fail-under=80
+                SUMMARY=`coverage report -m | grep TOTAL`
+                # Save results as ENV var
+                COVERAGE=$(python -c "print('$SUMMARY'.split(' ')[-1])")
+                echo "COVERAGE=$COVERAGE" >> $GITHUB_ENV
+                # var REF = 'refs/pull/27/merge.json';
+                REF=${{ github.ref }}
+                # console.log('github.ref: ' + REF);
+                echo "github.ref: $REF"
+                # var PATHS = REF.split('/');
+                IFS='/' read -ra PATHS <<< "$REF"
+                # var BRANCH_NAME = PATHS[1] + PATHS[2];
+                BRANCH_NAME="${PATHS[1]}_${PATHS[2]}"
+                # console.log(BRANCH_NAME); // 'pull_27'
+                echo $BRANCH_NAME
+                # process.env.BRANCH = 'pull_27';
+                echo "BRANCH=$(echo ${BRANCH_NAME})" >> $GITHUB_ENV
+          - if: ${{ matrix.python-version == '3.9' }}
+            name: Create the Badge
+            uses: schneegans/dynamic-badges-action@v1.0.0
+            with:
+                auth: ${{ secrets.GIST_SECRET }}
+                gistID: 04da387b53bdb4a3aa31253789550a9f
+                filename: dataloom__${{ env.BRANCH }}.json
+                label: Coverage
+                message: ${{ env.COVERAGE }}
+                color: green
+
+
diff --git a/tests/data/testing.cpg b/tests/data/testing.cpg
new file mode 100644
index 0000000..cd89cb9
--- /dev/null
+++ b/tests/data/testing.cpg
@@ -0,0 +1,1 @@
+ISO-8859-1
\ No newline at end of file
diff --git a/tests/data/testing.dbf b/tests/data/testing.dbf
new file mode 100644
index 0000000..ea9015f
Binary files /dev/null and b/tests/data/testing.dbf differ
diff --git a/tests/data/testing.prj b/tests/data/testing.prj
new file mode 100644
index 0000000..f45cbad
--- /dev/null
+++ b/tests/data/testing.prj
@@ -0,0 +1,1 @@
+GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137.0,298.257223563]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]]
\ No newline at end of file
diff --git a/tests/data/testing.shp b/tests/data/testing.shp
new file mode 100644
index 0000000..50e2c38
Binary files /dev/null and b/tests/data/testing.shp differ
diff --git a/tests/data/testing.shx b/tests/data/testing.shx
new file mode 100644
index 0000000..dc404ec
Binary files /dev/null and b/tests/data/testing.shx differ
diff --git a/tests/test_dataframe_utils.py b/tests/test_dataframe_utils.py
new file mode 100644
index 0000000..d65666d
--- /dev/null
+++ b/tests/test_dataframe_utils.py
@@ -0,0 +1,36 @@
+import pandas as pd
+import pytest
+
+from dataloom.dataframe_utils import join_df, append_df
+
+df1 = pd.DataFrame.from_records([{"foo": 12.0}, {"foo": 11.0}])
+df2 = pd.DataFrame.from_records([{"bar": 1.0}, {"bar": 1.0}])
+df3 = pd.DataFrame.from_records([{"foo": 12.0, "bar": 1.0}, {"foo": 11.0, "bar": 1.0}])
+df4 = pd.DataFrame.from_records(
+    [
+        {"foo": 12.0},
+        {"foo": 11.0},
+        {"foo": 12.0},
+        {"foo": 11.0},
+    ],
+    index=[0, 1, 0, 1],
+)
+
+
+@pytest.mark.parametrize(
+    "set1, set2, expected", [(df1, df2, df3), (df1, None, df1), (None, df2, df2)]
+)
+def test_join_df(set1, set2, expected):
+    pd.testing.assert_frame_equal(join_df(set1, set2), expected)
+
+
+def test_join_df_failure():
+    with pytest.raises(AttributeError):
+        join_df(df1, "bad value")
+
+
+@pytest.mark.parametrize(
+    "set1, set2, expected", [(df1, None, df1), (None, df2, df2), (df1, df1, df4)]
+)
+def test_append_df(set1, set2, expected):
+    pd.testing.assert_frame_equal(append_df(set1, set2), expected)
diff --git a/tests/test_dataloom.py b/tests/test_dataloom.py
deleted file mode 100644
index 95c3e84..0000000
--- a/tests/test_dataloom.py
+++ /dev/null
@@ -1,24 +0,0 @@
-#!/usr/bin/env python
-
-"""Tests for `dataloom` package."""
-
-import pytest
-
-
-from dataloom import dataloom
-
-
-@pytest.fixture
-def response():
-    """Sample pytest fixture.
-
-    See more at: http://doc.pytest.org/en/latest/fixture.html
-    """
-    # import requests
-    # return requests.get('https://github.com/audreyr/cookiecutter-pypackage')
-
-
-def test_content(response):
-    """Sample pytest test function with the pytest fixture as an argument."""
-    # from bs4 import BeautifulSoup
-    # assert 'GitHub' in BeautifulSoup(response.content).title.string
diff --git a/tests/test_point_data.py b/tests/test_point_data.py
new file mode 100644
index 0000000..dd08851
--- /dev/null
+++ b/tests/test_point_data.py
@@ -0,0 +1,549 @@
+import numpy as np
+import pytest
+from unittest.mock import patch, MagicMock
+from datetime import datetime, timezone, timedelta
+from copy import deepcopy
+import geopandas as gpd
+import pandas as pd
+import pytz
+from os import path
+from climata import snotel
+
+from dataloom.pointdata import (
+    CDECPointData,
+    PointDataCollection,
+    PointData,
+    SnotelPointData,
+)
+from dataloom.variables import CdecStationVariables, SnotelVariables
+
+
+def side_effect_error(*args):
+    raise ValueError("Testing error")
+
+
+class TestPointData(object):
+    @pytest.fixture(scope="class")
+    def data_dir(self):
+        this_dir = path.dirname(__file__)
+        return path.join(this_dir, "data")
+
+    @pytest.fixture(scope="class")
+    def shape_obj(self, data_dir):
+        fp = path.join(data_dir, "testing.shp")
+        return gpd.read_file(fp)
+
+    @staticmethod
+    def expected_response(dates, vals, var, unit, station, points):
+        obj = []
+        for dt, v in zip(dates, vals):
+            obj.append(
+                {
+                    "datetime": pd.Timestamp(dt, tz="UTC"),
+                    "measurementDate": pd.Timestamp(dt, tz="UTC"),
+                    var: v,
+                    f"{var}_units": unit,
+                    "site": station.id,
+                }
+            )
+        df = gpd.GeoDataFrame.from_dict(
+            obj,
+            geometry=[points] * len(dates),
+        )
+        # needed to reorder the columns for the pd testing compare
+        df = df.filter(
+            ["datetime", "geometry", "site", "measurementDate", var, f"{var}_units"]
+        )
+        df.set_index(keys=["datetime", "site"], inplace=True)
+        return df
+
+
+class TestCDECStation(TestPointData):
+    @staticmethod
+    def cdec_daily_response():
+        return [
+            {
+                "stationId": "TNY",
+                "durCode": "D",
+                "SENSOR_NUM": 3,
+                "sensorType": "SNOW WC",
+                "date": "2021-5-16 00:00",
+                "obsDate": "2021-5-16 00:00",
+                "value": -0.11,
+                "dataFlag": " ",
+                "units": "INCHES",
+            },
+            {
+                "stationId": "TNY",
+                "durCode": "D",
+                "SENSOR_NUM": 3,
+                "sensorType": "SNOW WC",
+                "date": "2021-5-17 00:00",
+                "obsDate": "2021-5-17 00:00",
+                "value": -0.10,
+                "dataFlag": " ",
+                "units": "INCHES",
+            },
+            {
+                "stationId": "TNY",
+                "durCode": "D",
+                "SENSOR_NUM": 3,
+                "sensorType": "SNOW WC",
+                "date": "2021-5-18 00:00",
+                "obsDate": "2021-5-18 00:00",
+                "value": -0.10,
+                "dataFlag": " ",
+                "units": "INCHES",
+            },
+        ]
+
+    @pytest.fixture(scope="function")
+    def tny_station(self):
+        return CDECPointData("TNY", "Tenaya Lake")
+
+    @pytest.fixture(scope="class")
+    def tny_daily_expected(self):
+        points = gpd.points_from_xy([-119.0], [42.0], z=[1000.0])
+        df = gpd.GeoDataFrame.from_dict(
+            [
+                {
+                    "datetime": pd.Timestamp("2021-05-16 07:00:00+0000", tz="UTC"),
+                    "measurementDate": pd.Timestamp(
+                        "2021-05-16 07:00:00+0000", tz="UTC"
+                    ),
+                    "ACCUMULATED PRECIPITATION": -0.11,
+                    "ACCUMULATED PRECIPITATION_units": "INCHES",
+                    "site": "TNY",
+                },
+                {
+                    "datetime": pd.Timestamp("2021-05-17 07:00:00+0000", tz="UTC"),
+                    "measurementDate": pd.Timestamp(
+                        "2021-05-17 07:00:00+0000", tz="UTC"
+                    ),
+                    "ACCUMULATED PRECIPITATION": -0.10,
+                    "ACCUMULATED PRECIPITATION_units": "INCHES",
+                    "site": "TNY",
+                },
+                {
+                    "datetime": pd.Timestamp("2021-05-18 07:00:00+0000", tz="UTC"),
+                    "measurementDate": pd.Timestamp(
+                        "2021-05-18 07:00:00+0000", tz="UTC"
+                    ),
+                    "ACCUMULATED PRECIPITATION": -0.10,
+                    "ACCUMULATED PRECIPITATION_units": "INCHES",
+                    "site": "TNY",
+                },
+            ],
+            geometry=[points[0]] * 3,
+        )
+        # needed to reorder the columns for the pd testing compare
+        df = df.filter(
+            [
+                "datetime",
+                "geometry",
+                "site",
+                "measurementDate",
+                "ACCUMULATED PRECIPITATION",
+                "ACCUMULATED PRECIPITATION_units",
+            ]
+        )
+        df.set_index(keys=["datetime", "site"], inplace=True)
+        return df
+
+    @classmethod
+    def tny_side_effect(cls, url, **kwargs):
+        mock = MagicMock()
+        if kwargs["params"].get("dur_code") == "D":
+            mock.json.return_value = cls.cdec_daily_response()
+        elif kwargs["params"].get("dur_code") == "H":
+            raise NotImplementedError()
+        elif "getStationInfo" in url:
+            mock.json.return_value = {
+                "STATION": [
+                    {
+                        "SENS_LONG_NAME": "SNOW, WATER CONTENT",
+                        "ELEVATION": 1000.0,
+                        "LATITUDE": 42.0,
+                        "LONGITUDE": -119.0,
+                    }
+                ]
+            }
+        else:
+            raise ValueError("unknown scenario")
+        return mock
+
+    @pytest.fixture(scope="class")
+    def station_search_response(self):
+        return [
+            pd.DataFrame.from_records(
+                [
+                    (
+                        "GIN",
+                        "GIN FLAT",
+                        "MERCED R",
+                        "MARIPOSA",
+                        -119.773,
+                        37.767,
+                        7050,
+                        "CA Dept of Water Resources/DFM-Hydro-SMN",
+                        np.nan,
+                    ),
+                    (
+                        "DAN",
+                        "DANA MEADOWS",
+                        "TUOLUMNE R",
+                        "TUOLUMNE",
+                        -119.257,
+                        37.897,
+                        9800,
+                        "CA Dept of Water Resources/DFM-Hydro-SMN",
+                        np.nan,
+                    ),
+                    (
+                        "TNY",
+                        "TENAYA LAKE",
+                        "MERCED R",
+                        "MARIPOSA",
+                        -119.448,
+                        37.838,
+                        8150,
+                        "CA Dept of Water Resources/DFM-Hydro-SMN",
+                        np.nan,
+                    ),
+                    (
+                        "GFL",
+                        "GIN FLAT (COURSE)",
+                        "MERCED R",
+                        "MARIPOSA",
+                        -119.773,
+                        37.765,
+                        7000,
+                        "Yosemite National Park",
+                        np.nan,
+                    ),
+                    (
+                        "TUM",
+                        "TUOLUMNE MEADOWS",
+                        "TUOLUMNE R",
+                        "TUOLUMNE",
+                        -119.350,
+                        37.873,
+                        8600,
+                        "CA Dept of Water Resources/DFM-Hydro-SMN",
+                        np.nan,
+                    ),
+                    (
+                        "SLI",
+                        "SLIDE CANYON",
+                        "TUOLUMNE R",
+                        "TUOLUMNE",
+                        -119.43,
+                        38.092,
+                        9200,
+                        "CA Dept of Water Resources/DFM-Hydro-SMN",
+                        np.nan,
+                    ),
+                ],
+                columns=[
+                    "ID",
+                    "Station Name",
+                    "River Basin",
+                    "County",
+                    "Longitude",
+                    "Latitude",
+                    "ElevationFeet",
+                    "Operator",
+                    "Map",
+                ],
+            )
+        ]
+
+    def test_class_variables(self):
+        assert CDECPointData("no", "no").tzinfo == pytz.timezone("US/Pacific")
+        # Base implementation should fail
+        with pytest.raises(AttributeError):
+            PointData("foo", "bar").tzinfo
+
+    def test_get_metadata(self, tny_station):
+        with patch("dataloom.pointdata.cdec.requests") as mock_requests:
+            mock_requests.get.side_effect = self.tny_side_effect
+            metadata = tny_station.metadata
+            mock_get = mock_requests.get
+            assert mock_get.call_count == 1
+            mock_get.assert_called_with(
+                "http://cdec.water.ca.gov/cdecstation2/CDecServlet/" "getStationInfo",
+                params={"stationID": "TNY"},
+            )
+        expected = gpd.points_from_xy([-119.0], [42.0], z=[1000.0])[0]
+        assert expected == metadata
+
+    def test_get_daily_data(self, tny_station, tny_daily_expected):
+        with patch("dataloom.pointdata.cdec.requests") as mock_requests:
+            mock_requests.get.side_effect = self.tny_side_effect
+            response = tny_station.get_daily_data(
+                datetime(2021, 5, 16),
+                datetime(2021, 5, 18),
+                [CdecStationVariables.PRECIPITATIONACCUM],
+            )
+            mock_get = mock_requests.get
+            mock_get.assert_any_call(
+                "http://cdec.water.ca.gov/dynamicapp/req/JSONDataServlet",
+                params={
+                    "Stations": "TNY",
+                    "dur_code": "D",
+                    "Start": "2021-05-16T00:00:00",
+                    "End": "2021-05-18T00:00:00",
+                    "SensorNums": "2",
+                },
+            )
+            assert mock_get.call_count == 2
+        pd.testing.assert_frame_equal(response, tny_daily_expected)
+
+    def test_points_from_geometry(self, shape_obj, station_search_response):
+        expected_url = (
+            "https://cdec.water.ca.gov/dynamicapp/staSearch?"
+            "sta=&sensor_chk=on&sensor=3"
+            "&collect=NONE+SPECIFIED&dur="
+            "&active_chk=on&active=Y"
+            "&loc_chk=on&lon1=-119.8"
+            "&lon2=-119.2&lat1=37.7"
+            "&lat2=38.2"
+            "&elev1=-5&elev2=99000&nearby=&basin=NONE+SPECIFIED"
+            "&hydro=NONE+SPECIFIED&county=NONE+SPECIFIED"
+            "&agency_num=160&display=sta"
+        )
+        with patch("dataloom.pointdata.cdec.pd.read_html") as mock_table_read:
+            mock_table_read.return_value = station_search_response
+            result = CDECPointData.points_from_geometry(
+                shape_obj, [CdecStationVariables.SWE]
+            )
+            mock_table_read.assert_called_with(expected_url)
+            assert len(result) == 5
+            assert [st.id for st in result] == ["GIN", "DAN", "TNY", "TUM", "SLI"]
+
+    def test_points_from_geometry_fail(self, shape_obj):
+        with patch("dataloom.pointdata.cdec.pd") as mock_pd:
+            mock_pd.read_html.side_effect = side_effect_error
+            result = CDECPointData.points_from_geometry(
+                shape_obj, [CdecStationVariables.SWE]
+            )
+            assert result.points == []
+
+    def test_point_collection_to_dataframe(self, shape_obj, station_search_response):
+        with patch("dataloom.pointdata.cdec.pd.read_html") as mock_table_read:
+            mock_table_read.return_value = station_search_response
+            result = CDECPointData.points_from_geometry(
+                shape_obj, [CdecStationVariables.SWE]
+            )
+            assert isinstance(result, PointDataCollection)
+            points_df = result.to_dataframe()
+            for idp, point in enumerate(result):
+                point_row = points_df.iloc[idp]
+                assert point.name == point_row["name"]
+                assert point.id == point_row["id"]
+                assert point.metadata == point_row["geometry"]
+
+
+class MockSnotelIO(snotel.SnotelIO):
+    """
+    Mock the data structure that the climata.snotel classes return
+    """
+
+    def __init__(self, mock_obj, *args, **kwargs):
+        """
+        Args:
+            mock_obj: a list of dictionaries that will be stored
+        """
+        self.mock_dict = mock_obj
+        super(MockSnotelIO, self).__init__(*args, **kwargs)
+
+    def load(self):
+        self.data = self.mock_dict
+        transformed = []
+        # store the nested 'data' attribute as the same class
+        # to allow attribute level access of variables
+        if len(self.data) > 0 and self.data[0].get("data", False):
+            for row in self.data:
+                inner_data = MockSnotelIO(row["data"])
+                new_row = deepcopy(row)
+                new_row["data"] = inner_data
+                transformed += [new_row]
+            self.data = transformed
+
+
+class TestSnotelPointData(TestPointData):
+    @pytest.fixture(scope="function")
+    def station(self):
+        return SnotelPointData("538:CO:SNTL", "TestSite")
+
+    @pytest.fixture(scope="class")
+    def points(self):
+        return gpd.points_from_xy([-107.67552], [37.9339], z=[9800.0])[0]
+
+    @pytest.fixture(scope="class")
+    def meta_response_dict(self):
+        return {
+            "actonId": "07M27S",
+            "beginDate": "1979-10-01 00:00:00",
+            "countyName": "Ouray",
+            "elevation": 9800.0,
+            "endDate": "2100-01-01 00:00:00",
+            "fipsCountryCd": "US",
+            "fipsCountyCd": "091",
+            "fipsStateNumber": "08",
+            "huc": "140200060201",
+            "hud": "14020006",
+            "latitude": 37.9339,
+            "longitude": -107.67552,
+            "name": "Idarado",
+            "shefId": "IDRC2",
+            "stationTriplet": "538:CO:SNTL",
+        }
+
+    @pytest.fixture(scope="class")
+    def mock_meta_response(self, meta_response_dict):
+        return MockSnotelIO([{"stationDataTimeZone": -8.0, **meta_response_dict}])
+
+    @pytest.fixture(scope="class")
+    def mock_coursemeta_response(self, meta_response_dict):
+        return MockSnotelIO([meta_response_dict])
+
+    @pytest.fixture(scope="function")
+    def standard_snotel_return(self):
+        return [{"elementCd": SnotelVariables.SWE.code, "storedunitcd": "in"}]
+
+    def test_metadata(self, mock_meta_response):
+        with patch("dataloom.pointdata.snotel.snotel.StationMetaIO") as mock_snotel:
+            mock_snotel.return_value = mock_meta_response
+            obj = SnotelPointData("538:CO:SNTL", "eh")
+            assert (
+                obj.metadata
+                == gpd.points_from_xy([-107.67552], [37.9339], z=[9800.0])[0]
+            )
+            assert obj.tzinfo == timezone(timedelta(hours=-8.0))
+
+    @pytest.mark.parametrize(
+        "mocked_class, date_name, dts, expected_dts, vals, d1, d2, fn_name",
+        [
+            (
+                "snotel.StationHourlyDataIO",
+                "dateTime",
+                ["2020-03-20 00:00", "2020-03-20 01:00", "2020-03-20 02:00"],
+                ["2020-03-20 08:00", "2020-03-20 09:00", "2020-03-20 10:00"],
+                [13.19, 13.17, 13.14],
+                datetime(2020, 3, 20, 0),
+                datetime(2020, 3, 20, 2),
+                "get_hourly_data",
+            ),
+            (
+                "snotel.StationDailyDataIO",
+                "date",
+                ["2020-03-20", "2020-03-21", "2020-03-22"],
+                ["2020-03-20 08:00", "2020-03-21 08:00", "2020-03-22 08:00"],
+                [13.19, 13.17, 13.14],
+                datetime(2020, 3, 20),
+                datetime(2020, 3, 22),
+                "get_daily_data",
+            ),
+            (
+                "StationMonthlyDataIO",
+                "date",
+                ["2020-01-28", "2020-02-27"],
+                ["2020-01-28 00:00", "2020-02-27 00:00"],
+                [13.19, 13.17],
+                datetime(2020, 1, 20),
+                datetime(2020, 3, 15),
+                "get_snow_course_data",
+            ),
+        ],
+    )
+    def test_get_data_methods(
+        self,
+        mocked_class,
+        date_name,
+        dts,
+        expected_dts,
+        vals,
+        d1,
+        d2,
+        fn_name,
+        station,
+        standard_snotel_return,
+        mock_meta_response,
+        mock_coursemeta_response,
+        points,
+    ):
+        with patch(f"dataloom.pointdata.snotel.{mocked_class}") as mock_snotel, patch(
+            "dataloom.pointdata.snotel.snotel.StationMetaIO"
+        ) as mock_meta:
+            if "snow_course" in fn_name:
+                mock_meta.return_value = mock_coursemeta_response
+            else:
+                mock_meta.return_value = mock_meta_response
+            standard_snotel_return[0]["data"] = [
+                {date_name: dt, "flag": "V", "value": v} for dt, v, in zip(dts, vals)
+            ]
+            mock_snotel.return_value = MockSnotelIO(standard_snotel_return)
+
+            vrs = [SnotelVariables.SWE]
+            fn = getattr(station, fn_name)
+            result = fn(d1, d2, vrs)
+            expected = self.expected_response(
+                expected_dts, vals, SnotelVariables.SWE.name, "in", station, points
+            )
+            pd.testing.assert_frame_equal(result, expected)
+
+    def test_points_from_geometry(self, shape_obj):
+        with patch("dataloom.pointdata.snotel.LoomStationIO") as mock_search:
+            mock_search.return_value = MockSnotelIO(
+                [
+                    {
+                        "actonId": None,
+                        "beginDate": "1930-02-01 00:00:00",
+                        "countyName": "Tuolumne",
+                        "elevation": 6500.0,
+                        "endDate": "2100-01-01 00:00:00",
+                        "fipsCountryCd": "US",
+                        "fipsCountyCd": "109",
+                        "fipsStateNumber": "06",
+                        "huc": "180400090302",
+                        "latitude": 37.995,
+                        "longitude": -119.78,
+                        "name": "Fake1",
+                        "shefId": None,
+                        "stationTriplet": "FFF:CA:SNOW",
+                    },
+                    {
+                        "actonId": None,
+                        "beginDate": "1948-02-01 00:00:00",
+                        "countyName": "Tuolumne",
+                        "elevation": 9300.0,
+                        "endDate": "2100-01-01 00:00:00",
+                        "fipsCountryCd": "US",
+                        "fipsCountyCd": "109",
+                        "fipsStateNumber": "06",
+                        "huc": "180400090402",
+                        "hud": "18040009",
+                        "latitude": 38.18333,
+                        "longitude": -119.61667,
+                        "name": "Fake2",
+                        "shefId": None,
+                        "stationTriplet": "BBB:CA:SNOW",
+                    },
+                ]
+            )
+            result = SnotelPointData.points_from_geometry(
+                shape_obj, [SnotelVariables.SWE], snow_courses=True
+            )
+            ids = [point.id for point in result]
+            names = [point.name for point in result]
+            assert ids == ["FFF:CA:SNOW", "BBB:CA:SNOW"]
+            assert names == ["Fake1", "Fake2"]
+
+    def test_points_from_geometry_fail(self, shape_obj):
+        with patch("dataloom.pointdata.snotel.LoomStationIO") as mock_search:
+            mock_search.return_value = MockSnotelIO([])
+            result = SnotelPointData.points_from_geometry(
+                shape_obj, [SnotelVariables.SWE], snow_courses=True
+            )
+            assert len(result) == 0
diff --git a/tests/test_variables.py b/tests/test_variables.py
new file mode 100644
index 0000000..f0b2dcd
--- /dev/null
+++ b/tests/test_variables.py
@@ -0,0 +1,27 @@
+import pytest
+
+from dataloom.variables import VariableBase, CdecStationVariables
+
+
+class TestBaseVariables:
+    def test_validate_variable_fails(self):
+        with pytest.raises(ValueError):
+            VariableBase.from_code(-1)
+
+
+class TestCDECStationVariables:
+    @pytest.mark.parametrize(
+        "code, expected",
+        [
+            (2, CdecStationVariables.PRECIPITATIONACCUM),
+            ("2", CdecStationVariables.PRECIPITATIONACCUM),
+            ("3", CdecStationVariables.SWE),
+            (30, CdecStationVariables.TEMPAVG),
+        ],
+    )
+    def test_from_code(self, code, expected):
+        assert CdecStationVariables.from_code(code) == expected
+
+    def test_from_code_failure(self):
+        with pytest.raises(ValueError):
+            CdecStationVariables.from_code(-9999)