Repository: regro/regolith

Issue:
helper: contact lister
finds a person in contacts and lists brief info if they are there.  Simon has a template so ask him for it.

Discussion:
shogho0: Fix #359 
codecov[bot]: # [Codecov](https://codecov.io/gh/regro/regolith/pull/370?src=pr&el=h1) Report
> Merging [#370](https://codecov.io/gh/regro/regolith/pull/370?src=pr&el=desc) into [master](https://codecov.io/gh/regro/regolith/commit/e3321a64d839fd230dd5e16d296c99dae2cc2a6e&el=desc) will **decrease** coverage by `0.41%`.
> The diff coverage is `34.69%`.

[![Impacted file tree graph](https://codecov.io/gh/regro/regolith/pull/370/graphs/tree.svg?width=650&height=150&src=pr&token=YOWfAqEPVS)](https://codecov.io/gh/regro/regolith/pull/370?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #370      +/-   ##
==========================================
- Coverage   63.81%   63.39%   -0.42%     
==========================================
  Files          44       45       +1     
  Lines        3410     3459      +49     
==========================================
+ Hits         2176     2193      +17     
- Misses       1234     1266      +32     
```


| [Impacted Files](https://codecov.io/gh/regro/regolith/pull/370?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [regolith/helpers/l\_contactshelper.py](https://codecov.io/gh/regro/regolith/pull/370/diff?src=pr&el=tree#diff-cmVnb2xpdGgvaGVscGVycy9sX2NvbnRhY3RzaGVscGVyLnB5) | `33.33% <33.33%> (ø)` | |
| [regolith/helper.py](https://codecov.io/gh/regro/regolith/pull/370/diff?src=pr&el=tree#diff-cmVnb2xpdGgvaGVscGVyLnB5) | `100.00% <100.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/regro/regolith/pull/370?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/regro/regolith/pull/370?src=pr&el=footer). Last update [e3321a6...a71a0e3](https://codecov.io/gh/regro/regolith/pull/370?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

sbillinge: > 
> 
> Fix #359

Please see how I did this in the topmost comment.  In this way the issue will be automatically closed whe the project is merged.
shogho0: Added code that completes UC1 (user can search for name fragment and helper will print info about everyone who has that fragment in their name)
shogho0: Helper now works for all use cases. Only action item left is to add linking to institutions.yml database.
sbillinge: The comments always appear right below the line they are referring to.

On Sat, May 30, 2020 at 8:35 PM shogho0 <notifications@github.com> wrote:

> *@shogho0* commented on this pull request.
> ------------------------------
>
> In regolith/helpers/l_contactshelper.py
> <https://github.com/regro/regolith/pull/370#discussion_r432898484>:
>
> > +from dateutil.relativedelta import relativedelta
> +import sys
> +
> +from pathlib import Path
> +from regolith.dates import get_due_date
> +from regolith.helpers.basehelper import SoutHelperBase
> +from regolith.fsclient import _id_key
> +from regolith.tools import (
> +    all_docs_from_collection,
> +    get_pi_id,
> +)
> +
> +TARGET_COLL = "contacts"
> +HELPER_TARGET = "l_contacts"
> +
> +THIS_DIR = Path('.')
>
> I assume you're referring to lines 23 and 24 here, right?
>
> —
> You are receiving this because you commented.
>
>
> Reply to this email directly, view it on GitHub
> <https://github.com/regro/regolith/pull/370#discussion_r432898484>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABAOWULVB7WYRK5JSBAERC3RUGQ3LANCNFSM4NNMRNIQ>
> .
>
-- 
Professor Simon Billinge
Columbia University

sbillinge: What do you think?

On Sat, May 30, 2020 at 8:35 PM shogho0 <notifications@github.com> wrote:

> *@shogho0* commented on this pull request.
> ------------------------------
>
> In tests/test_helpers.py
> <https://github.com/regro/regolith/pull/370#discussion_r432898534>:
>
> > @@ -35,7 +35,15 @@
>      (["helper", "l_grants", "-c", "-d", "2020-05-25"],
>       "sym2.0, awardnr: , acctn: , 2019-06-01 to 2030-12-31\n, awardnr: , acctn: GG012345, 2015-10-01 to 2025-09-30\n"
>       ),
> -
> +     (["helper", "l_contacts", "-n", "ny"],
>
> Unfortunately not, because there's no date information for the contact in
> the sample database file. Should I modify that file and add in that
> information?
>
> —
> You are receiving this because you commented.
>
>
> Reply to this email directly, view it on GitHub
> <https://github.com/regro/regolith/pull/370#discussion_r432898534>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABAOWUP5DBFM6OIZN7UEKWTRUGQ65ANCNFSM4NNMRNIQ>
> .
>
-- 
Professor Simon Billinge
Columbia University

shogho0: Made updates according to the review comments on 5-30-2020
shogho0: Uploaded an updated version-made changes off the comments

sbillinge: hi @shogho0 so I thought about this some more.  Here are my inclinations:

1. I am not convinced that we need to handle the list of fragments.  It seems to me that that is a different UC. i.e., (a) show me the full listing of a contact (or all the contacts that match) a fragment of information I remember and (b) I have a list of names or name fragments and I want a summary of info about them returned.  So part of our struggle is delivering two different functionalities in the same helper.  I suggest we change the name to l_contact (singular) and just do the first functionality.
1. I think the functionality that then filters for this could be function in tools as we may want to reuse it many places.  Right now we have ``fuzzy_retrieval`` that matches a given string with the values of a list of fields in a collection and returns the document if it finds a match.  What we need here is something like ``fuzzy_fuzzy_retrieval``, or ``fragment_retrieval`` or sthg that does the same, but accepts a "hit" if the fragment is found anywhere in those values, and returns the full documents of all the hits.  So we could do something like
```
hits = fragment_retrieval(gtx["contacts], ["name","aka","_id"],"fragment")
```
and it will return all documents in contacts.yml where billi appears in ``name``, ``aka`` or ``_id``.

We could then run this for all items we want to search over and use the properties of sets to do the logic (set(union(lists)) would be OR and intersection would be AND.

This logic could be used in many different helpers.

There may be a better way to do this if you google around too.

shogho0: Couldn't both functionalities hypothetically be bundled into the same helper? If the inputs apply to multiple documents, multiple documents could be returned, while if the inputs only apply to one contact, only one contact would be returned. 

I like the fragment_retrieval idea as I definitely have a lot of repeated code in the helper. My only question is how would I be able to integrate the date into that?
sbillinge: On Tue, Jun 2, 2020 at 5:12 PM Shomik Ghose <notifications@github.com>
wrote:

> Couldn't both functionalities hypothetically be bundled into the same
> helper? If the inputs apply to multiple documents, multiple documents could
> be returned, while if the inputs only apply to one contact, only one
> contact would be returned.
>
this is what we tried and it was making the code/project a bit complicated,
so agile programming would say implement the simple thing, do the
complicated thing only if it is needed.

> I like the fragment_retrieval idea as I definitely have a lot of repeated
> code in the helper. My only question is how would I be able to integrate
> the date into that?
>
right.  So we can make a much nicer and more compact code.

So the decision is to contact lister (singular) by implementing
fragment_retrieval.     But please make a separate PR and branch for
fragment retrieval.  Let's get that merged then work on the lister.

> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/regro/regolith/pull/370#issuecomment-637809302>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABAOWUNLHFVZUOHE6IC5KY3RUVTLJANCNFSM4NNMRNIQ>
> .
>


-- 
Professor Simon Billinge
Columbia University

shogho0: Got it-will do
sbillinge: @shogho0 fyi I just merged Michael's ``is_current`` so you can use it.  Please use these steps:
first, make sure all your recent edits have been committed and pushed to origin.  Then, 
```
git checkout master
git pull upstream master
git checkout contact_lister
```
Then using PyCharm, "rebase current onto selected" where you select upstream master as the branch to rebase onto.  Please get  help with this if you haven't done it before.  You will then have to force-push to your fork.  Then you can go back to editing, committing and pushing normally.  Only ever force-push after a rebase.  If git won't let you push normally it means something is corrupted in your repo, which is bad.  The rebase changes the git history and requires the force push.
shogho0: Just rebased
sbillinge: did you push your commits.  The latest commit seems to be from 5 days ago.....
shogho0: Not yet-I was just marking the comments I had covered while I was going along
sbillinge: cool, sorry!  I guess I get a notification when you add to the conversation
.  No worries, let me know when you are ready for review by @ me.

On Sat, Jun 6, 2020 at 3:11 PM Shomik Ghose <notifications@github.com>
wrote:

> Not yet-I was just marking the comments I had covered while I was going
> along
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/regro/regolith/pull/370#issuecomment-640105322>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABAOWUISDQ4TK35GKT3SUBLRVKIELANCNFSM4NNMRNIQ>
> .
>


-- 
Professor Simon Billinge
Columbia University


Patch:
diff --git a/news/l_contactshelper.rst b/news/l_contactshelper.rst
new file mode 100644
index 00000000..a5af8f85
--- /dev/null
+++ b/news/l_contactshelper.rst
@@ -0,0 +1,12 @@
+**Added:**
+ * helper that lists all contacts corresponding to a name, institution, date, or note fragment
+
+**Changed:** None
+
+**Deprecated:** None
+
+**Removed:** None
+
+**Fixed:** None
+
+**Security:** None
diff --git a/regolith/helper.py b/regolith/helper.py
index 95b69cda..2f05c18d 100644
--- a/regolith/helper.py
+++ b/regolith/helper.py
@@ -8,6 +8,7 @@ from regolith.helpers import l_milestoneshelper as l_milestone
 from regolith.helpers import l_projectahelper as l_projecta
 from regolith.helpers import l_grantshelper as l_grants
 from regolith.helpers import l_membershelper as l_members
+from regolith.helpers import l_contactshelper as l_contacts
 
 HELPERS = {
     "hello": (hello.HelloHelper, hello.subparser),
@@ -18,6 +19,7 @@ HELPERS = {
     "l_projecta": (l_projecta.ProjectaListerHelper, l_projecta.subparser),
     "l_grants": (l_grants.GrantsListerHelper, l_grants.subparser),
     "l_members": (l_members.MembersListerHelper, l_members.subparser),
+    "l_contacts": (l_contacts.ContactsListerHelper, l_contacts.subparser),
 }
 
 
diff --git a/regolith/helpers/l_contactshelper.py b/regolith/helpers/l_contactshelper.py
new file mode 100644
index 00000000..be2f14a6
--- /dev/null
+++ b/regolith/helpers/l_contactshelper.py
@@ -0,0 +1,130 @@
+"""Helper for finding and listing contacts from the contacts.yml database.
+Prints name, institution, and email (if applicable) of the contact.
+"""
+import dateutil
+import dateutil.parser as date_parser
+from regolith.dates import is_current
+from regolith.helpers.basehelper import SoutHelperBase
+from regolith.fsclient import _id_key
+from regolith.tools import (
+    all_docs_from_collection,
+    get_pi_id,
+    fragment_retrieval
+)
+
+TARGET_COLL = "contacts"
+HELPER_TARGET = "l_contacts"
+
+
+def subparser(subpi):
+    subpi.add_argument(
+        "-n",
+        "--name",
+        help='name or name fragment (single argument only) to use to find contacts')
+    subpi.add_argument(
+        "-i",
+        "--inst",
+        help='institution or an institution fragment (single argument only) to use to find contacts')
+    subpi.add_argument(
+        "-d",
+        "--date",
+        help='approximate date in ISO format (YYYY-MM-DD) corresponding to when the contact was entered in the database. Comes with a default range of 4 months centered around the date; change range using --range argument')
+    subpi.add_argument(
+        "-r",
+        "--range",
+        help='range (in months) centered around date d specified by --date, i.e. (d +/- r/2)',
+        default=4)
+    subpi.add_argument(
+        "-o",
+        "--notes",
+        help='fragment (single argument only) to be found in the notes section of a contact'
+    )
+    return subpi
+
+
+def stringify(con):
+    return f"name: {con.get('name')}, institution: {con.get('institution')}, email: {con.get('email','missing')}"
+
+
+class ContactsListerHelper(SoutHelperBase):
+    """Helper for finding and listing contacts from the contacts.yml file
+    """
+    # btype must be the same as helper target in helper.py
+    btype = HELPER_TARGET
+    needed_dbs = [f'{TARGET_COLL}']
+
+    def construct_global_ctx(self):
+        """Constructs the global context"""
+        super().construct_global_ctx()
+        gtx = self.gtx
+        rc = self.rc
+        if "groups" in self.needed_dbs:
+            rc.pi_id = get_pi_id(rc)
+        rc.coll = f"{TARGET_COLL}"
+        try:
+            if not rc.database:
+                rc.database = rc.databases[0]["name"]
+        except BaseException:
+            pass
+        colls = [
+            sorted(
+                all_docs_from_collection(rc.client, collname), key=_id_key
+            )
+            for collname in self.needed_dbs
+        ]
+        for db, coll in zip(self.needed_dbs, colls):
+            gtx[db] = coll
+        gtx["all_docs_from_collection"] = all_docs_from_collection
+        gtx["float"] = float
+        gtx["str"] = str
+        gtx["zip"] = zip
+
+    def sout(self):
+        rc = self.rc
+        def_l = set(stringify(i) for i in
+                    self.gtx['contacts'])
+        if rc.name:
+            namel = set(stringify(i) for i in
+                        fragment_retrieval(
+                self.gtx['contacts'], [
+                        "_id", "aka", "name"], rc.name))
+        else:
+            namel = def_l
+        if rc.inst:
+            instl = set(stringify(i) for i in
+                        fragment_retrieval(
+                self.gtx['contacts'],
+                ["institution"],
+                rc.inst))
+        else:
+            instl = def_l
+        if rc.notes:
+            notel = set(stringify(i) for i in
+                        fragment_retrieval(
+                self.gtx['contacts'], [
+                        "notes"], rc.notes))
+        else:
+            notel = def_l
+        if rc.date:
+            date_list = []
+            temp_dat = date_parser.parse(rc.date).date()
+            for contact in self.gtx["contacts"]:
+                if rc.date:
+                    temp_dict = {
+                        "begin_date": (temp_dat -
+                                       dateutil.relativedelta.relativedelta(
+                                           months=int(
+                                               rc.range))).isoformat(),
+                        "end_date": (temp_dat +
+                                     dateutil.relativedelta.relativedelta(
+                                         months=int(
+                                             rc.range))).isoformat()}
+                    if is_current(temp_dict, now=temp_dat):
+                        date_list.append(stringify(contact))
+            datel = set(date_list)
+        else:
+            datel = def_l
+        res_l = set.intersection(namel, instl, notel, datel)
+        for item in res_l:
+            print(item)
+        return
diff --git a/regolith/schemas.py b/regolith/schemas.py
index 7ea469c4..4d9bff31 100644
--- a/regolith/schemas.py
+++ b/regolith/schemas.py
@@ -96,6 +96,8 @@ EXEMPLARS = {
         "name": "Anthony B Friend",
         "notes": "The guy I meet for coffee sometimes",
         "title": "Mr.",
+        "month": 1,
+        "year": 2020,
     },
     "courses": {
         "_id": "EMCH-552-2016-F",
@@ -1150,7 +1152,7 @@ SCHEMAS = {
             "type": "string"
         },
         "name": {
-            "description": "the person canonical name",
+            "description": "the person's canonical name",
             "required": True,
             "type": "string",
         },
@@ -1164,6 +1166,16 @@ SCHEMAS = {
             "required": False,
             "type": "string",
         },
+        "month": {
+            "description": "the month when the entry was created",
+            "required": False,
+            "type": "integer",
+        },
+        "year": {
+            "description": "the year when the entry was created",
+            "required": False,
+            "type": "integer",
+        },                
     },
     "grades": {
         "_description": {


Test patch:
diff --git a/tests/test_helpers.py b/tests/test_helpers.py
index 0b8e7766..6d86e730 100644
--- a/tests/test_helpers.py
+++ b/tests/test_helpers.py
@@ -8,8 +8,8 @@ from regolith.main import main
 helper_map = [
     (["helper", "hello", "--person", "Simon"], "hello Simon\n"),
     (["helper", "a_proprev", "A. Einstein", "nsf", "2020-04-08", "-q",
-      "Tess Guebre","-s", "downloaded", "-t", "A flat world theory"],
-      "A. Einstein proposal has been added/updated in proposal reviews\n"),
+      "Tess Guebre", "-s", "downloaded", "-t", "A flat world theory"],
+     "A. Einstein proposal has been added/updated in proposal reviews\n"),
     (["helper", "a_grppub_readlist", "test the lister",
       "A list to test the lister", "pdf", "--purpose", "Test the lister"],
      "test_the_lister has been added in reading_lists\n"),
@@ -38,7 +38,9 @@ helper_map = [
     (["helper", "l_members", "-v"],
      "Anthony Scopatz, professor | group_id: scopatz | orcid: 0000-0002-9432-4248 | github_id: ascopatz\n"
      ),
-
+    (["helper", "l_contacts", "-n", "ny", "-i", "col", "-o", "coffee", "-d", "2020-01-15", "-r", "2"],
+     "name: Anthony B Friend, institution: columbiau, email: friend@deed.com\n"
+     ),
 ]
 
 
@@ -58,14 +60,14 @@ def test_helper_python(hm, make_db, capsys):
     if are_outfiles and not expecteddir.is_dir():
         print("WARNING: there are built outputs that are not being tested")
     if are_outfiles and expecteddir.is_dir():
-        assert_outputs(builddir,expecteddir)
+        assert_outputs(builddir, expecteddir)
 
     builddir = repo / "db"
     if expecteddir.is_dir():
-        assert_outputs(builddir,expecteddir)
+        assert_outputs(builddir, expecteddir)
 
 
-def assert_outputs(builddir,expecteddir):
+def assert_outputs(builddir, expecteddir):
     """
 
     Parameters
@@ -88,11 +90,11 @@ def assert_outputs(builddir,expecteddir):
 
                 # Skip because of a date time in
                 if file != "rss.xml":
-                    # Fixme proper fix for testing hard coded filepaths on windows
+                    # Fixme proper fix for testing hard coded filepaths on
+                    # windows
                     if os.name == "nt":
                         if "tmp" not in expected:
                             if "../.." not in expected:
                                 assert expected == actual
                     else:
                         assert expected == actual
-