giseldo/neodataset huggingface dataset:
Refactor frontend for new international ip handling

Acceptance Criteria:
* Make a check on new backend ip location endpoint on page load of every stage of open student sign up, not partners
  * Redirect to new international sign up denial page
* Partner pages call partner backend API route, open pages call open backend API route
* If backend sign up API endpoints return an "international ip" error, redirect to new denial page

## Tasks
- [ ] Code review
- [ ] Staging smoke tests
- [ ] Merge to main
- [ ] Deploy to prod

## Smoke tests
- [x] Student/volunteer can login
- [x] Student/volunteer can make/join session
- [x] Student/volunteer can use whiteboard and chat in session
- [x] Use VPN to have international IP and try to visit each sign up page
- [x] Then turn off VPN and try to visit each sign up page
- [x] Partner sign up should always work

Story points: 5.0
Refactor the backend student signup API endpoints to handle new international ip flow

Acceptance Criteria:
* check IP location and return new error if its international
  * unless the student is coming from a partner
* Remove current ip banning code paths and code

## Tasks
- [x] Code review
- [x] Staging smoke tests
- [x] Merge to main
- [ ] Deploy to prod

## Smoke tests
- [x] Student/volunteer can login
- [x] Student/volunteer can make/join session
- [x] Student/volunteer can use whiteboard and chat in session

Story points: 2.0
Make international student sign up denial page

Mockup: https://gitlab.com/upchieve/staff-requests/-/issues/87

## Tasks
- [ ] Code review
- [ ] Staging smoke tests
- [ ] Merge to main
- [ ] Deploy to prod

## Smoke tests
- [ ] Student/volunteer can login
- [ ] Student/volunteer can make/join session
- [ ] Student/volunteer can use whiteboard and chat in session

Story points: 2.0
Refactor API contract for student registration

- Student registration calls are documented in Swagger
- In a separate file that is $ref'd into the main one for readability
- Main purpose of refactor is to determine contract around US and non-US IP signups
  - Existing student sign up endpoints need to make their own IP location checks for each request, so they should have a new potential return value, which would be an error saying that the request couldn't be honored because it's from an international IP.
  - New endpoint for the frontend to be able to make IP location checks separately from other requests.

## Tasks
- [x] Code review
- [x] Merge to main

Story points: 2.0
Server - split student registration routes between partner and open

On all partner student registration api calls, just go through the regular flow as it exists now in the backend.

On all open student registration api calls, check the header for the source IP, and perform a who-is call. If it is a non-us ip, send back an 403 error code, backend should have a custom error type of NonUSStudent or something similar.

Wrap all student registration calls in the feature flag `ip-ban-overhaul`.

## Tasks
- [x] Code review
- [x] Staging smoke tests
- [x] Merge to main
- [x] Deploy to prod

## Smoke tests
- [x] Student/volunteer can login
- [x] Student/volunteer can make/join session
- [x] Student/volunteer can use whiteboard and chat in session
- [x] student sign up through front end works like before
  - test both open and partner student sign up
- [x] New partner route works
  - both for US and international IPs
- [x] New open route works for US ip
- [x] New open route 403 for international ip

Story points: 2.0
Add two new options to post-session survey

Volunteers have two new options in the post session survey, "The student was rude or inappropriate" and "The student was only looking for answers", in the obstacles section.

They should be added to the end of the list, so we don't mess up current integer/obstacle key/value associations. String values/checkboxes need to be added to FeedbackView and FeedbackPreview.

On backend, add const enum that maps string values to integers so we can see what the values should be and validate in the future, this enum should be shared by frontend and backend by importing the enum from consts.

Feedback schema needs to be bumped to v3.

## Tasks
- [x] Code review
- [x] Staging smoke tests
- [x] Merge to main
- [x] Deploy to prod

## Smoke tests
- [x] Student/volunteer can login
- [x] Student/volunteer can make/join session
- [x] Student/volunteer can use whiteboard and chat in session
- [x] Make sure new feedback options get stored correctly in db

Story points: 2.0
Update Zwibbler source strings in config

To test this, change local config to point at CDN location of new file.

Look through minified zwibbler package to find the websocket connection variable (`zwibblerContainer.xy.zz.ab...`) and update `src/views/SessionView/Whiteboard.vue:564` to use the new package. Put this change behind a feature flag.

To test in staging, change value in Doppler.

To update in production, change value in Doppler.

## Tasks
- [ ] Code review
- [ ] Staging smoke tests
- [ ] Merge to main
- [ ] Deploy to prod

## Smoke tests
- [ ] Student/volunteer can login
- [ ] Student/volunteer can make/join session
- [ ] Student/volunteer can use whiteboard and chat in session
- [ ] Student/volunteer can use all whiteboard tools
- [ ] Mobile Student/volunteer can use all whiteboard tools

Story points: 2.0
App should exit if database is not available

At the moment, if the database is not available, connections fail and just stay failed, but the app does not crash. The same occurs in the worker.

In the connections instantiated by both the web and worker processes, the connection is not done with an `try...await...catch` or `return...catch`. We need to catch errors and exit if the db is not available.

The resulting connection provides a `.on('error')` handler. Similarly here we should log the error exit the process or have some kind of exponential back off (like this one: https://www.npmjs.com/package/@kessler/exponential-backoff) of trying to reconnect before finally exiting.

Backoff retries should not last longer than 60 seconds

## Tasks
- [x] Code review
- [x] Staging smoke tests
- [x] Merge to main
- [x] Deploy to prod

## Smoke tests
- [x] Starting app without mongo running, it retries for 60 seconds and exits
- [x] Starting app and then killing mongo process, app retries for 60 seconds and exits
- [x] In staging, starting app with kubernetes IP blocked on staging mongo database, app retries for 60 seconds and exits, container should stop, and kubernetes should start a new one
- [x] In staging, starting app with mongo access, then removing kubernetes IP from allowlist, app retries for 60 seconds and exits, container should stop and kubernetes should start a new one
- [x] Student/volunteer can login
- [x] Student/volunteer can make/join session
- [x] Student/volunteer can use whiteboard and chat in session

Story points: 3.0
Remove hoursTutored references from backend

hoursTutored is referenced in

* session service tests
* userService in `parseUser`
* endSession
* buildVolunteer generator

These references should be removed so we're no longer doing anything with hoursTutored

## Tasks
- [ ] Code review
- [ ] Staging smoke tests
- [ ] Merge to main
- [ ] Deploy to prod

## Smoke tests
- [ ] Student/volunteer can login
- [ ] Student/volunteer can make/join session
- [ ] Student/volunteer can use whiteboard and chat in session

Story points: 2.0
Update volunteer dashboard impact summary to use timeTutored

Refactor src/views/DashboardView/VolunteerDashboard/index.vue, the getImpactStats, to use timeTutored instead of hours, and convert ms to hours.

## Tasks
- [ ] Code review
- [ ] Staging smoke tests
- [ ] Merge to main
- [ ] Deploy to prod

## Smoke tests
- [ ] Student/volunteer can login
- [ ] Student/volunteer can make/join session
- [ ] Student/volunteer can use whiteboard and chat in session
- [ ] Dashboard shows accurate data in impact summary before and after change

Story points: 1.0
Capture ASSIStments query params for a logged-in student

Vertical slice of making sure this works.

* Be logged in as an ASSISTments student
* Visit the site via a url containing the query parameters we will receive
* Land on the dashboard
* Refresh
* Start a session
* New assistments document is created with correct data

## Tasks
- [x] Code review
- [x] Staging smoke tests
- [x] Merge to main
- [x] Deploy to prod

## Smoke tests
- [x] Student/volunteer can login
- [x] Student/volunteer can make/join session
- [x] Student/volunteer can use whiteboard and chat in session
- [x] Assistments link new student flow works (confirm via compass)
  - staging.upchieve.org/integration?partner=assistments&studentId=fb2e790c-514d-4936-a6f5-2ef20afe4cf0&assignmentId=a63c5239-b2e4-4760-a6b8-fb55b7bb938b&problemId=1234
- [x] Assistments existing, not logged in student flow works (confirm via compass)
  - staging.upchieve.org/integration?partner=assistments&studentId=e608f3b7-3918-43d8-adc7-20ff5d5f1973&assignmentId=7b5070c0-512d-48ff-9b5e-e613de7d590d&problemId=1234
- [x] Assistments existing, logged in student flow works (confirm via compass)
  - staging.upchieve.org/integration?partner=assistments&studentId=dd979e07-33e8-4c88-99ea-a693a3d66862&assignmentId=43f3bb3d-fb93-470e-a651-57f2f9275daa&problemId=1234
- [x] Bad assistments link does not create data object and logs error but still creates session object
  - staging.upchieve.org/integration?partner=assistments&studentId=hello&assignmentId=world&problemId=foobar
  - staging.upchieve.org/integration?partner=assistments&studentId=hello&assignmentId=world
  - staging.upchieve.org/integration?studentId=hello&assignmentId=world&problemId=foobar

Story points: 5.0
Create assistments data document collection that stores assistments information

This document needs to hold:

* problemId
* assignmentId
* sessionId as a reference to the session

This is a great chance to start the repo design style on a new model.

Methods that we probably need to expose:
* Creating
* GetAll (mainly for testing)

Specific analytics reads are something we can write later when we're writing the integration for retrieving data for Assistments.

Need to ask assistments what format these ids are in so we can validate the fields.

## Tasks
- [ ] Code review
- [ ] Merge to main

Story points: 1.0
UX design for better invalid input on sign up

We're going to try to work in better UX on invalid inputs into the ASSISTments sign up / log in forms.

Need wireframes and prototypes in Figma for:
* Where do we show errors (global at top or bottom or under each field, for example)
* Colors, shape, placement
* Behavior (how fast and when should we show the error feedback)

Story points: 5.0
Whiteboard line drawing tool has multiple issues

1.  When selecting the line drawing tool, it becomes active and the cursor changes but upon clicking it reverts. Selecting the line drawing tool again works as expected. Then, after I draw a line with the line drawing tool, the tool is still selected. When I next use it, upon clicking, the currently selected tool reverts to the pencil and I'm therefore not drawing a line anymore. The line drawing tool effectively works every other try.

2.  Sometimes the line drawing tool is not sensitive to my choice of color.

3.  Sometimes the drawn lines are insensitive to undo and redo. If I use the pencil, draw a line, use the pencil, then press revert twice, the pencil drawings disappear and the line is still there.



* * *



Ticket created from a Slack [message](https://upchieveaccommunity.slack.com/archives/CE3V4J8KF/p1621022206152500) by [Dave Sudia](https://upchieveaccommunity.slack.com/team/U01B0ATT8AK) in #bug-reporting.

gz#5

Story points: 1.0
Write SessionService function to generate and save heatmap data in cache, and retrieve it from cache

Write the actual function that generates the heatmap data structure according to the API contract spec (#679)

Go through the sessions for the last week, compute wait time, average them for each hour and save it for that hour.

Should include unmatched sessions.

Ignore sessions (matched and unmatched) that lasted less than 1 minute

**To calculate wait time:**

volunteerJoinedAt - createdAt if the session is matched

endedAt - createdAt if the session is not matched

## Tasks
- [x] Code review
- [x] Staging smoke tests
- [x] Merge to main
- [ ] Deploy to prod
- [ ] Prod smoke tests

## Smoke tests
- [x] Student/volunteer can login
- [x] Student/volunteer can make/join session
- [x] Student/volunteer can use whiteboard and chat in session

Story points: 1.0
Refactor SessionService

Refactor session service to have cleaner code and follow new repo/svc/router architecture. Will involve touching many other files that import session service.

## Tasks
- [ ] Code review
- [ ] Staging smoke tests
- [ ] Merge to main
- [ ] Deploy to prod
- [ ] Prod smoke tests

## Smoke tests
- [ ] Student/volunteer can login
- [ ] Student/volunteer can make/join session
- [ ] Student/volunteer can use whiteboard and chat in session

Story points: 1.0
Resolve missing seconday texts

## Tasks
- [x] Staging smoke tests
- [x] Code review
- [x] Merge to main
- [x] Deploy to prod
- [x] Prod smoke tests

## Smoke tests
- [x] Student/volunteer can login
- [x] Student/volunteer can make/join session
- [x] Student/volunteer can use whiteboard and chat in session
- [x] Secondary text actually gets sent 
  - [x] Locally via worker logs
  - [ ] Test on staging

Story points: 2.0
Searching for any specific school on the admin pages shows all schools instead of the searched school

Admin school search input box does not actually change search results. Programs team needs to be able to search schools to be able to conduct their work.

## Investigation
- `src/views/Admin/AdminSchool.vue`
- `SchoolService.getSchools`

## Tasks
- [ ] Code review
- [x] Staging smoke tests
- [ ] Merge to main
- [ ] Deploy to prod
- [ ] Prod smoke tests

## Smoke tests
- [x] Student/volunteer can login
- [x] Student/volunteer can make/join session
- [x] Student/volunteer can use whiteboard and chat in session
- [x] Admin school search works

Story points: 2.0
Misspelled word in the Academic Integrity policy

There's a misspelled word in the Academic Integrity Policy. Here's the updated version: https://docs.google.com/document/d/1PXzRpXOvxQiZPv0fYcVM10tbUdYB1_qSfzITzathW5o/

Tasks
- [x] Download updated policy as a PDF
- [x] Upload the updated policy as `academic-integrity.pdf` and set the current version to the new uploaded policy

Story points: 0.0
Switch ordering of waiting students list

Students who have been waiting the longest should show up at the top of the list as opposed to the bottom of the list

Rationale: We hope this will help ensure volunteers pick up the oldest requests first. It will also minimize cases where a volunteer selects the wrong student because a new student made a request right before they clicked (pushing down the list).

## Tasks
- [x] Code review
- [x] Staging smoke tests
- [x] Merge to main
- [x] Deploy to prod
- [x] Prod smoke tests

## Smoke tests
- [x] Student/volunteer can login
- [x] Student/volunteer can make/join session
- [x] Student/volunteer can use whiteboard and chat in session
- [x] Volunteer dashboard shows students sorted by wait time decreasing

Story points: 1.0
Bootstrap Storybook.js for components

Get Storybook installed into our application: https://storybook.js.org/docs/vue/get-started/introduction

Get one component created (header, as will be used in #678) as an example with 1 story and 1 test as an exemplar.

We may need to make our own header component that's not just an h1, that encapsulates our styling for an h1, outside of the global css

* unit test
* [storyshot test](https://storybook.js.org/docs/vue/workflows/snapshot-testing)

* write up a wiki doc on basic storybook stuff, why, how

Story points: 3.0
Frontend components and views for volunteer availability heat map

* modifying the schedule view to show the alarm icons from the design issue.
* create grid-cell and grid components in Storybook that represent the schedule grid and show state of selected/waittime, etc.
* grid-cell and grid components should have unit tests
* add to network service to get data
* refactor schedule view to hydrate the data

Design story w/ final mockups: https://gitlab.com/upchieve/subway/-/issues/689

Design: #689

Story points: 5.0
Endpoint for retrieving wait times for volunteer availability heat map

* Create new `stats` router to host endpoints for user stats
* should call new SessionService function to retrieve cached data (#705)
* return data map for frontend to display
* only available to logged-in volunteers
  * this may require new middleware to see if someone is a volunteer

## Tasks
- [x] Code review
- [x] Staging smoke tests
- [ ] Merge to main
- [ ] Deploy to prod
- [ ] Prod smoke tests

## Smoke tests
- [ ] Student/volunteer can login
- [ ] Student/volunteer can make/join session
- [ ] Student/volunteer can use whiteboard and chat in session
- [ ] New endpoint works

Story points: 2.0
Job that computes wait times for volunteer availabililty heat map

* Executes weekly - update at 8am ET on Mondays and look at wait times from Mon-Sun from the prior week
* puts values into Redis under a key of "allsubjectswaittimes" and a json-stringified data map of the data to retrieve for that subject to fill the frontend
* job should execute immediately (maybe manually) when it is first deployed just to get things populated right off the bat
* calls the function written in #705 

## Tasks
- [ ] Code review
- [x] Staging smoke tests
- [ ] Merge to main
- [ ] Deploy to prod
- [ ] Add cron job with one off initialization
- [ ] Prod smoke tests

## Smoke tests
- [x] Student/volunteer can login
- [x] Student/volunteer can make/join session
- [x] Student/volunteer can use whiteboard and chat in session
- [x] Make sure heatmaps are actually populated correctly

Story points: 2.0
Implement redis pub/sub in whiteboard socket server

## Acceptance Criteria

* Whiteboard communication works over multiple instances of the app
* By using redis pub/sub to communicate websocket messages across instances

## Testing notes

* Can test this locally by spinning up two version of the dev server with different ports
* Increase instance count in staging to ensure high probability of landing on different servers
* Test with one instance to make sure that scenario continues to function

## Tasks

- [ ] Pass code review
- [ ] Deploy to staging to validate
- [ ] Merge to main
- [ ] Deploy to production

Story points: 5.0
Corporate Partner analytics spreadsheet

Sample of what I'd like to be able to pull: https://docs.google.com/spreadsheets/d/14crzaXpEZxQRsWDOG7-R7xxL0INxIZHvwbyYv-ieSr4/edit?usp=sharing

It is 1 report with all the data needed. I'll go in and delete columns as needed.


Acceptance Criteria:
* Report can be pulled via the admin panel like student and volunteer partner reports.

Story points: 8.0
Update telecom reporting to be accurate to new spec

## Acceptance criteria

**Report columns**
* Name - firstname and lastname - correctly capitalized
* Email
* event id (4003)
* Date volunteered: max one row per date
* hours credited - take the sum of
  * certification hours (1 hour per quiz passed on a given date)
  * tutoring hours (total tutoring time on a given date, rounded up to 15 min increment) , in decimal format (eg 1 hour and 15 mins is 1.25)
  * elapsed availability hours (1:1 ratio for elapsed availability hours BUT ONLY on days where they had a tutoring session and the hours did not overlap with the time they tutored)

**Frontend**
* Generate Volunteer Partner Report button should be changed to Generate Telecom Partner Report

## Tasks
- [x] Pass code review
- [x] Deploy to staging and validate
- [x] Merge to main
- [x] Deploy to production

Story points: 5.0
Jobs catch and log their own errors internally making it difficult to track when they fail

We should have jobs bubble up errors to the queue executing the job so that all job have a consistent mechanism for logging an error state. This can help us easily detect and monitor job performance.

- [x] Deploy to staging and check logs after jobs have run
- [x] Merge
- [x] Make Port Authority MR with main commit sha
- [x] Deploy to Production
- [x] Set up NewRelic alerts for jobs

Story points: 3.0
Unmatched sessions are not auto-ended after 45 minutes

Problem:
Students first see a modal after waiting 15 minutes that we are having trouble matching their session. After 45 minutes, they should see another modal saying that we were unable to match their session. At the moment the modal is shown, the session should be auto-ended. However, there are currently unmatched sessions that are lasting longer than this 45 minute mark.

Desired outcome: 
- Ensure that the modals are shown at the 15 and 45 minute mark of an unmatched session
- The session should auto-end at the 45 minute mark

Story points: 5.0
SendGrid Volunteer contact list and segments are inaccurate

The total number of contacts under "Volunteers" in SendGrid does not match the total number of volunteers that we have on the platform. 

The segment lists "Subscribed Volunteers" and "Newsletter - Subscribed Volunteers" under the contact list "Volunteers" also do not accurately align with the total number of expected volunteers that match the segment's filters

Desired outcome:
- The "Volunteers" contact list will have the correct total number of volunteers as there are on our platform
- The "Subscribed Volunteers" segment should have the correct total number of volunteers given a filter that is reproducible in MongoDB

Story points: 5.0
minor changes to contact form

## Purpose

- Reduce duplicate submissions
- Reduce "Other" category submissions

##  Vision/Outcome
- Add new dropdown choice to the top of the list called "General question"
- Change "Feedback" to "General feedback"
- Replace everything on the contact form page under the header + help text (see screenshot below) with a simple text message saying "Thank you for contacting us! We'll get back to you as soon as possible" when the user submits the form (thereby preventing multiple submissions)
![image](/uploads/19603aaf2a35dbb024a37787381735be/image.png)

## Tasks

- [x] Pass code review
- [x] Deploy to staging and test
- [x] Deploy to production

Story points: 2.0
Store contact form submissions in db

## Purpose

To help us better monitor trends in contact form submissions and make it possible to export into excel all the submissions over a specific time range. 


##  Vision/Outcome
- The objects in the collection should contain: createdAt, user id, user first name, user partner org (if it exists), user email, user type (student/volunteer), user banned status (true/false), topic of the submission, text of the submission
- We should have a new dashboard on retool which lets us track contact form submissions.. how many did we get over a specific time frame? what was the breakdown of topics? what % came from students vs volunteers? and a table that can be downloaded to excel with all of the data.

Story points: 5.0
Remove v2 compatibility from socket.io server

After the v3 client upgrade has been released to prod, wait one day and then change server setting to
```
const io = require("socket.io")({
  allowEIO3: false
});
```

## Tasks

Smoke tests:

- [ ] Dashboard works, specifically the waiting sessions queue
- [ ] Volunteer can join session
- [ ] Chat works
- [ ] End a session ends the session for both users, when ended by student, or by volunteer

Story points: 1.0