fixes account reports zip export to have .csv on files

Change-Id: I0e419a448b19fb9278e0f7f4676942ba5de08770
Reviewed-on: https://gerrit.instructure.com/11581
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
don't try to link account rubrics to assessments; fixes #8864

testing steps:
 * try to create a rubric associated with an account
   (not a course or assignment)
 * it should work

Change-Id: I98e7ed77b5ee4c4c6c7feb2557174d1edc052185
Reviewed-on: https://gerrit.instructure.com/11579
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
properly copy all day dates for assignments

The all-dayness of an assignment is inferred from the datetime,
trying to set it explicitely makes it go to the wrong day. So
this just lets the normal before_save's take care of the all-day
date

Test Plan:
 * Create an assignment due at 11:59 for some day
 * Copy the course
 * Look at your calendar, the assignment on the new course should be on the same day as the assignment for the old course.

closes #8533

Change-Id: I7a951c5663ea198fd5c42769c0969ec88c13cbdc
Reviewed-on: https://gerrit.instructure.com/11506
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
don't show un-needed text in additional comments dialog

fixes #8613

test plan:
  1. Create a Rubric in a course and use it on an Assignment
     for grading and set it to allow free-form commenting
  2. As a Student, submit the assignment used in Step 1
  3. As the Teacher, go to the Speedgrader for the
     assignment/student used in Step 2, add comments to some
     criterions and Save
  4. From the right-hand column of the Speedgrader, click a
     comment you left in the Rubric (should display a popup
     dialogue about the Rubric)
  5. Notice, only the title of the criterion is displayed
     in addtion to the comments

Change-Id: Ia0cd884ee6468bdac85e5c597ae28670e78d7f26
Reviewed-on: https://gerrit.instructure.com/10834
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Joe Tanner <joe@instructure.com>
flag the following apis as beta

adds support for flagging individual endpoints, not just resources, as
beta.

test plan: generate the api docs and verify that the tagged endpoints
have a beta message.

Change-Id: Iba815db309ee71482e1f471eaf613527cd7114ec
Reviewed-on: https://gerrit.instructure.com/11585
Reviewed-by: Simon Williams <simon@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
allow plugins to register account services

refs #9000

Also allow services to be specified as "service" or "setting" so that
they'll show up under the relevant heading in the account settings ui.
Sort the services by name, as well, so their ordering isn't random on
every page reload.

test plan: visit the account settings page, and verify you can still
change service settings as before (including the User Avatar service,
which should now be moved to the Features portion of the page).

Change-Id: Ib4542015659c34de719c5a2fd8d281696fa35cf8
Reviewed-on: https://gerrit.instructure.com/11603
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Brian Palmer <brianp@instructure.com>
ensure that temp file handles are closed when uploading .zip files

Temp files are automatically closed when they are garbage
collected, but waiting for that to happen can cause too
many files to be open at once.

Test Plan:
 * Import a .zip or course import with lots of files
 * They should all be imported

closes #8726

Change-Id: I72b5a445259a330e27eccc645df4957a92ea1381
Reviewed-on: https://gerrit.instructure.com/11582
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Forgot password now matches email without case consideration. Fixes #8861

Updates forgot_password to use newer (and already existing) named scopes
for better searching. Uses a case-insensitive search by email that is
supported on multiple databases. Additional minor refactor to clean up code
and only do the search if an email is given. Blanks don't count anymore.

Testing Notes:
* Create a user with a CommunicationChannel email that has mixed case like
  "Tom.Thumb@example.com"
* Doing a "forgot password" from login should now match and generate an email.

Change-Id: I1bec847fbae8b43e949210b010aa2cfad438ce55
Reviewed-on: https://gerrit.instructure.com/11193
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Joe Tanner <joe@instructure.com>
implement delayed job singletons without the delete/insert

Instead, do a select, and only insert if no non-running job is found on
the strand.

test plan: do something that generates singleton jobs, like comment on a
discussion or trigger a grade recalculation. check /jobs to verify that
no more than one non-running job exists on the strand.

Change-Id: I3c293fa4d58d53f3b9aa388da5c9bcf6900c1316
Reviewed-on: https://gerrit.instructure.com/11564
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
drop unused course section columns

test plan:
 * ensure current specs pass, and no functionality changes

Change-Id: I7296f63421eb8041c1f33277244e6b745197e972
Reviewed-on: https://gerrit.instructure.com/11619
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
fix sis_course_id in course create api; fixes #9063

This fixes an tricky condition where setting the sis_course_id in the course
create api only worked if you were also passing the offer => true parameter.
This was because offering the course caused the workflow state to change, and
the object to be saved without validation first, which set up the root account
needed for the real save which validates the uniqueness of the sis_course_id
using that root account. Since validations happen before before_saves, when
offer wasn't present and the first save tried to do validations, they failed.

test plan:
- hit the api course create endpoint with an sis_course_id and without the
  offere parameter
- it should succeed

Change-Id: I8fdfa5654c5a5a06609418224e96ee6aef0628da
Reviewed-on: https://gerrit.instructure.com/11642
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
items and followers counts for collections

aggregate and cache the number of items and number of followers of
a collection. return these counts in the api response.

test plan:
- run the migration
- query existing collections and make sure counts look accurate
- follow/unfollow collections and make sure count changes
- pin/delete items and make sure the count changes

Change-Id: Ic3a96682b550c168f505f419ffc4b619f859f044
Reviewed-on: https://gerrit.instructure.com/11490
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
process sis batches on a singleton strand

And process all pending batches in the job, don't require one job per
batch.

This way we don't pile up jobs for accounts that send frequent sis
updates.

There is also an optional setting to add a delay before the job starts,
so that we can process more batches per job invocation if they are being
sent frequently.

test plan: make sure that sis batches are still processed, including
batches that were queued before the code update. verify that no matter
how many batches are queued, only one job per strand will be in the
queue (sometimes 2, if one is already running and a second is added).

Change-Id: I4d551ef9fa45d9d332244ab29ca8d061fa5d7b9c
Reviewed-on: https://gerrit.instructure.com/11465
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
remove "page views over time" from courses/X/statistics

This is superceded by the analytics data, which is much more efficient
to query.

test plan: visit /courses/X/statistics, and verify the page views graph
is no longer on the page, but the rest of the data loads correctly.

Change-Id: Ic9991f661071dc9eb77f39a8a601fde644c9bc13
Reviewed-on: https://gerrit.instructure.com/11653
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
force creation_pending state for students invited via api

fixes #8750

they will automatically transition to invited when the course is published

test plan:
 * invite a student to an unpublished course via the api
 * the student shouldn't get a notification
 * publish the course
 * the student should get a notification

Change-Id: I16795085acd4198b54ec5cf440335845f733c195
Reviewed-on: https://gerrit.instructure.com/11664
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Implement option for "no submission comments in inbox" for teachers. Closes #8084

Backend implementation that, when set, keeps submission comments from
showing up in the teacher's inbox. This commit also adds documentation
comments to related methods that I had to research for understanding behavior.
This currently has no visual interface and can only be enabled from the console.
The UI is part of the Notification rewrite/redesign slated for next.

Testing Notes:
=============
Console commands to change:
  u = User.find_by_email('user@example.com')
  u.preferences[:no_submission_comments_inbox] = true  # to turn option on (defaults to off)
  u.preferences[:no_submission_comments_inbox] = false # to turn option off again
  u.save
=============
Disabled first
* After using console commands to disable
* Submit an assignment as a student with a comment
* Teacher's inbox does not show the comment.
* Comment does show on Course Activity and the Submission view

Disabled after
* While option is disabled, submit an assignment with a comment
* The comment should appear in teacher's inbox (as it has in past)
* Mark all comments as read (or just 'read' it)
* Enable the feature in the console
* As student, add another comment to submission
* As teacher, no unread conversations in inbox and
  selecting the existing thread will show the new comment

Change-Id: Ib9406c9d85bccb33b137af0e60c9803ca6cb3280
Reviewed-on: https://gerrit.instructure.com/11144
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Joe Tanner <joe@instructure.com>
remove unused context_module_legend html dialog

this was being rendered on every page that contained a module item, but
wasn't being used.

test plan: load module items (assignments, files, etc that are
included in a course module), and verify the page still renders.

Change-Id: I7b4a17b608c6d0484d58c2d80e7228b14cf5d24d
Reviewed-on: https://gerrit.instructure.com/11670
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
optimize courses list in accounts

also reduce limit to 50 courses (no one scrolls that far anyway)

test plan:
 * go to account home page
 * it should list number of students in course, and names of unique
   teachers accurately

Change-Id: I517c66b404987b00157129df145beea889b95136
Reviewed-on: https://gerrit.instructure.com/11621
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
remove to_json from js_env submission object

fixes #8872

ENV.SUBMISSION.submission was a JSON string
instead of an object, which caused the grade to
reset to nothing, then adding a comment would
remove the grade from the page

other bugs may be fixed by this commit as well

Change-Id: I03ad06d26e959fbd52129377f2ba7a3a76faa01f
Reviewed-on: https://gerrit.instructure.com/11687
Reviewed-by: Jon Jensen <jon@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
don't copy deleted assignments linked to topics or quizzes

Test Plan:
 * Create a topic for grading
 * Change it to not be for grading anymore
 * Copy the course
 * There shouldn't be an assignment for the topic

closes #8948

Change-Id: Ieeda35ab22988e8ed173ff25dea76b4cf8b354fe
Reviewed-on: https://gerrit.instructure.com/11652
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>