limit students with no submissions report to two weeks

closes CNVS-8167

test plan
 - report should only report on a 2 weeks time range
 - if time range is over two weeks it will use the end date and 2 weeks before it for the start date
 - report should not time out

Change-Id: I2750d53f358a41b351e36e718bfbbb04e61aa198
Reviewed-on: https://gerrit.instructure.com/24382
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
add ext param to generate file

default the param to csv
also support tab delimited parsing in report spec helper

refs PS-526

Change-Id: I962877d1937f1d88072861f456a29d2c31f2a3d4
Reviewed-on: https://gerrit.instructure.com/24330
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Jake Sorce <jake@instructure.com>
QA-Review: Jake Sorce <jake@instructure.com>
Reviewed-by: Jake Sorce <jake@instructure.com>
Tested-by: Jake Sorce <jake@instructure.com>
extract delayed_job redis lua utilities into a gem

We'd like to re-use some of this code elsewhere, and there's nothing
canvas-specific about it.

refs CNVS-7008

test plan: redis delayed job specs still pass

Change-Id: Iaa19ee60419d01bb451cccf2861a1015a7ba087a
Reviewed-on: https://gerrit.instructure.com/24255
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
user request throttling

Define the cost of a request as (user cpu + time spent in db queries),
then store that using a leaky bucket algorithm in redis. The algorithm
is slightly modified from a normal leaky bucket, see the comments in the
code.

The parameters (maximum, hwm, outflow) are configurable Settings.
Because this code path is hit on every request, I've tried to keep the
added latency to a minimum.

Uses lua scripting in redis to avoid the latency of an extra round trip
(two round trips with lua, one at the beginning of the request and one
at the end, vs at least three without lua).

closes CNVS-7008

test plan:
* Given the default params, you're not going to ever hit the throttling
  limit without making multiple requests in parallel. Normally local dev
  environments won't process parallel requests, so you'll want to tweak
  the config params.
  * Setting.set('request_throttle.hwm', '2')
  * Setting.set('request_throttle.outflow', '0.1')
* Make canvas http requests as a logged in user, and verify that after a
  few requests in quick succession, you get a 403 response. After
  waiting a second for your bucket to fall back below the threshold, you
  can make requests again.
* Do the same using the api and sending your access token in the query
  string or http auth header.
* This should work correctly with a clustered redis as well.
* If possible it'd be good to test parallel requests as well. The code
  should handle this correctly and combine the cost of the parallel
  requests, rather than stomping on the values.

Change-Id: I1fdfd4e009d81bd6525bcf45a93437f4c395f129
Reviewed-on: https://gerrit.instructure.com/24256
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
fix "jump to today" link for syllabus

test plan:
* create a course with an assignment due today
* set the syllabus as the course home page
* clicking on 'Jump to Today' should scroll down
to the bottom (just as on the main syllabus page)

* clicking on 'Jump to Today' or on a date on the
calendar sidebar will highlight the relevant date (and
moving the mouse will not disrupt the highlighting)

* clicking on 'Jump to Today' before the page has fully
loaded will not break the link from that point on

fixes #CNVS-3808 #CNVS-7665 #CNVS-7839

Change-Id: I0aa61c7f2640356314cff882f063421f9a967f2c
Reviewed-on: https://gerrit.instructure.com/24376
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Hannah Bottalla <hannah@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
try to only show wiki edit unsaved warning on visible changes

test plan:
* create a page
* edit the page
* confirm that adding and then removing a carriage return (i.e.
pressing "Enter" then "Backspace") in a way that results in
no visual changes will not trigger an 'unsaved changes' warning
when attempting to cancel or leave the page

closes #CNVS-8146

Change-Id: I01372583ffa476bfa8bebe282d8b6c8716f29a61
Reviewed-on: https://gerrit.instructure.com/24384
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
allow file deletion even without valid file data

test plan:
* upload an attachment
* in the console, run the following command (replacing for
the relevant id):

Attachment.where(:id => [ATTACHMENT_ID]).
update_all(:content_type => nil)

* should still be able to delete the file without an error

fixes #CNVS-8166

Change-Id: I9756f47043d9d23c32d760f0186c4c8f56342b88
Reviewed-on: https://gerrit.instructure.com/24421
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
QA-Review: August Thornton <august@instructure.com>
discussions: remove extraneous screenreader label on search

fixes CNVS-6653

test plan:
  * open a discussion topic
  * verify with your browser's DOM inspector that there is
      not a hidden label for the topic search box as well
      as an aria-label

Change-Id: I5c6a3c3089a3e71f96ad72f8869356bd3ee10358
Reviewed-on: https://gerrit.instructure.com/24395
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Joel Hough <joel@instructure.com>
QA-Review: Cam Theriault <cam@instructure.com>
Product-Review: Braden Anderson <banderson@instructure.com>
discussions: add aria-label for expand/collapse buttons

fixes CNVS-6654

test plan:
  * open a discussion topic with a screenreader
  * focus the expand and collapse replies buttons
  * verify that each is read aloud usefully

Change-Id: I10eb30d0ef6e9a71bc06e8a79cc118ef69b1b1d3
Reviewed-on: https://gerrit.instructure.com/24396
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Joel Hough <joel@instructure.com>
QA-Review: Cam Theriault <cam@instructure.com>
Product-Review: Braden Anderson <banderson@instructure.com>
fix pinned discussion sorting

the backbone update broke pinned discussion sorting (probably because the reset event was removed). CollectionView doesn't
insert the item views in the same order as the sorted collection
because backbone doesn't trigger the add events in sorted order.

fixes CNVS-8161

test plan:
- reorder some pinned discussions and verify sort order sticks
  after refreshing the page. try several different orderings.

Change-Id: If68e0abb16206e786953648949aec5b24774eb2b
Reviewed-on: https://gerrit.instructure.com/24417
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Zach Pendleton <zachp@instructure.com>
QA-Review: Cam Theriault <cam@instructure.com>
Product-Review: Jon Willesen <jonw@instructure.com>
make account reports only run once and low priority

fixes CNVS-8212

test plan 
 - run an account report
 - report should work

Change-Id: I38b1457d5e1428101273f640f62696ec13dc0d39
Reviewed-on: https://gerrit.instructure.com/24427
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
remove sleep(5) from drag/drop spec

fixes CNVS-8217

test plan:
 - run 'should reorder assignments with drag and drop'
   spec in spec/selenium/assignments_spec.rb
 - it should pass

Change-Id: Idc7e493206c69c72b72f15aa5b7b240cfb34ee3e
Reviewed-on: https://gerrit.instructure.com/24450
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
spec: fix grading_standard_spec creating/deleting

Change-Id: I2300f9669b188a76390be7ed35ec12aa71af640a
Reviewed-on: https://gerrit.instructure.com/24434
Reviewed-by: Bryan Madsen <bryan@instructure.com>
Product-Review: Bryan Madsen <bryan@instructure.com>
QA-Review: Bryan Madsen <bryan@instructure.com>
Tested-by: Bryan Madsen <bryan@instructure.com>
tweak tie-breaking logic for Accept-Language header

previously in the case of ties in the quality value we sorted by length (so en
beats en-US), and then alphabetically (so ar beats en).  this part of the spec
is somewhat ambiguous, but this ordering does not take into account the order
passed into the accept header.  so passing "en,ar" would have chosen 'ar'
because quality and length both tie, and 'ar' is first alphabetically.

to fix this we are incorporating the order of the types passed into the accept
header as one of the tie breaking criteria.

this fixes the bug with CutyCapt snapshots of canvas being rendered in arabic
because CutyCapt passes "en,*" as it's accept header.  up until recently, 'en'
happened to be alphabetically first in our list of supported languages, so
everything worked out.  but when we added support for arabic, a new
alphabetical winner emerged, exposing this bug.

fixes CNVS-3070

test plan:
- in an environment with CutyCapt enabled
- create a web submission assignment that allows url submissions
- submit a canvas url that has this fix applied (note that you can run this
  from anywhere, it's the url you submit that needs the fix)
- it should be in english, not arabic

Change-Id: I7cf6d9db02ec0e79ad425bc0479c4fc43942fa52
Reviewed-on: https://gerrit.instructure.com/24348
Reviewed-by: Jon Jensen <jon@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Liz Abinante <labinante@instructure.com>
QA-Review: Bryan Madsen <bryan@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
fix intermittent selenium failures

Change-Id: I2498ea7dfe546588c211459a2f85d0c33aa5ea3d
Reviewed-on: https://gerrit.instructure.com/24422
Reviewed-by: Bryan Madsen <bryan@instructure.com>
Product-Review: Bryan Madsen <bryan@instructure.com>
QA-Review: Bryan Madsen <bryan@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
fix cross-shard stream item dismissal, fixes CNVS-1404

1. make sure the returned stream item ids are relative to the user, not
   the domain, since we need to look up the instances by those ids from
   the user's shard later
2. make sure we actually handle shortened global ids, rather than
   asploding
3. just cache stream items on the user's shard, not on every shard the
   user visits. makes cache invalidation practical/possible

test plan:
1. set up canvas with redis and sharding
2. set up two additional shards (your user is in the initial/primary one)
3. enroll your user in a course in the second shard
4. as another user, do something that creates a stream item for the course
   (e.g. create an announcement)
5. as the original user, confirm that:
   1. you see the stream item on your shard's dashboard
   2. you can dismiss the stream item
   3. when you refresh the page, it is still dismissed
6. repeat step 4.
7. as the original user, confirm that:
   1. you can see the stream item on the shard 3 dashboard
   2. you can dismiss the stream item
   3. when you refresh the page, it is still dismissed
8. as the original user, confirm that the items are dismissed from your
   dashboard on all shards

Change-Id: I2c600685015640af36d9e33ac71e25cd536d7391
Reviewed-on: https://gerrit.instructure.com/24155
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Mark Ericksen <marke@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Jon Jensen <jon@instructure.com>
prevent deleting frozen assignments and their groups on index page

fixes CNVS-7669

test plan:
- enable draft mode
- create a course with frozen assignments
- from the assignments index page only
- verify that "delete" option does not appear under gear wheel for:
 * frozen assignments
 * groups with frozen assignments

Change-Id: I55c7a429efc58d014b3e28d7681bc6a245578d81
Reviewed-on: https://gerrit.instructure.com/24203
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
conversations beta: display message attachments

fixes CNVS-8111

test plan:
  * open conversations beta
  * select a conversation containing a media comment or file
  * verify that the attachment is displayed and can be opened

Change-Id: I3c18a6c4d87623af8260bf70741795f56b6f5e07
Reviewed-on: https://gerrit.instructure.com/24394
Reviewed-by: Joel Hough <joel@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Cam Theriault <cam@instructure.com>
Product-Review: Braden Anderson <banderson@instructure.com>
discussions: remove icon shadows

fixes CNVS-8228

test plan:
  * open a discussion with peer reviews enabled
  * open the topic gear menu
  * verify that the peer review and rubric link icons
      have no faint doubles
  * verify that the two links are still functional

I suspect the reason for these is that they used to be the only links
with icons in the menu, and when we added icons to the rest, we forgot
to remove these.

Change-Id: I6154c5e7c6d740868deb994264f55ee2f692b3fb
Reviewed-on: https://gerrit.instructure.com/24482
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Zach Pendleton <zachp@instructure.com>
QA-Review: Cam Theriault <cam@instructure.com>
Product-Review: Braden Anderson <banderson@instructure.com>
fix module_item_sequence when module position is not unique

test plan:
 - have a course with non-unique module positions
   (can change them in the Rails console, or find a course with this
    problem)
 - call the module_item_sequence API on an item in such a module
 - ensure that the previous and next items in the module are
   returned correctly

fixes CNVS-8223

Change-Id: I73d499e79078e132b20bad610abdef73f3de929f
Reviewed-on: https://gerrit.instructure.com/24470
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
Product-Review: Bracken Mosbacker <bracken@instructure.com>
QA-Review: August Thornton <august@instructure.com>