fix answer corruption when a question is regraded twice

fixes CNVS-7989
when a question is updated then regraded without refreshing the page,
the answer ids on the question are rewritten by the assessment question
id because of DOM corruption.  this mitigates that by ensuring the duped
id isn't present in the object that's applied to the answer templates.

Test plan:
1. Create a quiz with a multiple answer question
2. Take the quiz as a student
3. Edit the quiz and question as a teacher
4. Change the correct answer for the question
5. Choose the "update without regrading" option and update the question
6. Choose another regrade option and update the question
7. Save and publish the quiz
- The student's overview of their submission should have both the
  correct and selected answer labeled correctly.
- The quiz should work and be scored normally.

Change-Id: I9588197ec10a332e62356371235262c6fa64bf61
Reviewed-on: https://gerrit.instructure.com/24340
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Myller de Araujo <myller@instructure.com>
Reviewed-by: Derek DeVries <ddevries@instructure.com>
Product-Review: Josh Simpson <jsimpson@instructure.com>
discussions: allow subscribing after reply to topic requiring reply

fixes CNVS-8019

The ticket says that IE9 used to refresh the page when adding a reply to
a topic with "Users must post before seeing replies" turned on and that
it was useful because it displayed the subscription button. Instead of
restoring that behavior, this change just displays the subscription
button on the client side after adding a reply, so all browsers can see
it.

Test plan:
  * open a topic with "Users must post before seeing replies" enabled
      as a student who has not replied
  * add a reply
  * verify that, in addition to seeing the other replies, you now
      immediately see the subscription button

Change-Id: Ieb91938d2f320f996c1d2d48ca5eb26675c4e465
Reviewed-on: https://gerrit.instructure.com/24353
QA-Review: Cam Theriault <cam@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jon Willesen <jonw@instructure.com>
Product-Review: Braden Anderson <banderson@instructure.com>
stop using hashes in the tour guide

closes CNVS-8083

Change-Id: I3fa1bb635850c3985691f9970dc9957fea5c4789
Reviewed-on: https://gerrit.instructure.com/24291
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Zach Pendleton <zachp@instructure.com>
Product-Review: Ryan Florence <ryanf@instructure.com>
QA-Review: Ryan Florence <ryanf@instructure.com>
spec: selenium and firefox upgrade spec fixes

Change-Id: I7bd9b169e5905310b035b681e37bf4ee46056c30
Reviewed-on: https://gerrit.instructure.com/24060
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Bryan Madsen <bryan@instructure.com>
Product-Review: Bryan Madsen <bryan@instructure.com>
QA-Review: Bryan Madsen <bryan@instructure.com>
fix migration on mysql

only postgres can use the conditions, and mysql complains about us removing
the index supporting the FK, so only do it for postgres

Change-Id: I0700887169ef4e1efe54cf7bb70529fdb7c571ab
Reviewed-on: https://gerrit.instructure.com/24386
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
clear active connections after each job

in a sharded environment, this helps to clear out idle connections

this is similar to what
ActiveRecord::ConnectionAdapters::ConnectionManagement middleware
does after each web request

test plan:
 * run some jobs
 * they should not fail

Change-Id: I431205ff1e9197940bddf4028d3c4c7662a74056
Reviewed-on: https://gerrit.instructure.com/24285
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
fix links to static api doc files

test plan: links to the files api documentation, such as in
submissions#upload_file , should work correctly

Change-Id: I936a597b5a9e40bae06c753e036ca36aa1538cfd
Reviewed-on: https://gerrit.instructure.com/24279
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
module sequence footer on assignment show

fixes CNVS-8015

test plan
 - turn on draft state for a course (or account)
 - put assignments in a module for the course
 - navigate to an assignment
 - the bottom should have a new footer for the module sequence
 - verify that links to all parts of the module work

Change-Id: I74f4ea218571d3c04b44cdd2150d10dc924a27b9
Reviewed-on: https://gerrit.instructure.com/24252
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cameron Matheson <cameron@instructure.com>
QA-Review: Amber Taniuchi <amber@instructure.com>
Product-Review: Cameron Sutter <csutter@instructure.com>
limit students with no submissions report to two weeks

closes CNVS-8167

test plan
 - report should only report on a 2 weeks time range
 - if time range is over two weeks it will use the end date and 2 weeks before it for the start date
 - report should not time out

Change-Id: I2750d53f358a41b351e36e718bfbbb04e61aa198
Reviewed-on: https://gerrit.instructure.com/24382
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
add ext param to generate file

default the param to csv
also support tab delimited parsing in report spec helper

refs PS-526

Change-Id: I962877d1937f1d88072861f456a29d2c31f2a3d4
Reviewed-on: https://gerrit.instructure.com/24330
Reviewed-by: Cody Cutrer <cody@instructure.com>
Product-Review: Jake Sorce <jake@instructure.com>
QA-Review: Jake Sorce <jake@instructure.com>
Reviewed-by: Jake Sorce <jake@instructure.com>
Tested-by: Jake Sorce <jake@instructure.com>
extract delayed_job redis lua utilities into a gem

We'd like to re-use some of this code elsewhere, and there's nothing
canvas-specific about it.

refs CNVS-7008

test plan: redis delayed job specs still pass

Change-Id: Iaa19ee60419d01bb451cccf2861a1015a7ba087a
Reviewed-on: https://gerrit.instructure.com/24255
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
user request throttling

Define the cost of a request as (user cpu + time spent in db queries),
then store that using a leaky bucket algorithm in redis. The algorithm
is slightly modified from a normal leaky bucket, see the comments in the
code.

The parameters (maximum, hwm, outflow) are configurable Settings.
Because this code path is hit on every request, I've tried to keep the
added latency to a minimum.

Uses lua scripting in redis to avoid the latency of an extra round trip
(two round trips with lua, one at the beginning of the request and one
at the end, vs at least three without lua).

closes CNVS-7008

test plan:
* Given the default params, you're not going to ever hit the throttling
  limit without making multiple requests in parallel. Normally local dev
  environments won't process parallel requests, so you'll want to tweak
  the config params.
  * Setting.set('request_throttle.hwm', '2')
  * Setting.set('request_throttle.outflow', '0.1')
* Make canvas http requests as a logged in user, and verify that after a
  few requests in quick succession, you get a 403 response. After
  waiting a second for your bucket to fall back below the threshold, you
  can make requests again.
* Do the same using the api and sending your access token in the query
  string or http auth header.
* This should work correctly with a clustered redis as well.
* If possible it'd be good to test parallel requests as well. The code
  should handle this correctly and combine the cost of the parallel
  requests, rather than stomping on the values.

Change-Id: I1fdfd4e009d81bd6525bcf45a93437f4c395f129
Reviewed-on: https://gerrit.instructure.com/24256
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
fix "jump to today" link for syllabus

test plan:
* create a course with an assignment due today
* set the syllabus as the course home page
* clicking on 'Jump to Today' should scroll down
to the bottom (just as on the main syllabus page)

* clicking on 'Jump to Today' or on a date on the
calendar sidebar will highlight the relevant date (and
moving the mouse will not disrupt the highlighting)

* clicking on 'Jump to Today' before the page has fully
loaded will not break the link from that point on

fixes #CNVS-3808 #CNVS-7665 #CNVS-7839

Change-Id: I0aa61c7f2640356314cff882f063421f9a967f2c
Reviewed-on: https://gerrit.instructure.com/24376
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Hannah Bottalla <hannah@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
try to only show wiki edit unsaved warning on visible changes

test plan:
* create a page
* edit the page
* confirm that adding and then removing a carriage return (i.e.
pressing "Enter" then "Backspace") in a way that results in
no visual changes will not trigger an 'unsaved changes' warning
when attempting to cancel or leave the page

closes #CNVS-8146

Change-Id: I01372583ffa476bfa8bebe282d8b6c8716f29a61
Reviewed-on: https://gerrit.instructure.com/24384
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
QA-Review: August Thornton <august@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
allow file deletion even without valid file data

test plan:
* upload an attachment
* in the console, run the following command (replacing for
the relevant id):

Attachment.where(:id => [ATTACHMENT_ID]).
update_all(:content_type => nil)

* should still be able to delete the file without an error

fixes #CNVS-8166

Change-Id: I9756f47043d9d23c32d760f0186c4c8f56342b88
Reviewed-on: https://gerrit.instructure.com/24421
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Product-Review: Jeremy Stanley <jeremy@instructure.com>
QA-Review: August Thornton <august@instructure.com>
discussions: remove extraneous screenreader label on search

fixes CNVS-6653

test plan:
  * open a discussion topic
  * verify with your browser's DOM inspector that there is
      not a hidden label for the topic search box as well
      as an aria-label

Change-Id: I5c6a3c3089a3e71f96ad72f8869356bd3ee10358
Reviewed-on: https://gerrit.instructure.com/24395
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Joel Hough <joel@instructure.com>
QA-Review: Cam Theriault <cam@instructure.com>
Product-Review: Braden Anderson <banderson@instructure.com>
discussions: add aria-label for expand/collapse buttons

fixes CNVS-6654

test plan:
  * open a discussion topic with a screenreader
  * focus the expand and collapse replies buttons
  * verify that each is read aloud usefully

Change-Id: I10eb30d0ef6e9a71bc06e8a79cc118ef69b1b1d3
Reviewed-on: https://gerrit.instructure.com/24396
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Joel Hough <joel@instructure.com>
QA-Review: Cam Theriault <cam@instructure.com>
Product-Review: Braden Anderson <banderson@instructure.com>
fix pinned discussion sorting

the backbone update broke pinned discussion sorting (probably because the reset event was removed). CollectionView doesn't
insert the item views in the same order as the sorted collection
because backbone doesn't trigger the add events in sorted order.

fixes CNVS-8161

test plan:
- reorder some pinned discussions and verify sort order sticks
  after refreshing the page. try several different orderings.

Change-Id: If68e0abb16206e786953648949aec5b24774eb2b
Reviewed-on: https://gerrit.instructure.com/24417
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Zach Pendleton <zachp@instructure.com>
QA-Review: Cam Theriault <cam@instructure.com>
Product-Review: Jon Willesen <jonw@instructure.com>
make account reports only run once and low priority

fixes CNVS-8212

test plan 
 - run an account report
 - report should work

Change-Id: I38b1457d5e1428101273f640f62696ec13dc0d39
Reviewed-on: https://gerrit.instructure.com/24427
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Rob Orton <rob@instructure.com>
remove sleep(5) from drag/drop spec

fixes CNVS-8217

test plan:
 - run 'should reorder assignments with drag and drop'
   spec in spec/selenium/assignments_spec.rb
 - it should pass

Change-Id: Idc7e493206c69c72b72f15aa5b7b240cfb34ee3e
Reviewed-on: https://gerrit.instructure.com/24450
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>