strip invalid utf-8 from AR yaml columns serialized as binary by ruby 1.8

fixes CNVS-4174

Ruby 1.8 somtimes inserted invalid utf-8 into these columns using a
!binary tag, which 1.9 syck deserializes as ASCII-8BIT encoded. But then
it later blows up trying to append to a UTF-8 erb view or json output.

This adds a whitelisted set of columns for which we want to not only
strip invalid utf-8, but convert ascii encoded strings to utf-8 strings
and strip those too. These columns don't contain any binary data, so
that's an OK conversion to make.

Once Canvas only supports ruby 1.9, we shouldn't get any invalid utf-8
inserted even into serialized columns (YAML in 1.9 raises an exception
on serialization), so we won't need to keep this list up to date with
newly added columns, and we'll avoid accidentally converting a
serialized column that does contain binary data to utf-8.

test plan: rather than switching to ruby 1.8 and inserting bad data,
it'll be easier to just munge the database directly. create a
quiz_questions row with question_data that contains something like:

   question_name: !binary |
     oHRleHSg

then load the quiz in the UI using ruby 1.9. verify that you see a
question name "test" rather than a page error.

Change-Id: I8054026a06f28b1b61e05ba187bca8403a07fbb5
Reviewed-on: https://gerrit.instructure.com/18059
Reviewed-by: Paul Hinze <paulh@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
fix bad sql in refactored named scope

Change-Id: I6860fcb44c55e63cb84283061dd6c66582b72c8a
Reviewed-on: https://gerrit.instructure.com/18115
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Zach Pendleton <zachp@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
use mysql2 gem for mysql

closes CNVS-4199

Change-Id: I81a14efca365a61ff9e5d29892ec95ace64aabb3
Reviewed-on: https://gerrit.instructure.com/18089
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
fix migrations so they can run transactionally

refs CNVS-3902

even if the preference is that they don't

test plan:
 * bring up a fresh database
 * it should not fail

Change-Id: I5e7b2cf8479de4b3f1564ee65cd1cb456ae4674e
Reviewed-on: https://gerrit.instructure.com/17873
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
paginate and process includes for user_search api

closes CNVS-4115

test plan:
 * set up a course with lots of users that can be searched via this
   API
 * ensure limit param for user search API still works (set it really
   low), even though it's no longer documented
 * test normal pagination (per_page, multiple pages)
 * use an include[] from the list users api; ensure you get the
   expected data in the response

Change-Id: I0d2a44ce96bef5d61cdf666b3bd59c9422952f33
Reviewed-on: https://gerrit.instructure.com/18061
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Duane Johnson <duane@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
fix deliver_via_email for ruby 1.8.

fixes CNVS-4196

Timeout::Error doesn't inherit from StandardError in
< 1.9, causing some exceptions to not be properly handled
in deliver_via_email.

test plan:
  * run spec/lib/message_dispatcher_spec.rb with ruby 1.8
    and verify that specs pass.

Change-Id: I233c62a5960e27804ef37f529512bb39b8746239
Reviewed-on: https://gerrit.instructure.com/18079
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Joel Hough <joel@instructure.com>
QA-Review: Bryan Madsen <bryan@instructure.com>
Reviewed-by: Eric Berry <ericb@instructure.com>
Tested-by: Eric Berry <ericb@instructure.com>
specs: always reset the i18n locale to :en

fixes a race condition issue where some tests were being set to the
wrong locale.

test plan:
  - specs pass
  - make a spec that changes the global locale (there are several
    already that exist it seems). Run that spec, then run another spec
    asserting that the locale is "en" again, rather than the locale you
    set.

Change-Id: I3f925f1b59e7d8c975398429824432ab9c94ef86
Reviewed-on: https://gerrit.instructure.com/18153
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Bryan Madsen <bryan@instructure.com>
specs: fix student submission selenium specs

Fixes an issue where selenium wasn't waiting long enough for the flash
object to load, which was not letting the jquery modal dialog close for
unknown reasons.

Tried closing the dialog using jquery clicks, but failures were just as
sporadic with the old 0.5 timeout.

test plan:
  - specs pass
  - celebrate!

Change-Id: Ic0caee02c9ef68bc96b676215eb5ebe8e7440ae3
Reviewed-on: https://gerrit.instructure.com/18152
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Shawn Meredith <shawn@instructure.com>
QA-Review: Shawn Meredith <shawn@instructure.com>
add inaccessibility message for record video tinymce button

test_plan:
* using a screen reader, navigate to the video record
button in the tinymce editor
* should hear a message that the feature is inaccessible

closes #CNVS-1529

Change-Id: I555d2abaf09b25da2198d8198ff5c6db6578f98f
Reviewed-on: https://gerrit.instructure.com/17958
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
QA-Review: Adam Phillipps <adam@instructure.com>
Fix modules formatting on course home page

The styling for modules on the home page was broken, so we fixed it by
ensuring that the css for modules is now included where/when the context
module page is loaded.

Test Plan:
1. Go to your course home page.
2. Make sure your course home page settings are set to show modules
by default
3. Observe that your modules are styled correctly with css.

refs #CNVS-4201

Change-Id: Id4b550cd03196e396be11a41b9c659620fd2178e
Reviewed-on: https://gerrit.instructure.com/18111
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Ryan Florence <ryanf@instructure.com>
QA-Review: Adam Phillipps <adam@instructure.com>
tabbed interface for quiz edit page

Notes:
* Changed the quiz edit page into a tabbed interface
* Quiz settings were moved from the right sidebar to the default
  tab in the main content panel.
* Questions were moved to a second tab behind the quiz settings.
* Selenium specs were modified to interact with the tabs
* Checkboxes that "enable" a text field now place focus on their
  respective text fields when checked
* Admin links ("gear" menu) added with assignment rubric and
  delete options
* Changed bottom form action buttons to Cancel and Save; save
  button now takes user back to quiz show screen
* Publish and preview buttons to be added to quiz show page

closes CNVS-2949

Change-Id: I7d8cd579ef5789e32f91610c290820cb5fd96568
Reviewed-on: https://gerrit.instructure.com/17146
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
add varied due date editor to quiz edit page

test plan:
  - new specs should pass
  - go through and make sure assignment overrides work
    for quizzes by using the editor on the quiz
    edit page.

fixes CNVS-469

Change-Id: I6b647575bb7db34235b2a1742558f6274b2605fc
Reviewed-on: https://gerrit.instructure.com/17437
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Myller de Araujo <myller@instructure.com>
updates to quiz display page

Notes:
* Added Preview and Publish to show page
* Re-styled page for teachers and students according to mockups

Test Plan:
* Publish/Preview buttons should appear on the page for teachers
  when the quiz is unpublished or has unpublished changes
* Teachers should see quiz settings details and all due dates
* Students should be able to see quiz details including their
  own due date
* All sidebar links should be styled into the Related Items menu

closes CNVS-3677

Change-Id: I4cbf1f8af887e0d36eb409a4f728e77a92a5cb89
Reviewed-on: https://gerrit.instructure.com/17754
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Myller de Araujo <myller@instructure.com>
Reviewed-by: Stanley Stuart <stanley@instructure.com>
don't show 'everyone else' when there is only one due date

Notes:
* When viewing an assignment or quiz as a teacher, the due date
  table at the bottom should show "Everyone" if there is only one
  due date for everyone, instead of "Everyone Else"
* Added integration specs to test this, and also to test that
  "Everyone" is not shown at all when all sections have specified
  due dates

fixes CNVS-4105

Change-Id: If0928bbfb1ecb3c7eedecb67b260ef2959b2ed43
Reviewed-on: https://gerrit.instructure.com/18039
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Simon Williams <simon@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
don't raise an error if an uploaded file has no original filename

Browsers tend to send a filename, but some API clients don't.

fixes CNVS-4233

test plan: do a multipart file upload to an api endpoint, such as a sis
api, and leave out the filename=/original/filename component of the
parameter. the upload should still be accepted and no error raised.

Change-Id: I9be8d425693cfb9d8d44644e57358f5dc9cee4c1
Reviewed-on: https://gerrit.instructure.com/18114
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Jacob Fugal <jacob@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
google docs front end

FIXES #CNVS-3196

this is the javascript bit that takes duanes
refactored server side for the google docs
submission and makes use of the new endpoint
to lazy-load the google docs into it's tab when
clicked.

TEST PLAN:

This should be a simple replacement of functionality
(with the addition of an ajax loader) so regression
tests on submitting google docs as assignment submissions
are all that should be necessary.

1) login as a user who has authorized with google docs
2) go to the submission page for an assignment
that allows a google doc submission
3) click the google docs tab, you should see a loader
indicator and then the google docs tree.
4) pick a document, it should not allow you to pick
more than one.
5) submit the document, it should give you a success message.

Change-Id: Idd6c85668a40ef37a09631c635f3e83db69178e9
Reviewed-on: https://gerrit.instructure.com/17915
Reviewed-by: Brian Palmer <brianp@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Hetherington <clare@instructure.com>
spec: fixing calendar2_spec.rb should render assignments due before midnight

Change-Id: I23d4e608a6d6bf7f9f21e110d657a638dc7a6414
Reviewed-on: https://gerrit.instructure.com/18141
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Shawn Meredith <shawn@instructure.com>
QA-Review: Shawn Meredith <shawn@instructure.com>
fix bug serializing an empty datepicker input

datepicker does not clear the data-date attribute when the input is cleared,
which means that you can't delete a date if you're using this method.

fixes CNVS-4005

test plan:
- with the graded discussion due date field
- set a date, save, edit, clear the date, save
- the date should be cleared

* note: you could also test this with assignment vdd, but there is another bug
that when a override date is cleared it inherits the default date from the
assignment.

Change-Id: Iad16b252d8be9a3c7120db88ef8c6783cd72c6c8
Reviewed-on: https://gerrit.instructure.com/18103
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cameron Matheson <cameron@instructure.com>
QA-Review: Amber Taniuchi <amber@instructure.com>
don't show 'use for grading' on /rubrics page

when making a rubric not associated with an assignment, 'use for grading'
should not be an option.

fixes CNVS-4224

test plan:
- go to /courses/:id/rubrics
- start creating a new rubric
- use for grading should not be an option

Change-Id: I8d43836152528beb37c7e6794de261835a36c43d
Reviewed-on: https://gerrit.instructure.com/18126
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Cameron Matheson <cameron@instructure.com>
QA-Review: Amber Taniuchi <amber@instructure.com>
ensure unread_entry_counts never drop below 0.

fixes CNVS-4028

add a before_save to discussion_topic_participant to
check/fix unread_entry_count updates that are negative.

test plan:
  * create a new discussion and post to it;
  * in the console, find the post author's associated
    discussion_topic_participant record;
  * attempt to set the record's unread_entry_count to
    a negative number and call save!;
  * verify that the unread_entry_count is saved as 0.

Change-Id: I43fe0c73814ea0493366d80c0f2dd4ee0af140ae
Reviewed-on: https://gerrit.instructure.com/18098
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Mark Ericksen <marke@instructure.com>
QA-Review: Marc LeGendre <marc@instructure.com>