spec: filter allowable_services by proc too

Change-Id: I61e43e09f9e3e67fb7cfa5815bfe48d13d52433c
Reviewed-on: https://gerrit.instructure.com/21299
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
activate the user's shard when consulting cassandra

refs CNVS-6018

test plan:
 * saving and loading of cassandra page views should work

Change-Id: Ic9730371bb1f4207cb06aa6da30baee9f4232d8f
Reviewed-on: https://gerrit.instructure.com/20979
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
optimize Setting.from_config

refs CNVS-6018

cache the entire file, not just for the current env

test plan:
 * canvas should still boot and stuff

Change-Id: I09068452c9bafb3b3208b49abe3d19cf9fce1e8a
Reviewed-on: https://gerrit.instructure.com/20976
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Duane Johnson <duane@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
improve Cassandra helper methods

refs CNVS-6018

 * allow specifying which environment you want to check is configured/
   get a connection for
 * properly cache nil results

test plan:
 * cassandra page views should still save and load

Change-Id: I14c667182a38da4a5c64e7685dc9db81d45fcf07
Reviewed-on: https://gerrit.instructure.com/20978
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Brian Palmer <brianp@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
add cassandra cluster info to CQL log lines

test plan:
  * enable cassandra page views
  * visit /users/X for a user with page views
  * search the rails log file for CQL lines, they should include the
    cluster name and the environment name like "[page_views development]"

Change-Id: I61903e55c01c818f33022f2971905951a31b8f0e
Reviewed-on: https://gerrit.instructure.com/21302
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Brian Palmer <brianp@instructure.com>
add permissions check to show the "new conference" button

the check was lost in the conferences interface upgrade

fixes CNVS-6195

test plan:
 - go to account permissions and turn off "create web
   conferences" for students.
 - go to student view of conferences index and make sure the
   "new conferences" button does not show.
 - turn the "create web conferences" permission on for students.
 - go to student view of conferences index and make sure the
   "new conferences" button is there.

Change-Id: I1aae399dcf39de61d104371e04aac24721a445bc
Reviewed-on: https://gerrit.instructure.com/21258
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Cam Theriault <cam@instructure.com>
Reviewed-by: Zach Pendleton <zachp@instructure.com>
Product-Review: Jon Willesen <jonw@instructure.com>
removing deleted accounts from the sub_accounts api call

this fixes a bug in the sub_accounts api that caused all
sub accounts to be returned, regardless of workflow state.
additionally fixes issues with the link headers that are
returned by this call.

test plan:
 * create an account with sub accounts
 * add a few sub accounts to the primary sub accounts
 * delete one or two of the accounts you just created
 * make an api call for sub-acccounts
   - use the recursive parameter
   - use the per_page param to be less then then number of active sub accounts
 * check to make sure that the deleted accounts do not show up in the results
 * check the link headers to make sure that they are being generated correctly

Change-Id: I8638f7ef18338c2aa9854c823fc5825834bb0c0c
Reviewed-on: https://gerrit.instructure.com/21230
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Strong <clare@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Product-Review: Brad Humphrey <brad@instructure.com>
add "pinned" discussions to index view.

fixes CNVS-5708

discussion topics can now be pinned to preserve their
position at the top of the discussion topics index page.
pinned discussions can be manually sorted, while all
other topics remain sorted by most recent activity.

test plan:
  * in a course with existing discussion topics, attempt
    to drag a topic into the pinned area and verify that
    it properly moves to that group and persists on page
    refresh;
  * move multiple topics into the pinned area and verify
    that they can be manually sorted and that the sort
    persists on page refresh;
  * drag a pinned topic to the open discussion area and
    verify that it properly moves there and persists on
    page refresh;
  * verify that pinned and open topics cannot be dragged
    into the locked area;
  * verify that open topics offer a pin option in their
    gear menus and that pinned topics offer an unpin
    option. verify that these options work as expected.

Change-Id: I65dccba5eed11dd8b0996ecd930b93a1d61dbebc
Reviewed-on: https://gerrit.instructure.com/21260
Reviewed-by: Zach Pendleton <zachp@instructure.com>
Product-Review: Zach Pendleton <zachp@instructure.com>
QA-Review: Zach Pendleton <zachp@instructure.com>
Tested-by: Zach Pendleton <zachp@instructure.com>
don't silently eat failure in cassandra page views

if recording a page view fails, we want to handle and stop the failure
so it doesn't fail out the whole batch. but we don't want to do so
silently; log the error instead. (in the case of cassandra page views,
this failure will actually be a failure to log the failure in the
event_stream_failures table.)

fixes CNVS-6199

test-plan:
  - turn off cassandra for page views but leave redis queueing on
  - queue a page view (via script/console) and note its request id
  - queue a second page view with the same request id
  - queue a third page view with a different request id
  - process the queue (PageView.process_cache_queue)
  - the first and third page views should be inserted into the database
  - there should be crit-level lines in the log about the failure of the
    second page view

Change-Id: Ie1849a73d8f625ab8f90c8c15ddff5f9150a0de4
Reviewed-on: https://gerrit.instructure.com/21263
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
add default_group_storage_quota setting to accounts

test plan:
* in account settings - quotas tab,
 set the default group storage quota
* attempt to upload files to a new group (without a set
quota) until the quota is hit or exceeded, confirming
that the set default is in effect

refs #CNVS-271

Change-Id: Ia4cbd84da19eb07d188dcc85c81c6b45bc2fa7ce
Reviewed-on: https://gerrit.instructure.com/21191
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
Product-Review: Bracken Mosbacker <bracken@instructure.com>
QA-Review: Clare Strong <clare@instructure.com>
change available_for? and locked_for? to be more consistent

test plan: no functional change

refs #CNVS-5837

Change-Id: I07d942840e30df15883d484fcc4c6d0f86ff477d
Reviewed-on: https://gerrit.instructure.com/21156
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Clare Strong <clare@instructure.com>
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
Product-Review: Bracken Mosbacker <bracken@instructure.com>
change APIs to return consistent locking information

affected apis:
 * assignments
 * discussions
 * files
 * pages
 * quizzes

test plan:
 * verify returned attributes:
   * locked - manually locked state of the item
   * locked_for_user - user's access to the item
     (false will always be returned for a teacher)
   * lock_explanation - explanation of the reason for the lock
     (if locked_for_user)
   * lock_info - detailed information about the reason for the lock
     (if locked_for_user)

fixes #CNVS-5837

Change-Id: I0c397a218180d58da7a2f710d9977d9ddb7745fc
Reviewed-on: https://gerrit.instructure.com/21212
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: James Williams  <jamesw@instructure.com>
QA-Review: Clare Strong <clare@instructure.com>
Product-Review: Bracken Mosbacker <bracken@instructure.com>
add section to set course and group quota directly from accounts

test plan:
* go to the accounts settings -> quotas tab
* note the new section on the bottom
 ('Manually Settable Quotas')
* should be able to select a course or group
 by their ID, and set their quota

closes #CNVS-271 #CNVS-6060

Change-Id: Iaa5336c792aa0447bc96f8b61260ee655ad40adc
Reviewed-on: https://gerrit.instructure.com/21285
QA-Review: Bracken Mosbacker <bracken@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
Product-Review: Bracken Mosbacker <bracken@instructure.com>
don't load entire submissions and quiz_submissions into memory

for user merge

test plan:
 * merge together some users that have both taken the same quiz
 * it should not fail

Change-Id: I57c9624b6f41d903b7e6bcc6436da348dddd1ccd
Reviewed-on: https://gerrit.instructure.com/20770
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
Allow clicking on label text to check checkbox

fixes CNVS-6125

Test Plan
1. Go to the new migrations page and migrate
something that you can select content.
2. Labels can be clicked to check the checkbox

Change-Id: I21244b7a89f5482797017962d03c089d4d378ea6
Reviewed-on: https://gerrit.instructure.com/21324
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
Product-Review: Bracken Mosbacker <bracken@instructure.com>
QA-Review: Bracken Mosbacker <bracken@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
return selective sub-item urls

for the convenience of api users

Test Plan:
 * The top-level return value of the selective api call
   should include urls to the sub-items lists
 * the top-level url is: api/v1/courses/:course_id/content_migrations/:id/selective_data

refs CNVS-6061

Change-Id: I35b9288a1471dcbe66462703636e9b45dcafa31e
Reviewed-on: https://gerrit.instructure.com/21249
QA-Review: Clare Strong <clare@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
Product-Review: Bracken Mosbacker <bracken@instructure.com>
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
add sublevel checkbox support to content migration

fixes #CNVS-6061

This adds sub level checkboxes that are dynamically loaded when
selecting a top level checkbox. For instance, unchecking the 'files'
toplevel checkbox now shows all of the files being migrated.

Test plan
1. Create a course to be migrated with nested assignments, files,
folders, modules etc.
2. Migrate that course by copying it and checking the 'select content'
checkbox, which will then allow you to click the 'select content'
button.
3. Uncheck the toplevel checkboxes to see the sub-level checkboxes. They
should all be there and labeled correctly.
4. Checking and unchecking a sub-level checkbox should check/uncheck
it's sub sub-level checkboxes

Change-Id: I0fe3be77d2c94f885269bb679280b4e8a5893ce5
Reviewed-on: https://gerrit.instructure.com/21305
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Bracken Mosbacker <bracken@instructure.com>
Product-Review: Bracken Mosbacker <bracken@instructure.com>
QA-Review: Bracken Mosbacker <bracken@instructure.com>
replace submissions.late column with .cached_due_date

refs CNVS-5805

with efficient calculation of all due dates for any submissions for a
given assignment when related records (the assignment, its overrides,
related enrollments, and related group memberships) changes.

compares this cached due date to the submitted_at or current time when
determining lateness.

populates the column for existing submissions in a post-deploy
data-fixup migration.

test-plan:
  - run lib/data_fixup/initialize_submission_cached_due_date.rb
  - all submissions' cached_due_dates should be updated over several
    jobs
  - enroll a student in a course and create submissions in that course
    - create a second enrollment in a second section; the
      cached_due_dates for the user's submissions should recalculate
    - destroy the second enrollment; the cached_due_dates for the user's
      submissions should recalculate
    - create a group assignment
      - add the student to a group in the assignment's category; the
        cached_due_dates for the user's submissions should recalculate
      - remove the student from the group; the cached_due_dates for the
        user's submissions should recalculate
  - enroll more students in the course
    - change an assignment's due date; the cached_due_dates for the
      assignment's submissions should recalculate
    - create an override for the assignment; the cached_due_dates for
      the assignment's submissions should recalculate
    - change the due date on the override; the cached_due_dates for the
      assignment's submissions should recalculate
    - delete the override; the cached_due_dates for the assignment's
      submissions should recalculate
  - during any of the above recalculations:
    - the most lenient applicable override should apply
    - if the most lenient applicable override is more stringent than the
      assignment due_at, it should still apply
    - the assignment due_at should apply if there are no applicable
      overrides

Change-Id: Ibacab27429a76755114dabb1e735d4b3d9bbd2fc
Reviewed-on: https://gerrit.instructure.com/21123
Reviewed-by: Brian Palmer <brianp@instructure.com>
Product-Review: Jacob Fugal <jacob@instructure.com>
QA-Review: Jacob Fugal <jacob@instructure.com>
Tested-by: Jacob Fugal <jacob@instructure.com>
query pg if a fk exists, instead of relying on failure

refs CNVS-6202

Change-Id: Iac287adea8456c3b8bb48df42be696720787043c
Reviewed-on: https://gerrit.instructure.com/21276
Tested-by: Jenkins <jenkins@instructure.com>
Reviewed-by: Brian Palmer <brianp@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
add foreign keys part 1

refs CNVS-6202

Change-Id: Iff57a7b76d1bccb911c0350f69b82daebf9e519f
Reviewed-on: https://gerrit.instructure.com/21266
Reviewed-by: Brian Palmer <brianp@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Tested-by: Cody Cutrer <cody@instructure.com>