DA - fix N+1 on visible_assignments

test plan:
  - with da off, with da on, as teacher, as student
    - assignment group index works
    - assignment group api wors
    - gradebooks works
    - grade summary page works

Change-Id: I73c26f7f754ab738f67e936dcddad7cdfa5b86b6
Reviewed-on: https://gerrit.instructure.com/38365
Reviewed-by: Cameron Matheson <cameron@instructure.com>
Tested-by: Jenkins <jenkins@instructure.com>
QA-Review: Anna Koalenz <akoalenz@instructure.com>
Product-Review: Simon Williams <simon@instructure.com>