Repository: greenriver/hmis-warehouse
Pull request #5179: 7348 active client details performance, add pii protection
Author: login: ttoomey
Created: 2025-03-04T01:52:49Z | Merged: 2025-03-08T04:38:08Z

## _Merging this PR_
- use the squash-merge strategy for PRs targeting a release-X branch

## Description
Fix performance issue on active clients report. It doesn't seem reasonable to render thousands of clients on page
* replace async rendering with paginated list of clients
* fix can_view_clients check
* will change the sort order to be by date, rather than client_id, date
* implements pii protection

## Type of change
Bug/performance fix

## Checklist before requesting review
[//]: # (Remove any items that are not applicable)
- [x] I have performed a self-review of my code
- [x] I have run the code that is being changed under ideal conditions, and it doesn't fail



Diff:
diff --git a/app/controllers/warehouse_reports/client_details/actives_controller.rb b/app/controllers/warehouse_reports/client_details/actives_controller.rb
index 81b73de4ac8..24da1c483d1 100644
--- a/app/controllers/warehouse_reports/client_details/actives_controller.rb
+++ b/app/controllers/warehouse_reports/client_details/actives_controller.rb
@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 ###
 # Copyright 2016 - 2025 Green River Data Analysis, LLC
 #
@@ -14,13 +16,6 @@ class ActivesController < ApplicationController
     before_action :set_limited, only: [:index]
     before_action :set_filter
 
-    background_render_action :render_section, ::BackgroundRender::ActiveClientsReportJob do
-      {
-        filter: @filter.for_params.to_json,
-        user_id: current_user.id,
-      }
-    end
-
     def index
       @report = report_source.new(filter: @filter, user: current_user)
       @filter.errors.add(:project_type_codes, message: 'are required') if @filter.project_type_codes.blank?
diff --git a/app/models/active_client_report.rb b/app/models/active_client_report.rb
index 862270ba763..cf74d7bb0c3 100644
--- a/app/models/active_client_report.rb
+++ b/app/models/active_client_report.rb
@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 ###
 # Copyright 2016 - 2025 Green River Data Analysis, LLC
 #
@@ -32,27 +34,12 @@ def value_for_display(_key, value)
     value
   end
 
-  def enrollments
-    @enrollments ||= active_client_service_history
-  end
-
-  def clients
-    @clients ||= GrdaWarehouse::Hud::Client.where(id: enrollments.keys).
-      preload(:source_clients).
-      index_by(&:id)
-  end
-
-  def active_client_service_history
-    enrollment_scope.distinct.group_by(&:client_id)
-  end
-
-  private def enrollment_scope
+  def enrollment_scope
     residential_service_history_source.joins(:client, :enrollment, :project).
-      includes(:client, :enrollment, :project).
       with_service_between(start_date: @filter.start, end_date: @filter.end).
       open_between(start_date: @filter.start, end_date: @filter.end).
       distinct.
-      order(first_date_in_program: :asc)
+      order(first_date_in_program: :asc, last_date_in_program: :asc)
   end
 
   def residential_service_history_source
diff --git a/app/models/grda_warehouse/auth_policies/deny_pii_policy.rb b/app/models/grda_warehouse/auth_policies/deny_pii_policy.rb
new file mode 100644
index 00000000000..2e1910658c9
--- /dev/null
+++ b/app/models/grda_warehouse/auth_policies/deny_pii_policy.rb
@@ -0,0 +1,26 @@
+# frozen_string_literal: true
+
+###
+# Copyright 2016 - 2025 Green River Data Analysis, LLC
+#
+# License detail: https://github.com/greenriver/hmis-warehouse/blob/production/LICENSE.md
+###
+
+# Policy that always denys PII access.
+# Used in cases where we never want to show PII (see include_pii_in_detail_downloads config)
+class GrdaWarehouse::AuthPolicies::DenyPiiPolicy
+  include Singleton
+  [
+    :can_view,
+    :can_view_name,
+    :can_view_photo,
+    :can_view_full_dob,
+    :can_view_full_ssn,
+    :can_view_hiv_status,
+  ].each do |permission|
+    method_name = :"#{permission}?"
+    define_method(method_name) do
+      false
+    end
+  end
+end
diff --git a/app/models/grda_warehouse/auth_policies/project_pii_policy.rb b/app/models/grda_warehouse/auth_policies/project_pii_policy.rb
index 9827b3d6478..82052296c70 100644
--- a/app/models/grda_warehouse/auth_policies/project_pii_policy.rb
+++ b/app/models/grda_warehouse/auth_policies/project_pii_policy.rb
@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 ###
 # Copyright 2016 - 2025 Green River Data Analysis, LLC
 #
@@ -21,6 +23,11 @@ class GrdaWarehouse::AuthPolicies::ProjectPiiPolicy < GrdaWarehouse::AuthPolicie
     end
   end
 
+  def initialize(context:, resource:)
+    resource_id = resource.is_a?(GrdaWarehouse::Hud::Project) ? resource.id : resource
+    super(context: context, resource: resource_id)
+  end
+
   protected
 
   def validate_resource!(arg)
diff --git a/app/models/grda_warehouse/hud/client.rb b/app/models/grda_warehouse/hud/client.rb
index bc211ba599f..29f0844d460 100644
--- a/app/models/grda_warehouse/hud/client.rb
+++ b/app/models/grda_warehouse/hud/client.rb
@@ -1318,11 +1318,29 @@ def policy_class
       end
     end
 
+    # pii provider for use on client dashboard
     memoize def pii_provider(user:)
       policy = user.policy_for(self)
       GrdaWarehouse::PiiProvider.new(self, policy: policy)
     end
 
+    # pii provider for use in reports and bulk view
+    def project_pii_provider(project:, user:, mode:)
+      allowed = false
+      case mode.to_sym
+      when :download
+        allowed = ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads)
+      when :browse
+        allowed = true
+      else
+        raise ArgumentError, "Bad mode #{mode}"
+      end
+
+      policy = user.policy_for(project, policy_class: GrdaWarehouse::AuthPolicies::ProjectPiiPolicy) if allowed
+      policy ||= GrdaWarehouse::AuthPolicies::DenyPiiPolicy.instance
+      GrdaWarehouse::PiiProvider.new(self, policy: policy)
+    end
+
     def name
       # Deprecated
       # skip deprecations to avoid test failures. Suggest uncommenting when we are ready to implement pii globally
diff --git a/app/views/warehouse_reports/client_details/actives/_list.haml b/app/views/warehouse_reports/client_details/actives/_list.haml
new file mode 100644
index 00000000000..386955859d3
--- /dev/null
+++ b/app/views/warehouse_reports/client_details/actives/_list.haml
@@ -0,0 +1,29 @@
+.card
+  %table.mb-0.table.table-striped
+    %thead
+      %tr
+        %th Client
+        %th Project Type
+        %th Project
+        %th Entry Date
+        %th Exit Date
+    %tbody
+      :ruby
+        list = list.preload(:client)
+        current_user.policy_context.preload_project_dependencies(list.map { |e| e.project.id })
+      - list.each do |enrollment|
+        :ruby
+          client = enrollment.client
+          project = enrollment.project
+          pii = client.project_pii_provider(project: project, user: current_user, mode: :browse)
+        %tr
+          %td
+            -# use the full client policy to check visibility
+            - if current_user.policy_for(client).can_view?
+              = link_to pii.brief_name, appropriate_client_path(id: client.id)
+            -else
+              = pii.brief_name
+          %td= HudUtility2024.project_type(enrollment.project_type)
+          %td= project.name(current_user)
+          %td= enrollment.first_date_in_program
+          %td= enrollment.last_date_in_program
diff --git a/app/views/warehouse_reports/client_details/actives/_report.haml b/app/views/warehouse_reports/client_details/actives/_report.haml
deleted file mode 100644
index 3b01778836f..00000000000
--- a/app/views/warehouse_reports/client_details/actives/_report.haml
+++ /dev/null
@@ -1,26 +0,0 @@
-.d-flex
-  %p
-    = "The following #{number_with_delimiter(@report.enrollments.count)} clients with #{number_with_delimiter(@report.enrollments.values.flatten(1).count)} enrollments, have at least one day of service between #{@filter.start} and #{@filter.end}."
-    %br
-    This report was built to provide supporting details to the "Active Clients" section of the Population Dashboards.
-
-= render 'warehouse_reports/limited_by_project_note'
-.card
-  %table.mb-0.table.table-striped
-    %thead
-      %tr
-        %th Client
-        %th Project Type
-        %th Project
-        %th Entry Date
-        %th Exit Date
-    %tbody
-      - @report.enrollments.each do |client_id, enrollments|
-        - enrollments.each do |enrollment|
-          - client = enrollment.client
-          %tr
-            %td= link_to_if current_user.can_view_clients?, "#{client.FirstName} #{client.LastName}", appropriate_client_path(id: client_id)
-            %td= HudUtility2024.project_type(enrollment.project_type)
-            %td= enrollment.project.name(current_user)
-            %td= enrollment.first_date_in_program
-            %td= enrollment.last_date_in_program
diff --git a/app/views/warehouse_reports/client_details/actives/index.haml b/app/views/warehouse_reports/client_details/actives/index.haml
index 7354a48a720..2acd0df6f9d 100644
--- a/app/views/warehouse_reports/client_details/actives/index.haml
+++ b/app/views/warehouse_reports/client_details/actives/index.haml
@@ -5,14 +5,17 @@
   .col-sm-8
     %h1= title
 = render 'filter'
-.d-flex
-  .ml-auto.mb-2
-    - if can_view_clients?
-      = render 'report_downloads/report_download', export: @pdf_export, excel_download_path: warehouse_reports_client_details_actives_path({format: :xlsx}.merge(link_params)), excel_download_tooltip: 'Download includes the following additional fields: Length of Stay (to-date), Living Situation, Destination, Race.'
-= render 'background_render', url: render_section_warehouse_reports_client_details_actives_path, fetch_params: { filter: @filter.for_params[:filters] } do
-  .rollup
-    .rollup-container.well
 
-= render 'common/section_not_loading', path_to_section: section_warehouse_reports_client_details_actives_path(link_params)
+.row.mb-5
+  .col-8
+    %p
+      - unique_clients = @report.enrollment_scope.distinct.count(:client_id)
+      = "The following #{number_with_delimiter(unique_clients)} clients with #{number_with_delimiter(@report.enrollment_scope.count)} enrollments, have at least one day of service between #{@filter.start} and #{@filter.end}."
+      %br
+      This report was built to provide supporting details to the "Active Clients" section of the Population Dashboards.
+  - if can_view_clients?
+    .col-4.text-right
+      = render 'report_downloads/report_download', export: @pdf_export, excel_download_path: warehouse_reports_client_details_actives_path({format: :xlsx}.merge(link_params)), excel_download_tooltip: 'Download includes the following additional fields: Length of Stay (to-date), Living Situation, Destination, Race.'
 
--# = render 'report'
+.my-5= render 'warehouse_reports/limited_by_project_note'
+= render_paginated_list(scope: @report.enrollment_scope, item_name: 'data set', list_partial: 'list')
diff --git a/app/views/warehouse_reports/client_details/actives/index.xlsx.axlsx b/app/views/warehouse_reports/client_details/actives/index.xlsx.axlsx
index 67ee99f78ae..d25c413dc1f 100644
--- a/app/views/warehouse_reports/client_details/actives/index.xlsx.axlsx
+++ b/app/views/warehouse_reports/client_details/actives/index.xlsx.axlsx
@@ -1,7 +1,8 @@
+# frozen_string_literal: true
 
 wb = xlsx_package.workbook
 wb.add_worksheet(name: "Active #{GrdaWarehouse::WarehouseReports::Dashboard::Base.available_sub_populations.invert[@filter.sub_population]}"[0..30]) do |sheet|
-  title = sheet.styles.add_style(sz: 12, b: true, alignment: {horizontal: :center})
+  title = sheet.styles.add_style(sz: 12, b: true, alignment: { horizontal: :center })
   sheet.add_row([
     'Warehouse Client ID',
     'First Name',
@@ -15,9 +16,14 @@ wb.add_worksheet(name: "Active #{GrdaWarehouse::WarehouseReports::Dashboard::Bas
     'Living Situation',
     'Destination',
   ] + GrdaWarehouse::Hud::Client.race_fields.map { |m| HudUtility2024.race(m).gsub('None', 'Race None') }, style: title)
-  @report.enrollments.each do |client_id, enrollments|
-    enrollments.each do |enrollment|
+
+  @report.enrollment_scope.preload(:client, :project, :enrollment).in_batches(of: 5_000).each do |batch|
+    current_user.policy_context.preload_project_dependencies(batch.map { |e| e.project.id })
+    batch.each do |enrollment|
       client = enrollment.client
+      project = enrollment.project
+      pii = client.project_pii_provider(project: project, user: current_user, mode: :download)
+
       length_of_stay = if enrollment.last_date_in_program
         enrollment.last_date_in_program - enrollment.first_date_in_program
       else
@@ -25,11 +31,11 @@ wb.add_worksheet(name: "Active #{GrdaWarehouse::WarehouseReports::Dashboard::Bas
       end
 
       row = [
-        client_id,
-        ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads) ? client.FirstName : 'Redacted',
-        ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads) ? client.LastName : 'Redacted',
+        enrollment.client_id,
+        pii.first_name,
+        pii.last_name,
         HudUtility2024.project_type(enrollment.project_type),
-        enrollment.project.name(current_user),
+        project.name(current_user),
         enrollment.first_date_in_program,
         enrollment.last_date_in_program,
         length_of_stay.to_i,
diff --git a/app/views/warehouse_reports/client_details/actives/section.haml b/app/views/warehouse_reports/client_details/actives/section.haml
deleted file mode 100644
index b6fe01ee90e..00000000000
--- a/app/views/warehouse_reports/client_details/actives/section.haml
+++ /dev/null
@@ -1 +0,0 @@
-= render 'report'
diff --git a/spec/models/auth_policies/deny_pii_policy_spec.rb b/spec/models/auth_policies/deny_pii_policy_spec.rb
new file mode 100644
index 00000000000..8aa9928d405
--- /dev/null
+++ b/spec/models/auth_policies/deny_pii_policy_spec.rb
@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+
+require 'rails_helper'
+
+RSpec.describe GrdaWarehouse::AuthPolicies::DenyPiiPolicy, type: :model do
+  let(:policy) { described_class.instance }
+  it 'denies all PII permissions' do
+    expect(policy.can_view_name?).to be false
+    expect(policy.can_view_photo?).to be false
+    expect(policy.can_view_full_dob?).to be false
+    expect(policy.can_view_full_ssn?).to be false
+    expect(policy.can_view_hiv_status?).to be false
+  end
+end


Reviews:
user: login: ttoomey
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T01:55:33Z
user: login: eanders
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T02:29:49Z
user: login: ttoomey
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T02:47:20Z
user: login: eanders
body: Just a few notes from our discussion, this looks great, thank you!
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T21:32:32Z
user: login: ttoomey
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T23:54:52Z
user: login: eanders
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-05T00:04:06Z
user: login: ttoomey
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-05T02:23:46Z
user: login: ttoomey
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-06T03:08:15Z
user: login: ttoomey
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-06T04:40:42Z
user: login: eanders
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-07T13:37:29Z
user: login: eanders
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-07T13:48:55Z
user: login: eanders
body: Thanks @ttoomey   

state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-07T13:52:10Z
user: login: ttoomey
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-07T16:40:06Z

Review comments:
diff_hunk: @@ -5,14 +5,16 @@
   .col-sm-8
     %h1= title
 = render 'filter'
-.d-flex
-  .ml-auto.mb-2
-    - if can_view_clients?
-      = render 'report_downloads/report_download', export: @pdf_export, excel_download_path: warehouse_reports_client_details_actives_path({format: :xlsx}.merge(link_params)), excel_download_tooltip: 'Download includes the following additional fields: Length of Stay (to-date), Living Situation, Destination, Race.'
-= render 'background_render', url: render_section_warehouse_reports_client_details_actives_path, fetch_params: { filter: @filter.for_params[:filters] } do
-  .rollup
-    .rollup-container.well
 
-= render 'common/section_not_loading', path_to_section: section_warehouse_reports_client_details_actives_path(link_params)
+- if can_view_clients?
path: app/views/warehouse_reports/client_details/actives/index.haml
user: login: ttoomey
body: this puts the download button and the list of clients behind can_view_clients
created_at: 2025-03-04T01:55:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -5,14 +5,16 @@
   .col-sm-8
     %h1= title
 = render 'filter'
-.d-flex
-  .ml-auto.mb-2
-    - if can_view_clients?
-      = render 'report_downloads/report_download', export: @pdf_export, excel_download_path: warehouse_reports_client_details_actives_path({format: :xlsx}.merge(link_params)), excel_download_tooltip: 'Download includes the following additional fields: Length of Stay (to-date), Living Situation, Destination, Race.'
-= render 'background_render', url: render_section_warehouse_reports_client_details_actives_path, fetch_params: { filter: @filter.for_params[:filters] } do
-  .rollup
-    .rollup-container.well
 
-= render 'common/section_not_loading', path_to_section: section_warehouse_reports_client_details_actives_path(link_params)
+- if can_view_clients?
path: app/views/warehouse_reports/client_details/actives/index.haml
user: login: eanders
body: Does this mean the entire report is behind `can_view_clients?`
I don't think that's the intention, though it does show clients, it is ok that someone sees clients within the context of a report without having `can_view_clients?`.  `can_view_clients?` is intended to cover the client dashboard.  Reports are assigned explicitly, so can potentially expose some client data without the report runner having access to the full client dashboard.  
created_at: 2025-03-04T02:29:49Z
author_association: CONTRIBUTOR
diff_hunk: @@ -5,14 +5,16 @@
   .col-sm-8
     %h1= title
 = render 'filter'
-.d-flex
-  .ml-auto.mb-2
-    - if can_view_clients?
-      = render 'report_downloads/report_download', export: @pdf_export, excel_download_path: warehouse_reports_client_details_actives_path({format: :xlsx}.merge(link_params)), excel_download_tooltip: 'Download includes the following additional fields: Length of Stay (to-date), Living Situation, Destination, Race.'
-= render 'background_render', url: render_section_warehouse_reports_client_details_actives_path, fetch_params: { filter: @filter.for_params[:filters] } do
-  .rollup
-    .rollup-container.well
 
-= render 'common/section_not_loading', path_to_section: section_warehouse_reports_client_details_actives_path(link_params)
+- if can_view_clients?
path: app/views/warehouse_reports/client_details/actives/index.haml
user: login: ttoomey
body: Thanks for the explanation, I'll adjust it back so that only the download button is protected by the condition
created_at: 2025-03-04T02:47:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,18 @@
+.card
+  %table.mb-0.table.table-striped
+    %thead
+      %tr
+        %th Client
+        %th Project Type
+        %th Project
+        %th Entry Date
+        %th Exit Date
+    %tbody
+      - list.preload(:client).each do |enrollment|
+        - client = enrollment.client
+        %tr
+          %td= link_to_if current_user.can_view_clients?, "#{client.FirstName} #{client.LastName}", appropriate_client_path(id: client.id)
path: app/views/warehouse_reports/client_details/actives/_list.haml
user: login: eanders
body: Let's implement the PII provider for name
created_at: 2025-03-04T21:10:50Z
author_association: CONTRIBUTOR
diff_hunk: @@ -32,23 +34,8 @@ def value_for_display(_key, value)
     value
   end
 
-  def enrollments
-    @enrollments ||= active_client_service_history
-  end
-
-  def clients
-    @clients ||= GrdaWarehouse::Hud::Client.where(id: enrollments.keys).
-      preload(:source_clients).
-      index_by(&:id)
-  end
-
-  def active_client_service_history
-    enrollment_scope.distinct.group_by(&:client_id)
-  end
-
-  private def enrollment_scope
+  def enrollment_scope
path: app/models/active_client_report.rb
user: login: eanders
body: Change order to `first_date_in_program, last_date_in_program`
created_at: 2025-03-04T21:11:28Z
author_association: CONTRIBUTOR
diff_hunk: @@ -15,29 +15,27 @@ wb.add_worksheet(name: "Active #{GrdaWarehouse::WarehouseReports::Dashboard::Bas
     'Living Situation',
     'Destination',
   ] + GrdaWarehouse::Hud::Client.race_fields.map { |m| HudUtility2024.race(m).gsub('None', 'Race None') }, style: title)
-  @report.enrollments.each do |client_id, enrollments|
-    enrollments.each do |enrollment|
-      client = enrollment.client
-      length_of_stay = if enrollment.last_date_in_program
-        enrollment.last_date_in_program - enrollment.first_date_in_program
-      else
-        Date.current - enrollment.first_date_in_program
-      end
-
-      row = [
-        client_id,
-        ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads) ? client.FirstName : 'Redacted',
-        ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads) ? client.LastName : 'Redacted',
-        HudUtility2024.project_type(enrollment.project_type),
-        enrollment.project.name(current_user),
-        enrollment.first_date_in_program,
-        enrollment.last_date_in_program,
-        length_of_stay.to_i,
-        enrollment.enrollment_group_id,
-        HudUtility2024.living_situation(enrollment.enrollment&.LivingSituation),
-        HudUtility2024.destination(enrollment.destination),
-      ] + GrdaWarehouse::Hud::Client.race_fields.map { |m| HudUtility2024.no_yes_reasons_for_missing_data(client.public_send(m.to_sym)&.to_i) }
-      sheet.add_row(row)
+  @report.enrollment_scope.each do |enrollment|
+    client = enrollment.client
+    length_of_stay = if enrollment.last_date_in_program
+      enrollment.last_date_in_program - enrollment.first_date_in_program
+    else
+      Date.current - enrollment.first_date_in_program
     end
+
+    row = [
+      enrollment.client_id,
+      ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads) ? client.FirstName : 'Redacted',
+      ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads) ? client.LastName : 'Redacted',
path: app/views/warehouse_reports/client_details/actives/index.xlsx.axlsx
user: login: eanders
body: This should use PII provider too
created_at: 2025-03-04T21:12:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -15,29 +16,30 @@ wb.add_worksheet(name: "Active #{GrdaWarehouse::WarehouseReports::Dashboard::Bas
     'Living Situation',
     'Destination',
   ] + GrdaWarehouse::Hud::Client.race_fields.map { |m| HudUtility2024.race(m).gsub('None', 'Race None') }, style: title)
-  @report.enrollments.each do |client_id, enrollments|
-    enrollments.each do |enrollment|
-      client = enrollment.client
-      length_of_stay = if enrollment.last_date_in_program
-        enrollment.last_date_in_program - enrollment.first_date_in_program
-      else
-        Date.current - enrollment.first_date_in_program
-      end
 
-      row = [
-        client_id,
-        ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads) ? client.FirstName : 'Redacted',
-        ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads) ? client.LastName : 'Redacted',
-        HudUtility2024.project_type(enrollment.project_type),
-        enrollment.project.name(current_user),
-        enrollment.first_date_in_program,
-        enrollment.last_date_in_program,
-        length_of_stay.to_i,
-        enrollment.enrollment_group_id,
-        HudUtility2024.living_situation(enrollment.enrollment&.LivingSituation),
-        HudUtility2024.destination(enrollment.destination),
-      ] + GrdaWarehouse::Hud::Client.race_fields.map { |m| HudUtility2024.no_yes_reasons_for_missing_data(client.public_send(m.to_sym)&.to_i) }
-      sheet.add_row(row)
+  @report.enrollment_scope.preload(:client, :project, :enrollment).find_each do |enrollment|
+    client = enrollment.client
+    pii = client.pii_provider(user: current_user, context: :download)
path: app/views/warehouse_reports/client_details/actives/index.xlsx.axlsx
user: login: ttoomey
body: this introduces a lot of n+1 queries. Perhaps we should use ProjectPiiPolicy ?
created_at: 2025-03-04T23:54:52Z
author_association: CONTRIBUTOR
diff_hunk: @@ -15,29 +16,30 @@ wb.add_worksheet(name: "Active #{GrdaWarehouse::WarehouseReports::Dashboard::Bas
     'Living Situation',
     'Destination',
   ] + GrdaWarehouse::Hud::Client.race_fields.map { |m| HudUtility2024.race(m).gsub('None', 'Race None') }, style: title)
-  @report.enrollments.each do |client_id, enrollments|
-    enrollments.each do |enrollment|
-      client = enrollment.client
-      length_of_stay = if enrollment.last_date_in_program
-        enrollment.last_date_in_program - enrollment.first_date_in_program
-      else
-        Date.current - enrollment.first_date_in_program
-      end
 
-      row = [
-        client_id,
-        ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads) ? client.FirstName : 'Redacted',
-        ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads) ? client.LastName : 'Redacted',
-        HudUtility2024.project_type(enrollment.project_type),
-        enrollment.project.name(current_user),
-        enrollment.first_date_in_program,
-        enrollment.last_date_in_program,
-        length_of_stay.to_i,
-        enrollment.enrollment_group_id,
-        HudUtility2024.living_situation(enrollment.enrollment&.LivingSituation),
-        HudUtility2024.destination(enrollment.destination),
-      ] + GrdaWarehouse::Hud::Client.race_fields.map { |m| HudUtility2024.no_yes_reasons_for_missing_data(client.public_send(m.to_sym)&.to_i) }
-      sheet.add_row(row)
+  @report.enrollment_scope.preload(:client, :project, :enrollment).find_each do |enrollment|
+    client = enrollment.client
+    pii = client.pii_provider(user: current_user, context: :download)
path: app/views/warehouse_reports/client_details/actives/index.xlsx.axlsx
user: login: eanders
body: That works for me.  I wonder if we should plan to use ProjectPiiPolicy for reports for consistency.
created_at: 2025-03-05T00:04:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -15,29 +16,30 @@ wb.add_worksheet(name: "Active #{GrdaWarehouse::WarehouseReports::Dashboard::Bas
     'Living Situation',
     'Destination',
   ] + GrdaWarehouse::Hud::Client.race_fields.map { |m| HudUtility2024.race(m).gsub('None', 'Race None') }, style: title)
-  @report.enrollments.each do |client_id, enrollments|
-    enrollments.each do |enrollment|
-      client = enrollment.client
-      length_of_stay = if enrollment.last_date_in_program
-        enrollment.last_date_in_program - enrollment.first_date_in_program
-      else
-        Date.current - enrollment.first_date_in_program
-      end
 
-      row = [
-        client_id,
-        ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads) ? client.FirstName : 'Redacted',
-        ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads) ? client.LastName : 'Redacted',
-        HudUtility2024.project_type(enrollment.project_type),
-        enrollment.project.name(current_user),
-        enrollment.first_date_in_program,
-        enrollment.last_date_in_program,
-        length_of_stay.to_i,
-        enrollment.enrollment_group_id,
-        HudUtility2024.living_situation(enrollment.enrollment&.LivingSituation),
-        HudUtility2024.destination(enrollment.destination),
-      ] + GrdaWarehouse::Hud::Client.race_fields.map { |m| HudUtility2024.no_yes_reasons_for_missing_data(client.public_send(m.to_sym)&.to_i) }
-      sheet.add_row(row)
+  @report.enrollment_scope.preload(:client, :project, :enrollment).find_each do |enrollment|
+    client = enrollment.client
+    pii = client.pii_provider(user: current_user, context: :download)
path: app/views/warehouse_reports/client_details/actives/index.xlsx.axlsx
user: login: ttoomey
body: yeah, we could do that- I also wonder if we should plan implement bulk-preload for SouceClientPolicy and then replace ProjectPiiPolicy. If that seems reasonable I might leave this as-is for now. But if that seems unrealistic then I'll switch this to ProjectPiiPolicy, the only issue is that it doesn't handle all the other auth logic (roi, etc)
created_at: 2025-03-05T02:23:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -15,29 +16,30 @@ wb.add_worksheet(name: "Active #{GrdaWarehouse::WarehouseReports::Dashboard::Bas
     'Living Situation',
     'Destination',
   ] + GrdaWarehouse::Hud::Client.race_fields.map { |m| HudUtility2024.race(m).gsub('None', 'Race None') }, style: title)
-  @report.enrollments.each do |client_id, enrollments|
-    enrollments.each do |enrollment|
-      client = enrollment.client
-      length_of_stay = if enrollment.last_date_in_program
-        enrollment.last_date_in_program - enrollment.first_date_in_program
-      else
-        Date.current - enrollment.first_date_in_program
-      end
 
-      row = [
-        client_id,
-        ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads) ? client.FirstName : 'Redacted',
-        ::GrdaWarehouse::Config.get(:include_pii_in_detail_downloads) ? client.LastName : 'Redacted',
-        HudUtility2024.project_type(enrollment.project_type),
-        enrollment.project.name(current_user),
-        enrollment.first_date_in_program,
-        enrollment.last_date_in_program,
-        length_of_stay.to_i,
-        enrollment.enrollment_group_id,
-        HudUtility2024.living_situation(enrollment.enrollment&.LivingSituation),
-        HudUtility2024.destination(enrollment.destination),
-      ] + GrdaWarehouse::Hud::Client.race_fields.map { |m| HudUtility2024.no_yes_reasons_for_missing_data(client.public_send(m.to_sym)&.to_i) }
-      sheet.add_row(row)
+  @report.enrollment_scope.preload(:client, :project, :enrollment).find_each do |enrollment|
+    client = enrollment.client
+    pii = client.pii_provider(user: current_user, context: :download)
path: app/views/warehouse_reports/client_details/actives/index.xlsx.axlsx
user: login: ttoomey
body: per team check in about reporting perms, the ProjectPiiPolicy sounds like the best option
created_at: 2025-03-06T03:08:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -21,6 +23,11 @@ class GrdaWarehouse::AuthPolicies::ProjectPiiPolicy < GrdaWarehouse::AuthPolicie
     end
   end
 
+  def initialize(context:, resource:)
path: app/models/grda_warehouse/auth_policies/project_pii_policy.rb
user: login: ttoomey
body: allow this policy to take a project instead of just an id
created_at: 2025-03-06T04:40:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+###
+# Copyright 2016 - 2025 Green River Data Analysis, LLC
+#
+# License detail: https://github.com/greenriver/hmis-warehouse/blob/production/LICENSE.md
+###
+
+class GrdaWarehouse::AuthPolicies::NullPiiPolicy
path: app/models/grda_warehouse/auth_policies/null_pii_policy.rb
user: login: eanders
body: Is this really a `false` policy?  I would think a `null` policy might have unknown consequences.

Either way, can you add a comment about what this is doing and when it is appropriate to use?
created_at: 2025-03-07T13:37:29Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1318,11 +1318,29 @@ def policy_class
       end
     end
 
+    # pii provider for use on client dashboard
     memoize def pii_provider(user:)
       policy = user.policy_for(self)
       GrdaWarehouse::PiiProvider.new(self, policy: policy)
     end
 
+    # pii provider for use in reports and bulk view
path: app/models/grda_warehouse/hud/client.rb
user: login: eanders
body: This looks like a really nice helper, thanks!
created_at: 2025-03-07T13:48:55Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+###
+# Copyright 2016 - 2025 Green River Data Analysis, LLC
+#
+# License detail: https://github.com/greenriver/hmis-warehouse/blob/production/LICENSE.md
+###
+
+class GrdaWarehouse::AuthPolicies::NullPiiPolicy
path: app/models/grda_warehouse/auth_policies/null_pii_policy.rb
user: login: ttoomey
body: Sure thing, added a comment, renamed to to DenyPiiPolicy to better match the behavior
created_at: 2025-03-07T16:40:05Z
author_association: CONTRIBUTOR

PR comments:
user: login: ttoomey
created_at: 2025-03-04T02:52:18Z
body: Trivy is still throwing an error even though we have updated the vulnerable gems. Probably a cache but not sure how to clear it.
author_association: CONTRIBUTOR
user: login: ttoomey
created_at: 2025-03-04T13:45:29Z
body: moving back to draft
author_association: CONTRIBUTOR
user: login: ttoomey
created_at: 2025-03-07T16:42:55Z
body: @eanders my inclination would be to merge this to 154 and send to QA; or do you think this should go in as a hotfix to 153?
author_association: CONTRIBUTOR
user: login: eanders
created_at: 2025-03-07T17:02:41Z
body: > @eanders my inclination would be to merge this to 154 and send to QA; or do you think this should go in as a hotfix to 153?

I agree, let's retarget 154.  Looks good to me!
author_association: CONTRIBUTOR