# States’ Risk Management Practices, Tools and Resources & Sub-Grantee Risk-based Monitoring

Seven state risk management tools, resources and practices are highlighted below. These examples spotlight a combination of approaches states have taken in efforts to identify risk and better understand how sub-grantees are actually managing funds. In some cases, states have designed a prescriptive and detailed method of assigning and calculating risk levels; in other cases, states have established general monitoring practices and developed accompanying protocols, tools and instruments to conduct monitoring and oversight activities.  

### Georgia:Financial Risk Assessment Rubric 

Georgia Department of Education (GADOE) developed a risk assessment rubric in 2011. The risk rubric ranks all 195 Local Education Agencies (LEAs). GADOE’s risk assessment rubric contains 19 categories or risk areas, many of which are audit related data, including: number of financial statement findings, number of Federal award findings, auditor’s opinion (from prior year audit), material weakness, and fund deficit. In FY2016, GADOE will add three risk areas: change in audit organization, audit issues and FY14 operating deficit. Points are assigned to each risk category, and a composite score determines the LEAs overall risk level. In 2015, GADOE identified 14 high-risk LEAs using this risk assessment rubric. This risk rubric is a comprehensive system that uses both Single Audit data and program monitoring findings to rate all of the SEA’s federal grant programs. The Alabama State Department of Education (ALSDE) has benchmarked Georgia’s financial risk rubric to support its efforts to develop a similar LEA risk rating system.  

### Arkansas:Financial and Programmatic Monitoring Protocol & Prioritized Monitoring of Designated High-risk LEAs

Arkansas Department of Education (ADE) established a high-risk grantee designation for its LEAs. ADE conducts an internal review of its LEAs and assigns high risk to entities that have significant compliance and accountability issues. Based on this designation, ADE conducts more frequent and rigorous monitoring of those select LEAs, outside of its general six year monitoring review schedule. ADE reviews both programmatic and financial areas of sub-grantees’ profiles in order to make these high-risk determinations. As a part of the LEA monitoring process, Arkansas Department of Education also conducts a review of LEAs’ internal control systems, particularly as they relate to implementing the grant Assurances that sub-grantees attest to in receipt of federal grant funds. ADE has a comprehensive financial and programmatic monitoring protocol it uses during federal grant program monitoring activities. (http://www.arkansased.gov/public/userfiles/Public_School_Accountability/FederalState_Monitoring/2014_2015MonitoringProtocol.pdf) 

### Michigan:Risk Factors and Reference Tool

Michigan Department of Education (MDE) developed risk factors and an accompanying reference tool in 2010, to score and rank all LEAs. The reference tool identifies ten scoring categories or risk areas, including: Title I Allocation, Deficit Districts, Single Audit Findings, and Graduation Rate. These ten categories are broken out into two sub-groups: financial factors and academic risk factors. Each of the 10 risk areas is assigned a score with values from 0-4, and blank. (A blank submission represents no score assigned and/or risk area found to be not applicable.) LEAs are then assigned composite scores based on the numbers/responses, and are then ranked from highest risk score (33) to lowest risk score (0). Michigan uses this risk assessment technique to determine when and which LEAs will be visited for detailed monitoring. MDE also identifies LEAs most in need of review based on financial and/or academic risk areas and can schedule appropriate visits based on either financial and/or academic area; whichever demonstrates higher risk. MDE does visit all LEAs every five years, but the use of this tool-- which is updated annually--helps the SEA to better prioritize need, and determine if the LEA needs a financial or academic review (or both). Maryland has identified Michigan’s risk reference tool and procedures as a resource for the development of its Internal Audit office. 

### Minnesota:

(https://www.charterschoolcenter.org/sites/default/files/files/field_webinar_attachments/Data%20Management%20Tools%20for%20Risk%20Based%20Monitoring%20%20_0.pdf) 

### Serve DC:Grant Administration Policies and Procedures—Grantee Evaluation/Risk Level Assessment 

Serve DC, The Mayor’s Office on Volunteerism, of the District of Columbia (DC), developed and implemented a risk based monitoring approach for local education agencies in February 2009. The risk-based monitoring system utilizes a risk scoring rating from low to high. Risk levels are determined by site visits, desk audits, progress reports and/or recommendations from the Corporation for National and Community Service (CNCS) or the Department of Homeland Security. ED federal grant programs do not use this risk rating system. The scoring rubric is used to analyze CNCS grantees during the review and/or monitoring activity. The risk-based monitoring tool incorporates three general categories: program, participant and financial management. Each category contains 6-10 elements that are each scored numerically from 1-3, with 3 indicating high risk and 1 representing low risk. Scores are then tallied up to demonstrate a combined final score. The low-risk category scoring range is 26-39, moderate is 40-60 and high-risk is 61-78. Serve DC uses this risk assessment system to determine what, if any, significant risk grantees may possess, and follows up as needed with additional technical assistance and in some cases sanctions. 

(http://serve.dc.gov/sites/default/files/dc/sites/serve/publication/attachments/Grant_Admin_Policies_Procedures_2008-2009.pdf)

Similarly, the DC Office of the State Superintendent of Education (DC OSSE) developed a comprehensive monitoring guidance for federal grant programs.

(https://osse.dc.gov/sites/default/files/dc/sites/osse/publication/attachments/Risk-Based%20Monitoring%20Guidance.pdf)

### California:California Compliance Monitoring---California Monitoring Tool 

The California Department of Education (CDE) has two web based resources for managing its sub-grantees’ monitoring information: the California Monitoring Tool (CMT) and the California Accountability and Improvement System (CAIS).  However, the CAIS will be phased out soon, with the CMT operating as the central web-based portal for housing LEAs’ monitoring information. The CMT should be fully operational by late summer 2015. The CMT web based portal serves as the infrastructure of CDE’s overall risk-based monitoring approach. CDE is able to identify which local education agencies will receive a Federal Program Monitoring (FPM) review by the Federal Program Monitoring Office and is able to classify reviews by monitoring cycles A, B, C or D. This monitoring cycle classification allows CDE to use risk factors to make monitoring cycle and frequency decisions. Risk factors considered are: academic achievement, fiscal analysis, program size, and compliance history. The risk-based monitoring approach is supported by a wealth of online resources, including: the monitoring instrument, review cycle, review type, frequently asked questions, monitoring selection criteria, trainings and a database for unresolved findings. CDE maintains a transparent and comprehensive website with multiple grants management and monitoring resources, and access to the web-based oversight tools. The CMT is permission access only so users have to establish appropriate login credentials. 

(http://www.cde.ca.gov/ta/cr/selectioncriteria1112.asp)

(http://www.cde.ca.gov/ta/cr/#CMT)

(http://www.cde.ca.gov/ta/cr/#CAIS)

### Texas: Division of Federal Fiscal Monitoring

The Texas Education Agency (TEA) has an established Division of Federal Fiscal Monitoring (DFFM). DFFM selects federal sub-grantees to review and utilizes single audit data findings to determine risk levels and identify grantees in most need for review.  DFFM has online resources for sub-grantees to better understand what financial areas TEA will review and baseline financial management standards, including: federal reporting and accounting, payroll samples and reimbursement requests training manual for high-risk grantees. TEA also has a *Financial Accountability System Resource Guide*(*FASRG*) that describes the rules of financial accounting for school districts, charter schools, and education service centers. This serves as a resource for local education agencies to follow and guide in the development of their financial systems and controls. Once sub-grantees have been reviewed, the TEA follows up with a corrective action plan. TEA’s Office for Grants and Federal Fiscal Compliance (OGFFC), located within the DFFM, has a formalized corrective action framework that describes the areas of noncompliance identified; the required corrective action, including any enforcement actions; and the required completion date. TEA has a transparent monitoring system, with identified standards, expectations and supportive resources.

(http://tea.texas.gov/Finance_and_Grants/Grants/Federal_Fiscal_Monitoring/Federal_Fiscal_Monitoring_Division/)

### Florida:Quality Assurance and Compliance Monitoring

The Florida Department of Education (FLDOE), Division of Career and Adult Education, has implemented an annual monitoring training for sub-grantees. FLDOE developed a compliance tool for this training. The tool outlines the SEA’s risk assessment process, goals, visions and objectives. Annual assessment factors are generated and a risk matrix is developed for each sub-grantee. FLDOE risk assessment considers sub-grantees’ number of grants, amount of agency funding received, prior audit/monitoring findings, changes in Program Director and the existence of a Program Improvement Plan (PIP). Metrics are weighted and scores are assigned to denote the level of risk. Based on the total risk score, FLDOE makes a determination as to whether the sub-grantee presents high-risk and will be monitored during the upcoming fiscal year. FLDOE makes its monitoring and oversight process very transparent, providing a list of districts and/or colleges that need focused attention and what areas the state will be looking at. FLDOE also provides information about sub-grantees’ most common monitoring findings, thus providing an additional level of technical assistance and training on discrete grants management and operational areas.   

(http://www.fldoe.org/core/fileparse.php/5423/urlt/providertraningspr15.pdf)

### Recommendations

Based on the analysis of current state practices, the consideration of States’ resources and existing statutory requirements, agencies can and should implement some form of risk-based monitoring. The details and dynamics of the risk based monitoring system need not be overly prescriptive, but at the very least, it should be developed, articulated and executed consistently. 

The following recommendations are suggestions for potential steps State Education Agencies (SEAs) can take in efforts to improve their sub-grantee oversight activities. 

· ***Develop a comprehensive risk-based monitoring strategy***

1.) Consider automation and web-based resources in managing the risk-based monitoring.

2.) Utilize a risk approach to prioritize and rank grantees selected for monitoring activities. The risk approach should include a risk rubric that facilitates a systemized and transparent ranking and scoring system. 

a. Identify grantees with the greatest need and prioritize monitoring activities based on the initial risk assessment results.

b. Use the risk approach and/or tool(s) consistently 

3.) Provide training, technical assistance and support to grantees about the risk assessment process.

a. Ensure a high-level of visibility and transparency of the process for sub-grantees. 

4.) Implement the risk-based monitoring system, using tools, rubric, checklists and other supportive materials.

a. Develop a sub-grantee monitoring plan that includes policies, procedures and standards for verification. Standards for sub-grantee evaluation should be clearly established within these resources. (The types of documents required for submission along with standards for appropriate documentation should be clearly discernable.)

b. Establish and execute a cyclical monitoring schedule and determine appropriate types of monitoring activities for high-medium-and low-risk level sub-grantees (onsite, targeted, desk, virtual, telephone, etc.)

c. Develop materials and resources for sub-grantees to help them prepare for monitoring engagements. Materials should be written in plain language, citing applicable statutory requirements. Sub-grantees should also be given a clear understanding of what documents will be required for submission, in preparation for the monitoring activities, including by when and how documents should be submitted.

d. Consider separating financial monitoring from programmatic (and/or have SEA financial staff accompany on the monitoring visit to conduct in depth financial reviews)

5.) Create and maintain a data repository system that can capture and store information about grantees to inform future technical assistance and/or monitoring efforts.

a. Document sub-grantees’ weaknesses, findings and challenges identified during monitoring activities and corrective action plans.

b. Develop and record corrective action plans with deliverables and timelines.

c. Share benchmarking examples as a resource for engaging sub-recipients with similar issues and potential corrections.

6.) Regularly review and analyze the risk-based monitoring system, including continuous review and updating of monitoring plans, checklists, and protocols making adjustments as needed. 

a. Develop and implement a process for sub-recipient follow up.

· ***Identify Key General Risk Areas***

Consider risk areas that influence the sub-grantees’ ability to effectively manage federal funds and conduct assessments of each risk area to gather a comprehensive picture of the grantee’s overall operational health.  

1. New to managing grant funds, high staff turnover, specialization and experience of staff, major operational or organizational changes.

2. Prior audit (late or missing audits) and monitoring history: past and current internal and/or external audit findings (including recurring findings), programmatic and/or financial findings, etc.

3. Financial systems assessment (sub-grantee should have an established financial management system that allows for proper accounting and cash management over funds)

4. Programmatic assessment (including high dollar grant award, new programmatic requirements and existing programmatic waivers.)

5. Other Areas

#### Strengthening Sub-Recipient Monitoring & OMB Uniform Guidance

**RISK-BASED MONITORING STRATEGY**

The risk-based monitoring strategy is a partnership between the State Education Agencies (SEAs) and the sub-grantees. Both entities should work collaboratively towards the common goal of improving student performance and strengthening program outcomes.  This partnership is pivotal, particularly in light of the new Uniform Administrative Requirements, Cost Principles and Audit Requirements for Federal Awards (Uniform Guidance) that now expressly directs pass through entities (including SEAs) to perform pre and post award sub-recipient risk assessments for each sub-grantee. (Section 200.331(b)) The nature and intent of these risk assessments are to determine precisely what types of technical assistance and/or interventions sub-grantees may need. Conducting mandatory risk assessments for sub-grantees is a new requirement introduced by the Uniform Guidance; it necessitates stronger and more collaborative SEA/Sub-grantee grants management relationships with more built in accountability and oversight structures on both the front and back ends of grants management. 

The risk-based approach should (to the extent possible) focus on identifying the likelihood of non-compliance, calculating the potential impacts of these occurrences and requiring remediation, as needed. The risk-based approach should also, in part, help determine the appropriateness and responsiveness of pass-through entities’ overall sub-recipient monitoring strategies and the responsiveness of the strategies in mitigating risk and reducing the occurrence of improper grants management practices.

In efforts to be more responsive to the Uniform Guidance requirements in establishing and implementing sub-recipient risk-based monitoring, SEAs will need to evaluate and strengthen their own internal processes, practices and guidelines. 

In many instances, this may require that SEAs must:

1. Revise and update policies and procedures to account for additional risk-based sub-recipient monitoring activities, including procedures for handling high-risk grantees and/or risk areas.

2. Develop, communicate and distribute these revised internal practices and guidelines that the SEA staff should follow in implementing the new sub-recipient monitoring tools and standards.

3. Identify enforcement practices for major non-compliance areas (and/or severe findings of fraud, theft, abuse, etc.) when serious issues are identified and establish accompanying protocols SEA staff should follow in those situations.

The Uniform Guidance further requires that sub-recipients have established internal controls over federal funds. (Section 200.303(a)) SEAs should incorporate internal controls into their risk-based monitoring activities. 

The Department of Education should continue to provide technical assistance to states in their efforts to implement a comprehensive risk-based monitoring framework and be compliant with the Uniform Guidance and other statutory requirements. This collaboration benefits the Department, states and their grant recipients.

Risk Management Service, September 2015 (Rev. July 2017)