Securities Regulation: SEC Can Further Enhance Its Oversight Program of FINRA
Released: 2015-04-30T00:00:00

Letter

Background

The Exchange Act provides SEC with broad authority over the securities industry, including the power to register, regulate, and oversee participants in securities markets. Section 15A of the Exchange Act established a system of SEC registration for national securities associations and includes general criteria that a national securities association must satisfy in order to be registered and to maintain its registration. These criteria include that the association must be so organized and have the capacity to carry out the purposes of the Exchange Act and to enforce compliance by its members and persons associated with its members with the rules and regulations of the Exchange Act and the rules of the association. Section 19 of the Exchange Act outlines the registration, responsibilities, and oversight of SROs, including the review and approval or disapproval of SRO rules, and includes registered securities associations in the definition of “self- regulatory organization.” One key aspect of SEC’s oversight of FINRA, a registered national securities association, is the authority provided to SEC under Section 17 of the Exchange Act to require, by rule, FINRA to make, and keep such records as SEC prescribes as necessary and appropriate and to make all records of FINRA available to SEC for examination.

SEC oversees SROs primarily through the review of SRO rules, which is delegated to and coordinated by the Division of Trading and Markets (Trading and Markets) and through inspections led by OCIE. Specifically, Trading and Markets takes the lead role in SEC’s review and subsequent approval or disapproval of SRO proposed rules and proposed changes to existing rules, including those submitted by FINRA, based on findings that a rule or rule change is consistent with the Exchange Act and the rules and regulations thereunder applicable to the SRO.

OCIE administers SEC’s nationwide examination and inspection program for registered SROs, broker-dealers, transfer agents, clearing agencies, investment companies, and investment advisers. OCIE conducts examinations and inspections to improve compliance, prevent fraud, monitor risk, and inform policy. Individual groups in OCIE have oversight responsibility for the various registered entities (see fig. 1). Market Oversight in OCIE oversees 18 exchanges and four other entities, including FINRA, primarily through inspections and monitoring of SRO programs and operations. As of March 1, 2015, 34 OCIE employees were assigned to Market Oversight.

In our 2012 report on the extent to which SEC conducted oversight of the areas listed in Section 964 of the Dodd-Frank Act, we found that oversight varied and that OCIE was moving from routine, or cycle-based inspections—inspections that occur at regular intervals of time—to a more risk-based approach. With the transition to a more risk-based inspection process, OCIE officials stated that they were focusing resources on the most critical and high-risk areas for oversight of FINRA and other SROs.

At the time of our report, OCIE was analyzing information it collected on FINRA’s regulatory programs and operations (including Section 964 areas) and planned to implement its enhanced risk-based oversight of FINRA later in 2012. We also found that OCIE’s approach to developing and implementing a risk-based approach to oversight of FINRA did not follow all the components of a risk-management framework identified in our prior work. We recommended that OCIE follow all components of a risk-management framework as it developed its approach. SEC generally agreed with our recommendation and has taken some steps to further incorporate the components into its risk-based approach, such as its risk-assessment process described later in this report.

Our risk-management framework for federal oversight efforts has five components (described as follows and shown in fig. 2).

Strategic goals, objectives, and constraints identification: identifying the strategic goals that an agency is trying to achieve and the steps needed to attain those goals, including determining limitations or constraints that can affect the desired outcomes.

Risk assessment: identifying the key aspects of potential risk.

Alternatives evaluation: considering measures to reduce the identified risks.

Management selection: management selecting where resources and investments will be made based on selecting the appropriate alternatives for reducing risks. Implementation and monitoring: applying and monitoring the selected alternatives for reducing risk to help ensure ongoing effectiveness, including the implementation of new policies, procedures, and controls and how these procedures are documented and maintained.

We developed the framework based on criteria that include GAO best practices, OMB circulars, Standards for Internal Control in the Federal Government, and the Government Performance and Results Act of 1993 (GPRA). The risk-management framework is designed to be applied at various organizational levels, including that of a program. Risk management is a widely endorsed strategy for helping managers to make decisions about allocating finite resources and take actions under conditions of uncertainty. The process of risk management provides the rigor and structure necessary to identify and select among alternative risk responses (the cumulative effect of which is intended to reduce risk). Risk management interrelates to an entity’s governance, performance management, and internal controls.

Other federal financial regulators, such as the Federal Reserve and FHFA, also employ a risk-based approach to oversight and examinations. Under a risk-based approach, those activities judged to pose the highest risk are to receive the most scrutiny by examiners. As we have reported in the past, regulators face a number of challenges in implementing a risk-based approach. One key challenge, inherent in the design of a risk-based approach, is how to identify the aspects of an entity on which the examiner’s attention should be concentrated. Another is ensuring that examiners’ risk assessments are not overly influenced by the regulated entity’s risk-management systems on which they, at least in part, rely.

FINRA Oversight Program Lacks Elements of an Effective Risk- Management Framework

Since our 2012 report, Market Oversight has made further progress in transitioning its FINRA oversight program to a risk-based approach, including assessing the risk of various FINRA programs and using the assessment results to inform its oversight activities. However, the oversight program continues to lack elements of certain components of GAO’s risk-management framework, namely setting specific performance goals and measures, establishing procedures for certain elements of its process, and conducting assessments of internal risks.

Market Oversight Assesses FINRA Program Risks to Inform Its Oversight Activities

Market Oversight uses an annual risk assessment to make the initial determinations on which FINRA programs and operations will receive oversight and of what type of oversight, such as inspections and monitoring. Potential programs and operations that receive scrutiny include, for example, market regulation programs, enforcement programs, and areas listed under Section 964 of the Dodd-Frank Act. Under a risk- based approach, the programs and operations judged to pose the greatest risk generally receive the most scrutiny. For example, a program or operation deemed high-risk might receive an inspection, while a low- risk program or operation might receive monitoring. According to Market Oversight’s guidelines for assessing FINRA program risks, the annual FINRA risk assessment encompasses (1) risk identification and analysis, (2) control environment assessment, and (3) risk-level determination (see fig. 3).

As risks are identified and assessed, Market Oversight documents its risk-level determinations and oversight recommendations in a risk- assessment document. Staff and management then work together to develop the annual inspection plan. According to Market Oversight staff, in developing the inspection plan, they review a variety of information, including the FINRA risk-assessment document; obtain input from Market Oversight management, other SEC offices and divisions, and the Commission; and consider Market Oversight’s available resources. The inspection plan documents the SRO programs and operations, including FINRA programs and operations, that Market Oversight plans to inspect that year. The associate director of Market Oversight approves the inspection plan and discusses the plan with OCIE senior management. Other oversight determinations, such as monitoring, are documented on the risk-assessment document.

Throughout the year, Market Oversight staff and management determine necessary adjustments to planned FINRA oversight activities based on information they collect. Staff stated that if, for example, FINRA’s enforcement program was not included in the original inspection plan, but staff later observed a change in the pattern or trend of new FINRA enforcement cases, staff and management might determine it necessary to inspect FINRA’s enforcement program. The information that Market Oversight collects throughout the year could include findings from Market Oversight’s inspections, FINRA internal audit reports, tips, complaints, and referrals, and news stories.

Market Oversight’s oversight activities of FINRA for fiscal year 2014 generally corresponded with planned oversight activities. As shown in figure 4, Market Oversight initially identified 12 areas for oversight in 2014. During the course of the year, they added two FINRA areas for inspection and determined not to conduct inspections of three FINRA areas that were in the original inspection plan. Market Oversight added the two FINRA areas for inspection based on information that it had collected during its monitoring of these two areas. According to Market Oversight staff, two of the three FINRA areas were removed from the inspection plan because insufficient time had elapsed since FINRA initiated the program to start an inspection in fiscal year 2014. The third planned inspection was moved to fiscal year 2015 because Market Oversight was obtaining additional input from the Commission during 2014.

Market Oversight’s risk-assessment process is generally similar to the risk-assessment component outlined in our risk-management framework as well as the risk-assessment processes of other federal financial regulators. For example, the Federal Reserve and FHFA require examiners to identify the regulated entity’s risks, review and assess the controls that the entity has in place to mitigate those risks, and determine the level of risk that cannot be mitigated with existing controls. Market Oversight also assesses the likelihood that the identified risks will occur and the impact of such occurrence; defines each potential level of risk that can be assigned; and uses the results of the assessments in determining oversight plans—most of which are common elements of our risk-management framework or processes of other regulators.

Market Oversight Has Not Established Performance Goals and Certain Oversight Procedures, or Assessed Internal Program Risks

In our 2012 report, we found that Market Oversight had adopted and began transitioning to a risk-based approach for its FINRA oversight program in 2010. As part of this transition, beginning with its fiscal year 2014 oversight, Market Oversight implemented a risk-based approach, which it developed in part by modifying some parts of the enterprise risk- management framework of the Committee of Sponsoring Organizations of the Treadway Commission (COSO framework). However, as stated earlier, in 2012, we also found that Market Oversight’s approach to developing and implementing its risk-based approach to oversee FINRA did not contain all the elements of a risk-management framework. While Market Oversight’s October 2013 risk-assessment guidelines state that Market Oversight has incorporated all elements of a risk-management framework (as we recommended in 2012), the FINRA oversight program that Market Oversight has since implemented continues to lack important elements of risk management. In particular, the program lacks specific performance goals and measures, documentation of procedures and decision making, and comprehensive assessments to identify internal and external risks.

No Performance Goals and Few Performance Measures of FINRA Oversight

Although Market Oversight has taken some steps to assess its oversight of FINRA, it lacks performance goals or measures specific to assessing whether it has met its stated goal of enhancing oversight of FINRA. Performance goals set a target level of performance over time expressed as a tangible, measurable objective against which actual achievement can be compared. When performance goals are not self-measuring, performance measures seek to translate those goals into concrete, observable conditions that determine what data to collect to learn whether progress was made towards achieving goals. Performance measures can include measures of outputs or outcomes. For example, under its strategic goal for the safe and sound housing government-sponsored enterprises, FHFA has created the performance goal of improving the condition of the regulated entities and developed four performance measures for measuring its progress in reaching this goal. These performance measures include both output measures—a measure of the timeliness of its response to a regulated entity’s remedial action plan— and outcome measures—a measure of a specific financial ratio that provides an indication of the condition of each regulated entity.

As we reported in 2012, Market Oversight has a stated goal of enhancing its oversight of FINRA. This goal is in line with SEC’s strategic goal to foster and enforce compliance with federal securities laws and OCIE’s goal to further implement its risk-focused examination strategy and enhance its risk-assessment efforts for its nationwide examination program. Market Oversight has also established three output-based performance measures that track the number and timeliness of inspections conducted, and the review of tips, complaints and referrals related to FINRA. However, Market Oversight has not established performance goals or measures, both output and outcome, that would translate its goal of enhancing FINRA oversight into tangible, measurable activities and timelines. Furthermore, it has not yet determined targets for any of the performance measures it has established.

According to our risk-management framework, management decisions should be made in the context of the organization’s strategic plan, with clearly articulated goals and objectives that flow from the plan. Performance measures must be clear, concise, and measurable and can be used to measure progress toward these goals. Furthermore, leading practices in federal performance management that we previously identified state that performance measures should have appropriate targets. Internal control standards in the federal government call for agencies to develop control activities to help ensure that management’s directives are carried out, for example, by establishing performance measures that align with their mission and compare performance against targets.

In 2012, Market Oversight officials explained that in developing their risk- based approach they decided to modify the COSO framework, in part to customize the process to Market Oversight’s needs and expertise. Further, Market Oversight staff told us that because fiscal year 2014 was the first year it was implemented, staff continue to review and improve its risk-management framework. However, as discussed earlier, our 2012 report described the components of our risk-management framework and recommended that as Market Oversight developed its risk-based approach, it should follow all components of a risk-management framework. Until specific performance goals and related measures that address the range of its FINRA oversight program activities are established, Market Oversight risks not having the information necessary to evaluate the progress and results of its risk-based oversight, determine whether its FINRA oversight has been working as intended, and ensure accountability.

No Formal Process for Documenting Decision-making

While Market Oversight’s guidelines on FINRA risk assessments describe its process to identify and assess FINRA program risks, Market Oversight lacks written procedures describing where oversight determinations are to be documented and what that documentation should include. For instance, Market Oversight’s guidelines on risk assessment do not describe where or how determinations of oversight activities are to be documented. Market Oversight also continues to lack formal procedures for determining if and when a change to the initial inspection plan is necessary and for documenting the changes and associated rationales in relevant documents, such as the risk-assessment document and the inspection plan. For example, Market Oversight staff told us that in fiscal year 2014 they were monitoring a specific FINRA operation, but that operation was not initially marked or subsequently updated for monitoring on the risk-assessment document. In addition, Market Oversight’s fiscal year 2014 inspection plan listed only the names of the FINRA programs and operations selected for inspection but did not document the reasoning behind the determinations. Market Oversight staff stated that fiscal year 2014 was the first year in which they conducted oversight of FINRA under the risk-based approach and they had yet to formalize a documentation system for their decision-making process and oversight activities.

During the course of our review, Market Oversight has made some improvements in documenting its fiscal year 2015 oversight determinations and the rationales that supported the determinations. Specifically, Market Oversight’s fiscal year 2015 inspection plan contains separate discussions of FINRA areas selected for inspection, potential inspections subject to available resources, and areas considered but not selected, and the related rationale for each determination. Also during fiscal year 2015, Market Oversight began formally documenting changes to the inspection plan by preparing an addendum to the plan. The addendum was submitted to the associate director for approval. Market Oversight staff told us that they are in the process of updating their procedures and expect to complete the revisions by the end of fiscal year 2015.

As previously discussed, our risk-management framework incorporates our standards for internal control. These standards call for agencies to develop detailed policies and procedures to help ensure that management’s directives are carried out. They further require agencies to clearly document all significant events. Such documentation also should be properly managed, maintained, periodically updated, and readily available for examination by management and others involved in evaluating and analyzing the process.

Without documented procedures for all processes of its FINRA oversight program, including documenting decisions, such as changes to annual planned oversight activities, and decision rationales, Market Oversight’s ability to provide reasonable assurances that its processes were being carried out as intended may be limited. Furthermore, Market Oversight may lack the information necessary to monitor and evaluate the quality of FINRA oversight program processes and ensure decisions are made consistently.

No Procedures to Assess Internal Risks

Market Oversight has conducted risk assessments of FINRA programs and operations—external risks—but its current FINRA risk-assessment process and guidelines do not require a comprehensive assessment of risks (that is, both internal and external) to its FINRA oversight program objectives. For example, the risk-assessment guidelines define risk as the risk of FINRA not successfully fulfilling its responsibilities as a self- regulatory organization. This definition does not include risks internal to Market Oversight not achieving its objectives for FINRA oversight. An internal risk assessment could include reviews of how factors relevant to Market Oversight—such as resources, staff availability, and competing priorities—could affect the FINRA oversight program’s ability to successfully achieve its goals and objectives.

Our risk-management framework states that risk assessment helps decision makers identify and evaluate potential risks so that countermeasures can be designed and implemented to prevent or mitigate the effects of the risks. Further, the Standards for Internal Control in the Federal Government (one of the bases for our framework) state that management should assess the risks a program faces from external and internal sources and comprehensively identify risks associated with achieving agency objectives. The standards also state that agencies should have the appropriate methods and information to identify and assess risks to the achievement of their objectives.

Additionally, SEC’s strategic plan for fiscal years 2014 through 2018 identifies several objectives and initiatives that encourage SEC’s divisions and offices to use resources efficiently, enhance methods and tools to more effectively identify and assess risks, and better leverage and integrate data into management decisions. For example, the strategic plan states that SEC will seek to obtain greater access to data from a variety of sources, including data from SROs.

As we reported in 2012, Market Oversight had started gathering information on and assessing the risks of FINRA programs and operations. Market Oversight has guidelines for conducting assessments of FINRA program risks. However, Market Oversight’s guidelines did not include plans or procedures to conduct assessments of risks internal to the oversight program objectives. Market Oversight staff told us they are continuing to review and improve Market Oversight’s risk- assessment process. Staff further explained that although they have discussed internal factors as part of their decision making, they have not formalized this process in Market Oversight’s risk-assessment procedures. In the absence of a comprehensive assessment of internal and external risks to the FINRA oversight program objectives, Market Oversight may be missing key information needed to establish, evaluate, and adjust when necessary its oversight program.

OCIE Has Taken Steps to Enhance Its FINRA Oversight

Since our 2012 report, OCIE—primarily through its Market Oversight program—has taken steps to enhance its FINRA oversight by incorporating oversight of certain Section 964 areas into inspections of other FINRA programs and operations (beginning with inspections initiated in fiscal year 2014). Our review of Market Oversight documents, such as scope and planning memorandums and document requests, from inspections not specific to Section 964 areas that were in progress as of June 2014 found evidence of inquiries into or plans to review certain Section 964 areas in all but one open inspection. Furthermore, plans to conduct some oversight of all but two of the Section 964 areas were included in at least one of these inspections. For example, during an inspection of a FINRA market regulation program, staff inquired into and requested documents related to Section 964 areas. Market Oversight staff stated that while an inspection focused on a Section 964 area may or may not be included in a given year’s inspection plan, Market Oversight tries to incorporate some oversight of these areas into its inspections of other FINRA programs and operations. For example, Market Oversight conducted several inspections focused on Section 964 areas in fiscal year 2014, and fiscal year 2015 planning documents indicate plans to inspect at least one specific Section 964 area.

In addition, Market Oversight started a monitoring effort to enhance its oversight of FINRA. According to Market Oversight staff, one purpose of monitoring is to actively track FINRA developments to inform future FINRA risk assessments and oversight activities. Through the risk- assessment process, Market Oversight management may select a specific FINRA program(s) for monitoring. For example, in fiscal year 2014, Market Oversight selected an area for monitoring—which included the review of FINRA reports and meetings with FINRA officials. Similarly, completed inspections can also inform which issues are selected for future monitoring.

Furthermore, OCIE created and filled, within Market Oversight, the position of Senior Special Counsel-FINRA and New Markets to monitor FINRA generally and coordinate its FINRA oversight activities. According to Market Oversight staff, a primary objective of the position is to coordinate with Market Oversight management in gathering information on FINRA from a variety of sources to inform the risk-assessment process and oversight decisions. The Senior Special Counsel reviews the information collected and provided by FINRA and works with Market Oversight management to identify any issues that may require further oversight and create a draft risk assessment. The draft risk assessment is then used by Market Oversight’s senior management (associate director and assistant directors) and the counsel to make final risk-assessment and oversight activity determinations.

OCIE also has transitioned its FINRA district office inspections to a risk- focused model and centralized responsibility for the inspections in Market Oversight. Previously, OCIE conducted comprehensive inspections of each office on a 3-year cycle. Under the risk-focused model, Market Oversight staff now conduct risk-focused inspections targeting particular areas, district offices, or both. Market Oversight staff scope and plan the district office inspections and prepare the final reports. Staff from Market Oversight and SEC’s regional offices carry out the inspections and provide input throughout the process. According to Market Oversight documents, in making recommendations to management as to which FINRA district offices to inspect in the most recent round of inspections, staff reviewed information and data requested from FINRA regarding general organization, regulatory operations, enforcement program, cooperation with state securities regulators, and internal auditing of FINRA’s 15 district offices, among others. The staff also spoke with various SEC offices and divisions to learn about any material risks these offices noted in their work with FINRA. In selecting offices for inspection, staff considered individual risks to the office, types of risk, and location. Market Oversight documents also show that staff analyzed a variety of information in making their recommendations, such as the number of cause examinations and enforcement matters, the number of branch offices, and the number of high-risk firms in a district office’s jurisdiction.

Finally, OCIE recently began including a review of the outcomes of FINRA’s broker-dealer examinations as a factor in all of its broker-dealer examinations. Relevant information gathered from these examinations is shared with Market Oversight staff who use this information, along with other information collected, in its risk assessment process for FINRA.

Recent Inspections of Section 964 Areas Were Generally Consistent with Accepted Standards

Government Auditing Standards define performance audits as audits that provide findings or conclusions based on an evaluation of sufficient, appropriate evidence against criteria. Performance audit objectives can include assessments of program effectiveness, economy, and efficiency; internal control; compliance; and prospective analyses. OCIE’s inspections of SROs share many of the attributes of performance audits, including their objectives. For example, Market Oversight’s inspections of FINRA enable staff to evaluate FINRA’s compliance with applicable laws and regulations; FINRA rules, regulations, or by-laws; or both. Although OCIE is not required to follow Government Auditing Standards when inspecting SROs, the professional standards and guidance outlined in Government Auditing Standards provide a framework for conducting high- quality reviews that can serve as useful criteria in evaluating a regulatory agency’s examination or inspection programs.

OCIE policies and procedures that guided its 2011 through 2013 Section 964 area inspections generally were consistent with the requirements of Government Auditing Standards we determined were most relevant to assessing OCIE’s policies and procedures: independence, competence, quality control and assurance, planning, supervision, evidence, documentation, and reporting. Our review of 4 inspections (out of the 10) of the areas listed in Section 964 of the Dodd-Frank Act found that they were generally consistent with Government Auditing Standards and OCIE’s policies and procedures. We did not attempt to assess the validity of the inspection findings. This report only speaks to the results from our review of the 4 area inspections we reviewed.

Independence

Government Auditing Standards require that in all matters relating to audit work, the audit organization and the individual auditor, whether government or public, must be independent. Furthermore, there should be documentation of independence. According to OCIE staff, OCIE examiners do not regularly attest their independence. Conflicts of interest for federal employees are governed by relevant government ethics laws and regulations. As federal employees, OCIE examiners are subject to ethics restrictions concerning conflicts of interest and financial disclosure, including a criminal financial conflict of interest statute and executive branch-wide impartiality regulations. SEC supplemental ethics regulations restrict the securities holdings and transactions of all SEC employees, including OCIE examiners, and require all employees to report their securities holdings and transactions. Further, OCIE examiners must attend regular ethics training. Moreover, OCIE policies and procedures hold examination staff to additional standards of accountability because of the special nature of the examination process. OCIE guidance states that examiners must avoid any situation involving not only a conflict of interest, either financial or personal, but also the appearance of a conflict of interest. The OCIE guidance discusses a number of situations that examination staff may encounter, for example, whether an examiner may accept food or beverages from a registrant during an examination.

In addition, according to OCIE guidance, OCIE examiners are expected to declare any potential conflicts to SEC’s Office of the Ethics Counsel, which is responsible for advising employees on ethics and conflicts of interest arising under federal conflict of interest statutes and executive branch ethics regulations, and to their supervisors. The office provides guidance on issues such as personal and financial conflicts of interest, post-employment restrictions, securities holdings and transactions, the receipt of gifts, and adequate financial disclosure. OCIE’s Office of the Chief Counsel advises employees on any issues arising under OCIE’s guidance. Similarly, OCIE and ethics office training materials provide staff with information on dealing with conflict-of-interest situations. In 2011 all but one of the OCIE staff assigned to the Section 964 area inspections attended OCIE-provided ethics training. All assigned staff attended in 2012. According to OCIE staff, OCIE did not hold in-person ethics training in 2013. OCIE is currently revamping its ethics training and plans to roll out an enhanced program. In 2013, SEC’s Office of the Ethics Counsel required in-person ethics training for financial disclosure filers. Of the staff members assigned to the Section 964 inspections, one was not employed by SEC in 2013 and two staff members were not financial disclosure filers. All other staff assigned to the Section 964 inspections completed the Office of the Ethics Counsel provided training.

Competence

Government Auditing Standards state that audit staff collectively should possess adequate professional competence and technical knowledge, skills, and experience. According to OCIE policies and procedures, SRO inspections are generally conducted in teams. The size of the inspection team is determined by the size and complexity of the SRO regulatory program being reviewed. An exam manager is generally assigned as the immediate supervisor on the inspection and a member of the staff is generally designated as the lead for each inspection. The lead staff member and other members of the inspection team perform inspections under the supervision of the exam manager, assistant director, and associate director. Market Oversight’s inspections of each of the Section 964 area inspections were staffed by at least two examiners and overseen by an exam manager, assistant director, and associate director. Based on data we reviewed, on average, the inspection teams had combined OCIE experience of over 60 years, and the median experience of the lead examiners averaged 6.9 years.

OCIE also provides new examiners with training intended to provide knowledge necessary to conduct adequate examinations. OCIE’s new examiner training covers areas such as securities laws, securities products, fraud, compliance, registration issues, types of information disclosure, reporting, and filings, and SEC internal guidelines, policies, and procedures. Market Oversight also provides to its staff ongoing training, including opportunities on current capital markets topics such as consolidated audit trails and market data feeds. Finally, staff stated that OCIE consults with subject-matter experts and other offices/divisions of SEC when necessary.

Quality Control and Assurance

Government Auditing Standards state that an organization should establish a system of quality control designed to provide the organization with reasonable assurance that its personnel comply with professional standards and legal requirements. Furthermore, monitoring of quality is an ongoing, periodic assessment of work completed on audits. It is designed to provide management of the audit organization with reasonable assurance that the policies and procedures related to the quality control system are suitably designed and operating effectively. OCIE policies and procedures describe a report approval process that uses management review and ongoing input throughout the inspection process to achieve quality control and assurance. Specifically, SRO inspection reports generally must be reviewed and approved by the exam manager, assistant director, associate director, OCIE director, and the Commission before release.

Our review of documents, such as scope memorandums, planning memorandums, intra-office presentations, document requests, status memorandums, memorandums to the Commission, and final report letters to FINRA from 4 of the 10 Section 964 area inspections, indicate the inspections were conducted in a manner consistent with OCIE guidelines. Further, these documents demonstrate OCIE management and the Commission had multiple opportunities to review staff work and provide feedback.

Planning

Government Auditing Standards require a written audit plan for each audit. The form and content of the written audit plan may vary among audits. Plans may include an audit strategy, audit program, project plan, or other appropriate documentation of key decisions about the audit objectives, scope, and methodology and the auditor’s basis for those decisions. OCIE policies and procedures state that the inspection process begins with an initial risk and scope analysis of the regulatory program to be reviewed. Staff are instructed to review past inspection reports, information on past findings and SRO efforts to address those findings, prior enforcement actions, and other information sources. Staff then prepare a scope memorandum that outlines the inspection objectives and methodology, and inform the associate director that the inspection is commencing. After analyzing information a SRO provides in response to an initial document request, staff may also prepare a planning memorandum to describe the regulatory program being reviewed and the specific plan for the inspection.

For all 10 Section 964 area inspections, Market Oversight management prepared a single, broad scope memorandum and inspection teams developed planning memorandums for individual inspection areas. Both the scope and planning memorandums contained descriptions of the scope, objectives, and potential methodologies for the inspections. The planning memorandums we reviewed also contained background research on the specific Section 964 areas, the COSO standards for risk management, and specific areas for inspection based on a review of documents from FINRA. Market Oversight’s planning memorandums also described the background research completed and team’s decisions on scope, objectives, and methodology. Market Oversight also submitted one initial document request for all of the Section 964 area inspections.

Supervision

Government Auditing Standards state that audit supervision involves providing sufficient guidance and direction to staff assigned to the audit to address the audit objectives and follow applicable requirements, while staying informed about significant problems encountered, reviewing the work performed, and providing effective on-the-job training. OCIE policies and procedures require that staff submit write-ups to supervisors for review, analysis, and discussion at sensible intervals. Staff stated that such intervals vary depending on the facts and circumstances of the specific inspection. Supervisors are responsible for ensuring proper and consistent analysis across the examination team.

Documentation from the 4 (out of the 10) Section 964 area inspections that we reviewed demonstrated supervisory review throughout the process. Specifically, the associate director reviewed and signed the initial document request to FINRA, multiple memorandums to the Commission, and the final report to FINRA. Furthermore, assistant directors overseeing the inspections co-wrote the memorandums to the Commission and lead examiners addressed planning memorandums to their supervisors. Finally, Market Oversight staff said inspection staff routinely met as a whole to discuss status and present findings.

Evidence

Government Auditing Standards state that auditors must obtain sufficient, appropriate evidence to provide a reasonable basis for their findings and conclusions. The standards also state that auditors must use professional judgment in performing audits, including assessing the sufficiency and appropriateness of evidence. OCIE policies and procedures require the lead examiner and exam manager to discuss the findings and ensure that adequate evidence supports each finding.

Our review of overall scope memorandum and planning memorandums from 4 (out of 10) Section 964 inspections found that staff planned to use reasonable methodologies, such as document requests and reviews and interviews with FINRA officials. Market Oversight’s document requests and documentation of receiving the requested documents provide further support of OCIE receiving the requested materials and evidence. Finally, the summary and status memorandums for the 4 inspections we reviewed generally contained evidence that staff reviewed the areas outlined in the planning memorandums and indicated evidence for specific statements by providing citations to source documents.

Audit Documentation

Government Auditing Standards state that auditors must prepare audit documentation related to planning, conducting, and reporting for each audit. Auditors should document the following: objective, scope, and methodology of the audit; work performed and evidence obtained to support significant judgments and conclusions; and supervisory review (before the audit report is issued) of evidence that supports the findings, conclusions, and recommendations in the audit report. As stated earlier, OCIE policies and procedures for scope and planning memorandums state that these documents should contain information on the objectives, scope, and methodology of the inspection. Furthermore, OCIE’s policies and procedures for supervision and evidence help ensure documentation of supervisory review. For instance, workpapers from an inspection must be filed and subsequently archived in a summary file folder. The summary folder should contain all key inspection documents, including the scope memorandum, planning memorandum, document requests, summary of findings, and final reports.

Market Oversight followed a different process for documenting its Section 964 area inspections. Rather than archiving them in a summary file folder, which would be most consistent with OCIE’s policies and procedures, it instead stored each inspection’s documentation on a shared OCIE drive. Market Oversight staff stated that they used this approach because the inspections were all related to Section 964 and conducted in a coordinated manner. As a result, they considered this method to be generally consistent with the guidelines. According to staff, Market Oversight has since fully transitioned to using OCIE’s examination and inspection tracking database, which acts as a central repository for all documentation related to examinations and inspections. Market Oversight was able to access and make available all the documents listed in the summary folder in a reasonable amount of time, as required by the standards.

Reporting

Government Auditing Standards state that auditors must issue audit reports communicating the results of each completed performance audit. OCIE policies and procedures generally require SRO inspection reports to be submitted to the SRO and the Commission.

For the Section 964 area inspections, OCIE issued a single letter to FINRA describing its observations and requesting information on FINRA’s plans to address the findings. OCIE also sent an inspection close-out memorandum to the Commission and has been tracking FINRA’s responses to its findings.

Conclusions

Ensuring that FINRA is properly carrying out its wide-ranging regulatory responsibilities is critical to SEC’s mission of protecting investors, maintaining fair, orderly, and efficient markets, and facilitating capital formation. SEC has taken steps to enhance its oversight of FINRA, which include completing inspections of all 10 Section 964 areas, incorporating oversight of these areas into inspections of other FINRA programs and operations, and conducting risk assessments to inform its oversight activities. SEC also dedicated a staff person to coordinate FINRA oversight.

However, the FINRA oversight program continues to lack specific performance goals and measures, documented procedures for certain processes, and an assessment of internal risks to the program. As we recommended in our 2012 report, following all elements of a risk- management framework would allow Market Oversight to better measure the effectiveness of its risk-based approach, improve accountability, and assist management in directing its limited resources toward FINRA areas determined to be highest risk. Furthermore, a more formalized and systematically documented oversight process would help Market Oversight accumulate institutional knowledge, learn from past results, and make improvements to the oversight process. Because the implementation of the risk-management framework remains a work in progress, Market Oversight has the opportunity to improve upon its current processes and procedures by more fully implementing the elements of each risk-management component while maintaining a customized approach to fit its needs and expertise.

Recommendations for Executive Action

To improve SEC’s FINRA oversight program, the SEC Chair should direct the appropriate offices and divisions to incorporate additional risk- management practices by taking several actions, including: establishing specific performance goals for the program and performance measures and related targets to assess Market Oversight’s progress in meeting those goals; formalizing documentation of procedures, including procedures for making changes to the annual planned oversight activities and decision-making rationales; and modifying existing risk-assessment procedures to require an assessment of internal risks to successfully meeting the FINRA oversight program’s goals and objectives.

Agency Comments and Our Evaluation

We provided a draft of this report to SEC for their review and comment. In its comment letter, which is reprinted in appendix II, SEC described how they are implementing or plan to implement the actions listed in our recommendation. In response to our recommendation that SEC establish specific goals and measures, OCIE stated that Market Oversight plans to work with OCIE senior management to conduct an analysis that it intends to use to develop specific performance goals for OCIE’s inspection program for FINRA and performance measures to assess its progress in meeting those goals. Regarding our recommendation to formalize documentation, OCIE has begun the process of drafting formal, written procedures to document its decision making. And finally, concerning our recommendation to modify existing risk assessment procedures, OCIE plans to document its consideration of internal risks in its existing risk- assessment procedures. SEC also provided technical comments on the draft report, which we incorporated as appropriate.

We are sending copies of this report to the Chair of the Securities and Exchange Commission, the appropriate congressional committees, and other interested parties. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-8678 or clowersa@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to (1) assess the Securities and Exchanges Commission’s (SEC) implementation of its risk-based approach for overseeing the Financial Industry Regulatory Authority (FINRA), (2) review oversight of FINRA operations and programs, and (3) assess recent inspections of areas listed in Section 964 of the Dodd-Frank Wall Street Reform and Consumer Protection Act (Dodd-Frank Act).

To assess SEC’s implementation of its risk-based FINRA oversight program, we reviewed documents, guidance, and procedures from the Market Oversight program of SEC’s Office of Compliance Inspections and Examinations (OCIE). Specifically, we requested documentation to demonstrate how Market Oversight conducted its FINRA risk assessment and determined its planned oversight activities for fiscal year 2014. For example, we reviewed Market Oversight’s risk-assessment document and inspection plan for fiscal year 2014 and Market Oversight’s guidelines on conducting assessments of FINRA program risks. We asked for documentation of the steps described in Market Oversight’s guidelines, including documents Market Oversight reviewed and inputs Market Oversight obtained from others in conducting its FINRA risk assessment and determining oversight activities. We also asked for documentation of discussions within Market Oversight management and discussions held with others, including any underlying materials used in or reports generated for these discussions pertaining to FINRA oversight in fiscal years 2014 and 2015. Market Oversight provided us meeting agendas and appointments for meetings between Market Oversight staff and management, other SEC offices and divisions, and FINRA. Market Oversight also provided other documentation, including quarterly FINRA statistics on arbitration cases and a screenshot of its internal website that collects information related to FINRA. We also interviewed Market Oversight staff to understand processes for conducting the FINRA risk assessment and determining oversight activities.

In addition, to assess the extent to which Market Oversight’s actual oversight activities followed its planned oversight for fiscal year 2014, we reviewed and compared the FINRA risk-assessment document and the inspection plan for the planned oversight activities, and Market Oversight documents related to its inspections and other oversight activities, such as memorandums for inspection, for the activities conducted. For the FINRA programs and operations in which we identified differences between planned and actual activities, we clarified with Market Oversight to determine the actual activities and rationales for changes.

Finally, we compared Market Oversight’s risk-assessment process with the framework on risk management developed in prior GAO work, internal control standards for the federal government, the internal control framework of the Committee of Sponsoring Organizations of the Treadway Commission (COSO), and risk-assessment processes used by the Board of Governors of the Federal Reserve System (Federal Reserve) and the Federal Housing Finance Agency (FHFA) in their risk- focused examination approach. We selected these two financial regulators because they have established written guidance for conducting their risk-focused examinations of regulated entities and have specific guidance for programs and operations similar to some of the selected FINRA programs and operations under Section 964 of the Dodd-Frank Act.

In addition, we compared Market Oversight’s FINRA oversight program procedures and processes to elements outlined in our risk-management framework and the underlying criteria used to develop this framework. The underlying criteria include the internal control standards in the federal government, Office of Management and Budget circulars, and, the Government Performance and Results Act of 1993, which was modified by the GPRA Modernization Act of 2010. Additionally, we reviewed leading practices in federal performance management that we previously identified for setting appropriate targets for performance measures. We reviewed Market Oversight’s performance reporting documents to describe the steps Market Oversight has taken to assess the progress and results of its FINRA oversight program. We also reviewed Market Oversight’s procedural guidelines and documentation to determine the extent to which Market Oversight has formal procedures to document oversight determinations, changes to planned oversight activities, and processes to conduct a comprehensive risk assessment. Furthermore, we reviewed Market Oversight’s fiscal year 2015 FINRA risk assessment and inspection plan to identify improvements Market Oversight has made in documenting its oversight determinations and supporting rationales. We also reviewed SEC’s strategic plan for fiscal years 2014-2018 to understand SEC’s strategic objectives and initiatives related to risk identification and assessment. We also interviewed OCIE and Market Oversight officials to understand processes for conducting FINRA oversight. Finally, we interviewed officials of the Federal Reserve and FHFA to understand their risk-based oversight processes and reviewed performance reports of the Federal Reserve and FHFA to obtain information on performance goals and measures other federal regulators have established for their oversight programs.

To identify steps OCIE has taken to enhance its oversight of FINRA, we reviewed inspection documentation to determine the extent to which Section 964 areas had been incorporated in inspections of other FINRA programs and operations. We also reviewed procedures for FINRA district office inspections and memorandums for district office selection. We also reviewed e-mail discussions between OCIE and SEC’s Division of Economic and Risk Analysis regarding sampling of specific FINRA examinations to review. We reviewed OCIE policies and procedures to better understand changes made to OCIE’s broker-dealer examinations.

We interviewed the Senior Special Counsel-FINRA and New Markets to obtain information on the position, its responsibilities, and how those responsibilities were being carried out. Finally, we interviewed FINRA officials to obtain general information on SEC’s oversight of FINRA.

To assess OCIE’s oversight of FINRA programs and operations listed under Section 964 of the Dodd-Frank Act, we analyzed OCIE policies and procedures (for conducting inspections of self-regulatory organizations) that guided the inspections of the FINRA programs and operations listed in Section 964 (Section 964 area inspections) to determine if they were consistent with Government Auditing Standards. We focused our analysis on those elements of the standards we determined were most relevant to assessing OCIE’s policies and procedures. These elements were independence, competence, quality control and assurance, planning, supervision, evidence, audit documentation, and reporting. In addition, to obtain further assurance as to the competence, independence, and quality control elements, we reviewed OCIE examiner training materials, ethics guidelines and training materials, and evidence of the years of experience and attendance at ethics training for each OCIE examiner involved in the Section 964 area inspections. We also obtained general ethics information and SEC ethics training material from SEC’s Office of Ethics Counsel. We also asked the Office of Ethics Counsel if it was aware of a conflict or potential conflict for staff assigned to the Section 964 inspections and it did not identify any. To determine if OCIE followed its policies and procedures and Government Auditing Standards in conducting the Section 964 area inspections, we reviewed the planning memorandums, status and summary memorandums provided to OCIE management and the Commission, document requests, and OCIE presentations of 4 Section 964 area inspections. We also reviewed the scope memorandum and final report that covered all 10 Section 964 area inspections. We selected 4 area inspections for more detailed review—governance, transparency of governance, funding, and executive compensation—based on the frequency of SEC oversight of the FINRA programs and operations listed in Section 964 as found by our previous report (GAO-12-625), SEC Inspector General examination plans for fiscal years 2014 and 2015, and OCIE’s fiscal year 2014 risk assessment. In addition, discussions with OCIE staff and our review of documents indicated that all 10 inspections were likely conducted in a similar manner. This report only speaks to the results from our review of the 4 area inspections we reviewed, which illustrate alignment of actual SEC inspections with Government Auditing Standards and OCIE’s policies and procedures. Finally, we interviewed OCIE management to obtain additional information on Market Oversight’s FINRA oversight program and OCIE’s recent efforts to enhance that oversight.

We conducted this performance audit from May 2014 to April 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Securities and Exchange Commission

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Karen Tremba (Assistant Director), Matthew Keeler (Analyst-in-Charge), David Ahn, William R. Chatlos, Anna Chung, Kristen Kociolek, Marc Molino, Barbara Roesmann, and Jessica Sandler made key contributions to this report.