Mutual Fund Industry: SEC's Revised Examination Approach Offers Potential Benefits, but Significant Oversight Challenges Remain
Published: 2005-08-17T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: As the frontline regulator of mutual funds, the Securities and Exchange Commission (SEC) plays a key role in protecting the nearly half of all U.S. households owning mutual funds, valued around $8 trillion in 2005. Mutual fund abuses raised questions about the integrity of the industry and quality of oversight provided by SEC and self-regulatory organizations (SRO) that regulate broker-dealers selling funds. This report assesses (1) changes SEC has made to, or is planning for, its mutual fund exam program; (2) key aspects of SEC's quality control framework for routine fund exams; and (3) the adequacy of SEC's oversight of NASD and the New York Stock Exchange in protecting shareholders from mutual fund sales abuses.
section_title: What GAO Found
paragraphs: SEC is initiating several changes intended to strengthen its mutual fund exam program but faces challenges overseeing the fund industry. In the wake of the fund abuses, SEC has revised its past approach of primarily conducting routine exams of all funds on a regular schedule. It concluded these exams were not the best tool for identifying emerging problems, since funds were not selected for examination based on risk. To quickly identify problems, SEC is shifting resources away from routine exams to targeted exams that focus on specific risks. It will conduct routine exams on a regular schedule but only of funds deemed high risk. SEC also is forming teams to monitor some of the largest groups of advisers and funds. Although SEC is seeking to focus its resources on higher risk funds and activities, the resource tradeoffs it made in revising its oversight approach raise significant challenges. The tradeoffs may limit SEC's capacity not only to examine funds considered lower risk within a 10-year period but also to accurately identify which funds pose higher risk and effectively target them for routine examination. Potentially taxing its resources further, SEC recently adopted a rule to require advisers to hedge funds (investment vehicles generally not widely available to the public) to register with it. This rule is expected to increase SEC's exam workload, but the precise extent is not yet known. SEC has integrated some quality controls into its routine exams, but certain aspects of its framework could be improved. It relies on experienced staff to oversee all exam stages but does not expressly require supervisors to review work papers or document their review. GAO found deficiencies in key SEC exam work papers, raising questions about the quality of supervisory review. SEC also does not require examiners to prepare written exam plans, though they use considerable judgment in customizing each exam. Written plans could serve as a guide for conducting exams and reviewing whether exams were completed as planned. As done by other regulators, SEC also could review a sample of work papers to test compliance with its standards. A primary tool that SEC uses to assess the adequacy of SRO oversight of broker-dealers offering mutual funds provides limited information for achieving its objective and imposes duplicative costs on firms. To assess SRO oversight, SEC reviews SRO exam programs and conducts oversight exams of broker-dealers, including their mutual fund sales practices. SEC's oversight exams take place 6 to 12 months after SROs conduct their exams and serve to assess the quality of SRO exams. However, GAO reported in 1991 that SEC's oversight exams provided limited information in helping SROs to improve their exam quality, because SEC and the SROs used different exam guidelines and their exams often covered different periods. GAO found that these problems remain, raising questions about the considerable resources SEC devotes to oversight exams. GAO also found that SEC has not developed an automated system to track the full scope of work done during its oversight exams. Thus, SEC cannot readily determine the extent to which these exams assess mutual fund sales practices.

section_title: Letter
paragraphs: Nearly half of all U.S. households own mutual funds, with total assets of  about $8 trillion as of February 2005, and roughly one-third of the total  assets are held in retirement accounts. Recent trading abuses implicating  well-known mutual funds have called into question the integrity of the  mutual fund industry and quality of oversight provided by the Securities  and Exchange Commission (SEC), which is the industry’s frontline  regulator. State regulators rather than SEC were the first to uncover in the  summer of 2003 abuses involving market timing, which typically involves  the frequent buying and selling of mutual fund shares by sophisticated  investors seeking opportunities to profit from differences in prices  between overseas and U.S. markets. Although market timing is not itself  illegal, it can constitute illegal conduct if, for example, investment advisers  (firms that manage mutual funds) enter into undisclosed agreements with  favored customers, such as hedge funds, permitting them to trade  frequently and in contravention of fund prospectuses—as certain advisers  did before September 2003. Another type of abuse commonly referred to  as late trading was significant but less widespread than market timing  abuses. Late trading occurs when investors place orders to buy or sell  mutual fund shares after the mutual fund has calculated the price of its  shares, usually once daily at the 4:00 p.m. Eastern Time (ET) market close,  but receive that day’s fund share price. Investors who are permitted to late  trade can profit from their knowledge of events in the financial markets  that take place after 4:00 p.m., an opportunity that other fund shareholders  do not have. Although late trading can involve fund personnel, late trading  violations typically have occurred at intermediaries, such as broker-dealers  that offer mutual funds to their customers, before these institutions submit  their daily aggregate orders to mutual funds for final settlement.
SEC’s initial inability to detect the market timing abuses before late 2003  raised questions about the agency’s mutual fund examination program,  which is the agency’s primary means of detecting deficiencies and  violations and thereby protecting investors. However, following the  detection of the mutual fund trading abuses, SEC initiated a series of  examinations to determine the extent of the abuses, vigorously pursued  enforcement actions against violators of securities laws, and issued new  rules to overhaul the regulatory framework in which funds operate. The  agency also initiated changes to its examination program that are intended  to make it more focused on detecting abuses and emerging problems more  quickly.
In addition, the involvement of some broker-dealers in the recent mutual  fund trading abuses has raised concerns about regulatory oversight of that  industry. NASD and the New York Stock Exchange (NYSE) are self- regulatory organizations (SRO) and have primary responsibility for  regulating and examining their member broker-dealers, including their  mutual fund sales practices. Although NASD is primarily responsible for  assessing broker-dealer mutual fund sales practices, it did not detect the  trading abuses through examinations or other means. SEC is responsible  for overseeing the quality of SRO regulation of broker-dealers and does so  through on-site inspections of SRO regulatory programs. SEC also  conducts oversight examinations of broker-dealers recently examined by  SROs to assess the quality of their examination programs.
This report responds to your request that we review various issues  concerning SEC’s oversight of the mutual fund industry. Specifically, you  asked us to (1) identify and assess changes SEC has made to, or is planning  for, its mutual fund examination program; (2) assess key aspects of the  quality control framework of SEC’s routine mutual fund examinations; and  (3) determine the adequacy of SEC’s oversight of NASD and NYSE,  particularly in regard to the SROs’ oversight of mutual fund sales practices.
To accomplish our reporting objectives, we reviewed policies, procedures,  and other guidance applicable to SEC’s mutual fund examination program  as well as laws and regulations related to mutual funds. We also reviewed  policies, procedures, and other guidance applicable to SEC’s oversight of  NASD’s and NYSE’s broker-dealer examination programs. At three SEC  field offices that accounted for the largest number of completed routine  fund examinations in fiscal year 2004, we reviewed all routine  examinations of funds completed that year. In addition, we interviewed  officials at SEC headquarters and four field offices, NASD, NYSE, the  Investment Company Institute, and other industry participants about SEC’s  oversight of mutual funds or broker-dealers. Finally, we interviewed  officials from the Federal Deposit Insurance Corporation (FDIC), the  Board of Governors of the Federal Reserve System, the Office of the  Comptroller of the Currency (OCC), NASD, NYSE, and the National  Futures Association (NFA) about their examination programs. We also  reviewed guidance related to those programs. We performed our work in  Boston, Massachusetts; Chicago, Illinois; New York, New York;  Philadelphia, Pennsylvania; and Washington, D.C. We conducted our work  between February 2004 and July 2005 in accordance with generally  accepted government auditing standards. Appendix I provides a detailed  description of our scope and methodology.
subsections: section_title: Background
paragraphs: SEC oversees mutual funds primarily through its Office of Compliance  Inspections and Examinations (OCIE), Division of Investment  Management, and Division of Enforcement. OCIE examines mutual funds  to evaluate their compliance with the federal securities laws, to determine  if they are operating in accordance with disclosures made to investors, and  to assess the effectiveness of their compliance control systems. The  Division of Investment Management administers the securities laws  affecting funds and advisers. It reviews disclosure documents that mutual  funds are required to file with SEC and engages in other regulatory  activities, such as rulemaking, responding to requests for exemptions from  federal securities laws, and providing interpretation of those laws. Finally,  SEC’s Division of Enforcement investigates and prosecutes violations of  securities laws related to mutual funds.
SEC regulates mutual funds under the Investment Company Act of 1940,  the Investment Advisers Act of 1940, the Securities Act of 1933, and the  Securities Exchange Act of 1934. The Investment Company Act was passed  specifically to regulate mutual funds and other types of investment  companies. Under the act, mutual funds are required to register with SEC,  subjecting their activities to SEC regulation. The act also imposes  requirements on the operation and structure of mutual funds. Its core  objectives are to  ensure that investors receive adequate and accurate information about  mutual funds,   protect the integrity of fund assets,   prohibit abusive forms of self-dealing,   prevent the issuance of securities that have inequitable or   ensure the fair valuation of investor purchases and redemptions.
The Investment Advisers Act requires mutual fund advisers to register with  SEC, imposes reporting requirements on them, and prohibits them from  engaging in fraudulent, deceptive, or manipulative practices. The Securities  Act requires fund shares offered to the public to be registered with SEC and  regulates mutual fund advertising. Under the Securities Act and Investment  Company Act, SEC has adopted rules to require mutual funds to make  extensive disclosures in their prospectuses. The Securities Exchange Act,  among other things, regulates how funds are sold and requires persons  distributing funds or executing fund transactions to be registered with SEC  as broker-dealers.
SEC, NASD, and NYSE regulate broker-dealers, including their mutual fund  sales practices, by examining their operations and reviewing customer  complaints. Broker-dealers that are members of NYSE and do business  with the public are typically also required to be members of NASD.  Historically, NASD has conducted the mutual fund sales practice portions  of examinations for firms that are dually registered with it and NYSE. As a  result, NYSE generally plays a lesser role in examining broker-dealers for  mutual fund sales practices. NASD has established specific rules of  conduct for its members that provide, among other things, standards for  advertising and sales literature, including filing requirements, review  procedures, approval and recordkeeping obligations, and general  standards. NASD also tests members to certify their qualifications as  registered representatives.
SEC evaluates the quality of NASD and NYSE oversight in enforcing their  member compliance with federal securities laws through SRO oversight  inspections and broker-dealer oversight examinations. SROs are private  organizations with statutory responsibility to regulate their own members  through the adoption and enforcement of rules of conduct for fair, ethical,  and efficient practices. As part of this responsibility, they conduct  examinations of the sales practices of their broker-dealer members. SEC’s  SRO oversight inspections cover all aspects of an SRO’s compliance,  examination, and enforcement programs. The inspections determine  whether an SRO is (1) adequately assessing risks and targeting its  examinations to address those risks, (2) following its examination  procedures and documenting its work, and (3) referring cases to  enforcement authorities when appropriate. Under its broker-dealer  oversight examinations, SEC examines some of the broker-dealers that  SROs recently examined. SEC conducts these examinations to assess the  adequacy of the SRO examination programs. In addition to its oversight  examinations, SEC conducts cause, special, and surveillance examinations  of broker-dealers, but these examinations do not serve to assess the quality  of SRO examinations.
section_title: SEC’s Revised Mutual Fund Examination Program Offers Potential Benefits but also Poses Significant Oversight Challenges
paragraphs: Since the detection of the mutual fund trading abuses in the summer of  2003, SEC has made significant changes to its traditional examination  approach, which generally focused on conducting routine examinations of  all funds on an established schedule. To better detect potential violations,  SEC has reallocated or plans to reallocate its staff to conducting targeted  examinations focusing on specific risks and monitoring larger funds on a  continuous basis. SEC’s revised examination approach offers the potential  for the agency to more quickly identify emerging risks and better  understand the operations of large and complex funds, although it is too  soon to reach definitive judgments. However, due to the limited number of  SEC’s examination staff relative to the number of mutual funds and  advisers for which the agency has oversight responsibility, the decision to  focus examination resources on particular areas involved tradeoffs that  raise regulatory challenges. In particular, SEC’s capacity to examine lower  risk advisers and funds within a reasonable time period and develop  industry risk ratings has been limited.
subsections: section_title: SEC Has Revised Its Traditional Mutual Fund Examination Approach in the Wake of the Mutual Fund Trading Abuses
paragraphs: Historically, routine examinations of mutual fund complexes—groups or  families of funds sharing the same adviser or underwriter—have served as  the cornerstone of SEC’s mutual fund oversight, accounting for 85 percent  of the total fund examinations done from 1998 through 2003. During that  period, SEC generally tried to examine each complex at least once every 5  years. Due to resource constraints, SEC examinations typically focused on  discrete areas that staff viewed as representing the highest risks of  presenting compliance problems that could harm investors. Major areas of  review have included portfolio management, order execution, allocation of  trades, and advertising returns. In late 2002, SEC implemented a revised  approach to conducting routine examinations that included a systematic  process for documenting and assessing risks and controls for managing  those risks in a range of areas related to the asset management function.  Besides routine examinations, SEC conducts sweep examinations to probe  specific activities of a sample of funds identified through tips, complaints,  the media, or other information. The agency also conducts cause  examinations when it has reason to believe something is wrong at a  particular fund. Sweep and cause examinations accounted for about 5 and  10 percent, respectively, of the total examinations done during 1998  through 2003.
After the detection of the market timing and late trading abuses in the  summer of 2003, SEC officials concluded that the agency’s traditional focus  on routine examinations had limitations. In particular, SEC staff said that  routine examinations were not the best tool for broadly identifying  emerging compliance problems, since funds were selected for examination  based largely on the passage of time, not based on their particular risk  characteristics. In addition, SEC officials stated that they concluded the  growth in the number of mutual fund companies and the breadth of their  operations, combined with the need to perform more in-depth  examinations of discrete areas, did not allow SEC to maintain its existing  routine examination cycle.
To focus its resources on issues and funds presenting the greatest risk of  having compliance problems that may harm investors, SEC has made  significant revisions to its examination priorities and oversight processes  as described below:  First, SEC is placing a higher priority on sweep and cause examinations  and a lower priority on routine examinations. SEC has directed its 10  field offices that conduct fund examinations to give priority to initiating,  as warranted, sweep examinations of funds or advisers, focusing  particularly on operational or compliance issues. To address the  market timing and late trading abuses surfacing in late 2003, SEC shifted  resources away from routine examinations to support sweep and cause  examinations, according to SEC officials. As a result, sweep and cause  examinations accounted for 87 percent of the 690 fund examinations  completed in fiscal year 2004. SEC officials said that about 17 percent of  these examinations resulted in referrals to the agency’s Division of  Enforcement for potential violations of securities laws and regulations.  (We note that the large increase in the number of sweep and cause  examinations in fiscal year 2004 as well as the number of referrals was  likely due to SEC’s focusing a substantial amount of resources on  detecting market timing and late trading abuses.)
Second, SEC no longer will routinely examine all funds and advisers on  a regular basis, but it will conduct routine examinations of funds and  advisers perceived to be high risk, once every 2 to 3 years. In addition,  SEC will randomly select a sample of advisers and their affiliated funds  perceived to be low risk for routine examination each year. Because  these firms will be selected randomly, each firm will have an equal  chance of being examined each year. According to SEC officials, the  random selection process will enable agency staff to project the  examination findings to the population of firms deemed low risk and  assess the possible existence of problems within the population.
Third, SEC plans to provide more continuous and in-depth oversight of  the largest mutual funds. Specifically, SEC is creating teams of  examiners dedicated to regularly interacting with and closely  monitoring and examining the activities of firms in the largest and most  complex groups of affiliated advisers and mutual funds. SEC initially  plans to form teams under a pilot program to monitor 10 large advisory  groups. Any decision to form additional monitoring teams will depend  on how the pilot program develops, according to an SEC official. SEC  officials said that the monitoring teams are loosely modeled on the  federal bank regulators’ use of on-site teams to continuously monitor  operations of large banks. However, unlike the bank regulator approach,  SEC staff said the monitoring teams would not be located on-site at  large mutual fund companies.
Fourth, an SEC task force is considering the development of a  surveillance program to support the agency’s oversight of all funds and  advisers. The purpose of this program is to obtain from firms  information that would enable examiners to identify aberrant patterns  in fund and adviser activities and the possible existence of fraud or  abusive schemes that require follow-up through examinations. In its  fiscal 2006 budget request, SEC reported that the agency expects the  surveillance system to begin operations during the second half of 2006.
Fifth, SEC has promulgated rules that require investment advisers and  investment companies to appoint independent chief compliance officers  (CCO) who are responsible for ensuring that their companies adopt  policies and procedures designed to prevent violations of federal  securities laws and regulations. Fund CCOs are also responsible for  preparing annual reports that must, among other things, identify any  material compliance matter at the company since the date of the last  report. SEC staff said that they plan to review such annual compliance  reports while conducting examinations to assist in identifying problems  at mutual funds and determine whether the funds have taken corrective  actions. (As described later in this report, SEC is missing opportunities  to take full advantage of CCO compliance reports to detect potential  violations in the mutual fund industry.)
Finally, SEC has established the Office of Risk Assessment (ORA) to  assist the agency in carrying out its overall oversight responsibilities,  including mutual fund oversight. ORA’s director reports directly to the  SEC Chairman. According to SEC staff, ORA will enable the agency to  analyze risk across divisional boundaries, focusing on new or resurgent  forms of fraudulent, illegal, or questionable behavior or products. ORA’s  duties include (1) gathering and maintaining data on new trends and  risks from external experts, domestic and foreign agencies, surveys,  focus groups, and other market data; (2) analyzing data to identify and  assess new areas of concern across professions, industries, and  markets; and (3) preparing assessments of the agency’s risk  environment. ORA is to work in coordination with internal risk teams  established in each of the agency’s major program areas—including  OCIE—and a Risk Management Committee responsible for reviewing  implications of identified risks and recommending appropriate courses  of action.
section_title: SEC’s Revised Oversight Approach Reflects Some of the Lessons Learned from the Recent Mutual Fund Scandals
paragraphs: As we recently reported, the market timing and late trading abuses that  surfaced in 2003 revealed weaknesses in SEC’s mutual fund oversight  approach. We noted in the report that lessons can be learned from SEC  not having detected market timing arrangements at an earlier stage. The  key initiatives that SEC is taking to strengthen its mutual fund oversight  program are largely intended to focus the agency’s resources on the largest  and highest risk funds and activities. Although it is too soon to assess the  effectiveness of the initiatives in light of their recent or planned  implementation, the initiatives are consistent with some of the lessons  learned concerning the importance of (1) conducting independent  assessments of the adequacy of controls over areas such as market timing,  (2) developing the institutional capability to identify and analyze evidence  of potential risks, and (3) ensuring the independence and effectiveness of  company compliance staff and potentially using their work to benefit the  agency’s oversight program.
By placing greater priority on sweep examinations, SEC may be better  positioned to independently assess, as needed, the adequacy of fund  controls designed to prevent and detect abusive practices. As we reported,  SEC staff did not examine mutual funds for market timing abuses before  late 2003, because they viewed market timing as a relatively lower risk area  since agency staff believed that funds had adequate financial incentives to  establish effective controls for it. In that regard, we noted the importance  for SEC to conduct independent assessments of controls at a sample of  funds, at a minimum, to verify that areas viewed as low risk, such as market  timing, are in fact low risk and effective controls are in place. SEC’s revised  examination priorities, particularly their emphasis on initiating sweep  examinations that focus on operational or compliance issues, may provide  the agency with greater opportunity to conduct independent assessments  of controls for emerging risks, in part to validate critical assumptions about  such risks and confirm the adequacy of controls in place to address those  risks.
By forming examiner teams dedicated to monitoring the largest and most  complex groups of affiliated advisers and funds, SEC may have the  opportunity to more efficiently or effectively use its resources and help  ensure the independence and effectiveness of the monitored firms’  compliance staff. SEC estimates that the 100 largest advisory groups of  affiliate advisers and funds accounted for about $7.1 trillion, or 85 percent,  of the fund assets under management as of the end of September 2004.  Thus, focusing on the largest advisory groups may enable SEC to attain the  greatest dollar coverage with its limited examination resources. Focusing  on the largest advisory groups may also be appropriate due to the control  deficiencies that have been found at such companies. For example, SEC  determined that nearly 50 percent of the 80 largest mutual funds had  entered into undisclosed arrangements permitting certain shareholders to  engage in market timing that appeared to be inconsistent with the funds’  policies, prospectus disclosures, or fiduciary obligations. In our earlier  mutual fund work, we also found that compliance staff at some funds  identified market timing but lacked the independence or authority  necessary to control it. This finding suggested that routine communications  with fund compliance staff could enhance SEC’s capacity to detect  potential violations at an earlier stage, if compliance staff are effective and  forthcoming about the problems they detect. SEC’s monitoring teams will  provide agency staff with the opportunity to be in routine communication  with fund compliance staff, including CCOs. Furthermore, such  communications, combined with examinations, could help SEC ensure that  fund CCOs, as required under SEC’s compliance rules, are in a position of  authority to compel others to adhere to applicable compliance policies and  procedures.
By creating ORA, SEC is laying an important part of the foundation for  developing the institutional capability to identify and analyze evidence of  potential risks. SEC staff said that ORA will seek to ensure that SEC will  have the information necessary to make better, more informed decisions  on regulation. Working with other SEC offices, ORA staff expect to identify  new technologies, such as data mining systems, that can help agency staff  detect and track risks. SEC’s compliance rules create opportunities for  ORA to leverage the knowledge of fund CCOs, including their annual  compliance reports. Although ORA may help SEC be more proactive and  better identify emerging risks, it is too soon to assess its effectiveness. In  this regard, we note that as of February 2005, ORA had established an  executive team of 5 individuals but still planned to hire an additional 10  staff to assist in carrying out its responsibilities.
Finally, SEC’s fund and adviser surveillance system is in the exploratory  stage but, if properly designed and implemented, may help the agency to  leverage its limited resources to augment its examinations and oversee  funds and advisers. Federal bank and other regulators use off-site  surveillance programs to complement their on-site examinations. Each  federal bank regulator has an off-site surveillance program to monitor the  financial condition of banks between examinations. Information from off- site monitoring is used in setting bank examination schedules and  determining the allocation of examiner resources for higher risk banks.  Similarly, a recently deployed NASD surveillance program is used to  analyze trends in broker-dealer activities and identify unusual patterns that  indicate potential problems. NASD uses surveillance analyses to initiate  cause examinations and to help its examiners focus on high-risk areas  during their routine broker-dealers examinations.
section_title: SEC’s Revised Examination Approach Raises Oversight Challenges
paragraphs: SEC’s planned changes to its mutual fund examination program offer  potential advantages, but they also involve significant tradeoffs that raise  important regulatory challenges for the agency. In comparison to federal  bank regulators, SEC has significantly less examiners relative to the  number of entities it regulates (see fig. 1), although bank and mutual fund  regulatory regimes, including their examinations, differ from each other.  As reflected in SEC’s revised oversight approach, any decision by SEC to  focus additional examination resources on one or more fund areas involves  tradeoffs that could result in less oversight of, or create a regulatory gap in,  other areas. We are particularly concerned about SEC’s capacity going  forward to review the operations of firms considered to be lower risk,  conduct risk assessments of the industry, and potentially oversee the hedge  fund industry.
By shifting examination resources to targeted sweep and cause  examinations as well as monitoring teams for larger funds, SEC may be  limiting its capacity to examine the operations of funds perceived to pose  lower risk (generally smaller funds) within a reasonable period. As stated  previously, between 1998 and 2003, SEC generally sought to conduct  routine examinations of all funds once every 5 years and shortened the  cycle to 2 or 4 years in fiscal year 2004 following an increase in resources.  However, under SEC’s revised examination program, some mutual funds  may not be examined within a 10-year period. This is because SEC plans to  annually review the operations of 10 percent of the funds deemed lower  risk on a random basis. While reviewing funds on a random basis means  each firm will have an equal chance of being reviewed annually, it is not  clear that this approach will have more of an effect in deterring abuses than  if each fund was assured of being examined every 5 years or less. Moreover,  if SEC lacks sufficient resources to annually examine 10 percent of the  funds deemed lower risk, its approach would have less of a deterrent  effect. We recognize that through sweep examinations, SEC may review  particular facets of funds deemed lower risk much more frequently than  every 10 years or more. At the same time, sweep examinations are much  more narrowly scoped than routine examinations and may exclude other  potential areas of noncompliance at individual firms.
Similarly, SEC’s inability to conduct examinations of all mutual funds  within a reasonable period may limit its capacity to accurately distinguish  relatively higher risk funds from lower risk funds and effectively conduct  routine examinations of higher risk funds. Between late 2002 and October  2004, SEC routinely examined 297, or 30 percent, of the existing fund  complexes and used its revised examination guidelines to assess the  effectiveness of the funds’ compliance controls in deterring and preventing  abuses and to assign the funds risk ratings of low, medium, or high. Had  SEC not decided in late 2003 and 2004 to shift examination resources to  sweep and cause examinations, it might have been able to assign risk  ratings to all 982 fund complexes within the following 3 years in  accordance with its routine examination cycle. Completing risk ratings for  all fund complexes would have provided SEC with an additional basis for  allocating resources to the highest risk firms. Over time, SEC’s risk ratings  can become outdated, or stale, raising the possibility for funds deemed  lower risk to become higher risk. For example, changes in a fund’s  management, such as CCO, could lead to changes that weaken the fund’s  compliance culture and controls. However, because SEC may not examine  all fund complexes within a 10-year period under its revised examination  program, its ability to assign risk ratings to all fund complexes and  routinely examine all higher risk funds may be limited.
In a previous report, we found that SEC may be missing opportunities to  obtain useful information about the compliance controls of mutual funds,  including those perceived to represent lower risks and may not be  examined within a reasonable period of time. While SEC plans to review  investment company CCO annual compliance reports during examinations,  the agency has not developed a plan to receive and review the reports on  an ongoing basis. Obtaining access to such annual reports and reviewing  them on an annual basis could provide SEC examiners with insights into  the operations of all mutual funds, including those perceived to represent  lower risks, and could serve as a basis for initiating examinations to correct  potential deficiencies or violations. SEC noted that it is considering how  best to utilize the annual reports but noted any required filing of the reports  with SEC would require rulemaking by SEC.
A final oversight challenge facing SEC’s mutual fund examination program  involves a new rule requiring hedge fund advisers to register with the  agency. Issued in December 2004, the new rule requires hedge fund  advisers to register with SEC as investment advisers by February 2006. The  rule is designed, in part, to enhance SEC’s ability to deter or detect fraud by  unregistered hedge fund advisers, some of which were involved in the  recent mutual fund abuses. Once hedge fund advisers register, SEC will  have the authority to examine their activities. The rule is expected to  increase SEC’s examination workload, but because of data limitations the  precise extent will not be known until hedge fund advisers actually register.  Currently, comprehensive information on the number of hedge funds and  advisers is not available, but SEC estimates that from 690 to 1,260  additional hedge fund advisers may be required to register under the new  rule, increasing the pool of registered advisers by 8 to 15 percent.
SEC officials estimate that at least 1,000 hedge fund advisers have  previously registered as investment advisers with SEC to meet client needs  or requirements. Under its examination program, SEC has examined these  hedge fund advisers in the same way it has examined all other registered  advisers. According to SEC officials, it is anticipated that the additional  hedge fund advisers that register with SEC will be treated the same as all  other registered advisers under SEC’s examination program. SEC has  recognized that providing oversight of the additional registered hedge fund  advisers will pose a resource challenge and has identified options for  addressing the challenge. It could require fewer hedge fund advisers to  register with SEC by raising the threshold level of assets under  management required for adviser registration. It also has the option of  seeking additional resources from Congress for the increased workload  resulting from an increased number of registered advisers. Whatever  approach is ultimately taken, SEC will have to consider the potential  resource implications of the new rule for its oversight of mutual funds.
section_title: SEC Can Improve Certain Mutual Fund Examination Quality Control Measures
paragraphs: SEC has integrated quality controls into its routine examinations but could  benefit from additional controls to ensure that policies and procedures are  being implemented effectively and consistently throughout SEC field  offices. Under its new initiatives, SEC’s routine examinations will continue  to be the primary regulatory tool for determining whether all funds and  advisers are complying with the federal securities laws. Examination  quality controls provide, among other things, assurances that important  documents are provided supervisory review, and examinations are  conducted according to agency policies, procedures, and individual  examination plans. SEC could improve its quality control measures in  three areas: supervisory review of risk scorecards, preparation of written  examination plans, and review of completed examinations and work  papers. Bank and other financial regulators have quality control measures  that provide assurances above and beyond those measures used by SEC.
subsections: section_title: SEC Standards for Reviewing Mutual Fund Risk Scorecards Do Not Ensure Accuracy or Completeness
paragraphs: The risk scorecards prepared by SEC during each mutual fund examination  are critical work papers, providing the basis for determining areas to  review in depth and an overall risk rating for a fund. A set of individual  scorecards has been developed to assist examiners in assessing and  documenting a fund’s compliance controls in 13 strategic areas and to  determine the amount of additional testing examiners will do. (See table  1.)
If controls in an area are strong, examiners may do limited or no additional  testing to detect potential abuses, but if weak, additional testing is  expected to be performed. Collectively, the 13 areas reviewed with the set  of individual scorecards provides the basis for determining a mutual fund’s  overall risk rating, which OCIE uses to determine how frequently the fund  will be examined. While the risk scorecards currently cover 13 areas, SEC  officials stated that each scorecard serves, in concept, as a model for  assessing controls in a particular area of a firm’s activities. As such, SEC  staff could create additional scorecards to assist them in their review of  areas not covered by existing scorecards or modify existing scorecards not  suitable for reviewing the controls used by a firm in a critical area. OCIE  and field office officials told us that all applicable risk scorecards generally  should be completed during routine examinations, but if there are time  constraints due to extenuating factors, all scorecards may not be  completed.
Even though risk scorecards are important work papers for documenting  and assessing fund compliance controls, SEC standards do not expressly  require that they receive supervisory review. Current OCIE standards for  preparing examination work papers, including scorecards, specify that they  should be prepared in an organized manner facilitating supervisory review  and examination reporting. The standards do not provide further  supervisory review requirements such as who should do the review, how,  or when. While the review of scorecards is not expressly required, OCIE  headquarters and SEC field office officials stated that supervisors do  review scorecards and other examination work papers but typically do not  sign or initial them to document that they have been reviewed. In addition,  we were told that lead examiners and branch chiefs review work papers  throughout the examination process. These officials also review risk  scorecards and other work papers when reviewing final examination  reports, making sure that all findings are adequately supported and  summaries of the scorecard findings included in the examination reports  are accurate. After completing their review of examination reports, branch  chiefs sign a form to document their review.
In contrast to OCIE, federal bank and other regulators have standards  requiring supervisors to document that they have reviewed examination  work papers. Examples of the work paper standards include:  Federal Reserve guidance requires examiners-in-charge or other  experienced examiners to review all work papers as soon as practicable  and to sign or initial the applicable documents to evidence their review.
OCC guidance requires examiners-in-charge or other experienced  examiners to sign or initial work paper cover sheets to evidence their  review. The guidance allows reviewers to tailor the thoroughness of  their review based on the experience of the examiner preparing the  work paper.
According to NYSE and NFA officials, the organizations require senior  staff to review and sign work papers. NFA officials said that their work  papers are electronic, so staff mark a checkbox to evidence their work  paper review.
While SEC officials stated that the review of the scorecards is documented  indirectly by the supervisor’s signature on the examination report, without  the supervisor’s signature or initials on the scorecards themselves, there is  no way to readily verify that the scorecards were reviewed. Our review of  546 scorecards from 66 routine examinations of funds completed in fiscal  year 2004 by SEC’s Midwest Regional Office (MRO), Northeast Regional  Office (NERO), and Philadelphia District Office (PDO) disclosed a number  of deficiencies potentially stemming from quality control weaknesses. Most  of the scorecards did not contain evidence of supervisory review as  expected, based on statements by SEC officials, but 34 scorecards, or  about 6 percent, were signed or initialed as evidence of review. Regardless  of whether the completed scorecards were signed or initialed, we found  deficiencies in four areas that raise questions about the adequacy or  completeness of supervisory review.
First, each scorecard should be marked as to whether examiners rated  the compliance controls in the area as highly effective, effective, or  ineffective. We found 32, or about 6 percent, of the total scorecards  where the control rating was not marked.
Second, copies of scorecards should be included with the work papers  to facilitate supervisory review, but we found 11, or about 17 percent, of  the 66 examinations lacked any scorecards and 15, or about 23 percent,  were missing one or more scorecards.
Third, documentary evidence should be cited on scorecards to support  effective and highly effective ratings, but we found 25, or about 5  percent, of the total scorecards did not cite documentary evidence  supporting such ratings.
Fourth, scorecard ratings are included in examination reports, but we  found the ratings marked on 21, or about 4 percent, of the total  scorecards had ratings that differed from the ones in the examination  reports.
SEC supervisors document their review of examination reports, which  include a summary of the risk scorecard findings. Nonetheless, without  documenting that the scorecards themselves were reviewed, SEC does not  know if deficiencies resulted from a lack of or inadequate supervisory  review. The systematic supervisory review of work papers, particularly risk  scorecards, is essential for ensuring examination quality. Such reviews help  to ensure that the work is adequate and complete to support the  assessment of fund compliance controls as well as report findings and  conclusions. Likewise, documentation of the review is important to ensure  that all critical areas are reviewed. The reviewer’s initials or signature are  written verification that a specific employee checked the work.
section_title: Written Examination Plans for Fund Examinations Are Not Required but Would Be Useful for Documenting Agreements Reached on Review Areas
paragraphs: Written examination plans that document the scope and objectives of  routine examinations are not required by OCIE. Instead, OCIE officials  stated that written examination plans are optional. OCIE allows branch  chiefs and lead examiners to decide whether to prepare written plans, with  branch chiefs typically meeting with examination teams to discuss the  preliminary scope of examinations. Each routine examination is somewhat  different because of the risk-based approach used by OCIE. Under this  approach, all areas of compliance or fund business activities are not  reviewed and instead review areas are judgmentally selected based on their  degree of risk to shareholders. As a result, each examination is customized  to the activities of the particular fund under examination, with the success  of routine examinations depending, in part, on proper planning. The  documentation of this planning is important for tracking agreements  reached on examination scope and objectives and can be used as a guide  for the examination team. Furthermore, the plan can be used to determine  whether the examination was completed in accordance with the planned  scope. According to OCIE officials, written plans may be helpful in  planning examinations of large fund complexes, but many of the  examinations conducted are of small firms that have five or fewer  employees. For these small firms, the officials said that it may not be  necessary to prepare a written examination plan, especially if the  examination team conducting the work consists of one or two persons.
While OCIE does not require the preparation of written examination plans,  we found that SEC’s NERO requires examiners to prepare a planning  memorandum to document examination scope and objectives, including  firms to be examined within a fund complex, areas considered high risk,  and areas to be reviewed. NERO branch chiefs approve the memorandums  before the on-site work begins, and the memorandums effectively serve as  examination plans. In contrast, SEC’s MRO and PDO do not require  planning memorandums or examination plans. Instead, branch chiefs in  these two offices meet with the examination teams to discuss the scope of  examinations and then let the staff decide whether to prepare a written  plan, according to MRO and PDO officials. MRO officials said that some  branch chiefs will recommend that for large funds, teams prepare written  examination plans since it helps coordinate the work. For 66 routine  examinations we reviewed at these three offices, about half, or 53 percent,  had written planning memorandums or examination plans. Examinations  of the larger fund complexes that were managing more than $1 billion in  assets also had examination plans for about half, or 54 percent.
In contrast to OCIE, federal bank and other regulators require their staff to  prepare written examination plans before conducting examinations.  Examples of examination plan requirements include:  FDIC guidance requires the examiner-in-charge to prepare a scope  memorandum to document, among other things, the preliminary  examination scope; areas to be reviewed, including the reasons why;  and areas not to be included in the examination scope, including the  reasons why.
Federal Reserve guidance requires that a comprehensive risk-focused  supervisory plan be prepared annually for each banking organization.  The guidance also requires the examiner-in-charge, before going on-site,  to prepare a scope memorandum to document, among other things, the  objectives of the examination and activities and risks to be evaluated;  level of reliance on internal risk management systems and internal and  external audit findings; and the procedures that are to be performed. To  ensure consistency, the guidance requires the scope memorandum to be  reviewed and approved by Reserve Bank management.
OCC guidance requires the examiner-in-charge or portfolio manager to  develop and document a supervisory strategy for the bank that  integrates all examination areas and is tailored to the bank’s complexity  and risk profile. The strategy includes an estimate of resources that will  be needed to effectively supervise the bank and outlines the specific  strategy and examination activities that are planned for that supervisory  cycle. The strategies are reviewed and approved by the examiner-in- charge’s or portfolio manager’s supervisor.
NYSE and NFA officials told us that staff are required to prepare written  examination or audit plans. NYSE officials said that staff meet with  examination directors to reach agreement on the scope of their  examination plans. NFA officials said that staff complete a planning  module that includes a series of questions that staff answer to determine  the scope of the audit, and the completed planning module serves as the  audit plan.
Examination planning meetings between SEC branch chiefs and  examination teams are important for providing the opportunity to discuss  and reach decisions about critical areas of examination scope and  objectives. These discussions by themselves, however, do not provide a  record of the agreements reached and may not result in a clear and  complete understanding for examiners about the scope and objectives of a  particular examination. A written examination plan would provide such a  record—potentially enabling branch chiefs to better supervise  examinations and assisting lead examiners to better communicate the  examination strategy to the examination team. Such quality control is  especially important given that staff must exercise considerable judgment  for examination scope under SEC’s risk-based approach.
section_title: SEC Efforts to Ensure Quality Do Not Include Review of Work Papers of Completed Mutual Fund Examinations
paragraphs: SEC uses several methods to ensure the quality of its examinations but  does not review completed examinations and work papers as done by other  regulators to determine whether the examinations were conducted  according to procedures or done consistently across field offices. OCIE has  issued various policies and procedures to promote examination quality and  consistency across the 10 SEC field offices that conduct the majority of its  examinations. To help ensure that these policies and procedures are  followed, SEC relies on experienced supervisors in its field offices to  oversee all stages of routine examinations. Specifically, branch chiefs meet  with examination staff to discuss the preliminary scope of examinations,  advise staff during the fieldwork, and review all examination reports.  Assistant directors in SEC field offices also assist in overseeing  examinations and review all examination reports. Also, associate directors  and regional or district administrators in SEC field offices may review  examination reports. In addition, SEC field offices send each report and  deficiency letter, if any, to an OCIE liaison, who reviews them. Finally,  OCIE annually evaluates each field office examination program based on  factors such as the overall quality of the office’s examination selection and  findings; new initiatives and special projects; use of novel or effective risk  assessment approaches; and overall productivity, including achievement of  numerical examination goals.
In contrast to OCIE, we were told that federal bank and other regulators  have quality assurance programs that include reviews of completed  examinations or other activities. Examples of such reviews include:     FDIC guidance states that the agency reviews each regional office’s  compliance examination program every 2 years, in part, to evaluate the  consistency of supervision across the regions and compliance with  policies and procedures. According to the guidance, evaluations include  a review of examination reports and work papers.
Federal Reserve officials said that the agency conducts on-site  operations reviews of the banking supervision function of individual  Reserve Banks at least every 3 years. The review targets each Reserve  Bank’s risk-focused supervision program and includes a review of a  sample of examination reports, work papers, and other supporting  documentation. It also encompasses the bank’s ongoing quality  management function, or the processes, procedures, and activities the  bank uses to ensure that examination reports and related documents are  of high quality and comply with established policy.
OCC officials told us that the agency reviews its large bank examination  program, including specific examination procedures. It conducts  reviews to determine whether lead examiners are supervising banks  according to plans. It also assesses specific examination procedures  across samples of banks. Agency officials said that teams periodically  review how examiners are conducting certain procedures to ensure that  they are being implemented consistently throughout all field offices.
NASD conducts quality and peer reviews to improve the quality,  consistency, and effectiveness of its examination program. Under  quality reviews, each NASD district office annually evaluates its  performance in two or three areas. Under peer reviews, staff go on-site  to district offices to evaluate particular program areas.
NFA officials told us that the organization randomly selects completed  audits for review on a quarterly basis and, as part of the review,  supervisory teams review work papers to determine whether the audits  complied with established policies and procedures.
While OCIE staff evaluate all completed examinations by reviewing the  final examination report, they do not review a sample of completed  examinations and work papers to periodically assess examination quality  and consistency across SEC’s field offices. SEC officials stated that after- the-fact reviews of underlying work papers may not be a cost-effective use  of resources, given that key findings and evidentiary materials should be  discussed and described in the examination report itself, which is reviewed  by OCIE staff. Further, it would be difficult to second-guess decisions made  by examiners when on-site, since reviewers would not have access to the  same information. Finally, agency officials said that OCIE resources are  limited, and time spent reviewing completed examination work papers  would result in less time spent on conducting examinations. While  reviewing completed examination work papers involves resource  tradeoffs, it may yield important benefits. OCIE may be able to better  determine whether its examiners are complying with established policies  and procedures and whether its built-in quality controls are working. A  review of underlying work papers also may allow OCIE to better assess the  consistency of examination quality within and across SEC’s field offices as  well as the extent to which existing quality controls are helping to ensure  that quality is maintained.
According to SEC officials, the agency is implementing a computer-based  document management system. Under this system, it is anticipated that  most, if not all, of the work papers created during examinations will be  converted into electronic files, and these files will be maintained in a  consistent manner online for a number of years. SEC officials said that  when the system is fully operational, estimated to be some time in 2006, all  work papers created during an examination will be available electronically  to OCIE staff. At that point, OCIE liaisons could review electronic  examination work papers on a sample basis in conjunction with their  review of examination reports. In addition, electronic work papers would  eliminate the need to be on-site to review underlying examination  documentation and work papers across SEC’s examination program.
Importantly, deficiencies we found during our review of risk scorecards  highlight the need for OCIE to periodically assess the consistency of  examination staff’s use of scorecards and other steps being taken during  examinations. While the requirement to complete risk scorecards became  effective in October 2002, SEC has not yet evaluated, for instance, whether  the risk scorecards are being completed according to the guidance  provided, whether changes to the design of the scorecards are needed, and  whether additional guidance or training is needed. In March 2003, OCIE  provided one training course on the scorecards, which was attended by 98  examiners, or about 20 percent of the SEC examiners devoted to fund and  adviser examinations. According to SEC officials, two senior OCIE staff  visited each field office during the spring and summer of 2003 and provided  a full day of training on the scorecards to all examination staff.  Nevertheless, the scorecard deficiencies we found during our review may  indicate that additional training is needed.
In addition, the scorecards may have design weaknesses that result in  inconsistencies across SEC field offices. For example, field office officials  stated that scorecards are designed for investment companies organized as  mutual funds and do not readily apply to investment companies organized  as unit investment trusts. NERO examiners did not complete scorecards  for unit investment trusts, but MRO examiners did by modifying the  scorecards as needed. Similarly, SEC field office officials stated that while  the scorecards are designed to cover a broad range of fund compliance  controls, fund controls for detecting and preventing market timing do not  fall squarely under any of the 13 areas covered by the scorecards. As a  result, staff have used work papers other than the risk scorecards to  document their assessment of market timing controls. SEC officials said  that the scorecards are models created to assist examiners in assessing  fund controls. As such, scorecards are not intended to exist necessarily for  every conceivable control and examiners have the flexibility to modify the  scorecards as necessary. Moreover, the officials said that some  inconsistencies in the preparation of risk scorecards are expected because  not all funds and advisers are the same. In that regard, SEC officials told us  that the approach taken by MRO staff in modifying a scorecard to fit the  circumstances of an examination appears to be consistent with the  approach to scorecard use expected by OCIE.
section_title: SEC’s Oversight Examinations of Broker-Dealers Provide Limited Information on the Adequacy of SRO Oversight
paragraphs: To assess SRO oversight of broker-dealers, including their mutual fund  sales practices, SEC conducts examinations of broker-dealers shortly after  they have been examined by SROs. However, these SEC broker-dealer  examinations, which involve a significant commitment of agency  examination resources, provide limited information on the adequacy of  SRO oversight and impose duplicative regulatory costs on the securities  industry. SEC and SROs’ broker-dealer examinations often cover different  time periods, and generally employ different sampling methodologies and  use different examination guidelines. Consequently, SEC cannot reliably  determine whether its examination findings are due to weaknesses in SRO  examination procedures or some other factor. Another deficiency we found  regarding SEC’s SRO oversight of broker-dealer mutual fund sales practices  is that the agency does not have automated information on the full scope of  areas reviewed during its broker-dealer oversight examinations and,  therefore, cannot readily and reliably track useful examination  information.
subsections: section_title: SEC Often Cannot Attribute Broker-Dealer Oversight Violations It Finds to Weaknesses in SRO Examination Programs, Because Different Examination Procedures Are Used
paragraphs: SEC performs two types of activities to review the quality of SRO oversight  of broker-dealers, including their sales of mutual funds. First, SEC  conducts inspections of NASD and NYSE on a 3-year cycle that cover  various aspects of their compliance, examination, and enforcement  programs. These SRO oversight inspections are designed to determine  whether an SRO is (1) adequately assessing risks and targeting its  examinations to address those risks, (2) following its examination  procedures and documenting its work, and (3) referring cases to  enforcement authorities when appropriate. When conducting these  inspections, SEC reviews a sample of the SRO’s examination reports and  work papers to identify problems in examination scope or methods. As a  result of these inspections, SEC has identified deficiencies in SRO  examinations, including ones related to the SROs’ examinations of mutual  fund sales practices, and communicated those to the SRO to remedy the  problem. Second, SEC conducts broker-dealer oversight examinations,  during which it examines some broker-dealers from 6 to 12 months after an  SRO examines the firms. The purpose of broker-dealer oversight  examinations is to help the SROs improve their examination programs by  identifying violations that the SROs did not find and also by assisting them  in evaluating improvements in how SRO examiners perform their work.  SEC officials told us that a secondary goal of these examinations is to  supplement the SROs’ enforcement of broker-dealer compliance with  federal securities laws and regulations.
SEC’s broker-dealer oversight examinations involve a significant  commitment of agency resources and expose firms to duplicative  examinations and costs. In addition to conducting broker-dealer  examinations for the purposes of assessing SRO oversight (including for  mutual fund sales practices), SEC conducts cause, special, and surveillance  examinations of broker-dealers to directly assess broker-dealer compliance  with federal securities laws and regulations, including those related to  mutual fund sales. SEC currently has an internal goal of having oversight  examinations account for 40 percent of all broker-dealer examinations  each year. In 2004, 250, or 34 percent, of its 736 broker-dealer examinations  were oversight examinations. Broker-dealers that are subject to similar  SEC and SRO examinations that may take place within a 6 to 12 month  period incur the costs associated with assigning staff to respond to  examiner inquiries and to make available relevant records as requested.
Although SEC broker-dealer oversight examinations involve a significant  commitment of agency examination resources and impose costs on  securities firms, our past work questioned their cost-effectiveness. In a  1991 report, we found that the way SEC conducted oversight examinations  of broker-dealers provided limited information to help SROs improve the  quality of their broker-dealer examination programs. Specifically, during  its oversight examinations of broker-dealers, SEC often found violations  not identified by SROs and frequently could not attribute the violations it  found to weaknesses in SRO examination programs. Because SEC and  SROs used different examination procedures or covered different time  periods of broker-dealer activity, SEC examiners often could not determine  whether the violations they found resulted from the improper  implementation of procedures by SRO examiners or differences between  the procedures used or the activity period covered. We previously  recommended that SEC directly test SRO examination methods and  results. However, based on its efforts to replicate some examinations  conducted by SROs, the agency concluded that this was unproductive  because it only confirmed findings identified by SROs during their  examinations.
Our current review has shown that despite our 1991 findings, SEC  continues to conduct oversight examinations in a similar manner—by using  different examination guidelines and time periods. First, SEC continues to  review firm activities during the time between the completion of the SRO  examination and its own examination. Next, when SEC is reviewing a firm’s  transactions or customer accounts to identify potential abuses, it generally  does not duplicate the sampling technique used by the SRO, but instead  selects its own sample of transactions or customer accounts based on its  own procedures. Finally, SEC examiners ask different questions to identify  potential abuses. For example, although SEC and NASD both direct their  examiners to ask questions to assess potential weaknesses in a firm’s  internal controls to prevent market timing and late trading, their  procedures call for examiners to ask about different potential internal  control weaknesses.
According to SEC officials, its examiners do not use the same procedures  as SROs because using different procedures allows them to find violations  that would not otherwise be found if they just duplicated the SRO  procedures. Also, SEC officials stated that SEC has an obligation to review  the broker-dealer’s activities at the time of the SEC examination to ensure  compliance with securities laws at that time. However, as a result, SEC  often cannot determine the specific reason why the SRO did not find the  violations, limiting its ability to suggest improvements to SRO programs.  SEC routinely provides SROs copies of deficiency letters it sends to broker- dealers as a result of oversight examinations. These deficiency letters  sometimes include oversight comments that include steps the SRO can  take to enhance its program. SRO officials stated they can often identify the  reasons why SEC found the violations, but in many cases the reason is due  to SEC’s use of different procedures, such as different review periods or  samples. Consequently, SEC often cannot attribute a violation it finds to a  problem with the SRO’s examination program. SEC officials said in some  cases when SEC identifies a violation, it is able to determine whether the  violation was occurring at the time of the original examination and should  have been detected by the SRO. For example, in some cases when SEC  finds an error in a broker-dealer’s net capital calculation, it is able to trace  the error to previous calculations and determine whether it existed during  the SRO examination. Even in cases when SEC can attribute a violation it  found to a weakness in the SRO examination, it does not track this  information in its automated examination tracking system and, as a result,  cannot use it to identify trends in SRO problems it discovered during  oversight examinations. SEC officials stated that they have a staff  committee conducting a comprehensive review of oversight examination  procedures and plan to add a feature to SEC’s examination tracking system  to allow it to more systematically track identified weaknesses in SROs’  examination programs.
Although SEC’s oversight examinations continue to find violations at  broker-dealers and, thus, provide investor protection benefits, the  violations provide limited information for assessing the quality of the SRO  program. This information is particularly important given that the number  of violations that SEC has found during its oversight examinations and  determined as not found by NASD has increased in recent years. As shown  in figure 2, the number of these violations that SEC found but has  categorized as not found by NASD more than doubled between fiscal years  2002 and 2004.
Despite this significant increase, SEC officials could not explain why the  number of these violations increased but stated that the increase did not  necessarily represent a decrease in the quality of NASD’s examination  program. They said some of the increase is due to a significant increase in  the number of rules applicable to broker-dealers. SEC officials told us that  SRO officials have noted, and they agree, that the number of these  violations, alone, is not always an appropriate measure of the quality of  SRO examination programs. Accordingly, SEC officials told us that the  agency recently began tracking findings deemed to be significant to allow it  to better assess the materiality of an increase in the number of missed  violations. If SEC had tested NASD’s examination methods or better  tracked the reasons why NASD did not find a violation, SEC would have  more information to assess the quality of NASD’s examination program.
section_title: SEC Does Not Track the Full Scope of Work Performed during Its Oversight Examinations
paragraphs: Another deficiency we found regarding SEC’s SRO oversight is that the  agency cannot readily and reliably track key examination information. In  assessing the quality of SEC’s oversight of broker-dealer sales of mutual  funds, we asked SEC to provide data on which of its broker-dealer  oversight examinations in recent years included reviews of mutual fund  sales practices. The data would help determine the extent that SEC has  reviewed mutual fund sales practices. SEC was not able to provide this  information because it does not have automated information on the full  scope of areas reviewed during its broker-dealer oversight examinations.
SEC maintains a broad range of automated information about its  examinations in its Super Tracking and Reporting System (STARS),  including basic information about the firm, SEC staff assigned to conduct  the examination, and the deficiencies and violations found during the  examination. STARS identifies examinations that reviewed specific areas  of special interest to SEC, called “focus areas,” as identified by senior SEC  staff in headquarters, and new areas are added in part based on the  emergence of new abuses. For example, SEC added breakpoints as a focus  area in January 2003 and market timing and late trading in 2004. Although  focus area designations provide useful information about how often SEC  reviews some areas, focus areas do not cover all areas potentially reviewed  by SEC during its examinations. Without methodically tracking the full  scope of work performed during oversight examinations, SEC lacks  information for determining how effective its oversight is in two important  areas.
First, because SEC does not know how often it has reviewed particular  areas such as mutual fund sales practices during its oversight  examinations, it cannot ensure that it has adequately reviewed all areas it  considers important. When SEC reviews particular areas, its examiners  generally refer to a set of written procedures, known as examination  modules that provide information to guide examiners’ work. STARS does  not include data fields to track whether staff use the module on mutual  funds during an examination. Therefore, the extent of coverage of mutual  funds is unknown. As a result, SEC officials could not determine how many  of the approximately 1,400 broker-dealer oversight examinations  conducted between 2000 and 2004 included a review of mutual fund sales  practices. SEC officials stated that they have a separate database  containing examination reports that can be electronically searched to  identify relevant examinations containing a search term such as “mutual  fund,” which would yield an estimate of the number of examinations that  reviewed broker-dealer mutual fund sales practices. However, according to  an official, not all examinations covering mutual fund sales practices would  be captured because some examination reports that included reviews of  mutual fund sales practices would not necessarily include any mention of  mutual funds, especially if SEC identified no deficiencies or violations in  that area.
In contrast to SEC, both NASD and NYSE have systems with capability to  track the full scope of examinations including the use of mutual fund and  other examination modules. For NASD, some of its offices are able to track  which of its broker-dealer examinations were followed by an SEC oversight  examination. At 8 of its 15 district offices, which account for 55 percent of  its examinations, NASD tracked this information and SEC conducted  oversight examinations of approximately 5 percent of the 2,602 NASD  examinations conducted between January 1999 and August 2004 that  reviewed mutual fund sales practices. The remaining seven offices were  not able to track this information because, according to an NASD official,  the SEC field office conducting oversight examinations did not always  provide a letter informing them that an oversight examination was  conducted. With mutual fund sales practices being a regulatory priority, the  percentage of SEC examinations reviewing these practices would be a  useful measure for ensuring that the agency is addressing this priority.
Second, because SEC does not track the full scope of work performed  during its oversight examinations, it is limited in its ability to assess the  significance of deficiencies and violations it finds. Because SEC does not  know how often it has reviewed a particular area, the data it tracks on the  number of deficiencies and violations it finds in a particular area are less  meaningful. For example, it would be less significant if SEC found  violations in a particular area during 5 out of 100 examinations as opposed  to finding violations during 5 out of 5 examinations during which it  reviewed the area. Without knowing the full scope of each oversight  examination and therefore the number of times a particular area was  reviewed, data tracked by SEC on the number of deficiencies and  violations it finds are less meaningful.
In addition to conducting broker-dealer oversight examinations to evaluate  the adequacy of SRO activities, SEC conducts other types of examinations,  including cause and sweep examinations, which are designed to directly  assess broker-dealer compliance with the law. SEC tracks the number of  firms it targets during its examination sweeps along with the number of  violations and deficiencies it finds. SEC officials told us that the agency  tracks the number of findings from these examinations as a percentage of  the number of firms examined, and that tracking such information helps  SEC assess the prevalence of the findings relative to the number of firms.  However, without tracking the scope of work performed during its  oversight examinations, SEC is unable to make similar assessments about  the prevalence of violations and deficiencies identified during those  reviews.
Appendix II provides information you requested about (1) how SEC, NASD,  and NYSE share information, including written examination guidance,  related to their review of mutual fund sales practices and other  examination priorities; (2) how SEC distributes and stores examination  guidance for use by its broker-dealer examiners; and (3) what training SEC  has provided to broker-dealer examiners on mutual funds and other topics  and how it tracks and assesses such training.
section_title: Conclusions
paragraphs: In the wake of the market timing and late trading abuses, SEC staff  implemented significant changes to the agency’s mutual fund examination  program in the view that doing so would help ensure the earlier detection  and correction of violations. These changes—including conducting  additional sweep examinations and continuously monitoring large  companies—reflect a practical approach designed to focus SEC’s limited  resources on higher risk funds and activities and have the potential to  strengthen SEC’s oversight practices in certain regards. Nonetheless, the  changes also involve tradeoffs, such as limiting the agency’s capacity to  review funds perceived to be lower risk and conduct risk assessments of all  funds in a timely manner. Moreover, SEC’s capacity to effectively monitor  the hedge fund industry is open to question, given the tradeoffs that the  agency has had to make in overseeing the mutual fund industry. While we  recognize that SEC at some point may need to request additional resources  from Congress to carryout its mutual fund and other oversight  responsibilities, such requests should only occur after the agency has  explored and achieved all available efficiencies within its existing resource  limitations. Whether SEC’s utilization of resources under its revised  examination program will provide effective oversight remains to be seen.  Future adjustments by SEC to resources devoted to various oversight  activities, such as sweep examinations and randomly selected lower risk  fund examinations, are likely to occur as the agency gains experience  through conducting these oversight activities and changing conditions in  the mutual fund industry. However, SEC has had extensive experience with  its broker-dealer oversight examinations, and the effectiveness of these  examinations for improving the quality of SRO oversight remains unclear.  This situation raises concern, particularly in light of the significant level of  resources devoted to oversight examinations and the resource challenges  faced by SEC’s fund and adviser examination program.
We also identified basic weaknesses in SEC’s approaches to conducting  mutual fund and broker-dealer examinations. For mutual fund  examinations, SEC does not require staff to document their examination  plans to facilitate supervisory review. Second, SEC has issued work paper  standards but lacks guidance on their supervisory review. Moreover,  despite the importance of risk scorecards in determining the depth of work  done during examinations, SEC has not yet assessed whether they are  prepared according to standards since implementing the scorecards in  2002. For broker-dealer examinations, SEC has not developed an  automated system to track the full scope of work completed during  examinations and therefore lacks useful information about SRO oversight.  Without addressing these deficiencies, SEC’s capacity to effectively  oversee the mutual fund industry and SROs is reduced.
section_title: Recommendations for Executive Action
paragraphs: To improve SEC’s oversight of mutual funds and SRO oversight of broker- dealers that sell mutual funds, we are making four recommendations to the  SEC Chairman. First, we recommend that SEC periodically assess the level  of resources allocated to the various types of examinations in light of their  regulatory benefits to help ensure that the agency is using its resources  efficiently and effectively to oversee the mutual fund industry, including  broker-dealers that offer mutual funds. As part of this assessment, SEC  should seek to ensure that it allocates sufficient resources to mitigate any  regulatory gaps that may currently exist concerning the timely examination  of mutual funds perceived to represent lower risk, complete mutual fund  risk assessments within a more reasonable period, and fulfill its new  oversight responsibilities for the hedge fund industry. Second, in so doing,  we recommend that the agency assess its methodology for conducting  broker-dealer oversight examinations and whether some portion of the  resources currently devoted to these examinations could be better utilized  to perform mutual fund examinations.
Third, to strengthen SEC’s approach to mutual fund examinations, we  recommend that SEC  establish a policy or procedure for supervisory review of work papers  prepared during routine examinations and for documenting such  reviews;  establish a policy or procedure for preparing a written plan for each  routine examination, documenting at a minimum the preliminary  objectives and scope of the examination; and  consider reviewing on a sample basis completed routine examinations  and work papers to assess the quality and consistency of work within  and across the field offices conducting examinations.
Fourth, to assess and improve the effectiveness of SEC’s oversight of SRO  broker-dealer examination programs, we recommend that the Chairman,  SEC, electronically track information about the full scope of work  performed during broker-dealer oversight examinations, including all  major areas reviewed, to determine whether areas are receiving adequate  review and to more fully assess the significance of deficiencies and  violations found.
section_title: Agency Comments and Our Evaluation
paragraphs: SEC provided written comments on a draft of this report, which are  reprinted in appendix III. SEC also provided technical comments that we  incorporated into the final report, as appropriate. SEC focused most of its  comments on providing further elaboration on the potential benefits of its  examination strategy for overseeing mutual funds and investment advisers  and on the benefits obtained from its broker-dealer oversight examinations.  In addition, SEC briefly commented that it will consider our  recommendation directed at improving its quality controls for routine fund  examinations and that it has formed a working group to explore ways to  enhance the value of its broker-dealer oversight examinations, including  their ability to identify the reasons that violations may have been missed by  SRO examinations.
First, SEC stated that it is not possible for the agency to conduct timely,  comprehensive routine examinations of every mutual fund and adviser,  given the size of the industry and agency resources. Further, it expects its  risk-targeted examinations to provide an effective means of addressing  risks in the securities industry. Specifically, it believes that looking at the  same type of risk at a number of different firms is a better approach than  examining a single firm in depth. According to SEC, this approach will  provide benefits by promptly identifying emerging trends and compliance  problems, and individual firms can be compared to their industry peers.  The agency believes this approach has already yielded benefits in  identifying and addressing significant compliance problems before  becoming major crises. In addition, SEC stated that the program it is  developing to randomly select a sample of lower risk firms for routine  examination will address our concern that such firms may not be given  sufficient attention under its revised oversight strategy. According to SEC,  this approach will provide a deterrent effect, enable the agency to test  assumptions and techniques used throughout its examination program, and  allow the agency to draw inferences about compliance in the adviser  community, based on statistically valid sampling techniques.
We recognize that SEC’s revised examination strategy for mutual funds and  advisers offers potential benefits, including focusing its limited resources  on firms and activities that are perceived to pose higher risks. Nonetheless,  we continue to be concerned about SEC’s ability to examine all mutual  funds within a reasonable period and accurately assess the relative risk of  each fund on a timely basis. Unlike broker-dealers, mutual funds are  regulated and examined solely by SEC. Under SEC’s current plans to  randomly sample 10 percent of the firms perceived to be lower risk for  routine examination each year, it is possible that up to a third of the total  number of firms would not be selected for examination within a 10-year  period. We believe that this is a lengthy time period for firms to conduct  business without being examined. Similarly, SEC’s inability to conduct  examinations of all mutual funds within a reasonable period will limit its  capacity to accurately distinguish relatively higher risk funds from lower  risk funds and effectively target its limited examination resources on those  funds posing the highest risks. Therefore, we continue to believe that, as  recommended, SEC should periodically assess the level of resources  allocated to its various types of examinations and in so doing ensure that it  allocates sufficient resources to mitigate any regulatory gaps that currently  exist in the timely examination of funds perceived to represent lower risks  and to ensure that it completes mutual fund risk assessments within a more  reasonable time period.
Second, SEC stated that its broker-dealer oversight examinations provide  quality control over SRO examinations and serve other important goals.  For example, SEC stated that oversight examinations allow it to detect  violations that otherwise might not be detected, conduct routine  examinations of new products or services, and test and validate  assumptions and techniques used throughout the broker-dealer  examination program. In addition, SEC expressed concern about our  suggestion that it should reproduce SRO examinations if its oversight  examinations are to provide accurate quality control information. SEC  stated that this suggested approach would result in redundancies for  broker-dealers being examined and limit the agency’s ability to reach  conclusions about SRO examination programs. By conducting its  examinations as independent compliance reviews, SEC stated that it can  assess whether SRO procedures were followed and whether SRO  procedures need to be modified or enhanced. The agency stated that  through its oversight program it has identified SRO procedures that need to  be modified or enhanced and its examiners meet regularly with SRO  examiners to review the results of oversight examinations. Finally, SEC  commented that it has formed a working group to explore ways to gain  additional value from its broker-dealer oversight examinations, such as by  better identifying the reasons that a violation may not have been detected  by an SRO examination, aiding the SRO in improving its program, and  minimizing burden on the firm examined.
We recognize that SEC’s oversight examinations serve more than one goal  and provide investor protection benefits. While such examinations serve a  variety of purposes, one of their primary purposes is to assess the quality of  SRO examinations. In fulfilling this purpose, we remain concerned that  SEC’s approach provides limited ability to identify the reasons why an SRO  did not find violations that SEC found and, in turn, provide suggestions for  improving SRO examinations. SEC is responsible for overseeing SROs that  examine broker-dealers on a regular basis, and it conducts oversight  examinations of only a small percentage of the total number of broker- dealers. Thus, it is critical for SEC to ensure that SROs conduct effective  examinations. As discussed, SEC has formed a working group to evaluate  its oversight examinations. We believe this is a step in the right direction  and also provides the agency with the opportunity to evaluate its approach  and level of resources devoted to broker-dealer oversight examinations.
Finally, regarding our recommendation that SEC strengthen three aspects  of its quality control framework for routine fund examinations, the agency  stated it will fully consider the recommendation. Specifically, in 2006, the  agency plans to deploy an electronic system for work papers. In  preparation for this effort, it plans to review how new technology can be  used to improve the quality of examinations and it will consider our  recommendation in its review. While SEC did not directly comment on our  recommendation that it electronically track information about the full  scope of work performed during its broker-dealer oversight examinations,  we believe that this would provide SEC important information to determine  whether areas are receiving adequate review and the relative significance  of violations found in each area.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution of this report until 30  days from the report date. At that time, we will provide copies of this report  to the Chairman of the House Committee on Financial Services; the  Chairman of the Subcommittee on Capital Markets, Insurance and  Government Sponsored Enterprises, House Committee on Financial  Services; and the Chairman and Ranking Minority Member of the Senate  Committee on Banking, Housing, and Urban Affairs. We also will provide  copies of the report to SEC, FDIC, the Federal Reserve Board of  Governors, NASD, NYSE, and OCC and will make copies available to others  upon request. In addition, the report will be available at no cost on GAO’s  Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-8678 or hillmanr@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made key contributions to this report are  listed in appendix IV.
section_title: Scope and Methodology
paragraphs: To identify and assess the changes SEC has made to or is planning for its  mutual fund examination program, we reviewed SEC testimony, speeches,  reports, and other documents related to the agency’s mutual fund  examination program. We also reviewed federal securities laws and  regulations applicable to mutual funds and analyzed SEC data on the  number, types, and results of its fund and adviser examinations. We also  interviewed officials from SEC’s Office of Compliance Inspections and  Examinations (OCIE), Division of Investment Management, and Office of  Risk Assessment and representatives from the Investment Company  Institute to obtain information on the significance of planned changes. In  addition, we interviewed federal bank regulatory officials from the Federal  Deposit Insurance Corporation (FDIC), Board of Governors of the Federal  Reserve System, and Office of the Comptroller of the Currency (OCC) and  self-regulatory organization (SRO) officials from NASD, the New York  Stock Exchange (NYSE), and the National Futures Association (NFA) to  discuss their examination programs and supervisory tools.
To assess key aspects of the quality control framework of SEC’s routine  mutual fund examinations, we reviewed policies, procedures, and other  guidance applicable to those examinations. We also reviewed routine fund  examinations completed in fiscal year 2004 by SEC’s Midwest Regional  Office (MRO), Northeast Regional Office (NERO), and Philadelphia District  Office (PDO). We selected these field offices because they were the three  largest in the number of completed routine fund examinations in fiscal year  2004. The three offices completed 66 routine fund examinations,  accounting for about 72 percent of all routine fund examinations  completed in fiscal year 2004. Where appropriate, we also reviewed  examinations of advisers to the funds we reviewed. We used a  standardized data collection instrument to document the methods  examiners used to conduct examinations and areas examiners reviewed  during examinations. In addition, we interviewed officials from OCIE and  three SEC field offices—MRO, NERO, and PDO—about their examination  policies and procedures and representatives from a mutual fund company  and consulting firm about fund examinations. To gather information and  compare SEC examinations with those of other regulators, we interviewed  officials from FDIC, the Board of Governors of the Federal Reserve System,  OCC, NYSE, and NFA about their quality controls and reviewed some of  their quality control policies and procedures.
To determine the adequacy of SEC’s oversight of NASD and NYSE in  protecting shareholders from mutual fund sales practice abuses, we  reviewed SEC policies, procedures, and other guidance related to its  broker-dealer oversight examinations and inspections and interviewed  officials from SEC’s OCIE and Boston District Office, NASD, and NYSE. We  also reviewed judgmentally selected SEC broker-dealer oversight  examinations conducted by SEC’s Boston District Office in 2003 and 2004,  and reviewed all reports of SRO inspections conducted of NASD and NYSE  between 2001 and 2003. To gather information on SEC’s automated tracking  system, Super Tracking and Reporting System, we interviewed SEC staff  responsible for the system in headquarters and received an overview of the  system and its capabilities at the Boston District Office. In addition, we  reviewed reports generated from the system and training documents for  the system. To help assess the extent to which SEC, NASD, and NYSE have  shared written guidance, we compared and contrasted the examination  modules they used to examine for certain mutual fund sales practice  abuses. As part of our assessment of the training received by broker-dealer  examiners, we obtained and analyzed SEC’s training attendance rosters  and list of examiners employed by SEC since 1999.
To ensure that data provided about the number, nature, and results of  examinations conducted by SEC, NASD, and NYSE were reliable, we  reviewed written materials describing these systems and reviewed the data  provided to check for missing or inaccurate entries. We also interviewed  agency staff responsible for maintaining the information systems that track  such data. We determined that the data were sufficiently reliable for use in  this report.
We performed our work in Boston, Massachusetts; Chicago, Illinois; New  York, New York; Philadelphia, Pennsylvania; and Washington, D.C. We  conducted our work between February 2004 and July 2005 in accordance  with generally accepted government auditing standards.
section_title: Securities and Exchange Commission’s (SEC) Broker-Dealer Examination Guidance and Training
paragraphs: You asked us to provide information about aspects of SEC’s oversight of  the broker-dealer industry, including (1) how SEC, NASD, and the New  York Stock Exchange (NYSE) share information, including written  examination guidance, related to their review of mutual fund sales  practices and other examination priorities; (2) how SEC distributes and  stores examination guidance for use by its broker-dealer examiners; and  (3) what training SEC has provided to its broker-dealer examiners on  mutual funds and other topics, and how it tracks and assesses such  training.
subsections: section_title: Sharing of Written Mutual Fund Examination Guidance among Regulators
paragraphs: SEC, NASD, and NYSE have developed guidance for examiners to use in  assessing compliance by broker-dealers with mutual fund sales practice  rules. Each regulator has developed its own examination module, or set of  procedures, covering various topics related to mutual fund sales. Moreover,  all three regulators recently revised their modules to include procedures to  detect market timing and late trading abuses. In addition, the regulators  periodically have provided their staff with other written guidance related to  mutual fund sales. For example, SEC issued internal memorandums in 1997  and 2001 to inform staff about abuses related to breakpoints and other  mutual fund sales practices and provide them with procedures for  detecting such abuses.
Through its oversight role, SEC reviews aspects of the self-regulatory  organization (SRO) examination modules, including the mutual fund sales  practice module. First, SEC officials told us that NYSE and NASD e-mail  SEC copies of their examination modules when they make material  changes to them. Second, during SEC’s on-site inspections of SRO  examination programs, staff generally review SRO examination modules in  connection with their review of completed SRO broker-dealer  examinations and work papers. Third, as part of their broker-dealer  oversight examinations, SEC staff generally review the SRO broker-dealer  examinations and applicable examination modules before going on-site to  conduct examinations of such broker-dealers.
SEC and the SRO officials meet at least semiannually to discuss significant  examination findings, customer complaints, trends in the industry,  enforcement cases, and examination guidance. SEC officials told us that  agency staff have met with NASD and NYSE officials semiannually and  quarterly, respectively, to discuss, among other things, examination  findings and guidance. The officials also said that they hold frequent  telephone conversations to coordinate their examination efforts. For  example, SEC, NASD, and NYSE staff talked with each other immediately  following NASD’s discovery of breakpoint abuses in 2002, and established a  joint task force to determine the extent of the abuses by conducting  examinations of firms designed to identify failures to provide breakpoint  discounts. Similarly, SEC, NASD, and NYSE staff talked with each other in  their efforts to respond to the late trading and market timing abuses  uncovered in 2003. In addition, SEC and SRO staff jointly attended  conferences and training that included examination guidance as a topic of  discussion. Finally, SEC, NASD, and NYSE have jointly developed a number  of examination modules to enforce recent changes in laws and rules  applicable to broker-dealers.
Although SEC, NASD, and NYSE coordinate in these ways to oversee  broker-dealers, they generally do not provide copies of their written  examination materials to each other. That is, SEC typically does not  provide copies of its modules or other internal written guidance to the  SROs, nor do NASD and NYSE generally provide copies of such guidance to  each other. Officials at these agencies shared benefits and drawbacks of  providing written copies of examination materials to each other. The  regulators agreed that sharing information about their examination  approaches and outcomes is overall a positive way to more effectively  oversee the broker-dealer industry. They cautioned, however, that certain  drawbacks should be considered regarding the sharing of written  examination materials. SEC officials said that sharing SEC examination  modules could compromise its supervision of the SROs. According to the  officials, if SEC shared its modules, the SROs may be less innovative and  motivated to improve their methods. They said, for example, that the SROs  may view SEC’s procedures as the most that they would need to do. NASD  officials strongly disagreed with SEC’s assertions about the sharing of  examination modules, saying they always seek the most effective  examination procedures, regardless of those used by SEC; and an NYSE  official said that while NYSE understands the SEC’s position in this regard,  the sharing of SEC’s examination module would only enhance NYSE’s pre- existing examination procedures related to mutual funds. NASD and NYSE  officials said it would be helpful if SEC shared copies of its modules and  other guidance it shares with its own examiners. However, SEC and NASD  officials said that NASD and NYSE may not want to share their examination  modules with each other because of competitive reasons. For example, if  one SRO shared its modules with another SRO, it would run the risk that its  competitor could be able to adopt similar procedures without the cost of  developing them. Finally, NASD officials told us that differences exist  between NASD’s and NYSE’s membership, culture, priorities, and strategies  that can lead to differences in examination procedures, and the same is  true for financial institutions overseen by the banking regulators.
section_title: Distribution and Storage of Broker-Dealer Examination Guidance
paragraphs: SEC’s Office of Compliance Inspections and Examinations (OCIE)  oversees and directs SEC’s broker-dealer examination program, but SEC’s  11 field offices conduct the vast majority of the broker-dealer  examinations. Among other things, OCIE creates and updates broker- dealer examination modules, or policies and procedures; issues other  examination guidance; and reviews broker-dealer examination reports.  Currently, when OCIE develops and issues policy changes and examination  guidance, it typically distributes such guidance to the field offices by  e-mail. In turn, each field office separately stores the guidance on one of its  shared computer drives or in some other way to provide its examiners  access to the information. Field office examiners generally are responsible  for keeping abreast of changes in guidance and reviewing it as needed in  performing examinations.
To better ensure that SEC examiners across all field offices have access to  current and complete broker-dealer examination guidance, OCIE is  developing an internal Web site to serve as a central repository for all  broker-dealer examination guidance. According to agency officials, OCIE  launched its internal Web site in April 2005 on a pilot basis to select broker- dealer examiners nationwide to obtain their comments about its  organization and comprehensiveness. Subsequently, SEC made the Web  site available to all examiners in July 2005. According to SEC officials, the  Web site will allow broker-dealer examiners to access not only all guidance  at one location but also links to databases and numerous other examiner  tools.
section_title: Broker-Dealer Examiner Training and Tracking
paragraphs: SEC’s OCIE has a training branch that provides routine and specialized  training to its broker-dealer examiners, with some of the training related to  mutual funds. More specifically, OCIE’s training branch provides a two- phased training program for broker-dealer examiners that is designed to  teach examiners how to handle increasingly complex examination issues.  According to an SEC official, the phase-one course is designed for new  examiners and includes some training on mutual fund operations and  mutual fund sales practices of broker-dealers. OCIE’s training branch also  offers a range of specialized training delivered in a variety of formats. For  example, it offers classroom training sessions and videoconferences taught  by senior examiners or vendors, such as NASD, as well as training videos  that examiners can view when convenient. An SEC official told us that  since 1999 the training branch has offered over 25 training sessions that  have included mutual fund topics, such as breakpoints.
In addition, SEC periodically has coordinated its training efforts with SROs,  including NASD and NYSE. For example, examiners representing SEC,  NASD, NYSE, and other SROs, as well as state regulators have met annually  for a 3-day joint regulatory seminar to receive training about emerging and  recurring regulatory issues. In 2003 and 2004, the seminars provided  training on mutual fund sales practice abuses, including late trading,  market timing, and failure to provide breakpoint discounts. Finally, SEC  examiners attend or participate in external training, such as industry  conferences.
Separate from OCIE’s training branch, SEC has a central training center  called SEC University that oversees the agency’s training programs. SEC  University uses an electronic database to track training received by SEC  staff. According to SEC officials, the database has a number of weaknesses  that limits its usefulness in helping SEC to track and assess the training  received by examiners. For example, the database cannot be used to  generate reports on which examiners have taken or not taken a particular  course. Also, the database is not directly accessible to examiners or their  supervisors and, thus, does not allow them to review their training records  or enter external training they may have taken. Because of these  weaknesses, OCIE’s training branch uses training rosters as needed to  manually track which examiners have taken particular courses. SEC  training staff said that they are requesting that the agency purchase a  learning management system that would better enable it, including OCIE,  to track and assess all training and other developmental opportunities.  According to one of the officials, the initiative is currently tabled and may  or may not receive funding this year.
Despite challenges in its ability to track training in an automated way, OCIE  takes some steps to evaluate the training needs of its examiners. It gathers  and evaluates training participants’ reactions to and satisfaction with  training programs and uses that information to decide on what training to  offer in the future. Training branch staff told us that at the end of each  course, they hand out course evaluation forms to participants. These forms  include closed-ended questions about the extent to which participants  found the course helpful and open-ended questions about what additional  training needs they have. The training branch uses the information to  improve individual classes and the program as a whole. In addition, training  staff attend monthly meetings with management and staff from all field  offices, in part, to identify training needs and opportunities, and they also  attend yearly meetings with examination program managers to discuss the  examiners’ training needs.
section_title: Comments from the Securities and Exchange Commission
section_title: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, John Wanska, Randall Fasnacht,  Joel Grossman, Christine Houle, Marc Molino, Wesley Phillips, David  Pittman, Paul Thompson, Richard Tsuhara, and Mijo Vodopic made key  contributions to this report.