January 17, 2003

# ED-OIG/A09-C0018

## 

James T. Rogers, Executive Director

Southern Association of Colleges and Schools,

> Commission on Colleges

1866 Southern Lane

Decatur, Georgia 30033-4097

Dear Mr. Rogers:

This is the Office of Inspector General's **Final Management Information
Report**, entitled *Southern Association of Colleges and Schools,
Commission on Colleges' Accreditation Standards for Student Achievement
and Program Length*.

The purpose of our review of the Southern Association of Colleges and
Schools, Commission on Colleges (SACS) was to (1) identify SACS'
standards for success with respect to student achievement and measures
of program length as required by the Higher Education Act (HEA), Section
496(a)(5), and (2) evaluate SACS' management controls for ensuring that
institutions adhere to its standards for student achievement and
measures of program length and that consistent enforcement action is
taken when institutions are not in compliance with the standards. This
management information report, describing SACS' standards and its
monitoring and enforcement policies and procedures, is being provided to
assist the U.S. Department of Education (Department) in its oversight of
accrediting agencies. We also provide suggestions for SACS to consider
for enhancing its standards and management controls.

We received written comments from SACS on a draft of this report. SACS
disagreed with the OIG's conclusions that the agency's standards
covering student achievement and program length have inherent
limitations, but stated it would forward the OIG suggestions to its
Executive Council of the Commission on Colleges for consideration at its
December meeting. SACS' comments and our response are summarized at the
end of the REVIEW RESULTS section of the report. The full text of SACS'
comments is included as an attachment.

### REVIEW RESULTS

The Secretary of the Department recognizes SACS as a regional
accrediting agency. To achieve this recognition, the HEA requires
accrediting agencies to, among other requirements, establish standards
for student achievement and measures of program length.

SACS' standards that encompass student achievement are general, that is,
the standards do not include specific measures to be met by
institutions. The agency requires accredited institutions to develop
their own processes for assessing student achievement. We found that
SACS provided institutions with detailed guidance on establishing and
documenting their processes for evaluating educational programs and the
use of both quantitative and qualitative data to demonstrate student
achievement.

SACS' standards on program length require institutions to specify the
number of credit hours in an educational program and provide guidance on
program content. SACS did not define what constitutes a credit hour. The
Executive Director provided a definition, but SACS has not included the
definition in its publications.

Since SACS' standards do not contain specific measures for student
achievement, the agency has a limited ability to compare institutions'
performance. Also, since the standards do not explicitly state that
educational programs must meet the institution's expected educational
results or define what constitutes a credit hour, SACS may have limited
ability to take enforcement action when institutions' programs have not
demonstrated student achievement or when the program length is
questionable.

We concluded that SACS has policies and procedures in place to ensure
institutions adhere to its standards and that the agency enforces its
standards, except monitoring student achievement throughout the
institution's accreditation period is limited to institutions that were
previously identified as not meeting a standard. We offered suggestions
for enhancing SACS' standards and management controls.

### Standards for Student Achievement and Measures of Program Length

Section 496(a)(5) of the HEA requires accrediting agencies to establish
accreditation standards that assess the institution's---

(A) success with respect to student achievement in relation to the
    institution's mission, including, as appropriate, consideration of
    course completion, State licensing examinations, and job placement
    rates;\... \[and\]

> \(H\) measures of program length and the objectives of the degrees or
> credentials offered;\....

The regulations that address accrediting agencies' standards for student
achievement and measures of program length mirror the statutory
language.

###### SACS' Standard for Student Achievement

> SACS' standard for student achievement is embedded in its criterion on
> Institutional Effectiveness in its *1998 Criteria for Accreditation*.
> Section 3.l of Institutional Effectiveness states "\[t\]he institution
> must define its expected educational results and describe its methods
> for analyzing the results."

SACS expects each institution to develop a broad-based system to
determine institutional effectiveness appropriate to its own context and
purpose, to use the purpose statement as the foundation of planning and
evaluation, to employ a variety of assessment methods (quantitative and
qualitative), and to demonstrate use of the results of the planning and
evaluation process for the improvement of educational programs. SACS
emphasizes in its criteria that the institution must evaluate its
success with respect to student achievement including, as appropriate,
consideration of course completion, state licensing examinations, and
job placement rates.

SACS includes a list of potential measures in its criteria on
Institutional Effectiveness.

+---------------------------------+------------------------------------+
| **Measures Listed in 1998       |                                    |
| Criteria for Accreditation**    |                                    |
+---------------------------------+------------------------------------+
| -   evaluation of instructional | -   results of admissions tests    |
|     delivery                    |     for students applying to       |
|                                 |     graduate or professional       |
| -   adequacy of facilities and  |     schools                        |
|     equipment                   |                                    |
|                                 | -   job placement rates            |
| -   standardized tests          |                                    |
|                                 | -   results of licensing           |
| -   analysis of theses,         |     examinations                   |
|     portfolios and recitals     |                                    |
|                                 | -   evaluations by employers       |
| -   completion rates            |                                    |
|                                 | -   performance of student         |
| -   follow-up studies of alumni |     transfers at receiving         |
|                                 |     institutions                   |
+---------------------------------+------------------------------------+

SACS' *Resource Manual on Institutional Effectiveness* provides
guidelines for interpreting and responding to SACS' accreditation
criteria on institutional effectiveness. The *Resource Manual*
emphasizes that Section 3.1 of the 1998 Criteria for Accreditation
includes use of quantitative measures.

> Each institution is thus required to state its expectations regarding
> the important results of the educational process and then to describe
> the methods that will be used in discovering the extent to which those
> expectations are realized. Significant implications are that expected
> educational results may be stated in qualitative as well as
> quantitative terms and that the description of methods for
> ascertaining results should include information as to how, where,
> when, and by whom the assessment effort will be accomplished. While
> some important goals and educational results are not readily
> "measurable," this limitation should not necessarily preclude
> assessment of the extent to which most have been accomplished.

The *Resource Manual* also provides several examples of assessment
procedures (measures) for education programs with different educational
purposes.

In the preamble to the 1999 proposed rules, the Secretary stated that
accrediting agencies should establish quantitative standards---for
completion rates, job placement rates, and pass rates on State licensing
examinations---for vocational education programs. SACS has not
established quantitative standards for its institutions that offer
vocational education programs, but holds these institutions to the same
standards as all other accredited institutions. According to SACS'
Associate Executive Director for Commission Support, about 55 percent of
SACS-accredited institutions offer vocational and certificate programs.

SACS' new standards, which were approved in December 2001 and will
become effective in 2004, contain the same requirements as the *1998
Criteria for Accreditation*, except the new standards do not refer to
use of "quantitative and qualitative data" or contain a list of
potential measures. However, SACS' *Resource Manual on Institutional
Effectiveness*, which has not been revised, contains a similar reference
to "quantitative and qualitative data" and lists potential measures that
are similar to those listed in SACS' current standards.

SACS' standard for student achievement and its requirements for meeting
this standard have inherent limitations.

-   The agency may not be able to take enforcement action against
    institutions that are offering programs that have not demonstrated
    student achievement since the standards do not explicitly require
    that an institution's programs meet their defined expected
    educational results.

-   The agency may not be able to compare educational results of
    institutions with similar missions and purposes because the
    institutions may not use similar measures.

###### SACS' Standards for Measures of Program Length 

SACS addresses measures of program length within its criteria on
Educational Programs. Section 4.2 - Undergraduate Programs states---

> An institution **must** clearly define what is meant by a major or an
> area of concentration and **must** state the number of credits
> required for each. An adequate number of hours with appropriate
> prerequisites **must** be required in courses above the elementary
> level. \[Emphasis in original\]

SACS has additional requirements for courses offered in a concentrated
time period.

> Courses offered in non-traditional formats, e.g., concentrated or
> abbreviated time periods, **must** be designed to ensure an
> opportunity for preparation, reflection and analysis concerning the
> subject matter. At least one calendar week of reflection and analysis
> should be provided to students for each semester hour, or equivalent
> quarter hours, of undergraduate credit awarded. The institution
> **must** demonstrate that students completing these programs or
> courses have acquired equivalent levels of knowledge and competencies
> to those acquired in traditional formats. \[Emphasis in original\]

At the end of Section 4.2, SACS emphasizes that "\[t\]he institution
must demonstrate that program length, clock hours or credit hours, and
tuition and fee charges are appropriate for the degrees and credentials
it offers."

Neither the *1998 Criteria for Accreditation* nor its other documents
provided a definition or criteria for a "credit hour." According to
SACS' Executive Director, the following formulas are used when
evaluating program length: A three-credit course generally consists of
three one-hour classes per week for a full semester (20 to 22 weeks). If
the institution uses a quarter system, a five-credit course would
include about five 45-50 minute classes per week for a full quarter
(12 to 15 weeks). In addition to the class time hours, a student would
be required to spend time on homework, writing and reading assignments,
projects, lab work, etc. SACS does not require a specific amount of
homework and other outside class activities. Since the standards do not
define what constitutes a credit hour or specify the amount of outside
activities, SACS may have limited ability to take enforcement action
when an institution's program length or credit hour assignments are
questionable.

SACS' new standards, which will become effective in 2004, do not include
the section on additional requirements for courses offered in a
concentrated time period. Instead, the new standards state, "The
institution employs sound and acceptable practices for determining the
amount and level of credit awarded for courses, regardless of format or
mode of delivery." This requirement will apply to on-campus, off-campus,
and distance learning programs. The new standards do not define "sound
and acceptable practices." Thus, SACS' ability to enforce the new
standard may also be limited.

## **Systematic Review of Standards** 

The regulations at 34 C.F.R. § 602.21 address the requirements for an
accrediting agency's systematic review of its standards. Paragraph (a)
states "\[t\]he agency must maintain a systematic program of review that
demonstrates that its standards are adequate to evaluate the quality of
the education or training provided by the institutions and programs it
accredits and relevant to the educational or training needs of
students." Paragraph (b) lists the following required attributes of the
review:

> \[T\]he agency must ensure that its program of review:
>
> \(1\) Is comprehensive;
>
> (2) Occurs at regular, yet reasonable, intervals or on an ongoing
> basis; (3) Examines each of the agency's standards and the standards
> as a whole; and (4) Involves all of the agency's relevant
> constituencies in the review and affords them a meaningful opportunity
> to provide input into the review.

To meet this requirement, SACS' *Policies of the Commission on Colleges*
requires that a Review Committee, with representatives from each state,
every institutional level, and the public, be appointed every seven
years to conduct a comprehensive study of the standards to ascertain
whether they are adequate to evaluate the quality of the education or
training provided by the institutions and relevant to the educational
needs of students. The policy states that the Committee "will examine
each of the standards and the standards as a whole, will solicit
comments from the membership and other interested parties, and will
consider all comments and recommendations received since the last review
period."

In December 2001, the SACS' College Delegate Assembly approved new
standards that were the results of an examination conducted by such a
committee. We discussed the portions of the new standards that related
to student achievement and program length in the previous two sections
of this report.

**Suggestions for Enhancing the Standards for**

**Student Achievement and Program Length**

To enhance its established standards for student achievement and
measures of program length, we suggest that SACS consider---

-   Revising its new standard for student achievement to explicitly
    state that programs must fulfill the institution's defined expected
    educational results.

-   Analyzing the measures used in assessments at institutions with
    similar programs and developing measures for incorporation in the
    agency's standard for student achievement.

-   Establishing quantitative standards for completion rates, job
    placement rates, and pass rates on State licensing examinations, as
    applicable, for vocational education programs offered by its
    accredited institutions.

-   Providing institutions with written guidance on "sound and
    acceptable practices" for assigning credit hours to programs. The
    guidance should include the agency's formulas for assigning credit
    hours, required levels of outside preparation, and a requirement
    that institutions submit a written justification for any deviation
    from the guidance.

### MANAGEMENT CONTROLS FOR ENSURING adherence to STANDARDS

The regulations at 34 C.F.R. § 602.18 require an accrediting agency to
"consistently apply and enforce its standards to ensure that the
education or training offered by an institution or program\... is of
sufficient quality to achieve its stated objective for the duration of
any accreditation or preaccreditation period granted by the agency." In
addition, 34 C.F.R. § 602.19 requires the agency to (a) "reevaluate, at
regularly established intervals, the institutions or programs it has
accredited or preaccredited\... \[and\] (b)\... monitor institutions or
programs throughout their accreditation or preaccreditation period to
ensure that they remain in compliance with the agency\'s standards."

### SACS' Monitoring Activities and Procedures

SACS has policies and procedures to ensure institutions adhere to its
standards for student achievement and measures of program length
(including substantive changes), except monitoring institutions' systems
for determining institutional effectiveness may only be evaluated during
reaccreditations, unless the institutions were subject to follow-up
monitoring for previously identified deficiencies. This limits SACS'
ability to monitor an institution's adherence to the standard during the
accreditation period.

To monitor institutions, SACS uses institutional self-studies, visiting
committee reviews, and follow-up reports. Each institution is required
to conduct an institutional self-study and have a visiting committee
review at least every 10 years. SACS also requires institutions to
submit annual reports containing financial and student enrollment
information. The annual reports do not contain information on student
achievement or program length.

[Institutional Self-Studies]{.underline}. SACS' publication, *Handbook
for Institutional Self-Study*, provides institutions with detailed
instructions for conducting a self-study and preparing a self-study

report. The *Handbook* includes the following in its description of a
satisfactory self-study report:

> The self-study report deals sufficiently with all requirements of the
> *Criteria* in order to enable members of a visiting committee to
> evaluate the institution in relation to these requirements. In this
> regard, the institution should provide adequate information and
> analyses in the text of the report or in the Appendices to indicate
> its compliance with the *Criteria*. The narrative should include a
> balanced review of the strengths and weaknesses of the institution and
> sufficient data to provide the committee with information concerning
> compliance with the *Criteria*.

The *Handbook* specifically requires that the self-study includes an
examination of the institution's planning and evaluation process, which
encompasses student achievement. For distance learning activities, the
*Handbook* requires that the analyses be conducted in accord with the
general self-study guidelines and be conducted by those not directly
connected with the coordination and implementation function of the
distance learning activities.

Our review of 10 institutional files[^1] confirmed the institutional
self-studies contained information specific to SACS' standards covering
student achievement and program length. The following table provides a
partial list of the student assessment measures found in the
institutional self-study reports and other documents located in SACS'
files:

+-----------------------------------+-----------------------------------+
| **Student Assessment Measures**   |                                   |
+-----------------------------------+-----------------------------------+
| Measures of students' written and | Bar pass rates and other state    |
| oral communications skills        | and national licensing            |
|                                   | examinations                      |
| Projects, papers, or oral         |                                   |
| presentations                     | First time scores on a state exam |
|                                   |                                   |
| Course grades based on exams and  | National exams given at the end   |
| lab reports                       | of the second and third years of  |
|                                   | medical school                    |
| Completion of required "capstone" |                                   |
| courses at the end of the course  | Surveys of graduates in their     |
| sequence that require students to | first year of residency           |
| apply cumulative skills learned   |                                   |
|                                   | Residency directors' evaluations  |
| Standardized state and national   | of medical school students        |
| examinations                      |                                   |
|                                   | Job placement statistics and      |
| Student satisfaction              | placement rates                   |
|                                   |                                   |
| Senior assessment exam            | Successful employment in major    |
|                                   | field                             |
| Surveys utilizing a statistically |                                   |
| valid test instrument             | Increased employment success      |
|                                   |                                   |
| Graduate rates                    | Monitoring starting salaries      |
|                                   |                                   |
| Exit surveys                      | Sustained employment              |
|                                   |                                   |
| Alumni surveys                    |                                   |
+-----------------------------------+-----------------------------------+

For program length, the institution is required to describe how it
ensures that program length, clock hours or credit hours, tuition and
fee charges are appropriate for the degrees and credentials offered. For
example, one institution provided the following response in its
institutional self‑study:

> The Board of Directors must approve any new program or field of study
> offered. Program length, clock hours and credit hours are all reviewed
> in the approval process. College programs in traditional and
> non-traditional formats are comparable to those of similar
> institutions in the region in every respect contemplated in the
> Criterion. Refer to catalogs of various institutions and other
> documentation on file in the Display Room. The Academic Council and
> Registrar engage in the activity of comparing courses, formats,
> calendar length and clock time, along with a determination of
> competitive standards in the overall academic environment.

[Visiting Committee Review]{.underline}. The visiting committee (peer
evaluators) determines the extent to which the institution complied with
the *1998 Criteria for Accreditation* at the time of the visit and that
the institutional self-study met SACS' requirements. SACS' *Handbook for
Peer Evaluators* provides the peer evaluators with guidelines for
applying the *Criteria.* The guidelines, which list persons to
interview, questions to address, documents to review, and helpful
pointers, include the following related to student achievement and
measures of program length:

+-----------------------------------------------------------------------+
| ## **Peer Evaluator Questions and Instructions Related to**           |
|                                                                       |
| ## **Student Achievement and Measures of Program Length**             |
+-----------------------------------------------------------------------+
| Is the educational effectiveness of each program being measured by    |
| appropriate means? What are they?                                     |
|                                                                       |
| Does the institution have a plan for ascertaining the success of its  |
| graduates?                                                            |
|                                                                       |
| Review information on assessment procedures and results for programs  |
| and general education: student evaluations, test scores, exit data,   |
| employer surveys, program reviews, job placement statistics, surveys  |
| of graduates, etc.                                                    |
|                                                                       |
| Does the self-study provide adequate documentation of effectiveness   |
| of the educational program?                                           |
|                                                                       |
| Include in your evaluation any programs which are offered through     |
| non-traditional delivery methods or which are offered at off-campus   |
| sites.                                                                |
|                                                                       |
| Has the institution developed a reasonable plan for evaluating the    |
| effectiveness of its distance learning activities?                    |
|                                                                       |
| Are degree requirements clear, reasonable, and enforced consistently? |
|                                                                       |
| Review statements of goals and objectives for general education and   |
| for programs.                                                         |
|                                                                       |
| Are admissions, degree completion, curriculum, and instructional      |
| design policies and procedures \[for distance learning activities\]   |
| similar to those used for traditional campus-based programs?          |
|                                                                       |
| Are goals and objectives, and skills and competencies, the same for   |
| courses offered on the main campus as those offered through distance  |
| learning activities?                                                  |
|                                                                       |
| ##                                                                    |
+-----------------------------------------------------------------------+

## 

The guidelines do not explicitly state that peer evaluators are to
confirm that the institution used SACS' formulas for determining credit
hours or assess the validity of justifications for any deviation.

## SACS' *Handbook for Peer Evaluators* and its *Handbook for Committee Chairs* provide guidance and a suggested format for preparing the visiting committee report. The peer evaluators are instructed to include in the report comments, recommendations and suggestions addressing each section in the *1998 Criteria for Accreditation*. The guidance does not include a requirement for peer evaluators to report on institutional success with respect to student achievement for each educational program and the institution overall. 

## Our review of 10 institutional files confirmed that the visiting committee reports contained comments related to the standards for student achievement and program length and, if applicable, recommendations and suggestions for improvements. For 7 of the 10 institutions, the visiting committee reports identified deficiencies related to student assessments. The reports did not identify deficiencies for measures of program length. 

[Follow-up Reports and Special Committee Visits]{.underline}. To monitor
institutions with identified problems, SACS may require an institution
to submit follow-up reports. Follow-up reports may be required if the
Commission determines that an institution (1) could be in non-compliance
with accreditation standards in the future if steps are not taken to
correct the situation; (2) has failed to comply with accreditation
standards, but deficiencies are minor and can be corrected within a
short time; or (3) has failed to comply with agency policies and
procedures or failed to provide requested information. Also, SACS may
have a special committee conduct a site visit to determine whether the
institution has continued to comply with accreditation standards.
Institutions are allowed up to two years to remedy deficiencies.

SACS' data showed that 56 of the 83 institutions, which went through the
reaccreditation process during calendar year 2001, were required to
provide follow-up reports and/or have additional site visits.[^2] As
shown in the following table, SACS consistently required follow-up for a
significant number of institutions on areas related to its standard on
student achievement.

+------------------------------------------+------+-----+-----+-----+
| **Institutions with Follow-Up Required   |      |     |     |     |
| to Address Deficiencies**                |      |     |     |     |
|                                          |      |     |     |     |
| **Related to Section 3.1 of              |      |     |     |     |
| Institutional Effectiveness**            |      |     |     |     |
+------------------------------------------+------+-----+-----+-----+
|                                          | **19 | **  | **  | **  |
|                                          | 98** | 199 | 200 | 200 |
|                                          |      | 9** | 0** | 1** |
+------------------------------------------+------+-----+-----+-----+
| Institutions seeking reaccreditation     | 68   | 74  | 97  | 104 |
|                                          |      |     |     | ^a^ |
+------------------------------------------+------+-----+-----+-----+
| Institutions with follow-up required for | 37   | 48  | 60  | 56  |
| areas related to Section 3.1             |      |     |     |     |
+------------------------------------------+------+-----+-----+-----+
| Percentage                               | 54%  | 65% | 62% | 54% |
+------------------------------------------+------+-----+-----+-----+
| ^a^ Includes institutions previously     |      |     |     |     |
| denied reaccreditation. These            |      |     |     |     |
| institutions were not included in the    |      |     |     |     |
| universe used for the sample selection.  |      |     |     |     |
+------------------------------------------+------+-----+-----+-----+

**Substantive Changes**

The regulations at 34 C.F.R. § 602.22 address the requirements for an
accrediting agency's substantive change policy. Paragraph (a) of the
section states---

> If the agency accredits institutions, it must maintain adequate
> substantive change policies that ensure that any substantive change to
> the educational mission, program, or programs of an institution after
> the agency has accredited or preaccredited the institution does not
> adversely affect the capacity of the institution to continue to meet
> the agency's standards . . ..

Paragraph (a)(2) lists the types of changes that must be addressed by
the agency's policy. The following listed changes could impact an
institution's adherence to the accrediting agency's standards on student
achievement and measures of program length:

i.  Any change in the established mission or objectives of the
    institution...

```{=html}
<!-- -->
```
(iii) The addition of courses or programs that represent a significant
      departure, in either content or method of delivery, from those
      that were offered when the agency last evaluate the institution.

(iv) The addition of courses or programs at a degree or credential level
     above that which is included in the institution's current
     accreditation or preaccreditation.

(v) A change from clock hours to credit hours

(vi) A substantial increase in the number of clock or credit hours
     awarded for successful completion of a program...

###### 

SACS' substantive change policies complied with the Federal regulation.
Ten of the 15 institutional files that we reviewed included substantive
change requests. For these 10 institutions, we found that SACS followed
its substantive change policies and procedures.

To ensure that institutions report substantive changes in program
length, SACS provides training on the subject and staff review
institutions' annual reports for significant increases in student
enrollments, which may indicate the addition of a new program.

### Suggestions for Enhancing Management Controls for Ensuring Adherence to Standards

To enhance its management controls for evaluating and monitoring
institutions' performance, we suggest that SACS consider---

-   Requiring peer evaluators to confirm the institution used SACS'
    formulas for determining credit hours and report on the validity of
    justifications for any deviation.

-   Requiring peer evaluators to report on institutional success with
    respect to student achievement for each educational program and the
    institution overall.

-   Implementing procedures to monitor institutions' adherence to the
    standards for student achievement during the accreditation period.
    For example, SACS could consider adding a requirement in its
    substantive change policy that institutions inform SACS of changes
    in their systems for determining institutional effectiveness. SACS
    could also consider having institutions provide the results of their
    assessments of student achievement for each educational program as
    part of their annual reports.

### MANAGEMENT CONTROLS FOR ENSURING ENFORCEMENT OF STANDARDS

When an institution or program is not in compliance with the accrediting
agency's standards, the regulations at 34 C.F.R. § 602.20 require the
agency to either "immediately initiate adverse action against the
institution\... or\... require the institution\... to take appropriate
action to bring itself into compliance" within established timeframes.

SACS has policies and procedures that address its efforts to ensure
enforcement of its standards for student achievement and program length.
The time allowed by SACS' policies and procedures for an institution to
remedy a deficiency, together with the time for the Commission to make a
final decision, complies with timeframes specified in the regulations at
34 C.F.R. § 602.20(a).

SACS' policy provides for four types of enforcement actions that it may
impose according to the seriousness of the deficiencies or the length of
time the deficiencies have existed without correction by the
institution: notice, warning, probation, and removal from membership.
The Commission may place an institution on notice (a private sanction)
or warning (a public sanction) during the two years that institutions
are monitored by follow-up reports. For more serious deficiencies, the
Commission may place an institution on probation.

If an institution has not remedied deficiencies at the conclusion of a
two-year monitoring or probation period, the Commission must either
remove the institution from membership or continue accreditation for
"good cause." An institution's accreditation can be extended for "good
cause" if the Commission determines that the institution has---

-   demonstrated significant recent accomplishments in addressing
    non-compliance, and

-   documented that it has the "potential" to remedy all deficiencies
    within the extended period, and

-   provided assurance to the Commission that it is not aware of any
    other reasons why the institution could not be continued for "good
    cause."

During the period January 2000 through June 2002, SACS placed 61
institutions on notice, 61 institutions on warning, and 22 institutions
on probation. One institution was removed from membership.

For the five sampled institutions with enforcement actions,[^3] we
confirmed that SACS followed its enforcement policies and procedures.
For three institutions, corrective actions were taken within the
timeframes specified in 34 C.F.R. § 602.20(a). The allowed timeframe for
correction action has not yet expired for the other two institutions.

**SACS Comments and OIG Response**

In its comments on the draft report, SACS disagreed with the OIG's
conclusion that the agency's standards covering student achievement and
program length have inherent limitations. SACS did not provide specific
comments addressing the OIG suggestions for its standards and management
controls, but stated it would forward the OIG suggestions to its
Executive Council of the Commission on Colleges for consideration at its
Decembermeeting.

[SACS Comment]{.underline}. SACS stated that its standard for student
achievement does explicitly require that an institution's programs meet
their defined educational result and cited the applicable section of the
*1998 Criteria for Accreditation*. SACS also stated that information
provided to the OIG auditors demonstrates that the agency has been able
to take enforcement action against institutions offering programs which
did not demonstrate student achievement. SACS stated that the visiting
committee members from similar institutions are able to compare
educational results when reviewing the data made available at the
institution and making the initial determination about compliance with
the standard.

[OIG Response]{.underline}. The *1998 Criteria for Accreditation* and
SACS' new standards refer to goals, major aspects, and mission, but do
not make specific reference to the achievement of expected student
outcomes. The relevant sentence cited in SACS' comments was taken from
the introductory paragraphs of the section on institutional
effectiveness, which addresses educational programs, administrative and
educational support services, and institutional research. The sentence
states "\[e\]ducational quality will be judged finally by how
effectively the institution achieves its established goals." SACS' new
standards state under Principles of Accreditation that "an institution
is expected to document quality and effectiveness in all its major
aspects." Under Core Requirements, the new standards require a
systematic review of programs that "demonstrates that the institution is
effectively accomplishing its mission." While SACS has successfully
taken enforcement action against institutions with identified
deficiencies in the areas of institutional effectiveness, including
student achievement, our position remains that SACS' ability to take
such actions would be strengthened if its standards explicitly stated
that educational programs must meet the institution's defined expected
student outcomes.

Even though visiting committee members may be from similar institutions,
their institutions and the reviewed institution may not use the same
measures of student achievement. If institutions with similar programs
used the same measures, SACS and the visiting committee members could
better compare the educational results of the institutions accredited by
SACS.

[SACS Comment]{.underline}. SACS stated that the *1998 Criteria for
Accreditation* stress what is important regarding "measures of program
length." SACS stated "\[I\]t is not how much homework or how much seat
time is required; rather, the *Criteria* requires that an institution
demonstrates that learning takes place and when non-traditional formats
are used\... that institutions 'demonstrate that students completing
these abbreviated programs or courses have acquired equivalent levels of
knowledge and competencies to those acquired in traditional formats."
SACS stated that it has had no difficulty in enforcing its standard.

[OIG Response]{.underline}. The SACS Executive Director informed us
during our fieldwork that quantitative formulas are used when evaluating
program length and students are expected to spend time on activities
outside the classroom. The inclusion of these formulas, plus
clarification of the outside classroom work expected, in SACS' standards
or other publications would clearly communicate the agency's guidelines
for evaluating program length and, thus, enhance SACS' ability to
enforce its standard. Also, as we noted in the report, SACS' new
standard covering program length, which will be effective in 2004, does
not contain the language cited in SACS' comments. The enforcement of
SACS' new standard, which uses the term "sound and acceptable
practices," would also be enhanced by publishing guidelines in its
standards or other publications. SACS provided no support for its
statement that it has had no difficulty in enforcing its standard and
our review found no evidence to support the statement. Our position
remains that SACS' ability to take enforcement actions would be
strengthened if its standards defined "sound and acceptable practices"
in its new standards, including what constitutes a credit hour for both
traditional and non-traditional formats.

[SACS Comment]{.underline}. SACS made some general comments regarding
its agency and the OIG review. First, SACS stated that if its membership
or Commission does not approve the OIG suggested changes, SACS staff has
neither the prerogative nor an option to impose the change. Second, SACS
expressed its opinion that the general assumption inherent in the report
is that the only way quality can be verified is by mandating specific
measures for student achievement. SACS stated its concern that "\[b\]y
creating 'specific measures' or acceptable thresholds, the peer review
process is restricted to only those measures and abandons its ability to
look at an institution in its entirety in order to make a determination
about the comprehensive quality of an institution and not just a part of
the institution." Third, SACS stated that "\[d\]eveloping even more
specific measures than those already included in the *Criteria* puts the
Commission in a vulnerable litigation position that relies on 'measures'
and not quality as determined by professional judgments." Lastly, SACS
viewed the OIG review as redundant with that of the May 2001 review
conducted by the Office of Postsecondary Education's Accrediting Agency
Evaluation Unit. SACS stated that the May 2001 review deemed SACS in
compliance with applicable Federal regulations. \`

[OIG Response]{.underline}. The conclusions and suggestions contained in
our management information report are being provided to SACS and OPE to
assist them in their oversight responsibilities. SACS and OPE will have
to determine what actions they will take in response to this report.

We did not suggest that SACS solely use specific measures and thresholds
to evaluate success with respect to student achievement. We concluded
that the inclusions of specific measures and thresholds would strengthen
SACS' standards, and suggested that SACS include such measures and
thresholds in its standards for vocational education programs to ensure
the agency meets the expectation expressed by the Secretary in the
preamble to the 1999 proposed rules. SACS should be able to develop
standards with specific measures for vocational education programs that
supplement the professional judgments of its peer reviewers without
increasing the agency vulnerability to litigation. As we stated in our
initial letter to SACS, the Inspector General Act of 1978 authorizes the
OIG to conduct its own independent review of the agency.

**BACKGROUND**

Founded in 1895, SACS is the regional agency for the accreditation of
higher education institutions in 11 Southern states---Alabama, Florida,
Georgia, Kentucky, Louisiana, Mississippi, North Carolina, South
Carolina, Tennessee, Texas, and Virginia. SACS is governed by a College
Delegate Assembly, which meets annually and is comprised of one voting
representative from each accredited institution. The Assembly elects the
members of SACS' Commission on Colleges, which meets semiannually to
take final action on institutions' accreditation and approve revisions
to the agency's accreditation standards, policies and procedures. The
Commission is comprised of 77 chief executive officers, faculty, and
experienced administrators and public representatives. Thirteen of the
Commission members serve on the Executive Council, which acts for the
Commission while it is not in session. The remaining 64 Commission
members serve on the five Committees on Criteria and Reports. Each
Committee reviews institutional reports and recommends action to the
Commission. The Commission has 28 staff, which advise the Committees on
Criteria and Reports and coordinate efforts of educators and public
representatives who volunteer in the regional accreditation process.

##### SACS first gained Federal recognition in 1952. The Department most recently renewed the agency's recognition in December 2001 for a period of five years. During our review, SACS accredited 784 colleges and schools. 

##### 

##### **PURPOSE, SCOPE, AND METHODOLOGY**

The purpose of the review was to (1) identify SACS' standards for
success with respect to student achievement and measures of program
length as required under the HEA, Section 496, and (2) evaluate SACS'
management controls for ensuring that institutions adhere to its
standards for student achievement and measures of program length and
that consistent enforcement action is taken when institutions are not in
compliance with the standards.

For our review, we focused on the agency's current standards related to
the review objective and the procedures that SACS used for monitoring
and enforcing those standards. We reviewed applicable Federal laws and
regulations; reviewed SACS' policies, procedures, and guidance; and
interviewed the Executive Director and several staff.

We reviewed institutional files for a total of 15 institutions. To
evaluate SACS' conformity with its policies and procedures for
monitoring adherence to accreditation standards, we judgmentally
selected 10 institutions from the 83 institutions due for
reaccreditation in calendar year 2001. We stratified the 83 institutions
into three groups based on Commission action categories and judgmentally
selected institutions based on student enrollment totals (half with
higher enrollments and half with lower enrollments).

  --------------------------------------------- ------------ -------------
  **Commission Actions**                        **Total in   **Number
                                                Group**      Selected**

  Reaffirmed without follow up                  13           2

  Reaffirmed with follow up or on notice        60           6

  Deny reaffirmation with on warning            10           2
  --------------------------------------------- ------------ -------------

To evaluate SACS' conformity with its enforcement policies and
procedures, we judgmentally selected three institutions from the 26
 institutions on the Commission's Reports on Accreditation Actions that
had institutional effectiveness deficiencies and were denied
reaccreditation or on probation during calendar years 2000 and 2001. We
also judgmentally selected two institutions from a SACS-provided list of
five institutions that were removed from warning or probation during
calendar year 2001.

SACS provided a list of 784 affiliate institutions as of March 2002. To
confirm the completeness of SACS list we compared it to a Departmental
list that showed SACS as the primary accrediting agency for 781 Title IV
participant institutions. Since the difference between SACS list of
institutions and the Department's list was insignificant, we concluded
that SACS listing of accredited institutions was sufficiently complete
for our purposes. We confirmed that the sample institutions selected
from the universe of 83 institutions due for reaccreditation in 2001
were included in the Department list of 781 Title IV institutions.

We performed our fieldwork at SACS' office in Decatur, Georgia from
March 21 through March 28, 2002. We held an exit briefing with SACS
officials on September 5, 2002. Our review was performed in accordance
with generally accepted government auditing standards appropriate to the
scope of the review described.

##### **STATEMENT ON MANAGEMENT CONTROLS**

We assessed the system of management controls, policies, procedures, and
practices applicable to SACS' process for monitoring and enforcing
accreditation standards for student achievement and program length. We
performed our assessment to determine whether SACS' processes provided a
reasonable level of assurance that the agency ensured that institutions
adhered to established standards and, when institutions were
noncompliant, SACS took consistent enforcement action.

For the purpose of this report, we assessed and classified significant
controls related to SACS' accreditation standards into the following
categories:

-   Monitoring institutions' adherence to the standards, and

-   Taking enforcement action for noncompliant institutions.

The management of SACS is responsible for establishing and maintaining a
management control structure. In fulfilling this responsibility,
judgments by management are required to assess the expected benefits and
related costs of control procedures. The objectives of the system are to
provide management with reasonable, but not absolute, assurance that
institutions adhere to accreditation standards and that enforcement
action is taken when institutions are found to be noncompliant with the
standards.

Because of inherent limitations in any management control structure,
errors and irregularities may occur and not be detected. Also,
projection of any evaluation of the system to future periods is subject
to risk that procedures may become inadequate because of changes in
conditions, or that the degree of compliance with the procedures may
deteriorate.

In our opinion, the management control structure, taken as a whole, was
sufficient to meet the objectives stated above. However, we identified
areas where management controls for ensuring adherence to standards
could be strengthened and offered suggestions for enhancing the
controls, as discussed in the REVIEW RESULTS section.

# **ADMINISTRATIVE MATTERS**

Statements that managerial practices need improvements, as well as other
conclusions and suggestions in this report represent the opinions of the
Office of Inspector General. In accordance with the Freedom of
Information Act (5 U.S.C. § 552), reports issued by the Office of
Inspector General are made available, if requested, to members of the
press and general public to the extent information contained therein is
not subject to exemptions under the Act.

> Sincerely,
>
> /s/
>
> Gloria Pilotti
>
> Regional Inspector General for Audit

Attachment

**ATTACHMENT**

**SACS' Comments on the Draft Report**

**REPORT DISTRIBUTION LIST**

######### ED-OIG/A09-C0018

+--------------------------------------+---+---------------------------+
| **[Auditee]{.underline}**            | # |                           |
|                                      |   |                           |
|                                      | * |                           |
|                                      | * |                           |
|                                      | E |                           |
|                                      | D |                           |
|                                      |   |                           |
|                                      | A |                           |
|                                      | c |                           |
|                                      | t |                           |
|                                      | i |                           |
|                                      | o |                           |
|                                      | n |                           |
|                                      |   |                           |
|                                      | O |                           |
|                                      | f |                           |
|                                      | f |                           |
|                                      | i |                           |
|                                      | c |                           |
|                                      | i |                           |
|                                      | a |                           |
|                                      | l |                           |
|                                      | * |                           |
|                                      | * |                           |
+--------------------------------------+---+---------------------------+
| James T. Rogers, Executive Director  | # |                           |
|                                      |   |                           |
| Southern Association of Colleges and | S |                           |
| Schools,                             | a |                           |
|                                      | l |                           |
| > Commission on Colleges             | l |                           |
|                                      | y |                           |
| 1866 Southern Lane                   |   |                           |
|                                      | S |                           |
| Decatur, Georgia 30033-4097          | t |                           |
|                                      | r |                           |
|                                      | o |                           |
|                                      | u |                           |
|                                      | p |                           |
|                                      | , |                           |
|                                      |   |                           |
|                                      | A |                           |
|                                      | s |                           |
|                                      | s |                           |
|                                      | i |                           |
|                                      | s |                           |
|                                      | t |                           |
|                                      | a |                           |
|                                      | n |                           |
|                                      | t |                           |
|                                      |   |                           |
|                                      | S |                           |
|                                      | e |                           |
|                                      | c |                           |
|                                      | r |                           |
|                                      | e |                           |
|                                      | t |                           |
|                                      | a |                           |
|                                      | r |                           |
|                                      | y |                           |
|                                      |   |                           |
|                                      | # |                           |
|                                      |   |                           |
|                                      | O |                           |
|                                      | f |                           |
|                                      | f |                           |
|                                      | i |                           |
|                                      | c |                           |
|                                      | e |                           |
|                                      |   |                           |
|                                      | o |                           |
|                                      | f |                           |
|                                      |   |                           |
|                                      | P |                           |
|                                      | o |                           |
|                                      | s |                           |
|                                      | t |                           |
|                                      | s |                           |
|                                      | e |                           |
|                                      | c |                           |
|                                      | o |                           |
|                                      | n |                           |
|                                      | d |                           |
|                                      | a |                           |
|                                      | r |                           |
|                                      | y |                           |
|                                      |   |                           |
|                                      | E |                           |
|                                      | d |                           |
|                                      | u |                           |
|                                      | c |                           |
|                                      | a |                           |
|                                      | t |                           |
|                                      | i |                           |
|                                      | o |                           |
|                                      | n |                           |
+--------------------------------------+---+---------------------------+
| #                                    | # |                           |
|                                      |   |                           |
+--------------------------------------+---+---------------------------+
| # **Other Departmental               |   |                           |
|  Officials/Staff** (electronic copy) |   |                           |
+--------------------------------------+---+---------------------------+
| #                                    |   |                           |
+--------------------------------------+---+---------------------------+
| ## Audit Liaison Officer             |   | ##                        |
|                                      |   | Assistant General Counsel |
| ## Office of Postsecondary Education |   |                           |
|                                      |   | ## Offi                   |
|                                      |   | ce of the General Counsel |
+--------------------------------------+---+---------------------------+
| #                                    |   |                           |
+--------------------------------------+---+---------------------------+
| ## Accreditat                        |   | ## Deputy Secretary       |
| ion and State Liaison Staff Director |   |                           |
|                                      |   | ## Offic                  |
| ## Office of Postsecondary Education |   | e of the Deputy Secretary |
+--------------------------------------+---+---------------------------+
| #                                    |   |                           |
+--------------------------------------+---+---------------------------+
| ##                                   |   | ## Chief of Staff         |
|  Accrediting Agency Evaluation Chief |   |                           |
|                                      |   | #                         |
| ## Office of Postsecondary Education |   | # Office of the Secretary |
+--------------------------------------+---+---------------------------+
| #                                    |   | #                         |
+--------------------------------------+---+---------------------------+
| ## Assistant Secretary               |   | ## Under Secretary        |
|                                      |   |                           |
| ## Office of L                       |   | ## Offi                   |
| egislation and Congressional Affairs |   | ce of the Under Secretary |
+--------------------------------------+---+---------------------------+
| #                                    |   | #                         |
+--------------------------------------+---+---------------------------+
| ## Assistant Secretary               |   | ## Director               |
|                                      |   |                           |
| ## Office of Inter                   |   | ##                        |
| governmental and Interagency Affairs |   |  Office of Communications |
+--------------------------------------+---+---------------------------+
| #######                              |   | #                         |
+--------------------------------------+---+---------------------------+
| ## Director                          |   | #                         |
|                                      |   | # Chief Financial Officer |
| ## Financial I                       |   |                           |
| mprovement and Post Audit Operations |   | ## Office of th           |
|                                      |   | e Chief Financial Officer |
| ## O                                 |   |                           |
| ffice of the Chief Financial Officer |   |                           |
+--------------------------------------+---+---------------------------+
|                                      |   | ##                        |
+--------------------------------------+---+---------------------------+
| ## Post Audit Group Supervisor       |   | #                         |
|                                      |   | # Chief Operating Officer |
| ## Financial I                       |   |                           |
| mprovement and Post Audit Operations |   | > Federal Student Aid     |
|                                      |   |                           |
| ## O                                 |   |                           |
| ffice of the Chief Financial Officer |   |                           |
+--------------------------------------+---+---------------------------+
| #                                    |   | #                         |
+--------------------------------------+---+---------------------------+
| ##                                   |   | ## Correspondence Control |
|                                      |   |                           |
|                                      |   | ##                        |
|                                      |   | Office of General Counsel |
+--------------------------------------+---+---------------------------+
| ##                                   |   | #                         |
+--------------------------------------+---+---------------------------+

[^1]: The 10 institutions were judgmentally selected from the 83
    institutions due for reaccreditations in calendar year 2001. Since
    the sample was judgmentally selected, the results of our review may
    not be representative of the entire population. The PURPOSE, SCOPE,
    AND METHODOLOGY section of the report provides additional
    information on the sample selection methodology.

[^2]: Of the remaining 27 institutions, 13 institutions were reaffirmed
    without condition, 10 institutions were placed on warning, and 4
    institutions were placed on notice. The section of this report
    titled MANAGEMENT CONTROLS FOR ENSURING ENFORCEMENT OF STANDARDS
    provides information on the "warning" and "notice" enforcement
    actions.

[^3]: The five institutions were judgmentally selected from the
    institutions with Commission actions during calendar years 2000 and
    2001. Since the sample was judgmentally selected, the results of our
    review may not be representative of the entire population. The
    PURPOSE, SCOPE, AND METHODOLOGY section of the report provides
    additional information on the sample selection methodology.