National Organic Standards Board

Certification and Accreditation Subcommittee (CACS) 

Proposal

Personnel Performance Evaluations of Inspectors

December 13, 2016

I INTRODUCTION

Over the 26 years since the Organic Food Production Act (OFPA) was passed and the National Organic 

Program (NOP) was subsequently established to implement the Statute, many Rules, Guidance, 

Instruction, and other Policies and Procedures have been implemented. The organic industry has gown, 

and with that growth has come challenges for ensuring consistency and integrity in this process-based 

program which now serves producers worldwide. There are numerous levels of inspection and oversight 

with checks and balances. The producer submits an Organic System Plan (OSP) and must be inspected 

annually by the certifier; the inspector must be evaluated annually by the certifier; the certifier must 

submit documentation on all aspects of its operations to be accredited and is inspected/audited by NOP 

auditors every 5 years, with an interim audit at about 2-year intervals; the NOP auditors are evaluated 

every 3 years; the NOP is required to have a Peer Review evaluation annually. The fast growing nature of 

organic production has created some challenges.

Over the last year and a half, certifiers have been working to meet the requirements of NOP 2027, 

Instruction: Personnel Performance Evaluations, promulgated by the NOP on August 2, 2013 and revised 

effective March 31, 2016. This instruction requires that every inspector be evaluated not only based on 

inspection reports and written evaluation instruments every year, but also on in the field annual 

evaluations. Many certifiers have expressed considerable concern for the logistics and expense of 

meeting this instruction and the potential negative impact it will have on organic production in the 

marketplace over time. 

A Discussion Document on this topic was posted in Fall 2016, public comment was received, and the 

topic was discussed at the NOSB meeting in St Louis in November 2016. Public comment was 

substantive in nature and clearly identified challenges if full compliance with NOP 2027 is to be fully 

enforced. Most serious is the concern that full compliance for in-field evaluations of every organic 

inspector worldwide “at least annually” would have serious negative economic implications for organic 

agriculture in the US and overseas, and it doesn’t best serve the intended goal of continuous 

improvement for all involved. 

This Proposal is a recommendation that the NOP revise NOP 2027 as soon as possible to clarify and 

establish a standard that certifiers can meet in a sustainable and economic manner.

II BACKGROUND

History

On December 2, 2011, following receipt of public comment, the NOSB voted 13:1 with 1 abstention to 

“provide all inspectors with performance assessment and oversight: a. Witness audits by ACA to be 

conducted at a minimum every 300 inspections or 3 years, whichever is less. Results must be 

documented. Witness audits may be conducted by certification management, senior inspectors, or 

senior reviewers.” 

NOSB APRIL 2017 proposals and discussion documents 69/137
On August 2, 2013, the NOP issued Instruction 2027 stating that there should be in-field evaluations “at 

least annually.”

In 2014, the NOP issued non- compliances to certifiers who failed to evaluate every inspector in the field 

annually.

In 2014, the International Organic Inspectors Association (IOIA) explored a pilot program to assist 

certifiers in developing and managing an in-field inspector evaluation program. IOIA developed a lengthy 

and detailed evaluation form, recruited evaluators, and, in consultation with several certifiers, 

implemented a fee for service program in 2015. In 2016, IOIA expanded this program. 

On September 4, 2015, the Accredited Certifiers Association (ACA) submitted a letter to the NOP

providing some observations and concerns about the impact of this Instruction and provided several 

recommendations for improving this requirement. ACA referenced the 2011 NOSB recommendation and 

requested Instruction 2027 be amended to permit certifiers to develop risk-based plans to evaluate the 

inspectors they work with across a rotational cycle and not be limited to one calendar year. ACA also 

requested certifiers be permitted to share evaluations.

On December 8, 2015, the NOP issued NOP 2501 Evaluating Auditor Performance (of NOP Auditors), 

which requires in-field evaluation every 3 years. “5.2b A Witness Appraisal shall be conducted at least 

once every 3 years.” 

On February 15, 2016, the IOIA submitted a report to the NOP on the pilot in-field evaluation program. 

In response to concerns about implementation of the in-field evaluation of every inspector worldwide

every year, the NOP revised NOP 2027 effective March 31, 2016 to include a provision at 3.2 b ii that 

“Certifiers may use the field evaluations of another accredited certifier. The revised Instruction at 3.2 b 

iii also included a provision that “Certifiers may submit alternative proposals for field evaluation to their 

Accreditation Manager.” 

During spring and summer 2016, the IOIA reported that they worked with 10 certifiers to provide in-field 

evaluations of 106 inspectors. There are about 80 NOP accredited certifiers worldwide, with 48 based in 

the USA.

In Fall 2016, the CACS posted a Discussion Document with specific questions to seek public comment: 

1. For certifiers: To date, what have you observed about the benefits, costs, and logistics of 

meeting this requirement?

2. For certifiers: Have you been able to meet this requirement for inspectors in overseas locations?

3. For certifiers: If given an option to present alternative evaluation plans to the every inspector, 

every year requirement, what would these look like? If a risk-based approach, how do you 

define risk?

4. For certifiers and inspectors: What has been your experience sharing evaluation forms and 

processes? What have been the challenges associated with this sharing?

5. For inspectors: To date, what are the concerns and benefits that you have observed?

6. For organic operators: To date, what are the concerns and benefits that you have observed or 

experienced during in-field audits conducted on your operation?

NOSB APRIL 2017 proposals and discussion documents 70/137
7. For all stakeholders: What mechanisms are in place to ensure that client files being shared 

between evaluators and inspectors are taking place on completely secure computer systems?

8. For all stakeholders: What are the in-field audit requirements for auditors of other inspection or 

certification schemes such as GFSI, Global GAP, SQF etc.?

Substantive Pubic comment was received and is discussed below.

III RELEVANT AREAS OF THE RULE, NOP INSTRUCTION, NOSB RECOMMENDATION

USDA Organic Regulations 

7 CFR Part 205 § 205.501 General requirements for accreditation. 

(a) A private or governmental entity accredited as a certifying agent under this subpart must: ... 

(1) Have sufficient expertise in organic production or handling techniques to fully comply with 

and implement the terms and conditions of the organic certification program established under 

the Act and the regulations in this part;

(2) Demonstrate the ability to fully comply with the requirements for accreditation set forth in 

this subpart; ......

(4) Use a sufficient number of adequately trained personnel, including inspectors and 

certification review personnel, to comply with and implement the organic certification program 

established under the Act and the regulations in subpart E of this part; 

(5) Ensure that its responsibly connected persons, employees, and contractors with inspection, 

analysis, and decision-making responsibilities have sufficient expertise in organic production or 

handling techniques to successfully perform the duties assigned; 

(6) Conduct an annual performance evaluation of all persons who review applications for 

certification, perform on-site inspections, review certification documents, evaluate 

qualifications for certification, make recommendations concerning certification, or make 

certification decisions and implement measures to correct any deficiencies in certification 

services.

7 CFR 504 Evidence of Expertise and Ability. 

A private or governmental entity seeking accreditation as a certifying agent must submit the following 

documents and information to demonstrate its expertise in organic production or handling techniques; 

its ability to fully comply with and implement the organic certification program established in §§205.100 

and 205.101, §§205.201 through 205.203, §§205.300 through 205.303, §§205.400 through 205.406, and 

§§205.661 and 205.662; and its ability to comply with the requirements for accreditation set forth in 

§205.501:

(a) Personnel.

(1) A copy of the applicant's policies and procedures for training, evaluating, and 

supervising personnel;

 (2) The name and position description of all personnel to be used in the certification 

operation, including administrative staff, certification inspectors, members of any certification review 

and evaluation committees, contractors, and all parties responsibly connected to the certifying agent;

 (3) A description of the qualifications, including experience, training, and education in 

agriculture, organic production, and organic handling, for:

NOSB APRIL 2017 proposals and discussion documents 71/137
 (i) Each inspector to be used by the applicant and

 (ii) Each person to be designated by the applicant to review or evaluate 

applications for certification; 

 (4) A description of any training that the applicant has provided or intends to provide to 

personnel to ensure that they comply with and implement the requirements of the Act and the 

regulations in this part.

 (d) Current certification activities. An applicant who currently certifies production or handling 

operations must submit: (1) A list of all production and handling operations currently certified by the 

applicant; .......

 (2) Copies of at least 3 different inspection reports and certification evaluation 

documents for production or handling operations certified by the applicant during the previous year for 

each area of operation for which accreditation is requested; 

NOP Memorandum to NOSB, June 23, 2011, requesting a Proposal from NOSB on Inspector 

Qualification Requirements.

NOSB Recommendation, December 2, 2011, Inspector Qualification Requirements.

NOP Instruction 2027, August 2, 2013, and Revised March 31, 2016, Instruction: Personnel Performance 

Evaluation.

NOP Instruction 2501, December 8, 2015, Evaluating Auditor Performance

IV DISCUSSION

In the absence of specific regulatory requirements for the qualifications of organic inspectors, ACAs have 

instituted a wide range of requirements and criteria in their hiring process, training, and performance 

monitoring. While the CACS is aware of the requirements of a few ACAs in particular, the actual extent 

and range of these specific requirements among the 80 worldwide USDA-accredited ACAs is not fully 

known, and the total number of organic inspectors is also not known.

In response to public comment in spring 2016 the CACS reached out to both NOP staff and certifiers to 

better understand how NOP 2027 was impacting the certification process in the US and overseas. 

Stakeholders all agree that all inspectors must be professionally evaluated every year. Professional 

evaluation of inspectors takes place via the review of inspection reports and evaluations provided by 

certified operations. Stakeholders agree that in-field evaluations should take place, but many certifiers 

disagree about the requirement of an in-field evaluation of every inspector every year, worldwide. This 

disagreement is based on logistics, cost, and sustainability. Further, the push to fulfil the every inspector 

every year requirement may lead to evaluations chosen to simply check off completion versus selecting 

a location which would best demonstrate an inspector’s ability. 

The NOP expressed considerable concern about the quality of work of some inspectors that the NOP 

auditors witnessed during mid-term or 5 year review of some certifiers in recent years. The NOP 

NOSB APRIL 2017 proposals and discussion documents 72/137
interprets §205.501 to require in-field evaluation of every inspector worldwide every year and that this 

is necessary to ensure consistency in organic certification.

The ACA provided feedback from a range of certifiers. For some state-run certifiers the annual in-field 

inspections do not appear to be a serious concern either logistically or financially. Some certifiers see 

this requirement as beneficial in encouraging inspectors to increase their ongoing education.

For many other certifiers however, large and small, considerable concern has been raised regarding

costs, logistics, inequities between certifiers, and a range of other issues. 

The following is a sampling of public comment received from certifiers, the IOIA, and inspectors.

• It's unclear if NOP will approve alternative plans, but certifiers would appreciate the option.

• Evaluating an inspector used only a few times a year is not feasible. 

• Some certifiers are already relying heavily on shared evaluations and offering them to other 

certifiers at no charge. Others are trading evaluations or charging a fee.

• The most expensive evaluations are those in remote areas since the remote location is often the 

reason a certifier is using an outside inspector.

• Many certifiers believe strongly that in-field evaluations are a positive for their organization and 

the industry. Conducting them internally carries a high value. Evaluations received from other 

sources vary in quality and do not cover adherence to a certifier’s own policies, nor do they 

present an opportunity to observe personal interactive skills essential to good inspections. 

However, some find the cost-benefit just doesn't work out for every inspector every year. A riskbased approach would be more effective and efficient at achieving the desired results. 

• Some certifiers have seen a number of new applicants and have had to find new inspectors, or 

certifiers have had to turn to more inspectors than planned late in the year. This presents an 

even greater workload to ensure these inspectors are evaluated. The logistics and entire process 

is time-consuming for all involved and presents difficulties and disincentives. 

• NOP 2027 says in–field evaluations “should” be conducted, but NOP has issued non-compliances 

where 100% of all inspectors have not received in-field evaluations. NOP appears to interpret 

“should” as “shall”.

The following are more detailed excerpts from the public comments of one medium-sized certifier as 

illustrative of the concerns, with proposals for revisions for NOP 2027 which the CACS sees as sensible 

alternatives to the current every inspector, every year requirement. 

We certify approximately 2000 operations in more than 20 States; we employ 15 staff 

Certification Specialists who perform inspections, and we contract with approximately 50 

independent inspectors annually. 

In 2014, we were issued a noncompliance during our NOP audit for conducting onsite 

evaluations of less than 100% of our contract inspectors, per Standards section 205.501(a)(6). 

We addressed this. In 2015 and 2016 we conducted an onsite evaluation of 100% of the 65 

inspectors currently working for us. We’ve given many, many hours of consideration to NOP 

2027, and we are well-qualified to provide comments and feedback on this issue. 

In summary, we do not support NOP 2027 as written, specifically section 3.2.b which requires an 

annual onsite evaluation of all inspectors. We support a continuous improvement system that 

NOSB APRIL 2017 proposals and discussion documents 73/137
includes onsite evaluations as a tool for training and upholding performance criteria of 

inspectors. However, the requirement that each inspector be peer evaluated annually 

significantly increases the cost of inspections for certified operators, limits our ability to support 

the growth of the organic industry as well as the much needed growth of the inspector pool, and 

is unnecessarily prescriptive and is more stringent even than the evaluation requirement NOP 

has of its own auditors per NOP 2501 5.2.b. 

Overall, we estimate the onsite evaluation process for independent inspectors costs us roughly 

$25,000-30,000 annually. However, that can fluctuate upwards significantly based on travel 

requirements and evaluator availability. We estimate that salary costs to manage and 

coordinate evaluations and to evaluate staff inspectors roughly translates to an additional 

$10,000. 

Since 2014, the cost of our onsite evaluations has quadrupled, but we have not (yet) raised our 

certification fees. We’ve been trying to do more with less, i.e. using a fewer number of 

inspectors for an ever-growing number of operations, in order to keep the number of 

evaluations we must perform manageable. We’re well aware that this doesn’t serve the organic 

community in the long run, as it limits the number of new inspectors we take on, and it puts a 

cap on the overall number of inspections that can be done. This has the impact of depressing 

growth. In fact, in August 2016, we began turning away new client requests for crop certification, 

due in part to our lack of our capacity to perform an onsite evaluation of new inspectors. 

Requiring an annual onsite evaluation for each inspector has almost completely erased our 

ability to be strategic and thoughtful with these evaluations. In our commitment to remain 100% 

compliant and complete the annual evaluation cycle, we find it difficult to match the onsite visit 

with what would actually be, in our opinion, the best evaluation for the inspector. For example, 

we identify in #3 (risk criteria) below, that an inspector new to a scope would require an onsite 

evaluation. With the current non-risk-based requirement of evaluating 100% of inspectors 

annually, we cannot guarantee that we would be evaluating at a site which includes the new 

scope. We feel the requirement to complete all annual onsite evaluations downgrades the goal 

of assisting with strategic professional development of the inspector. This is contrary to the 

spirit of promoting continuous improvement and providing better value to all stakeholders in 

the organic industry. 

We are also left with the question about what happens when, despite our best efforts, we can’t 

perform an onsite evaluation of an inspector. For example, if a contractor performs several 

inspections for us but quits prior to his/her scheduled onsite evaluation, are we noncompliant 

with the 100% requirement in NOP 2027? 

We would like onsite evaluations to focus on and support inspector development as a tool to 

maintain a pool of qualified inspectors which grows at a pace commensurate with the growth of 

the organic industry, especially in underserved areas. 

Our proposal for a better-focused onsite evaluation process is to develop a risk-based approach. 

On an annual basis we would evaluate: 

• Novice inspectors 

• Inspectors in their first year of inspecting for us 

NOSB APRIL 2017 proposals and discussion documents 74/137
• Inspectors new to a scope 

• An inspector who, from Operator or our reviewer feedback, shows the need for 

additional guidance based upon our review criteria. 

For inspectors who are experienced and have demonstrated competency based upon their past 

onsite evaluations, reviewer feedback, and annual performance evaluations, we propose a 

three-year evaluation cycle, similar to the frequency with which evaluations are required for 

auditors working for the NOP. 

Costs and Benefits of NOP 2027, based on public comment overall:

Benefits of NOP 2027:

• ACAs report that they no longer hire poor performing inspectors.

• Allows ACAs to identify where further training is needed.

• Opened a broader dialogue between certification staff and inspectors. 

• Increases consistency between some certifiers.

• Increases oversight and accountability for inspectors.

Costs/Challenges of NOP 2027:

• Indicates gaps in consistency inasmuch as only 7 out of 80 ACAs provided comment. 

• The implementation of the IOIA field evaluation is only in its second year, and time will tell if the 

programs in place now will be sustainable over time without loss of inspectors or increased fees 

to clients at a time when there is increasing demand for certification of operations worldwide. 

• Disincentives to hire contract inspectors who conduct only a handful of inspections each year.

• Disincentive for ACAs to accept new clients locally or in distant locations. 

• Very expensive. Costs per inspection range from $400 to $2000 per inspection. Huge annual 

budget change for every certifier. 

• Not sustainable. 

• Not cost-effective: e.g. certifier with 70 inspectors who performs 4,800 inspections every year @ 

between 1 and 200 inspections per inspector.

• Logistically burdensome. 

• Cannot conduct international inspector in-field evaluations.

• Could not complete all 100% in 2015 or 2016

• IOIA found it very challenging to service 100 evaluations in one season.

• Sharing files through unsecure e-mails places client confidentiality at high risk.

• Sharing evaluations/witness audits between certifiers is inconsistent.

• Confusion over terms “personnel evaluation”, which is a normal aspect of the contract between 

certifier and independent contract inspector, versus “Witness Audit” which could be conducted 

without the overtones of a personnel evaluation by a peer.

• Places peer inspectors in a difficult professional relationship with fellow inspectors who may be 

competing for the same inspections or close friends over many years, with resultant 

inconsistency in evaluation. 

• Increases time, cost, and stress for clients and inspectors and ACA staff. 

• If ACAs adopt risk-based in-field witness audits over a period of three years, how will 

consistency be maintained between certifiers?

NOSB APRIL 2017 proposals and discussion documents 75/137
It is worthwhile re-visiting the NOSB Recommendation of December 2, 2011 on Inspector Qualifications, 

which was based on a range of public comment at that time: 

B. As continuing organic inspector criteria: 

1. Continuing Education

a. Annual training by Accredited Certifiers Association (ACA) to update on specific procedures of 

the ACA as well as National Organic Program (NOP) standards updates and guidelines.

b. Minimum 8 hours annual continuing education related to the type of inspection work 

performed. Each hour of curriculum time (e.g. class time, coursework, field study, testing), 

equals one hour of continuing education hours. Trainings conducted by ACAs and closed to the 

general inspection community do not apply toward continuing education hours given their 

tendency to focus on certifier procedures rather than broad knowledge such as agronomic and 

food industry practices and general auditing skills. 

c. In-depth training on the topic of recordkeeping and/or accounting must be included as part of 

continuing education, and IOIA is encouraged to develop a training to fulfill this need. 

d. Continuing education credits include webinars, seminars, workshops, and colleges and 

university extension programs related to the type of inspection work performed or new scope of 

inspection interest.

C. ACA accreditation criteria to ensure adequate monitoring and oversight of inspector 

qualifications: 

1. Annual attendance of NOP trainings. 

2. Documented inspector qualification monitoring program that readily provides verification 

that all inspectors employed or contracted in the service of the ACAs are qualified according to 

these criteria. 

3. Provide programmatic and consistent annual training to inspectors regarding processes, 

policies, and procedures specific to the ACA. Training materials used must be available for 

review during accreditation audits and be included in annual ACA updates to the NOP.

4. Provide all inspectors with performance assessment and oversight accordingly: 

a. Witness audits by ACA to be conducted at a minimum every 300 inspections or 3 years, 

whichever is less. Results must be documented. Witness audits may be conducted by 

certification management, senior inspectors, or senior reviewers. 

b. Evaluation of every inspection provided to the inspector. 

c. Annual performance evaluation provided to the inspector. 

d. All serious or persistent performance issues that arise during any of the above assessments 

must be documented by the ACA and must include documented corrective action and 

improvement measures as deemed necessary by the ACA. 

In Summary: 

Organic Inspectors are not licensed per se, and the flexibility of the present training and evaluation of 

inspectors has been helpful during these formative years of organic certification. But many of the 

concerns raised by the NOSB in 2011 continue to suggest that certifiers must work towards greater 

consistency in inspector training and evaluation. All inspectors must receive a personnel evaluation 

annually, but the inclusion of an in-field witness audit as part of this annual evaluation need only be 

conducted every 3 years, or more frequently where concerns have been raised about the individual 

inspector’s work. Witness audits should be conducted preferably by certifier staff, but alternatively 

Senior Peer Inspectors may be used provided that they have been properly trained in witness audits and 

are not placed in the position of conducting a personnel evaluation of a peer who is not in their employ.

NOSB APRIL 2017 proposals and discussion documents 76/137
Presently not all ACA’s attend annual NOP trainings, and yet they are critical to the overall success of the 

organic program. If ACA’s are unable to attend, they must be required to review all the written material 

and meeting reports, all of which must be readily available on the web. The NOP trainings must include 

clear direction as to inspector qualification, continuing education, scope of annual evaluations, and 

periodic in-field witness audits. 

Language in NOP 2027 appears to conflate annual personnel evaluations with witness audits. While a 

witness audit may be one part of the annual personnel evaluation, it is a witness audit which may be 

conducted by a peer if logistics dictate that a certifier staff person cannot attend.

If the IOIA continues to provide the service of in-field peer witness audits, there should be a mechanism 

in place to inspect and evaluate the IOIA. 

ACAs should work to clearly determine the total number of available organic inspectors so that logistics, 

costs, and market impact of conducting in-field witness audits, without unnecessary and costly 

replication, can be better understood.

V PROPOSAL

The NOSB Recommends that: 

1. All ACAs attend annual NOP trainings, and those trainings must include clear direction as to 

inspector qualifications, continuing education, annual evaluations, and periodic in-field witness 

audits. If the ACA is unable to attend, it is the ACA’S responsibility to review all training materials. 

2. All ACAs must include in their annual contracts with inspectors the number of hours of required 

continuing education, annual evaluation criteria, periodic in-field witness audits, and provide a 

written annual evaluation sent to the inspector with suggestions for improvement.

3. All ACAs must maintain in their files detailed procedures for how annual personnel evaluations 

of staff and contract inspectors are conducted and include a justification for the frequency of infield inspections for each inspector.

4. All inspectors must receive a personnel evaluation annually, but an in-field witness audit need 

only be conducted every three years, or more frequently where concerns have been raised 

about the individual inspector’s work or for a novice inspector. 

5. Witness audits should be conducted, preferably by certifier staff, but alternatively, by senior 

peer inspectors, provided that they have been properly trained in witness audits and are not 

placed in the position of “evaluating” a peer who is not in their employ.

VI VOTE:

Motion to approve that NOP 2027 be revised to incorporate the recommendations 1-5 listed above.

Motion by: Jean Richardson

Seconded by: Emily Oakley

Yes: 6 No: 0 Abstain: 0 Absent: 1 Recuse: 0 

Approved by Scott Rice, Subcommittee Chair, to transmit to NOSB December 13, 2016

NOSB APRIL 2017 proposals and discussion documents 77/137