Audit Report

OIG-20-030

BILL AND COIN MANUFACTURING

The United States Mint Numismatic Order 

Management System is Meeting User Needs 

But Improvements to Oversight are Needed

March 25, 2020

Office of Inspector General

Department of the Treasury
This Page Intentionally Left Blank
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) i

Contents

Audit Report

Results in Brief .............................................................................................2

Background..................................................................................................5

Launching of OMS-II .................................................................5

Roles of the Contractor and the Mint ..........................................6

Audit Results................................................................................................8

Finding 1 The Mint’s Oversight of the Numismatic Order Management System 

Contractor Needs Improvement..................................................8

Lack of Periodic Review of the QASP Performance Requirements and 

Corrective Action Plans .............................................................9

Lack of Sufficient Documentation Supporting the Mint’s Contract 

Oversight ..............................................................................11

The Mint Did Not Identify Loss of the Contractor as a Risk to the 

Numismatic Program...............................................................14

Recommendations ..................................................................15

Finding 2 OMS-II Is Working as Intended and Meeting User Needs..............18

OMS-II is Working as Intended .................................................19

OMS-II Is Meeting Most User Needs .........................................22

Numismatic Sales Have Not Expanded ......................................25

Recommendations ..................................................................27

Appendices

Appendix 1: Objectives, Scope, and Methodology...........................................29

Appendix 2: Management Response..............................................................34

Appendix 3: Major Contributors to This Report...............................................38

Appendix 4: Report Distribution....................................................................39

Abbreviations

CCC Customer Contact Center
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) ii

COR Contracting Officer’s Representative

CSR Customer Service Representative

FAR Federal Acquisition Regulation

GAO U.S. Government Accountability Office

GPRA Government Performance and Results Act 

JAMES Joint Audit Management Enterprise System

Mint United States Mint

OIG Office of Inspector General

OMB Office of Management and Budget

OMS-II The Mint’s Order Management System 

PFSWeb PFSWeb, Inc.

QASP Quality Assurance Surveillance Plan

SAM Sales and Marketing

Treasury Department of the Treasury
This Page Intentionally Left Blank
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 1

March 25, 2020 

David J. Ryder 

Director 

United States Mint 

This report presents the results of our audit of the United States 

Mint (Mint) numismatic order management system. The Mint is 

authorized to produce and sell numismatic products to the public. 

Numismatic products include precious metal coins, proof coins, 

uncirculated coins, commemorative coins, and Congressional 

medals. The numismatic program is self-sustaining and operates at 

no cost to the public under the United States Mint Public Enterprise 

Fund.1

Using the services of a contractor, PFSWeb, Inc. (PFSWeb), in 

October 2014, the Mint introduced a new catalog website and 

order management system (OMS-II). According to the Mint’s 

Capital Investment Plan, the objective of this new system is to 

provide Mint customers with an experience that keeps pace with 

advancements in both technology and business functionality. 

Our audit objective was to determine whether the Mint is 

effectively managing order processing for numismatic products, 

such as precious metal coins, commemorative coins, and medals 

for sale to the public. As part of this work, we planned to 

determine if OMS-II is working as intended and the numismatic 

program is meeting the needs of its users.2 This audit was included 

in our fiscal year 2017 annual audit plan. The scope of our audit 

 

1 Under 31 U.S.C. 5136, the Mint operates under the Public Enterprise Fund. This fund enables the 

Mint to operate without an appropriation using revenue generated through the sale of circulating 

coinage, numismatic products to the public, and bullion coins to authorized purchasers. 

2

 OMS-II, which is operated by PFSWeb, provides a full suite of applications that includes retail order 

management, warehouse management, customer service, and payment processing that supports the 

Mint’s numismatic operations and meets the eCommerce needs of the Mint’s retail sales operations. 
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 2

covered from the launch of OMS-II in October 2014 through 

September 2017. 

To accomplish our objective, we evaluated key contractor order

processing controls and controls used by the Mint in providing 

oversight of the contractor’s numismatic order management 

system. We reviewed Mint numismatic performance documentation 

and performed tests of system data collected by the Mint’s data 

warehouse to determine if OMS-II is working as intended. We also

reviewed customer survey results, Department of the Treasury 

(Treasury) Office of Inspector General (OIG) Office of Investigation 

complaints, and transcripts of online customer service

conversations to determine if the numismatic order processing 

program is meeting user needs. We conducted fieldwork from June

2017 through November 2018 at Mint facilities in Washington, DC,

and Philadelphia, Pennsylvania, and the contractor’s facilities in

Dallas Texas, and Memphis, Tennessee with additional follow-up in

June 2019. Appendix 1 contains a more detailed description of our 

objective, scope, and methodology.

Results in Brief

The Mint is effectively managing ordering processing for 

numismatic products using a contractor order management system;

however, the Mint’s oversight of its contractor needs improvement. 

The Mint contracted with a full-service provider to meet the order 

processing needs of its numismatic retail sales operations.3 The 

Mint provides oversight of the contractor through the Quality 

Assurance Surveillance Plan (QASP). The QASP established

contract performance requirements and the Mint’s methodologies 

for conducting contractor oversight. 

Specifically, the Mint did not make necessary adjustments or 

develop corrective action plans, as required by the QASP, to 

improve contractor performance in meeting established

performance requirements. In addition, the Mint did not maintain 

required quality assurance documentation of its monitoring 

 

3 An End-to-End eCommerce solution consolidates multiple facets of the eCommerce business with 

one trusted provider. This includes a team of technology professionals, distribution centers, logistics 

infrastructures, customer care operations, designers, and consultants. 
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 3

activities. The Mint also relied on contractor-provided summary 

reports to validate performance. Further, at the end of fieldwork, 

the Mint was in the process of developing a risk management 

program, but did not consider the risk of losing the numismatic 

order management contractor.

We also concluded that OMS-II, created and maintained by 

PFSWeb, is working as intended based on the contractor meeting 

most of its performance goals and successfully processing 

numismatic orders. In addition, based on Mint-contracted customer 

satisfaction surveys and our review of online customer service 

conversations, we determined that OMS-II is meeting the needs of 

a majority of its users. However, the contractor operated and 

maintained website and Customer Contact Center (CCC) do not 

offer services to those with limited English proficiency. 

Additionally, the Mint has not met targeted unit sales goals from

fiscal years 2015 through 2017, resulting in declining revenues.

We are recommending that the Director of the Mint ensure the 

Numismatic and Bullion Directorate (1) periodically reviews QASP 

performance measures to determine necessary adjustments and

develop corrective action plans, as required by the QASP, with the 

contractor, where needed; (2) retains evidence of the its monitoring 

activities outlined in the contract’s QASP to better document the 

Mint’s oversight of the contractor’s compliance with contract 

requirements; (3) obtains and validates contractor’s supporting 

documentation against monthly performance reports and retains

this as part of the Mint’s monthly monitoring documentation; 

(4) works with the newly appointed Chief Risk Officer to assess 

the need to include, the loss of the OMS-II contractor as a potential 

risk to the numismatic program as part of the Mint’s Risk 

Management Program; (5) performs an assessment regarding the 

impact of not having language accessibility on Mint’s numismatic 

program services and considers adding these services to OMS-II; 

and (6) performs analyses to determine the feasibility and potential 

impact of proposed actions to improve numismatic sales, performs

additional studies to enhance future sales, and reports the results

to stakeholders.

In a written response, management concurred with our six 

recommendations and outlined corrective actions, planned or taken, 
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 4

to address our recommendations. Specifically, in response to 

recommendation number 1, the Mint plans to address changes to 

the QASP during the contract option year negotiations and has 

established an executive steering committee to evaluate the current 

vendor and future needs during those option years. In response to 

recommendation number 2, management stated that the Mint

retains evidence and records of its monitoring of the CCC and 

performance related to customer service. Management also stated 

that the Mint plans to conduct a monthly review of all QASP 

elements, as well as the business dashboard, with each Division 

Chief and Branch manager. While we did find evidence of some of 

the Mint’s monitoring activities mentioned in management’s 

response, the Mint did not provide evidence of a monthly written 

report submitted to the Contracting Officer’s Representative (COR)

documenting the quality assurance monitoring of the contractor’s 

performance as required by the QASP. Similar to its retention of 

evidence and records of certain monitoring activities, the Mint 

should ensure that this monthly written report is prepared and 

submitted to the COR and retained as evidence in the records of 

the Mint’s monitoring activities.

In response to recommendation number 3, management plans to 

assess the feasibility of developing a system to capture and store 

the raw data elements to validate the contractor’s monthly 

performance reports, or contracting with a third party to conduct 

additional surveillance and/or audits. In response to 

recommendation number 4, management stated that the Mint’s 

Sales and Marketing (SAM) Directorate identified the loss of the 

OMS-II contractor as a risk in its risk inventory in 2018 and is 

currently evaluating available courses of action to mitigate this risk.

In response to recommendation number 5, in the next year, the 

Mint plans to reassess the costs and benefits of providing OMS-II 

system access with additional languages. In response to 

recommendation number 6, management stated that the Mint 

continues to perform formal and informal analyses of business 

opportunities to grow market share and gross revenue, as well as 

to identify new income sources for the program. 

Management’s response and its corrective actions, taken and 

planned, meet the intent of our recommendations. Management 

will need to record the estimated dates for completing its planned 
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 5

corrective actions as well as the actual dates of completed 

corrective actions in the Joint Audit Management Enterprise 

System (JAMES), Treasury’s audit recommendation tracking 

system.

Background

The Mint produces and sells numismatic products to the public

through its numismatic program.

4 Numismatic product offerings 

and maximum amounts minted are authorized by Congress.5 The 

program is self-sustaining and operates at no cost to taxpayers.

The numismatic program sold approximately 3.9 million units in 

fiscal year 2017, a decrease of 6 percent from the prior year. 

Numismatic revenues totaled $387.5 million in fiscal year 2017, a 

decrease of 6 percent from fiscal year 2016. Over half of the 

numismatic program’s revenue is generated through online sales. 

Bulk orders account for over 20 percent of revenue, but only 

accounted for 0.3 percent of all orders.6

Launching of OMS-II

In October 2014, the Mint launched OMS-II which included a new 

catalog website designed to provide Mint customers with an 

experience that keeps pace with advancements in both technology 

and business functionality. 

OMS-II, which is maintained by a contractor, supports the Mint’s 

numismatic retail sales operations mostly through eCommerce, 

allowing customers to place orders online and by phone.7 OMS-II 

provided a business solution for the Mint’s numismatic ordering 

processes to replace its legacy system. According to the Mint’s 

 

4 31 U.S.C. 5111 authorizes the Secretary of the Treasury to prepare and distribute numismatic items. 

Numismatic items include proof coins; uncirculated coins; commemorative coins; Congressional gold 

medals; platinum, gold, and silver bullion coins; and related products or accessories. 

5 31 U.S.C. 5112, Denominations, specifications, and design of coins.

6 To participate in the Numismatic Bulk Purchase Program, applicants must meet qualifying criteria and 

complete an agreement form and registration form.

7 Along with placing orders online and over the phone, customers can purchase coins at retail locations 

at the Mint Headquarters in Washington, DC and Mint-attended numismatic conventions. Mail orders 

for numismatic products ended on September 30, 2017.
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 6

Capital Investment Plan, the Mint’s old system had become 

obsolete, which resulted in an environment that failed to meet the 

Mint’s business needs, and posed unacceptable risks.

The OMS-II solution includes the following major components: 

eCommerce, Creative Design, Interactive Marketing Services, Order 

Management/Warehouse Management Systems, Information 

Technology Integration, CCC, and Fulfillment/Distribution Center. 

The contractor operates from its headquarters in Allen, Texas; the 

CCC is located in Dallas, Texas; and the Distribution Center is 

located in Memphis, Tennessee.

In addition to purchasing products online, over the phone, and in 

person at Mint Headquarters and the Mint kiosk at numismatic 

collector conventions, an enrollment program is available for

customers to sign up for certain products to be automatically 

delivered when they become available. The Mint also has a 

Numismatic Bulk Purchase Program where authorized dealers can 

order numismatic products in bulk and qualify for a discount on 

their purchases.

Roles of the Contractor and the Mint

In September 2013, the Mint entered into an initial 6 year 

performance-based contract with an eCommerce full-service 

provider to meet the needs of its numismatic retail sales 

operations. 8 As of November 2019, the Mint extended the 

PFSWeb contract until the end of fiscal year 2020. The contract 

set the requirements, roles and responsibilities of the contractor, 

and defined the scope of the contractor’s services.9 The contract is 

structured around “what” service or quality level is required, as 

opposed to “how” the contractor should perform the work. 

 

8 The Mint entered into a performance-based contract with PFSWeb for the OMS-II program with a 

potential performance period from September 2013 through September 2023. The contract includes 

a design and implementation year, five operations and maintenance years, and four operations and 

maintenance option years. According to the Federal Acquisition Regulation (FAR), performance-based 

contracting is the preferred method for acquiring services and is to be used to the maximum extent 

possible.

9 According to FAR 37.601(b), performance-based contracts shall include a performance work 

statement, measurable performance standards, and performance incentives where appropriate.
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 7

The contract also included a QASP that sets forth procedures and 

guidelines the Mint uses to ensure the contractor achieves required 

performance standards or service levels.10 The QASP identifies 

performance requirements the contractor must meet. These 

requirements fall into three categories—CCC, Distribution Center, 

and Systems—each with specific performance targets and 

minimum desired outcomes acceptable to the Mint. Quality levels 

that fall below the specified performance targets for the month are 

considered a “service failure.” Each month, the contractor provides 

the Mint with the results of these performance requirements. Some 

of these performance measures are tied to incentives for superior 

results or disincentives for not meeting the requirements. 

Incentives include monetary fees payable to the contractor when 

performance exceeds performance targets within a specified 

timeframe while disincentives results in “performance credits” 

which are paid to the Mint in the event of a Service Level Failure.

We noted that incentives and disincentives were applied to the 

invoice immediately following the month in which they were 

realized. From fiscal year 2014 through fiscal year 2017, the 

activities from the CCC and distribution center resulted in total net

disincentives of over $51,000.

11

The QASP also defines the performance management approach the 

Mint will use to monitor and manage the contractor’s performance 

to ensure expected outcomes or performance objectives 

communicated in the contract are achieved. As part of this 

approach, the QASP defines the methodologies the Mint uses to 

monitor and evaluate the contractor’s performance, describes Mint 

quality assurance documentation requirements, and describes the 

Mint’s analysis of quality assurance monitoring results. 

The methodologies to monitor performance identified in the QASP 

include 100 percent inspections and periodic inspections but does 

not specifically identify what those inspections entail. The QASP 

states that each month the Mint COR shall review monitoring 

 

10 According to FAR 46.401(a), quality assurance surveillance plans should be prepared in conjunction 

with the preparation of the statement of work and should specify all work requiring surveillance and 

the method of surveillance. 

11 Two percent of certain CCC and Distribution Center invoice amounts are available for 

incentives/disincentives to encourage the contractor to meet performance measures. Also, a 

disincentive of $10,000 per month is available if the contractor does not meet performance 

measures associated with the OMS-II system, such as system up-time requirements.
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 8

documentation and enter summary results into a Surveillance 

Activity Checklist. 

Mint officials and staff stated they used the following methods to 

conduct oversight of the contractor’s performance:

 Monthly reviews of the service level agreement results reported 

in the contractor’s QASP report. 

 Monthly reviews of contractor invoices for proper supporting 

documentation and approval.

 Quarterly inventories conducted with Mint oversight. 

 Periodic review and monitoring of the Distribution Center and 

CCC. 

 Daily live feed monitoring of website activity, including live-time 

traffic to the Mint’s website and products being bought.

 Daily report reviews including an Operational Dashboard.12

 Weekly meetings with the contractor to discuss CCC and 

distribution center operations.

 Reviews of all emails, chats, and call disposition summaries.

 Live call calibrations performed to ensure the contractor’s call 

quality assurance reviews are performed as required by the 

Mint.

 Reviews of the website and CCC customer satisfaction survey 

results. 

Audit Results

Finding 1 The Mint’s Oversight of the Numismatic Order 

Management System Contractor Needs Improvement

We concluded that key controls and procedures used by the Mint in 

providing oversight of the contractor’s numismatic ordering 

 

12 The Operational Dashboard includes revenue figures by order method, order method figures, freight 

option breakdowns, volume of units shipped, inbound activity (new inventory and sales returns), call 

center volume and service levels, and monthly customer counts.
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 9

management system could be improved. As discussed further 

below, we noted that the Mint did not periodically review or make

necessary adjustments to improve the QASP since the beginning of 

the contract. The Mint also did not develop corrective action plans

in conjunction with the contractor, as required by the QASP, to 

improve contractor performance in meeting established 

performance requirements.

We were also unable to verify the extent to which the Mint

performed monitoring activities per the QASP because the Mint did 

not maintain required quality assurance documentation. In addition, 

as of the end of fieldwork, the Mint was finalizing its risk 

management program. We noted that this program did not include

the risk associated with the potential loss of the numismatic order 

management contractor. 

Lack of Periodic Review of the QASP Performance 

Requirements and Corrective Action Plans

As discussed above, the QASP established contract performance 

requirements and the Mint’s methodologies for maintaining 

contractor oversight. We noted that the Mint did not periodically 

review QASP performance requirements, as necessary. The Mint 

also did not develop corrective action plans, as required by the 

QASP, when the contractor failed to meet established performance 

requirements. The performance measures put in place by the 

contract’s QASP are used to ensure customer service quality, 

which was identified by the Mint as a risk to OMS-II. Repeated 

failure to meet these performance measures places the Mint at 

greater risk of not ensuring customer service quality. 

While the contractor met most performance requirements from 

October 2014 through September 2017, as described further in 

finding 2 below, the contractor did not consistently meet the 

following CCC performance requirements over the period reviewed: 

• Agent Utilization – The contractor is required to maintain agent 

productivity to hours worked of 65 percent.

13 We found that 

 

13 Agent utilization is calculated by summing all agents’ total talk time, after call work, and training and 

quality coaching divided by the total system logged in time.
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 10

during the 3-year review period, the contractor only met this 

measure 39 percent of the time. 

• Peak Day Performance – The Mint identified high volume and 

coin release days as risks to the OMS-II. The Mint manages

these risks by identifying peak days with the contractor in 

advance to prepare staffing needs. The peak day performance 

measure goal is to meet all three performance requirements (i.e. 

80 percent of calls answered in 20 seconds or less, average 

speed of answer of 20 seconds or less, and abandon call rate of 

5 percent of less) during a designated peak day. There had been 

11 designated peak days over the three-year review period; the 

contractor met peak day performance requirements for only five 

of those days. 

• Email Response Time – The contractor is required to respond to 

99 percent of emails within one business day. The contractor 

only met this performance measure 17 percent of the time.

 CCC Satisfaction Survey –The contractor is required to maintain 

90 percent customer satisfaction. The contractor has never met

the CCC satisfaction survey measure over the 3-year review 

period—averaging 77 percent customer satisfaction per month.

The QASP states that if the contractor has not met the minimum

performance requirements, a corrective action plan is required to 

bring the contractor’s performance to service level requirements.

Mint personnel must coordinate and communicate with the 

contractor to resolve issues and concerns regarding marginal or 

unacceptable performance.

A Mint official told us that they have not developed any recent

corrective action plans with the contractor and that the

contractor’s performance has been strong and within acceptable 

ranges. 

At least one of the performance requirements may need to be

adjusted— CCC satisfaction survey requirement of 90 percent. 

Performance standard requirements should be appropriate and 

developed so that the contractor’s performance meets the contract 

terms and conditions. Because the contractor has never met this 

performance level requirement and the Mint has not developed a 

related corrective action plan, the required performance level may 
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 11

be too high or the survey may need to be modified. A Mint official 

told us that some performance measures such as the CCC survey 

may be too high. When asked about the CCC survey goal of 90

percent, a Mint employee told us that the goal was difficult to 

achieve and there was a need to figure out how to improve results. 

The Mint employee further stated that the customer survey was 

not truly reflective of the job CCC was doing. 

The QASP is a “living document” in that the Mint may review and 

revise it on a regular basis, with the consent of the contractor. As 

such, the QASP performance measures should be reviewed and 

adjusted, as appropriate. 

Based on the contractor’s inconsistency for meeting CCC 

performance requirements, the Mint should periodically review and 

refine QASP performance requirements and work with the 

contractor to develop corrective action plans to improve 

performance and ensure quality customer service is maintained.

The absence of updated performance requirements and corrective 

action plans leaves the Mint’s ordering system at risk of not 

achieving its goals and limits Mint management’s ability to 

effectively monitor contractor performance. Since most of these 

measures require proper staffing to achieve, the Mint should revisit 

its staffing forecast procedures with the contractor as part of these 

corrective action plans.

Lack of Sufficient Documentation Supporting the Mint’s

Contract Oversight

We were unable to verify the extent to which the Mint performed 

monitoring activities to ensure the contractor was meeting required 

performance requirements. Although Mint employees told us they 

monitor the contractor’s performance, they could not provide

sufficient documentation to support the Mint’s contract monitoring 

activity results, including activities specified in the QASP. A Mint 

employee told us there was no direct access to all information 

maintained by the contractor, and therefore, Mint personnel relied

on contractor-provided summary reports to validate QASP-related

performance. Specifically, the Mint did not prepare and maintain

reports supporting the overall results of its quality assurance 

monitoring of the contractor’s performance.
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 12

Per the QASP, quality assurance monitoring is to be performed by 

designated Mint technical monitors and reported using agreed upon 

monitoring forms. The forms are to be used to document the 

government’s assessment of the contractor’s performance under 

the contact to ensure that required results are being achieved. The 

COR is to retain copies of all completed quality assurance

surveillance forms. The QASP states that at the end of each 

month, technical monitors are to prepare a written report for the 

COR summarizing the overall results of the quality assurance 

monitoring of the contractor’s performance. This written report 

along with the contractor’s submitted weekly and monthly reports

and quality assurance monitoring forms are to become part of the 

quality assurance documentation.

We also noted that the Mint did not maintain documentation 

supporting the validation and approval of contractor invoices. The 

Mint COR told us that they do not maintain supporting 

documentation once the invoice has been approved. The only 

document maintained is a monthly email from the designated 

approver to the COR stating that they approve or disapprove of the 

totals on the invoice.

In addition, the Mint did not validate contractor performance 

reporting to raw source data. The Mint could not provide 

performance data to support the QASP performance results,

instead, we had to request that data directly from the contractor.

We also compared contractor performance reporting to the 

contractor provided supporting data and were able to validate most

performance results to supporting documentation. However, the 

contractor could not provide data supporting past performance of 

the agent utilization performance measure because they did not 

maintain and could not reproduce reports for the time period 

requested. We also noted that the contractor was not 

systematically tracking receiving and returns processing 

performance measures as required by the QASP, and its calculation 

of the shipping accuracy performance measure was not correct.

A Mint employee told us that the Mint was at a disadvantage 

because it does not receive raw data from the contractor, only 

contractor-produced reports. The Mint’s COR coordinates with 

specialists and managers from different operational areas to review 
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 13

performance results. This group looks at the performance results

summarized by the contractor in the monthly QASP report to see if 

it is consistent with daily figures. The Mint COR explained that 

these individuals look at the data on a daily basis and should know 

if the monthly performance results are accurate. 

The Government Accountability Office’s (GAO) Standards for 

Internal Control in the Federal Government,

14 states 

Management may engage external parties to perform certain 

operational processes for the entity, however, management 

retains responsibility for the performance of processes assigned 

to service organizations and needs to understand the controls 

each service organization has designed, implemented, and 

operates for the assigned operational process. 

Office of Management and Budget (OMB) Circular A-12315 states 

Management’s internal control considerations should include 

responsibility for the performance of third-party provided 

processes, establishing ‘user controls’, and service organization 

oversight. If the processes provided by the third-party service 

organization is significant to an Agency’s internal control 

objectives, then the Agency is responsible for establishing user 

controls that complement the service organization’s controls. 

Management retains overall responsibility and accountability for 

all controls related to the processes and must monitor the 

process as a whole to make sure it is effective.

Per OMB Circular A-123, user controls include:

 Input/output Controls – management must have access to the 

information processed by a service organization so that it can 

compare the data submitted by the service organization with 

reports or information received from the service organization.

 Performance Monitoring – Management must have a process for 

monitoring service level organization performance in relation to 

various metrics, typically defined in a service-level agreement.

 

14 GAO-14-704G, Standards for Internal Control in the Federal Government (September 2014).

15 OMB Circular No. A-123, Management’s Responsibility for Enterprise Risk Management and Internal 

Control (July 15, 2016).
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 14

 Process controls – user controls are closely tied to the service 

organizations processes and provide direct assurance over their 

operation. 

Although the Mint has processes in place for monitoring the 

contractor’s performance through the establishment of the QASP, 

the Mint should retain required documentation to support its 

contract monitoring. Without adequate documentation of its 

monitoring of the contractor’s performance, the Mint is at potential 

risk for not ensuring that oversight measures are being properly 

performed and required performance standards or service levels are 

achieved by the contractor. Supporting documentation retained 

should include obtaining information processed by the contractor to 

validate the data in the summary reports provided. 

GAO’s Standards for Internal Control in the Federal Government 

identifies the appropriate documentation of transactions and 

internal control as a common control activity category.16 The 

standard states:

Management clearly documents internal control and all 

transactions and other significant events in a manner that 

allows the documentation to be readily available for 

examination... Documentation and records are properly 

managed and maintained.

The Mint Did Not Identify Loss of the Contractor as a 

Risk to the Numismatic Program

As of the time of our audit fieldwork, the Mint did not have a 

formalized risk management program in place. In May 2017, the 

Mint initiated a risk management program and was in the process 

of forming a Concept of Operations document to capture details of 

the program and charters for two internal risk management 

governance bodies—the Risk Management Committee and the Risk 

Officer Community of Practice. As of June 2019, the Mint was in 

the process of hiring a Chief Risk Officer, finalizing the fiscal year 

2019 U.S. Mint Profile and Risk Appetite, and finalizing the 

Enterprise Risk Management Concept of Operations.

 

16 Management designs control activities in response to the entity’s objectives and risks to achieve an 

effective internal control system. 
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 15

As part of the risk management program, the Mint conducted a 

top-down risk assessment to create a risk profile. The Mint 

provided us with a list of strategic risks associated with its risk 

profile; however, the loss of the contractor was not identified as a

risk to the numismatic program.

GAO’s Standards for Internal Control in the Federal Government 

states “management should define objectives clearly to enable the 

identification of risks and define risk tolerances.” Further, 

“management should identify, analyze, and respond to risks related 

to achieving the defined objectives.” OMB Circular A-123 also 

requires agencies to complete a risk profile by June 2017 and to 

update that risk profile every June as part of its implementation of 

Enterprise Risk Management.17

Mint officials told us that they monitor the contractor’s ability to 

meet its contractual obligations and maintain a database of 

customer information that would help expedite the migration of 

work to an alternate vendor, if necessary. In the event the vendor 

could not continue to provide service to the Mint, the Mint may not 

be able to ensure the continuity of its numismatic program since it 

relies on the contractor to provide most of the order management 

services. We believe that the Mint should include the loss of the 

current numismatic contractor as part of its finalized risk 

management plan.

Recommendations

We recommend that the Mint Director ensures the Numismatic and 

Bullion Directorate:

1. Periodically reviews QASP performance measures to determine 

necessary adjustments and develop corrective action plans, as 

required by the QASP, with the contractor, where needed.

Management Response

Management concurred with our recommendation. Management 

stated that it has developed a business dashboard consisting of 

 

17 The primary purpose of a risk profile is to provide a thoughtful analysis of the risks an agency faces 

toward achieving its strategic objectives arising from its activities and operations and to identify 

appropriate options for addressing significant risks.
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 16

a robust set of variables and standards that are tracked by both 

the SAM group and the Mint’s OMS-II contractor on a daily

basis. The Mint believes these variables give the Mint a more 

reliable depiction of CCC and fulfillment operations. The 

leadership team of the Mint’s SAM Directorate monitors this 

data daily and meets monthly to discuss trends and impacts to 

the business. 

Management further stated that, on a quarterly basis, the Mint

conducts a formal business review with the contractor of OMSII, which consists of more data elements than the QASP. The 

data are reported on current terms and compared to historical 

data points to identify trends. During the review, the Mint and 

PFS collaborate on additional business standards or data points 

that need to be monitored. These steps are taken to ensure the

Mint and PFS are reacting to the needs of the Mint and 

stakeholders in a cost effective and time effective manner. 

Management also stated that changes to the QASP would 

require the Mint to formally negotiate terms of the contract and 

would be cost prohibitive and time consuming; and therefore, 

management will address changes to the QASP during the 

contract option year's negotiations. The Mint has also 

established an executive steering committee to evaluate its

current vendor and future needs during the option years on the 

existing contract.

OIG Comment

Management’s response and corrective actions, planned and 

taken, meet the intent of our recommendation. Management 

will need to record the estimated dates for completing its 

planned corrective actions as well as the actual dates of 

completed corrective actions in JAMES, Treasury’s audit 

recommendation tracking system.

2. Retains evidence of its monitoring activities outlined in the 

contract’s QASP to better document the Mint’s oversight of the 

contractor’s compliance with contract requirements. 

Management Response

Management concurred with the recommendation and stated 

that there have been on-going changes to monitoring activities. 
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 17

Management stated that the Mint has evidence and records 

showing that it monitors the CCC and performance related to 

customer service weekly and monthly, and the overall QASP is 

reviewed and retained on file monthly. In addition to these 

monitoring methods, the Mint conducts quality oversight 

through semi-annual CCC agent training. To improve employee 

awareness of the Mint's efforts, the Division Chief of Sales 

plans to conduct monthly reviews of all QASP elements, as well 

as the business dashboard, with each Division Chief and Branch 

manager. 

OIG Comment

Management’s response and planned corrective action meet the 

intent of our recommendation. While we did find evidence of 

some of the Mint’s monitoring activities mentioned in 

management’s response, the Mint did not provide evidence of a 

monthly written report submitted to the COR documenting the 

quality assurance monitoring of the contractor’s performance as 

required by the QASP. Similar to its retention of evidence and 

records of certain monitoring activities, the Mint needs to 

ensure that this monthly written report is prepared and 

submitted to the COR and retained as evidence in the records of 

the Mint’s monitoring activities. Management will need to 

record the estimated dates for completing its planned corrective 

actions as well as the actual dates of completed corrective 

actions in JAMES, Treasury’s audit recommendation tracking 

system.

3. Obtains and validates contractor’s supporting documentation 

against monthly performance reports and retains this as part of 

the Mint’s monthly monitoring documentation.

Management Response

Management concurred with the recommendation and

acknowledged that the Mint could have improved validating the 

contractor's documentation of performance against raw data. 

Management stated that the Mint has gone to great lengths to 

ensure customer service is monitored independent of the 

contractor and that records are kept on file. The Mint plans to

assess the feasibility of developing a system to capture and 
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 18

store the raw data elements, or of contracting a third party to 

conduct additional surveillance and/or audits.

OIG Comment

Management’s response and planned corrective actions meet 

the intent of our recommendation. Management will need to 

record the estimated dates for completing its planned corrective 

actions as well as the actual dates of completed corrective 

actions in JAMES, Treasury’s audit recommendation tracking 

system.

4. Assesses the need to include, the loss of the OMS-II contractor 

as a potential risk to the numismatic program as part of the 

Mint’s Risk Management Program.

Management Response

Management concurred with the recommendation and stated 

that the SAM Directorate identified a single source contractor as 

a risk in its risk inventory in 2018. The Mint is currently 

evaluating available courses of action to mitigate this risk.

OIG Comment

Management’s response and corrective actions, planned and 

taken, meet the intent of our recommendation. Management 

will need to record the estimated date for completing its 

planned corrective actions as well as the actual date of 

completed corrective actions in JAMES, Treasury’s audit 

recommendation tracking system.

Finding 2 OMS-II Is Working as Intended and Meeting User Needs

We concluded that the Mint OMS-II is working as intended based 

on the system meeting most of its performance goals and on our 

analysis of OMS-II numismatic order processing data. OMS-II is

meeting the needs of a majority of its users according to customer 

satisfaction surveys and our review of online customer service 

conversations. However, the Mint’s website and CCC are only 

offered in English. Additionally, the Mint has not met targeted unit 
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 19

sales goals for fiscal years 2015 through 2017, resulting in 

declining revenues. 

OMS-II is Working as Intended

OMS-II was intended to meet the Mint’s strategic goals of 

effectively managing bullion and numismatic programs and 

responsibly expanding the numismatic program. To meet its goals, 

the Mint contracted for a solution that uses existing online retail 

technology while providing reasonable levels of functionality. It is 

also not overly complex and can be used by the Mint and its 

employees. 

The Mint’s 2016 through 2018 Capital Investment Plans included, 

among others, the following metrics used to measure the success 

of the OMS-II:

 average time to ship orders,

 maintenance of eCommerce Website uptime, and

 bimonthly customer satisfaction measure from tracking survey.

The customer satisfaction metric and numismatic sales units are 

also used as performance metrics to gauge the Mint’s progress in 

achieving its strategic goals. Additionally, as previously mentioned,

the OMS-II contract established performance requirements. 

Table 1 below provides a summary of the performance 

requirements and the percentage of months the contractor met 

these performance requirements from October 2014 through 

September 2017.

As shown in Table 1, the contractor is generally meeting 

established performance requirements. The Mint’s website and 

order management systems have performed well since being 

implemented in October 2014, maintaining required uptime during 

the 3-year review period.18 The website is meeting its customer 

satisfaction goal and has only fallen under the required 

performance requirement twice since the measure started being 

tracked in March 2016. The contractor’s distribution center has 

 

18 Up-time measures the percentage of time the system is operational net of any scheduled systems 

maintenance.
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 20

also been excelling at meeting order fulfillment goals, while 

maintaining shipping accuracy and shipping priority and nonpriority 

shipments in a timely manner.19,20

 

19 Shipping Accuracy measures the accuracy of customer shipments and is defined as customers 

receiving the correct products and the correct number of units for their order.

20 The QASP requires that all priority orders be shipped on-time 99.5 percent of the time. On-time 

shipments for priority orders are considered to be on the same day if received prior to the 12:00 PM 

Eastern Standard Time cutoff or the next day if not received prior to that time. On-time shipments for 

non-priority orders are considered within one business day with a goal of meeting the target 99 

percent of the time.
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 21

Table 1: QASP Performance Requirements and Results

Source: QASP Workbooks (October 2014 through September 2017).

Area Measure

Measurement 

Period Target

% of 

Measurement 

Periods Met

Average Speed to Answer* Monthly ≤ 20 Seconds 56%

Agent Utilization* Monthly ≥ 65% 39%

Call - Customer Satisfaction 

Survey* Monthly ≥ 90% 0%

Web - Customer Satisfaction 

Survey* Monthly ≥ 75% 89%

Peak Day Performance * Peak Day(s) All 3 SLAs^ 45%

CSR Quality Score Monthly ≥ 90% 75%

Service Level (Answered witihin 

20 seconds) Monthly ≥ 80% 92%

Abandoned Call Rate Monthly ≤ 5% 92%

One Call Resolution (Resolved in 

1 call) Monthly ≥ 85% 94%

Email and Response Time (1 

business day) Monthly ≥ 99% 17%

On-Time Shipments - Priority 

(Same or next business day) * Monthly ≥ 99.5% 72%

On-Time Shipments - Non Priority 

(1 business day)* Monthly ≥ 99% 86%

Shipping Accuracy Monthly ≥ 99.5% 91%

Receiving Turnaround (1 business 

day) Monthly ≥ 99% 100%

Returns Processing (2 business 

days) Monthly 100% 94%

Freight Management - Audit 

Accuracy Monthly > 99% 91%

OMS Up-Time* Monthly ≥ 99.5% 100%

Web - Up-Time * Monthly ≥ 99.5% 100%

Incident Response on Critical 

Incidents (within 1 hour) Monthly ≥ 95% 100%

Load Test Certification Annually Pass/Fail 100%

Call Center

Fulfillment

Systems

^The Peak Day Performance Measure requires the contractor to meet three all service level 

agreements (SLA). The SLAs include maintaining an average answering speed of 20 seconds 

or less, answering 80% of calls in 20 seconds or less, and maintaining an abandoned call rate 

of less than 5%.

*Indicates a incentives/disincentive performance measure.
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 22

In Finding 1, we found the contractor has not consistently met 

agent utilization, peak day performance, email response, and call 

center customer satisfaction performance goals, however, the CCC 

is doing well in meeting other established performance goals. The 

contractor is meeting its goal of 80 percent of calls answered in 20 

seconds or less, having only missed this measure three times over 

the three-year period—all occurred within the first 4 months of 

operation. CCC has also improved its average answering speed 

over the life of the contract. CCC struggled with meeting its goal 

of average answering speed of 20 seconds or less during the first 

18 months of the contract, only meeting it five times. However, 

CCC showed significant improvement in meeting this measure 

during the period of April 2016 through September 2017, meeting 

the target 15 times. CCC has also successfully met its abandoned 

call rate goal by maintaining an average abandoned call percentage 

of only 2 percent over the reviewed period.21

In addition, we performed analytical tests of OMS-II sales order, 

credit card settlement, and shipping data we received for the 

month of September 2017 and found no anomalies.

22 Online, mail, 

and subscription transactions occurring within the audit period 

were traced through the OMS-II system data to determine if all 

orders processed and invoiced were settled by the customer and 

shipped from the warehouse; finding no significant exceptions.

Based on our review of OMS-II data, we found the system was 

meeting its purpose of accurately processing and fulfilling orders. 

OMS-II Is Meeting Most User Needs

Our review of customer satisfaction survey results and customer 

chat transcripts indicate that OMS-II is meeting the needs of the 

majority of its users. The Mint uses contractors to conduct 

customer satisfaction surveys for users of both CCC and the 

website.23 Mint officials told us that customer satisfaction surveys 

were the Mint’s primary means to determine customer satisfaction. 

 

21 Abandoned call rate is a measure of how frequently callers disconnect before they are serviced by a 

customer service representative (CSR). 

22 Anomalies included testing for zeros and blank fields when not appropriate, looking for outliers within 

the data, and testing for duplicate entries.

23 Desktop and mobile website surveys are conducted by Foresee and CCC surveys are conducted by 

the Claes Fornell International Group, both customer experience consultants.
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 23

As discussed in Finding 1, the QASP performance requirement for 

CCC customer satisfaction was never met by the contractor. Our

review of the customer satisfaction surveys from October 2014

through September 2017 found that, overall, customers are 

satisfied with the call center and there has been an upward trend in 

most performance categories over the 3-year period reviewed. 

The survey uses the American Customer Satisfaction Index 

methodology to measure overall customer satisfaction. Although 

they never met the 90 percent CCC customer satisfaction 

requirement established in the QASP, the CCC average customer 

satisfaction index of 76 is above private and public sector contact

center benchmarks of 68.24 CSRs have the largest impact on 

customer satisfaction and they are performing well above public 

and private sector industry benchmarks.25 Based on these results, 

customers were satisfied with the service they were receiving from 

the CSRs and the improvements to call waiting times.

Customer satisfaction survey results show that desktop and mobile 

users are overall satisfied with the website, based on the upward 

trends in satisfaction of all measured categories of the website

from the beginning to the end of the 3-year period reviewed. The 

Mint’s overall customer satisfaction measure has performed slightly 

above average for government desktop websites. 

We received approximately 66,000 CCC chat transcripts from 

fiscal year 2017. Because of the large volume of transcripts, we 

searched key words to filter through the transcripts for potential 

issues. The purpose of this review was to identify issues 

customers might be experiencing with OMS-II or the numismatic 

program, as well as to determine how those issues were 

addressed. We found no significant systemic issues with order 

processing or order fulfillment. In addition, CSRs generally handled 

chats properly resulting in customers being mostly satisfied with 

 

24 Private Sector benchmark is from the Contact Center Satisfaction Index and the Public Sector 

benchmark is from the Government Contact Center Satisfaction Index (GCCSI) issued from Claes 

Fornell International Group as of September 2017.

25 CSRs are rated on their knowledge, courteousness, communication skills, and ability to resolve 

issues. CSRs averaged a score of 88 for Knowledge, 92 for Courteousness, and 90 for 

Communication Skills during the three years reviewed. Public and private sectors scored 72 and 79, 

respectively, for Knowledge; 78 and 83, respectively, for Courteousness; and 77 and 82, 

respectively, for Communication Skills.
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 24

the service provided. Some of the most common issues identified 

in the chats were lost/forgotten password, sold out/household 

limits, damaged or missing products requiring a certificate of nonreceipt, and problems completing orders due to customer internet 

browser compatibility. 

However, we found that the Mint’s website and CCC are only 

offered in English to customers. Mint officials told us that most of 

the Mint’s user base is English speaking and internet browsers can 

translate the website into another language, if needed. With that 

said, as of 2017, the number of people in the United States 5 

years and older who spoke a language other than English neared 

22 percent.26 Of those, 39 percent spoke English less than “very 

well.” The Mint had not pursued adding foreign languages to its 

business solution because only a limited portion of its business 

comes from international sales. The Mint also had not performed 

any studies into offering its services in other languages. 

Federal agencies are required to take reasonable steps to provide 

meaningful access for Limited English Proficient individuals to 

federally conducted programs.27 Also, according to Guideline 7 of 

the “Guidelines for Improving Digital Services,” agencies are to 

ensure that digital services are easy to use and accessible, 

including for people with disabilities and those who are not 

proficient in English.28

We believe that the Mint should consider the feasibility of offering 

language services as part of its numismatic program based on the 

growing population of individuals who speak a language other than 

English and the requirement that all federal agencies must take 

reasonable steps to accommodate limited English proficient 

individuals. 

 

26 U.S. Census Bureau, 2017 American Community Survey 1-Year Estimates.

27 Executive Order 13166, Improving Access to Services For Persons With Limited English Proficiency

(August 11, 2000)

28 Digital Government: Building a 21st Century Platform to Better Serve the American People (May 23, 

2012) states that agencies are to ensure all new digital services follow digital services and customer 

experience improvement guidelines developed by Federal interagency groups. 
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 25

Numismatic Sales Have Not Expanded

The goal of OMS-II per the Mint’s Capital Investment Plan was to 

responsibly expand the numismatic program; however, numismatic 

sales since the inception of the contract reveal that the Mint is not 

meeting its units sold goal. Specifically, the numismatic sales unit 

metric, established as part of the Mint’s annual performance plan,

measures public demand for coin products sold from numismatic 

operations. Per the metric, as shown in Table 2, from fiscal years

2015 through 2017 the Mint missed its target unit sales goals—

during the same time target unit sales also decreased. For fiscal 

year 2018, the Mint target goal dropped to 3.5 million units.

Table 2: Numismatic Sales Units Fiscal Years 2013 – 2017

Numismatic Sales 

Units (In Millions) 2013 2014 2015 2016 2017

Actual Units Sold 5.5 5.7 5.4 4.2 3.9

Target Units Sold 5.2 5.4 5.6 4.6 4.4

Source: Excerpts from fiscal year 2013 through fiscal year 2017 President’s Budget and

fiscal years 2018 and 2019 Mint Congressional Justification and Annual Performance 

Report and Plan.

The Government Performance and Results Act (GPRA) 

Modernization Act of 2010 requires agencies to establish 

performance goals to define the level of performance to be 

achieved during the year as part of its annual performance plan.29

Each year the agency shall make available to the public an update 

on agency performance including a comparison of actual 

performance achieved with the performance goals established in 

the agency performance plan. Each update shall explain and 

describe where a performance goal has not been met, why the goal 

was not met, and the plans for achieving the established 

performance goal.

 

29 P.L. 111-352 The GPRA Modernization Act of 2010 (January 4, 2011).
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 26

As shown in the Table 3 below, lower unit sales resulted in lower 

revenues and income for the Mint numismatic program. From fiscal 

years 2013 to 2017, revenues decreased from approximately 

$512 million to $387 million (24 percent), while income decreased 

from approximately $96 million to $19 million (80 percent) during 

the same time. Numismatic revenues are projected to decrease in 

fiscal years 2018 and 2019 to $327 million and $305 million, 

respectively, due to decreased demand for numismatic products. 

Table 3: Numismatic Sales Fiscal Years 2013 – 2017

Numismatic 

Sales

(In 

thousands) 2013 2014 2015 2016 2017

Units Sold 5,509 5,725 5,379 4,159 3,905

Revenue $512,400 $504,500 $453,200 $413,000 $387,500

Net Income $95,800 $50,800 $66,800 $34,500 $19,200

Source: 2017 Mint Annual Report.

Along with lower profits, the numismatic customer base was also

steadily declining over the last few years. According to the 

contractor, in fiscal year 2013, Mint numismatic unique purchasers 

totaled over 575,000 customers. In fiscal year 2017, the number 

of unique purchasers fell to about 459,000 (20 percent). 

In its fiscal year 2017 annual report, the Mint attributed the lower 

than anticipated results in fiscal year 2017 on a shortfall in sales of 

annual precious metal coins because of the overabundance of coin 

products available on the secondary market, and the conclusion of 

the Presidential $1 Coin program. The Mint also attributed the 

shortfall to a lack of enthusiasm for commemorative coin offerings 

in fiscal year 2017.

Mint officials told us that the Mint is limited on the numismatic 

products it can offer to the public, and that they need 

Congressional approval and direction for new numismatic program

offerings. In its 2018 risk profile, the Mint identified the need to 
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 27

clearly articulate its brand and enhance the marketing of its 

products or be at risk of revenue from numismatic production being 

negatively impacted. At the time of our audit, the Mint was 

considering proposed actions to improve numismatic sales. 

Recommendations

We recommend that the Mint Director ensures the Numismatic and 

Bullion Directorate:

5. Performs an assessment regarding the impact of not having 

language accessibility for Mint’s numismatic program services and 

considers adding these services to OMS-II.

Management Response

Management concurred with the recommendation and stated that 

Executive Order 13166 requires agencies to assess the costs and 

benefits of providing access in additional languages. The Mint 

explored this with the contractor during initial implementation of 

OMS II. At that time, the Mint determined that it would not be

economically feasible nor cost effective to do so. Management 

further stated that, in the next 12 months, the SAM Directorate 

and the Equal Employment Opportunity Office will reassess this 

conclusion based on the current environment.

OIG Comment

Management’s response and planned corrective actions meet the 

intent of our recommendation. Management will need to record the 

estimated date for completing its planned corrective actions as well 

as the actual date of completed corrective actions in JAMES,

Treasury’s audit recommendation tracking system.

6. Performs analyses to determine the feasibility and potential impact 

of proposed actions to improve numismatic sales, performs 

additional studies to enhance future sales, and reports the results

to stakeholders.

Management Response

Management concurred with the recommendation. Potential 

strategies currently under evaluation include an improved loyalty 
The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 28

program, systems upgrades to allow for prepayment and embargo 

options for bulk customers, and alternative distribution channels 

that allow for greater reach and pricing flexibility. Management 

stated that the SAM Directorate has performed and continues to 

perform formal and informal analysis of business opportunities to 

grow the market share, gross revenue, and net income of the 

Mint's numismatic program. 

OIG Comment

Management’s response and corrective actions, planned and taken, 

meet the intent of our recommendation. Management will need to 

record the estimated dates for completing its planned corrective 

actions as well as the actual dates of completed corrective actions 

in JAMES, Treasury’s audit recommendation tracking system.

* * * * * * 

We appreciate the cooperation and courtesies extended to our staff 

during the audit. Major contributors to this report are listed in 

appendix 3. If you wish to discuss the report, you may contact me 

at (617) 223-8638 or Mark Ossinger, Audit Manager, at (617) 

223-8643. 

/s/ 

Sharon Torosian 

Audit Director, Manufacturing and Revenue 

Appendices 
Appendix 1: Objectives, Scope, and Methodology

The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 29

The objective of our audit was to determine whether the United 

States Mint (Mint) is effectively managing order processing for 

numismatic products, such as precious metal coins, 

commemorative coins, and medals for sale to the public. As 

part of this work, we planned to determine if the Mint’s order 

management system (OMS-II) is working as intended and the 

numismatic program is meeting the needs of its users. The 

scope of our audit covered from the launch of OMS-II in 

October 2014 through September 2017. We conducted our 

fieldwork from June 2017 through November 2018. We 

conducted additional follow-up with the Mint in June 2019.

To accomplish our objective, we interviewed Mint officials and 

staff at Mint’s Headquarters in Washington, DC and conducted 

tours of the Mint’s Philadelphia, Pennsylvania manufacturing 

facility. External to the Mint, we interviewed program 

management and staff from the Mint’s numismatic contractor. 

We also conducted tours of the contractor’s Customer Contact 

Center (CCC) in Dallas, Texas and its Distribution Center in 

Memphis, Tennessee. 

At the Mint Headquarters, we interviewed officials responsible 

for oversight of the Mint numismatic program and oversight of 

the OMS-II contractor to help gain an understanding of the 

Mint’s numismatic program order management system, as well 

as to determine how they oversee the Mint’s OMS-II contractor.

At the Mint’s Philadelphia, Pennsylvania manufacturing facility, 

we interviewed officials to obtain knowledge on how 

numismatic products are produced and the process for sending 

finished products to the numismatic contractor.

External to the Mint, we interviewed officials from the Mint’s

OMS-II contractor to gain an understanding of how they operate 

the OMS-II.

We reviewed applicable Treasury orders and directives, past 

Department of the Treasury (Treasury) Office of Inspector 

General (OIG) reports, applicable laws and regulations, and 

government-wide guidance applicable to the Mint’s numismatic 

program, including:
Appendix 1: Objectives, Scope, and Methodology

The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 30

 Office of Management and Budget Circular A-123 

Management’s Responsibility for Enterprise Risk 

Management and Internal Control

 Government Accountability Office Standards for Internal 

Control in the Federal Government (Green Book)

 Federal Acquisition Regulations

 Executive Order 13166, Improving Access to Services For 

Persons With Limited English Proficiency 

 Digital Government: Building a 21st Century Platform to 

Better Serve the American People

Through our interviews with Mint and contractor officials to 

gain an understanding of the numismatic program and the 

ordering processes, our walkthroughs of the contractor’s CCC 

and distribution center to gain an understanding of the 

contractor’s processes, and review of the Mint’s and 

contractor’s policies and procedures, we documented our 

understanding of the processes and controls over numismatic 

product order management and program oversight using flow 

charts. To validate our flow charts, we provided the flowcharts 

to the Mint and contractor to review and provide comments. A 

risk assessment was created to determine the processes and 

controls over the numismatic product ordering and program 

oversight that presented a high risk to the numismatic program. 

High risk process and oversight level controls were assessed 

further to determine if they were being implemented and were 

adequate. Although we identified physical security as a high-risk 

process-level risk, we did not assess the contractor’s physical 

security, which was outside the scope of the audit.

We reviewed the adequacy of established numismatic program 

performance measures against industry standards to determine

if the Mint contractor was meeting those established 

performance measures. 

We performed tests on transaction data from the Mint’s Data 

Warehouse and contractor data using data analysis software to 

determine if the system was working as intended and to 

determine the reliability of the system data. We requested and 

obtained sales order, credit card settlement, and shipping data 
Appendix 1: Objectives, Scope, and Methodology

The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 31

from the OMS-II for the month of September 2017, which was 

the most recent month in our audit period. To assess the 

integrity of key system data, we looked for anomalies in the 

data. This included reviewing each field in the data sets to 

ensure they met the following requirements: 

 Correct type of data (Character/Numeric),

 Presence (testing for zeros and blanks when not 

appropriate),

 Length (File was delineated based on established 

character length in the data definitions. Issues would be

present if field went over this length),

 Reasonableness (looking for outlier entries in the data), 

and

 Ensuring other requirements of the field established in the 

data definitions are met.

Within each data set, we performed the following:

 Tested for duplicate entries,

 Tested to ensure products ordered matched products 

shipped/returned,

 Tested to ensure the total price equaled the unit price 

multiplied by the units ordered and the price stays the 

same throughout the transaction,

 Tested the dates to ensure they occurred in the correct 

order, 

 Tested to ensure each record had a complete record as 

applicable, 

 Tested to ensure that all invoiced transactions had proper 

shipping identification fields.

In addition, we traced online, mail, and subscription transactions 

occurring through the OMS-II system from initial order through 

shipment to the customer. This included determining that orders 

that were processed and invoiced were ultimately paid (also 
Appendix 1: Objectives, Scope, and Methodology

The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 32

referred to as settled) by the customer and shipped from the 

warehouse. We performed the following:

 Compared products purchased to those in the active 

inventory to ensure they were valid,

 Compared the sales transaction amount to the settlement 

amount to ensure the customer was charged the correct 

amount, 

 Identified all invoiced orders from the sales transaction 

data to validate if the shipping data used to support the 

contractor’s invoice was correct,

 Identified all invoiced sales returns from the sales 

transaction data to validate if the returns data used to 

support the contractor’s invoice was correct,

 Identified all invoiced orders from the sales transaction 

data to validate if the data used to calculate QASP ontime shipping performance results was correct,

 Used sales return and appeasement data to validate if the 

data used to calculate the QASP shipping accuracy 

performance results was correct.

30

We also reviewed orders placed on hold to ensure they were 

eventually processed and we reviewed customer exchanges to 

ensure returned items were properly exchanged and returned to 

customers.

To determine if the system is meeting user needs, we reviewed 

customer satisfaction surveys, reviewed Treasury OIG Office of 

Investigation complaints regarding the Mint numismatic 

program, and reached out to two numismatic associations for

feedback on the Mint’s numismatic ordering processes. 

We also reviewed transcripts of the CCC’s online conversations 

with users referred to as chats. We received approximately 

66,000 CCC chat transcripts from calendar year 2017. Because 

of the large volume of transcripts, we searched key words to 

 

30 Customer Service Agents have the ability to refund the original shipping cost on original orders as 

a way to appease and satisfy customers.
Appendix 1: Objectives, Scope, and Methodology

The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 33

filter through the transcripts for potential issues. Examples of 

key words included broken, damaged, dissatisfied, horrible, 

issue, lost, missing, overcharge, problem, quality, terrible, 

worst, and wrong product.

We reviewed the transcripts to identify issues customers might 

be experiencing with OMS-II or the numismatic program, as well 

as to determine how the Mint addressed those issues.

We conducted this performance audit in accordance with 

generally accepted government auditing standards. Those 

standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis 

for our findings and conclusions based on our audit objectives. 

We believe that the evidence obtained provides a reasonable 

basis for our findings and conclusions based on our audit 

objectives.
Appendix 2: Management Response 

The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 34
Appendix 2: Management Response

The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 35
Appendix 2: Management Response

The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 36
Appendix 2: Management Response

The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 37
Appendix 3: Major Contributors to This Report 

The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 38

Mark Ossinger, Audit Manager

Patrick Arnold, Auditor-in-Charge

Timothy Cargill, Auditor

Justin Bruner, Audit

Jeffrey Dye, Referencer
Appendix 4: Report Distribution 

The United States Mint Numismatic Order Management System is 

Meeting User Needs But Improvements to Oversight are Needed 

(OIG-20-030) 39

Department of the Treasury

Secretary

Deputy Secretary 

Treasurer

Office of Strategic Planning and Performance Improvement

Office of the Deputy Chief Financial Officer, Risk and Control 

Group

United States Mint

Director

Audit Liaison

Office of Management and Budget

Office of Inspector General Budget Examiner

U.S. Senate

Chairman and Ranking Member

Committee on Banking, Housing and Urban Affairs

U.S. House of Representatives

Chairman and Ranking Member

Committee on Financial Services
This Page Intentionally Left Blank
REPORT WASTE, FRAUD, AND ABUSE

Treasury OIG Hotline: 1-800-359-3898

Hotline@oig.treas.gov

Gulf Coast Restoration Hotline: 1-855-584.GULF (4853)

gulfcoastrestorationhotline@oig.treas.gov

Access Treasury OIG reports and other information online: 

www.treasury.gov/about/organizational-structure/ig