UNITED STATES PATENT 

AND TRADEMARK 

OFFICE 

The Patent Hoteling 

Program Is Succeeding 

as a Business Strategy 

FINAL REPORT NO. OIG-12-018-A 

FEBRUARY 1, 2012 

U.S. Department of Commerce 

Office of Inspector General 

Office of Audit and Evaluation 

FOR PUBLIC RELEASE 
UNITED STATES DEPARTMENT OF COMMERCE 

Office of Inspector General 

Washington, D.C. 20230 

February I , 20 12 

MEMORANDUM FOR: David Kappas 

Under Secretary of Commerce for Intellectual Property 

and Director of the U.S. Patent and Trademark Office 

FROM: Ron Prevost 

Assistant Inspector ~;;r;r"Eco~omic and Statistical Program Assessment 

~ 

SUBJECT: The Patent Hoteling Program Is Succeeding as a Business Strategy 

Final Report No. OIG-12-0 18-A 

We have attached our final report related to our audit of USPTO's Patent Hoteling Program 

(PHP). This audit, part of the Office of Inspector General's Fiscal Year 20 I I audit plan, 

determined how USPTO measures productivity for its PHP participants and whether USPTO's 

data indicate improvement in productivity; the extent that USPTO has achieved its stated cost 

savings for the PHP program, including real estate savings; and the extent that USPTO policies 

and their implementation provide adequate management controls over the PHP. Our report 

presents the findings and recommendations of this audit, conducted under the authority of the 

Inspector General Act of 1978, as amended, and Department Organization Order I 0-13, dated 

August 3 I , 2006. 

We found that PHP participants review 3.5 more patent applications per year than their 

nonparticipating (but eligible) counterparts, and we attribute the additional output to PHP 

participants' allocating a greater proportion of their time to examining patents. We also found 

that while USPTO achieves real estate cost savings from PHP, it has not calculated more 

comprehensive savings and costs for the program. Our analysis showed that USPTO avoids 

costs of approximately $1,710 per PHP participant in the first year in the program and avoids 

costs of approximately $3,385 per PHP participant in each subsequent year, mainly due to real 

estate savings. We also found that the additional applications reviewed by each PHP participant 

can result in an additional $13,373 in expected future revenue for USPTO over a 14-year 

period. Finally, we found that USPTO's policies for managing PHP comply with applicable 

telework laws and provide reasonable controls and assurances that the program operates 

effectively and efficiently. 

Our report includes three recommendations to USPTO to (I) conduct a more comprehensive 

calculation of costs and savings from PHP; (2) work with the Department of Commerce to 

compare best practices from PHP that could aid Department telework programs; and (3) 

ensure that internal controls are in place to allow only eligible patent examiners to participate 

in PHP and documentation is maintained. 
Your January 18, 20 12, response concurs with our draft report's recommendations and outlines 

steps USPTO is taking to address these issues. We also received verbal technical comments 

and made changes as appropriate. In accordance with Department Administrative Order 213-5, 

within 60 days of the date of this memorandum, please provide us with an action plan that 

responds to all of the report recommendations. 

We thank USPTO personnel for the assistance and courtesies extended to my staff during the 

review. If you have any further questions or comments about the report, please feel free to 

contact me at (202) 482-3052 or Jill Schamberger, Project Manager, at (571) 272-5561. 

Attachment 

cc: 	 Margaret Focarino, Commissioner for Patents, USPTO 

Patricia M. Richter, Chief Administrative Officer, USPTO 

Bo Bounkong, Associate Commissioner for Patent Resources and Planning, USPTO 

Frederick R. Schmidt, Associate Commissioner for Patent Information Management, 

USPTO 


Anthony P. Scardino, Chief Financial Officer, USPTO 


Frank Murphy, Deputy Chief Financial Officer, USPTO 


Welton Lloyd, Audit Liaison, Office of Planning and Budget, USPTO 

Report In Brief 

FEBRUARY 1, 2012 

Background 

The U.S. Patent and Trademark Office (USPTO) offers a 

variety of telework options 

for its employees—from 1 to 

5 days a week. The largest of 

these is the Patent Hoteling 

Program (PHP), where employees work from home at 

least 4 days a week and reserve office space for when 

they need to be present at 

USPTO. At the end of fiscal 

year 2011, more than 2,600 

patent examiners were participating in PHP. Goals of 

PHP include reducing space 

and cost requirements while 

improving employee retention and job satisfaction. 

Why We Did This Review 

PHP is considered one of the 

leading government telework 

programs, yet no comprehensive and independent review 

of the program’s costs and 

benefits has ever been completed. Because, by law, participation in telework programs should not result in 

diminished federal employee 

performance, we specifically 

looked at the productivity of 

PHP participants. Also, because USPTO’s management 

challenges include addressing 

the backlog of about 700,000 

patent applications, we 

sought to determine if PHP 

can help reduce this backlog 

in a cost-effective way. 

U.S. PATENT AND TRADEMARK OFFICE 

The Patent Hoteling Program Is Succeeding 

as a Business Strategy 

OIG-12-018-A 

WHAT WE FOUND 

PHP participants review more patent applications than do examiners 

working at headquarters. Although both groups review patent applications 

at the same rate, PHP participants spend more time examining 

applications because they use less sick and administrative leave and charge 

less time to administrative tasks. As a result, the average PHP participant 

spends 66.3 more hours a year examining patents than does the average 

in-house examiner; this translates to reviewing about 3.5 more patent 

applications a year. 

USPTO avoids real estate costs through PHP but has not calculated a 

comprehensive cost analysis of the program. Whereas USPTO reported 

that it avoids $15.88 million annually in real estate costs by having PHP, 

we estimated that this amount is approximately $16.84 million as a result 

of the program. 

Although PHP incurs additional costs, mainly for IT infrastructure and 

hoteling support services, these costs are significantly offset by avoided 

real estate costs as well as revenue generated from the additional patent 

applications reviewed. 

USPTO has adequate controls over the patent hoteling program in key 

areas; however, close to 2 percent of a random sample of participants 

lacked documentation to support eligibility in the program. 

WHAT WE RECOMMEND 

We recommend that the Under Secretary of Commerce for Intellectual 

Property and Director, USPTO: 

1. Conduct a more comprehensive calculation for costs and cost 

avoidance related to PHP in order to obtain more accurate estimates of 

the cost and benefits affiliated with this program. 

2. Work with the Department of Commerce to compare best practices 

from the Patent Hoteling Program, which could aid telework programs 

within the rest of the Department. 

3. Ensure that internal controls are in place so that only eligible patent 

examiners participate in PHP and appropriate documentation is 

maintained. 
         

       

           

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Contents 

Introduction ....................................................................................................................................................... 1
 

Findings and Recommendations .................................................................................................................... 3
 

1.	 PHP Participants Review More Patent Applications Than Do Examiners Working at 

 Headquarters, but There Are No Differences in Efficiency .......................................................3
 

A.	 PHP participants review more patent applications ...................................................................3
 

B.	 PHP participants are no more efficient at reviewing patent applications than their 

 counterparts who work at headquarters ................................................................................... 4
 

2.	 USPTO Avoids Real Estate Costs Through PHP but Has Not Calculated a 

 Comprehensive Cost Analysis for the Program ...........................................................................5
 

A.	 Avoided real estate costs understated........................................................................................5
 

B.	 Additional and avoided costs associated with PHP ..................................................................5
 

3.	 Reviewing Additional Applications Generates Revenue That Offsets PHP Costs ................6
 

4. USPTO Has Adequate Controls over the Patent Hoteling Program in Key Areas ..............7
 

Conclusion ..................................................................................................................................................... 8
 

Recommendations........................................................................................................................................ 8
 

Summary of Department and OIG Comments ...................................................................................... 10
 

Appendix A: Objectives, Scope, and Methodology ................................................................................ 11
 

Appendix B: Quantitative Methodology ................................................................................................... 13
 

Data sources used in both econometric models ............................................................................... 13
 

Econometric model specification and methodology: Do PHP participants spend more 

 time examining patents?.......................................................................................................................... 13
 

Econometric model specification and methodology: Are PHP participants more efficient? .... 16
 

Limitations of our econometric model and data................................................................................ 18
 

Appendix C: Response to OIG’s Draft Report ...................................................................................... 20
 

COVER: Detail of fisheries pediment,

U.S. Department of Commerce headquarters,
 

by sculptor James Earle Fraser, 1934
 

FINAL REPORT NO. OIG-12-018-A 
 

                                                           

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Introduction 

The U.S. Patent and Trademark Office’s (USPTO’s) telework programs are part of an agency 

business strategy to meet its mission of issuing patents and awarding trademarks. USPTO 

currently offers a variety of telework options for its employees—from 1 to 5 days a week. The 

largest of these telework programs is the Patent Hoteling Program (PHP), in which more than 

2,600 patent examiners were participating at the end of fiscal year 2011. Goals of PHP include 

reducing space and cost requirements associated with increased hiring, while improving 

employee retention and job satisfaction. 

PHP is voluntary for eligible, participating patent examiners. “Hoteling” is a USPTO provides equipment and remote access to all relevant telework option systems; in return, the patent examiners work from home at where employees least 4 days a week and relinquish their office space at USPTO work from home headquarters. Before participating, patent examiners must meet at least 4 days a certain requirements (see table 1) and complete a 1-week week and reserve training course. USPTO began PHP in January 2006 with 500 

GS-14 and GS-151

 patent examiners. Since then, USPTO office space 

expanded eligibility to include GS-12 and GS-13 patent electronically for 

examiners. Expansion of the program currently is limited—to when they need 

about 500 additional patent examiners each year—because of to be present at 

infrastructure constraints, such as having the requisite training USPTO. 

courses and ensuring system capacity. 

Table 1. PHP Participation Requirements 

GS-12 or above ‘Fully Successful’ or higher 

performance rating 

Not suspended for disciplinary or adverse 

personnel action within the last 12 months 

Worked at least 2 

years at USPTO 

Passed the Certification Exam, the 

Registration Exam, or have Partial 

or Full Signatory Authoritya 

Not currently under an oral or written warning 

for performance 

Source: USPTO 

a

A Certification Exam is taken before promotion to GS-13 to ensure that the examiner has the knowledge, 

skills, and abilities to perform to the standards of this grade level. A Registration Exam is administered to 

attorneys and agents; those who pass are allowed to represent applicants with prospective or immediate 

business before USPTO in the preparation and prosecution of patent applications. An examiner with permanent 

or temporary Partial Signatory Authority may sign nonofficial actions, while a primary examiner with Full 

Signatory Authority is delegated to represent the Commissioner and sign all actions, including allowances. 

1

The federal government categorizes jobs, level of work, and pay using the General Schedule (GS). The higher the 

GS number, the higher the experience and pay. 

FINAL REPORT NO. OIG-12-018-A 1 
   

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

USPTO’s PHP is considered one of the leading government telework programs, yet no 

comprehensive and independent review of the program’s costs and benefits has ever been 

completed. Because, by law, participation in telework programs should not result in 

diminished federal employee performance, we specifically looked at the productivity of PHP 

participants. Also, because USPTO’s management challenges include addressing the backlog of 

about 700,000 patent applications, we sought to determine if the Patent Hoteling Program can 

help reduce the backlog in a cost-effective way. 

This audit’s objectives were to determine 1) how USPTO measures productivity for its PHP 

participants and whether USPTO’s data indicate improvement in productivity; 2) the extent 

that USPTO has achieved its stated cost savings for PHP, including real estate savings; and 3) 

the extent that USPTO policies and their implementation provide adequate management 

controls over PHP. Using USPTO data, we analyzed the production levels of PHP participants 

and the costs and savings associated with PHP. We also reviewed and tested policies and 

internal controls relevant to the PHP. For more details on the scope and methodology of our 

audit work, see appendix A. 

We conducted this performance audit in accordance with generally accepted government 

auditing standards. Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions 

based on our audit objectives. We believe that the evidence obtained provides a reasonable 

basis for our findings and conclusions based on our audit objectives. 

FINAL REPORT NO. OIG-12-018-A 2 
 

                                                           

 

  

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Findings and Recommendations 

1. PHP Participants Review More Patent Applications Than Do Examiners 

Working at Headquarters, but There Are No Differences in Efficiency 

A cornerstone of any telework program is that the teleworker must be no less productive at 

home than in the office.2 USPTO collects production data, such as the number of patent 

applications that an examiner reviews.3 It also sets production targets for the number of patent 

applications to be reviewed, based on an examiner’s experience level and the patents’ 

complexity. Using this and other USPTO data, we built statistical models that measure the 

effect of PHP participation on production and efficiency, while accounting for other influences 

such as performance rating, grade level, and technology center.4 We tested the reliability of 

the data, and while there were some limitations (see appendix B), we found the data to be 

reliable for use in our models. Based upon our analysis, we found that PHP participants review 

more patent applications than their nonparticipating counterparts, but PHP participants are no 

more efficient at reviewing applications. 

A. PHP participants review more patent applications 

Based on our model, we found participation in PHP to be statistically significant in explaining 

why these examiners spent more time examining patents. Specifically, we found that the 

average PHP participant spends 66.3 more hours5

 a year examining patents than a PHP-eligible 

examiner working at headquarters. In other words, an average PHP participant reviews about 

3.5 more applications6

 annually—or 4 percent—than a full-time examiner working at USPTO 

headquarters.7 

We found that the average PHP participant allocates a greater proportion of time to examining 

patents. PHP participants use less sick and administrative leave and charge less time to other 

activities not directly defined as examining patents. However, except for sick and 

administrative leave, among the various other activities not directly defined as examining 

2

 See Department of Transportation and Related Agencies Appropriations Act of 2001, Pub. L. No. 106-346, 

§ 359, 114 Stat. 1356, 1356A-36 (2000); 5 U.S.C. § 6502(b). 

3

 USPTO measures examiners’ production by the number of applications they review. Examiners earn partial 

credit for delivering a preliminary decision on the patentability of an application’s claims. They earn the remaining 

production credit after the review of an application is closed—because an applicant agreed with the examiner’s 

decision, abandoned the patent claim, or because the applicant exhausted the opportunities to persuade the 

examiner and the examiner issues a final rejection. 

4

 USPTO organizes various areas of technologic expertise into technology centers. USPTO currently has nine 

technology centers, including Communications, Chemical & Materials Engineering, and Computer Architecture 

and Software. 

5

 Based on our model, examiners did not work 66.3 more hours of overtime; they allocated 66.3 more hours to 

examining patents out of the total time they worked. 

6

 USPTO estimates that on average, an examiner eligible to participate in PHP spends 18.84 hours reviewing an 

application.

7

 Our analysis indicates that the average examiner working from headquarters at GS-12 level or above spends 

1,604 hours a year examining patents, and reviews approximately 85 patent applications per year. 

FINAL REPORT NO. OIG-12-018-A 3 
                                                           

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

patents, no single activity differed substantially between PHP participants and PHP-eligible 

participants who worked at headquarters. 

B. PHP participants are no more efficient at reviewing patent applications than their 

counterparts who work at headquarters 

Our second model tested whether there was a significant difference in examiner efficiency, as 

measured by the time it took PHP participants and eligible nonparticipants to review a patent 

application. As in the other model, we accounted for such variables as grade level and 

performance rating. We also controlled for USPTO’s production targets, which are based on 

technology area and the patent examiner’s experience level. We found no significant difference 

in the average efficiency of PHP participants and eligible examiners working at headquarters. 

However, we did find that USPTO’s production targets had a statistically significant and 

substantial effect on the amount of time it actually took to review applications (see appendix 

B). 

Although increased production was not a primary goal of PHP, our analysis found that the 

additional hours that a PHP participant devotes to examining patents could contribute to 

reducing the current backlog of patent applications. Our model estimates that a PHP 

participant removes 2.6 more new applications8 from the backlog each year than an examiner 

working at headquarters; therefore, approximately 2,600 PHP participants would review about 

6,700 additional patent applications in a given year. Given the current backlog of about 700,000 

patent applications (see figure 1), these additional reviews could have a small but nonetheless 

positive effect on reducing the backlog. 

Figure1. Increase in Patent Application Backlog from FY 2000-2010 

8

 To assess the impact of PHP on the backlog, USPTO estimates that 73 percent of the applications any examiner 

reviews are being reviewed for the first time, while the remaining 27 percent are applications that were refiled by 

the applicant. We found that the average PHP participant reviews about 3.5 more applications annually; thus, our 

model estimates that a PHP participant will review 2.6 more new applications each year. 

FINAL REPORT NO. OIG-12-018-A 4 
 

                                                           

 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

2. USPTO Avoids Real Estate Costs Through PHP but Has Not Calculated a 

Comprehensive Cost Analysis for the Program 

USPTO has not provided a complete representation of the costs and savings associated with 

PHP. USPTO understated the real estate costs it avoids and did not adequately account for 

additional costs of supporting the program. We calculated a more comprehensive comparison 

of the cost of supporting PHP participants with the cost of full-time examiners at the agency’s 

offices. We found that USPTO’s cost avoidance is $1,710 per PHP participant in the first year. 

In subsequent years, USPTO annually avoids costs of approximately $3,385 per PHP 

participant because of reduced real estate expenses such as rent, taxes, and utilities. USPTO’s 

cost avoidance in the first year is less than subsequent years due to additional costs incurred 

such as information technology (IT) setup and equipment costs to support a PHP participant. 

A. Avoided real estate costs understated 

USPTO has reported that because of its telework programs, it avoids approximately $19.88 

million annually for office space costs. Of this amount, approximately $15.88 million in real 

estate costs are avoided annually because of the Patent Hoteling Program. USPTO avoids this 

real estate cost because PHP participants relinquish their offices, thus freeing up office space 

and mitigating USPTO’s need to acquire additional office space. 

However, our analysis found that USPTO understated this $15.88 million in avoided real 

estate costs. We estimated that USPTO avoided approximately $16.84 million annually. This 

difference is due to USPTO omitting additional costs of $2.22 million for hoteling suites as well 

as omitting avoided costs of $3.18 million for common space such as hallways and cafeterias. 9 

B. Additional and avoided costs associated with PHP 

USPTO accounts for most of the real estate costs it avoids as a result of PHP, but the program 

also incurs as well as avoids other costs. As shown in table 2, additional costs include 

computer equipment, support services, and reimbursement for Internet service providers 

(ISP); avoided costs include real estate taxes as well as transit benefits, which are not paid to 

PHP participants. 

9

 The amount of common space needed is represented by a rentable/usable (R/U) ratio: the percentage of space 

in a building that is not usable plus a pro rata share of the building’s common areas, expressed as a percentage of 

usable area. USPTO’s current R/U ratio at the Alexandria, Virginia, campus is 1.2. 

FINAL REPORT NO. OIG-12-018-A 5 
 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Table 2. Comparison of FY 2010 Costs per Patent Examiner for PHP and Non-PHP 

Participants 

Cost Category PHP Non-PHP 

(at USPTO) 

First-Year 

Difference (using 

PHP as base) 

Subsequent-Year 

Difference (using 

PHP as base) 

Real estate, 

taxes, utilities $1,075 $9,835 -$8,760 -$8,760 

IT setup 1,050 +1,050 n/a 

Equipmenta 4,880 4,255 +625 n/a 

Virtual 

infrastructureb 4,450 +4,500c +4,500 

Hoteling support 

servicesc 1,675 +1,675 +1,675 

Transit benefits 230 1,100 -870 -870 

ISP 

reimbursement 70 +70 +70 

Total $13,480 $15,225 -$1,710 -$3,385 

Source: OIG analysis of USPTO data a

Equipment includes computer, dual monitors, printer, webcam, keyboard, speakers, and miscellaneous 

equipment. 

b

Virtual infrastructure costs are dynamic and reported on as incurred in 2010; however, USPTO is currently 

transitioning to a universal laptop program that will decrease these costs in the future. 

c

Hoteling support services include telework depot, engineer, VPN, storage, and backup. 

3. Reviewing Additional Applications Generates Revenue That Offsets PHP 

Costs 

While USPTO incurs additional costs to support PHP, mainly for IT infrastructure and hoteling 

support services, our analysis found that these costs are significantly offset by costs avoided by 

PHP and the expected future value of the extra applications reviewed by PHP participants. The 

expected future value for a patent application takes into account that any patent application 

USPTO reviews will generate revenue from a series of fees collected for reviewing and, if 

approved, for issuing and maintaining the patent. 

As shown in figure 2, the expected future revenue for a single patent application reviewed by 

any patent examiner, whether in PHP or not, is $3,924. Given that our model found that a 

PHP participant reviewed an additional 3.5 applications per year, USPTO could receive, for 

each year worked by a PHP participant, an additional $13,373 in future revenue. Because 

different fees are collected over time, the expected future revenue from these applications 

would be realized over 14 years. 

FINAL REPORT NO. OIG-12-018-A 6 
                                                           

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Figure 2. Expected Future Revenue for a Single Patent Application Reviewed by a Patent 

Examiner 

4. USPTO Has Adequate Controls over the Patent Hoteling Program in Key 

Areas 

USPTO’s policies for managing the PHP comply with applicable telework laws and provide 

reasonable controls and assurances that the program operates effectively and efficiently. The 

applicable laws, guidance, and USPTO policies address a variety of areas, including eligibility to 

telework, training, reimbursement for phone/Internet expenses, and remote access to the 

agency’s IT systems. We did not, however, test the adequacy of the IT policies. 

To implement the PHP, USPTO developed a program that complies with key federal statutes 

related to telework.10 The program addresses employee eligibility and participation, work 

schedules, the application process, and selection criteria. We reviewed a random sample of 

active PHP participants (see appendix A) and found that almost all met eligibility requirements 

for entering the program and USPTO had appropriate documentation to support their 

eligibility. Less than 2 percent of the participants sampled lacked documentation to support 

eligibility or to show that an exemption had been made for their participation. This indicates 

that fewer than 40 PHP participants would lack documentation to support their eligibility in 

the program. 

We examined USPTO’s controls over its Internet service provider (ISP) reimbursement policy 

for PHP participants and found that USPTO has adequate controls in place to prevent 

10This includes section 359 of the Department of Transportation and Related Agencies Appropriations Act for 

2001, Pub. L. No. 106-346, which requires each executive agency to establish a policy under which eligible 

employees may telework without diminishing employee performance. 

FINAL REPORT NO. OIG-12-018-A 7 
                                                           

 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

improper payments to PHP participants for these reimbursements. Under this policy, the 

agency is permitted to pay PHP participants either a full or partial reimbursement for ISP used 

at their homes for official USPTO purposes. The program allows for reimbursements of up to 

$100 per month. In FY 2010, 354 PHP participants, or approximately 17 percent, opted to 

receive this reimbursement, which cost USPTO a total of almost $152,000. Automated 

controls are in place for the ISP reimbursement system to ensure, among other things, that 

• only eligible PHP participants are able to submit reimbursement claims; 

• claims do not exceed the $100 monthly maximum; and 

• required documentation is attached to the claim. 

In addition, the ISP Reimbursement Facilitator reviews every reimbursement claim to ensure it 

complies with USPTO policy and performs weekly reconciliations to ensure payment accuracy. 

We also reviewed USPTO’s remote access policies to reduce security risks to USPTO systems 

and data. We did not identify any deficiencies in the policies when comparing them against 

government-wide guidance.11 However, we did not test the policies’ adequacy and therefore 

cannot attest to the security and effectiveness of the systems that support PHP. 

Conclusion 

While USPTO had not completed a comprehensive analysis of the costs associated with the 

Patent Hoteling Program, our analysis found that the program provides USPTO with a costeffective tool for meeting its business strategy and mission. We found that PHP participation 

increases production and can remove a small number of additional applications from the 

backlog of patent applications, thus providing a possible means to help reduce the patent 

backlog. USPTO’s production measurement and data systems provide the PHP with the means 

to assess its operations in detail. While these features may not exist to this extent at other 

agencies, the Patent Hoteling Program’s overall design, policies, and controls provide a 

framework for other telework programs to consider. 

Recommendations 

We recommend that the Under Secretary of Commerce for Intellectual Property and 

Director, USPTO: 

1. Conduct a more comprehensive calculation for costs and cost avoidance related to PHP in

 order to obtain more accurate estimates of the cost and benefits affiliated with this 

program. 

11 Federal agencies are required to be in compliance with the National Institute of Standards and Technology’s 

(NIST’s) information security standards for improving the security of federal information and information systems 

(NIST Special Publication 800-53, revision 3, August 2009. Recommended Security Controls for Federal Information 

Systems and Organizations). 

FINAL REPORT NO. OIG-12-018-A 8 
U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

2. Work with the Department of Commerce to compare best practices from the Patent 

 Hoteling Program, which could aid telework programs within the rest of the Department. 

3. Ensure that internal controls are in place so that only eligible patent examiners participate in

 PHP and appropriate documentation is maintained. 

FINAL REPORT NO. OIG-12-018-A 9 
U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Summary of Department and OIG Comments 

In responding to our draft report, USPTO concurred with all of our recommendations. We 

have included USPTO’s complete response as appendix C. Separately, the agency verbally 

provided technical comments that we addressed in the report where appropriate. 

FINAL REPORT NO. OIG-12-018-A 10 
 

                                                           

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Appendix A: Objectives, Scope, and 

Methodology 

The audit’s objectives were to determine: 1) how USPTO measures productivity for its PHP 

participants and whether the data show improvement in productivity; 2) the extent that 

USPTO has achieved its stated cost savings for the PHP, including in real estate; and 3) the 

extent that USPTO policies and their implementation provide adequate management controls 

over the PHP. We used USPTO data to analyze the production levels of PHP participants and 

their office equivalents, as well as costs and savings associated with PHP. We also reviewed 

and tested policies and internal controls relevant to the PHP. We conducted our work in 

accordance with generally accepted government auditing standards between March and 

November 2011 at USPTO headquarters in Alexandria, Virginia. 

To assess production levels for participants, we obtained USPTO data on all examiners eligible 

to participate in the program. We built statistical models to determine whether hoteling 

increased production and decreased the amount of time an examiner charges to nonexamining 

time. We collected data for production, performance ratings, time and attendance. We tested 

the reliability of the data, and limitations with the data are addressed in appendix B. The data 

came from January 2006—the official launch of the program—through March 31, 2011, and 

included information on many examiners from before and after the examiner joined the 

program. See appendix B for detailed descriptions of the models used for our analysis. 

To determine if USPTO achieved its stated cost savings, we obtained data on PHP’s costs from 

fiscal years 2008-10. The costs included real estate, equipment, and support services. We 

compared costs for an employee who is in PHP versus the costs of that same employee 

working at USPTO. We discussed the relevant costs with federal telework officials at the 

Office Personnel Management, General Services Administration, and Defense Information 

System Administration. In accordance with auditing standards, we verified these costs with 

USPTO and source documentation. We did not examine costs or savings to USPTO related to 

other factors such as attrition, retention, employee satisfaction, and environmental effects. 

To determine the extent to which USPTO policies and their implementation provide adequate 

management controls over the PHP, we assessed USPTO’s telework policies with applicable 

laws and regulations, verified participant compliance with PHP eligibility rules and training 

requirements, assessed internal controls associated with USPTO’s ISP reimbursement 

program, and reviewed USPTO’s IT security policies, with a focus on remote access. 

We assessed current telework policies as well as the various eligibility requirements that have 

been in place since the program began in January 2006 for compliance with applicable laws and 

regulations, including section 359 of the Department of Transportation and Related Agencies 

Appropriations Act for 2001, which requires each executive agency to establish a policy under 

which eligible employees may telework.12 To verify compliance with eligibility rules and training 

12Pub. L. No. 106-346, § 359, 114 Stat. 1356, 1356A-36 (2000). 

FINAL REPORT NO. OIG-12-018-A 11 
                                                           

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

requirements, we selected a representative random sample of 201 active participants in the 

program. 13 Using electronic records from various USPTO sources, we verified whether each 

individual in the sample was eligible to participate in PHP according to the program 

requirements in place at the time they entered the program and whether each of these 

individuals completed required training for the program. We tested the reliability of the data, 

and while there were some limitations (see appendix B), we found the data to be reliable for 

this work. 

To assess the internal controls associated with ISP reimbursement, we met with the 

appropriate agency officials and reviewed the reimbursement process and automated controls. 

We also reviewed USPTO’s IT security policies, with a focus on remote access, and met with 

USPTO officials responsible for enforcing these policies. We did not test the adequacy of these 

policies and therefore cannot attest to the security and effectiveness of the systems that 

support PHP. 

13 According to USPTO data, there were 2,344 active, full-time patent examiners in PHP as of March 31, 2011. 

FINAL REPORT NO. OIG-12-018-A 12 
 

                                                           

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Appendix B: Quantitative Methodology 

We developed two econometric models to explain how participation in PHP affects the 

number of patent applications an examiner reviews and the examiner’s efficiency. Because 

other factors can influence how much time examiners spend reviewing patents, and how 

efficiently, our models controlled for these factors. To test the robustness of the model 

results, we created multiple models using different specifications and measured the effects on a 

biweekly and annual basis; we found our results to be consistent. 

Data sources used in both econometric models 

For both models, we collected data from several USPTO business systems to track production 

and examiner characteristics from the program’s inception in January 2006 through March 31, 

2011. The data for our models came from PHP participants and patent examiners who were 

eligible to participate in PHP but did not to participate.14 The variables and data sources used 

in our models are shown in table 3. 

Econometric model specification and methodology: Do PHP participants spend 

more time examining patents? 

We developed this model to test whether PHP participants spend statistically significantly 

more time examining patents than those who are eligible but do not participate in the PHP 

program. We employed a fixed-effects model15 to control for effects specific to individuals and 

time periods. We controlled for several variables in our analysis because multiple factors could 

influence how much time an examiner spends reviewing patents. The additional variables 

contained in the model are described in table 4. 

Our regression model can be expressed by the following formula: 

YitExamining Hours = B0 + B1itPHP_Participation + B2itSatisfactory_Rating+ B3itGrade + 

B4itTechnology Center + uit 

- i represents the dummy variable added to control for effects specific to the 

individual. 

- t represents the dummy variable added to control for effects specific to the 

biweekly period. 

14 The models include all patent examiners at GS-12 and higher.
 15 Fixed-effects regression is a statistical method that controls for stable characteristics of individuals/entities, such
 

as work habits, and time periods to eliminate a large potential source of bias.
 

FINAL REPORT NO. OIG-12-018-A 13 
 

 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Table 3. Variables and Data Sources Used in Both Econometric Models 

Variable Description USPTO Data Source 

Hours spent 

examining patents 

USPTO tracks the time an examiner spends reviewing 

patents in a separate charge code. 

Data Warehouse. 

Collects and stores 

information from 

various business 

systems, including time 

and attendance records 

and performance 

ratings. 

Production units USPTO measures examiners’ production by the number of 

applications they review. 

Patent Application and 

Locator Monitoring 

System (PALM). Tracks, 

monitors, and reports 

on patent applications 

received by USPTO, 

including examiner 

production. 

PHP participation USPTO tracks when examiners take the training session to 

join PHP and when they leave the program. From these 

dates, we determined whether an examiner was 

participating in PHP for each biweekly period in our scope. 

Telework Central. 

Contains information 

from examiners who 

apply and are accepted 

into PHP, including 

when they join and 

leave the program. 

Grade GS-level PALM 

Technology center The technology center lists the overall organization unit 

where the examiner works. These organizations are 

grouped by broad technology areas, such as Chemical and 

Materials Engineering and Biotechnology and Organic 

Chemistry. 

PALM 

Individual fixed 

effects 

This is a dummy variablea for each examiner based on their 

examiner identification number. 

PALM, Data Warehouse 

Expected efficiency USPTO has a standard schedule that identifies how many 

hours an examiner should take to review a patent 

application. The expected efficiency variable identifies how 

many production units USPTO expects examiners to 

complete in an hour, given their grade and technology 

area. 

PALM 

Satisfactory rating Examiners are rated on a five-tier scale. To be eligible to 

participate in PHP, individuals must achieve a satisfactory 

rating. A satisfactory rating is a 3 or higher out of 5. 

Data Warehouse 

Time period, fixed 

effects 

This is a dummy variablea for each biweekly time period in 

the data sets. 

PALM, Data Warehouse 

a

A dummy variable takes the value of “0” or “1” to indicate the absence or presence of a characteristic. 

FINAL REPORT NO. OIG-12-018-A 14 
U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Table 4. Additional Variables in Model: Do PHP Participants Spend More Time Examining 

Patents? 

Type of 

Variable Variable Name Description / Other information 

Dependent 

Variablea 

Hours spent examining 

patents 

The time (in hours) charged to examining patents on a biweekly 

basis. USPTO tracks time spent examining patents separately in 

their charge codes. 

Primary Variable 

of Interest 

PHP_Participation Participation in PHP is coded as ‘1’ when the examiner is 

enrolled in PHP during the biweekly period and as ‘0’ otherwise. 

Other Explanatory Variables Controlled for: 

Grade As examiners progress up the GS scale, they may be required to 

spend more time helping train new examiners and in other nonexamining activities. The grade of the examiner could thus 

potentially negatively affect how much time they devote to 

examining patents. 

Technology center where the 

examiner works 

The technology center lists the overall organization unit where 

the examiner works. These organizations are grouped by broad 

technology areas. 

Satisfactory rating To be eligible to participate in the PHP program, individuals must 

achieve a satisfactory rating. We controlled for the effect of 

individuals who did not receive a satisfactory rating by creating a 

dummy variable coded as ‘0’ when the employee did not receive 

a satisfactory rating (1 or 2) and ‘1’ when the employee received 

a satisfactory rating (3 or above). No rating data were available 

for biweekly observations for FY 2011 because the annual 

performance period was not yet complete when we obtained the 

data. 

Individual fixed effects Controls for fixed effects unique to the examiner, such as 

established work habits. The model assumes that there is 

something unique about each person that could explain how 

much time they spend examining patents. 

Time period fixed effects We included a dummy variable for each biweekly period because 

certain periods, such as around holidays, could have a significant 

effect on the amount of time an individual devotes to examining 

patents. USPTO implemented changes in the overall performance 

measurement system in spring 2010. The time period fixed-effect 

variables also control for this change. 

a

 The dependent variable is the outcome that will be measured. We measure whether the primary variable of 

interest and other explanatory variables explain the outcome/dependent variable at a statistically significant level. 

Main Results 

As shown in table 5, the model estimates that a PHP participant spends 2.55 more hours per 

pay period examining patents than does a non-PHP participant. Since there are 26 pay periods 

each year, we estimate that a PHP participant spends 66.3 more hours examining patents over 

FINAL REPORT NO. OIG-12-018-A 15 
 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

the course of a year than does a non-PHP participant. We also found that examiners spend 

less time examining patents as they progress up the GS scale. This is consistent with our 

expectation that examiners at higher grades may devote more time to other activities such as 

training new examiners and substituting for supervisors. 

Table 5. Statistical Significance of Results of Model to Test Hours Devoted to Examining 

Patients (N = 378,270) 

Variable Coefficient Standard Errora 

PHP Participation 2.5582 0.1080 

GS-13 -2.8369 0.1186 

GS-14 -4.4951 0.1763 

GS-15 -30.6352 0.8316 

Satisfactory Rating 0.5459 0.1942 a

Results are statistically significant to 1 percent. 

Robustness Tests 

To ensure that the results were not a function of PHP employees working more hours than 

non-PHP employees, we re-ran the above model with percentage of time spent examining as 

the dependent variable. These results were consistent with our hypothesis that PHP 

participants spend a larger share of their time examining patents than do non-PHP participants. 

We also tested annual versions of the model, where we aggregated biweekly pay periods by 

individual and year (fiscal and calendar). In these specifications, PHP remained a strong, 

significant predictor of the total number of hours and percentage of time spent on examining 

patents. 

Econometric model specification and methodology: Are PHP participants more 

efficient? 

This model to test examiner efficiency shares many specifications with the previous model. We 

also used a fixed-effects model to control for effects specific to the individual and time period. 

The primary variable of interest in this model remains participation in the PHP program. This 

model includes the explanatory variables of grade and satisfactory rating that were described 

in the previous model description. 

In addition to the variables described in table 6, we also included the following explanatory 

variables as in the other model: Satisfactory Rating, Individual Fixed Effects, and Time Period 

Fixed Effects. 

FINAL REPORT NO. OIG-12-018-A 16 
 

 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Table 6. Additional Variable Descriptions in the Efficiency Model 

Type of 

Variable 

Variable Name Description/Other information 

Dependent 

Variable 

Examiner Efficiency We measured efficiency by the amount of production units an 

examiner produced per hour during a biweekly period. USPTO 

measures examiners’ production by the number of applications 

they review. Examiners earn partial credit when they deliver 

their preliminary decision on the patentability of an application’s 

claims and the remaining production credit after the application is 

closed. 

Primary Variable 

of Interest 

PHP_Participation The primary variable of interest in this model is participation in 

PHP, coded as ‘1’ when the examiner is enrolled in the PHP 

program during the biweekly period and as ‘0’ otherwise. 

Other 

Explanatory 

Variable 

Expected Efficiency USPTO uses an established schedule to identify how many hours 

an examiner should need to review an application based on the 

technology of the application and the grade of the examiner. To 

make it simpler to interpret the effect of USPTO expectations, 

we used the expected efficiency defined as the amount of 

production units USPTO expects an examiner to complete in an 

hour. 

Other 

Explanatory 

Variable 

Grade As examiners progress up the GS scale, they may become more 

effective at examining patents. USPTO does expect more 

efficiency from individuals at higher grades, but we wanted to 

control whether there was an additional effect related to the 

grade of the examiner. This variable was included in the previous 

model, but the expected effect is different. 

Our regression model can be expressed in the following formula: 

YitExaminer_Efficiency = B0 + B1itPHP_Participation + B2itSatisfactory_Rating+ B3itGrade + 

B4itExpected_Efficiency + uit 

- The dependent variable in this model is the number of production units earned in a 

pay period divided by the time spent examining patents. 

- i represents the dummy variable added to control for effects specific to the 

individual. 

- t represents the dummy variable added to control for effects specific to the 

biweekly period. 

Main Results 

As shown in table 7, our results found no statistically significant differences in efficiency 

between the two groups. The model did estimate that the Expected Efficiency Rate established 

by USPTO was a statistically significant and a substantial factor in explaining the actual 

efficiency of the examiner. To interpret the magnitude of the effect of USPTO expectations, 

we would say that, holding all other variables constant, a 1 percent increase in the expected 

FINAL REPORT NO. OIG-12-018-A 17 
U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

efficiency would result in a 0.72 percent increase in the actual efficiency. Thus, our model 

estimates that actual efficiency is highly sensitive to the expectations established by USPTO. 

Table 7. Statistical Significance of Results of Model to Test Efficiency (N = 374,690) 

Variable Coefficient Standard 

Error 

PHP Participant 0.0003 0.0005b 

Expected Efficiency Rate 0.7236 0.0748a 

Satisfactory Rating 0.0068 0.0010a 

GS-13 0.0008 0.0007b 

GS-14 0.0043 0.0013a 

GS-15 0.0164 0.0052a 

a

 Results are statistically significant to 1 percent. 

b

 Results are not statistically significant. 

Robustness Tests 

To ensure that the results were not due to anomalies introduced by measuring efficiency on a 

biweekly basis, we also tested annual versions of the model where we aggregated biweekly pay 

periods by individual and year (fiscal and calendar). Consistent with the results described in 

our report, PHP participation is not a statistically significant factor in predicting efficiency on an 

annual basis. 

Limitations of our econometric model and data 

We used data collected and recorded by USPTO for our models. Relying on this data 

presented three major limitations to our analysis. First, because we relied on self-reported 

data submitted into time and attendance records, we cannot confirm the actual number of 

exam hours and nonexam hours that an examiner worked in a biweekly period. While there 

may be instances where examiners did not accurately portray how they allocated their time, 

there is no basis for assuming that inaccuracies are more prevalent in the groups of examiners 

participating in the PHP versus examiners working from headquarters. 

Second, there is no independent source of information on examiner production and time 

spent examining patents other than the data that are entered into USPTO’s electronic 

systems. Thus, we could not perform tests to trace each transaction to paper documentation. 

We assessed the reliability of our data by conducting numerous electronic tests to ensure the 

data appeared complete and accurate. 

Finally, we did not test the IT security internal controls of each data system, and no published 

reports on data quality existed for these systems during our analysis. We reviewed the overall 

FINAL REPORT NO. OIG-12-018-A 18 
U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

data entry policies with system users to ensure that there were segregation of duties. 

However, we did not conduct an extensive test on the overall IT security. 

Given these limitations, we have no evidence that any potential inaccuracies or tampering 

would be more likely to occur with PHP participants versus eligible patent examiners who do 

not participate in PHP. We used the same data that USPTO management relies on to make 

management decisions. Therefore we believe that the data is sufficiently reliable to assess the 

differences between examiners in the PHP program and those who work at headquarters. 

When interpreting our productivity analysis, it is important to recognize the limits of 

generalizing our model’s results to other workplaces. USPTO has unique features, such as 

explicit, measurable performance expectations and extensive tracking of production. While 

our results explain the effect of the PHP program from January 2006 to March 31, 2011, the 

results of our work will need to be replicated in other hoteling programs before it is 

appropriate to generalize these findings to other workplaces. 

FINAL REPORT NO. OIG-12-018-A 19 
~ ~ .... ~~ 

UNITED STATRS I' A TENT AND TRADE"ARK OFFICE 

\IWOO StC. '1'.UV or CCIMMiat'l: POIIIIf~ Plttwt~rv AM> 

0.lCrultOFl1-IE l.N'tr.oSlAT'L'S PAft.'\-r AMb alt0ffw;e 

JAN I 8 2012 

MEMORANDUM FOR Ron Prevost 

Assistam Inspector General for Economic and 

Statistical ~ogr t 

D-......... 1(- r:.-

FROM: Margaret A. F rino ' 

Commissioner for Patents 

SUBJECT: Response to Draft Repon: "United States Patent and Trademark 

0/fict: The Paten! llottling Program is Succeeding as a Busintss 

Sirategy 

Executive S ummary 

Thank you for your n:vicw of the United States Patent and Trademark Office's (USPTO) 

teleworking program policies and implementation process. We appreciate the effon you and 

your •1afThave made in reviewing the Patent Hoteling Program (PHP), including the associated 

cost savings and business strategy for over 2,600 participants through fiscal year 20 II. We have 

carefully considen:d the three recommendations made in the subject draft repon. 

The USPTO believes our implementation and progn:ss in this telcworking program is recognized 

as o model for other federul agencies to consider as a best practice. We are prood of the 

contributions made thus far from our participants, which we believe will assist in achieving our 

strategic goals and mission. 

Our response to each n:commendation is discussed in detail below. We have no technical 

comments to provide for this report. 

es ~e to Resommsndationlf 

IG Recommtndutlon that the U11der Secretary qfCommtrcefor Jntel/octual Property and 

Director of USPTO (/): Conduct a more comprehensive calculation for costs and cost 

ovoidunce related to the PHP in order to obtain more accurate estimates of the cost and benefits 

affiliated with this program. 

USPTO Response: 

The USPTO concurs with this recommendation. Tbe USPTO bas recently undertal<en an 

assessment of the costs and cost avoidance associated with the PHP as part of the Cost Benefit 

Analysis for the Telework Enhancement Act Pilot Program that has recently been approved by 

the General Services Administration. We will review the dato we have collected as it applies to 

P.O. Boll t450, Ato.onortt_. Vlrgin1122313- ~ ·wm.us.,ro.oov 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

Appendix C: Response to OIG’s Draft Report 

FINAL REPORT NO. OIG-12-018-A 20 
the entire hoteling program as well as reassess our fmdings in other an:as of cost and cost 

avoidance in light of the findings of this Inspector General (!G) report. We appreciate the 

feedback provided by the IG. The infonnation will be helpful as the USPTO.seeks to continually 

improve our telework programs. 

JG Recommendation tit at the Undu Secretary for Commerce for Intellectual Proptrty and 

Director of USPTO (2): Work with the Department of Commerce to compare best practices 

from the Patent Hotcling Program, which could aid telework programs within the rest of the 

Department. 

USPTO Response: 

The USPTO concurs with this recommendation. The USPTO works regularly with the 

Department of Commerce (DOC) on depamnental and bureau-specific telework program issues 

and implementation. We have frequent informal interaction with various depamncntal and 

bureau staff members in addition to both hosting and participating in more formal events. 

To date, the USPTO has held two telework workshops ~-pecific lly designed for DOC 

organizations. These workshops were developed in response to the numerous questions received 

by the USPTO regarding the lessons learned from USPTO's telework and hoteling programs. 

The goal of each workshop \vas to better educate DOC telework points of contact with regard to 

gathering/evaluating pertinent telework data, creating bureau tclework policies and telework 

Web site, commWlicating telework infom1ation bureau-wide, developing data-collecting tools, 

and designing an infrastructure to support robust telework initiatives. 

Workshop anendees included representatives from DOC-· International Tmde Administration 

(IT A), Census, Economic Development Administration (ED A), National Oceanic and 

Atmospheric Administration (NOAA), and Nationallnstitute of Standards and Technology 

(NlST). Agenda items included an overview of tclework at the USPTO, a discussion of the 

enterprise-wide telework policy, telework tools, gathering and analy-Ling telework statistics, 

developing a telework Web site and communicating information agcncywidc, o demonstration of 

the telework database, developing an IT infrastructure to support a telework strategy, the Office 

of Personnel ~anageme t (OPM) Telework Data Call, and Q&A panels on emerging Federal 

telework program issues. 

The USPTO looks forward to continuing our work with depamnental colleagues in the 

furtherance oftelework throughout DOC. We have committed to planning and hosting quarterly 

departmental telework workshops going forward and will continue to be available to consult on 

an individual basis, as needed. 

IG Recommendation that the Under S ecretary for Commerce f or Jntel/ecfuaf Property and 

Director of USPTO (3): Ensure that intemal controls arc in place so that only eligible patent 

examiners participate in PIIP and appropriate documentation is maintained. 

USPTO Response: 

l'he USPTO concurs with this recommendation. Patents in is the process of evie,.~ng the 

eligibility, sign-up procedures, and records retention procedures for the PHP. We will ensure 

that there are procedures in place to fully reflect the eligibility information for all PHP 

participants. 

U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL
 

FINAL REPORT NO. OIG-12-018-A 21 
U.S. DEPARTMENT OF COMMERCE OFFICE OF INSPECTOR GENERAL

FINAL REPORT NO. OIG-12-018-A 22