9 Quality Management System that Takes into Consideration
the Evaluations Made by Patent Applicants and Agents
The purpose of this survey research is to conduct a survey as a means to analyze patentexamination objectively and cross-sectionally on various technical fields, and apply to improve
the quality management system based on the analysis results. This research, which focuses onapplicants’ and agents’ evaluations of the quality of patent examination, consists of the following
three parts: (1) Research on the current quality management systems of foreign patent offices,(2) Trial questionnaire-mailing survey on applicants and agents, and (3) Interview survey on
applicants and agents to obtain their evaluations of patent examination and their opinions on
survey methods. Based on the findings of this research and surveys, we identified and analyzed
issues concerning a quality management system that takes into consideration the “evaluations
by applicants and agents” and explored measures to solve those issues. We were pleased to findthat the response rate of this trial survey was extremely high in comparison with that of othersimilar surveys conducted by foreign patent offices. The fact that many interviewees subject tothe interview survey expressed positive views on the questionnaire survey suggests that many
applicants and agents recognize the need for this type of survey. It is therefore important for us
to conduct such a survey on a regular basis, while making efforts to further improve thequestionnaire based on the findings from the analysis and examination carried out in this survey
research.

I

Introduction

In an effort to improve the speed andefficiency of patent examination, theMinister of Economy, Trade and Industry
issuedtheAdvanced
MeasuresforAccelerating Reform toward Innovation Planin Patent Examination (AMARI Plan 2007).
The Plan focuses on, among other things,
how to enhance quality management in orderto maintain and improve the quality ofpatent examination.
The global trend of sharing the results ofprior art searches and patent examinations
is accelerating especially among trilateral
offices, namely the JPO-USPTO-EPO triad,
and the JPO-SIPO-KIPO triad. These patentoffices are all faced with the same challengeof establishing a system and procedure toensure high-quality patent examination,
which is a prerequisite for successful
information sharing. In the meantime,within the framework of Patent Cooperation
Treaty(PCT),
provisions
concerning
requirements for quality improvement of●

IIP Bulletin 2008

international
searchandpreliminary
examination were established (so-called PCT
Quality Framework) in Chapter 21 of thePCT International Search and Preliminary
Examination Guidelines (“PCT Guidelines”).
Under
these
provisions,
Preliminary
Examination Authorities and International
Searching Authorities are required to
improve their quality management systems.
The JPO manages the quality of patentexamination
based
on
the
PDCA
(Plan-Do-Check-Act)
cycle.
The
newly
established Quality Management Office is incharge of analyzing the results of patentexamination objectively and cross-sectionallyon various technical fields. The analysis
results are utilized to ensure continuous
improvement of the quality of JPO’s patentexamination as a whole.
In this survey research, we focused on
applicants’ and agents’ evaluations of thequality of patent examination and examinedthe current quality management systems offoreign patent offices. Furthermore, we
conducted a trial survey by sending
68 ●questionnaires to applicants and agents aswell as an interview survey to obtain theirevaluations of patent examination and theiropinions on survey methods. Based on thefindings from these surveys, we identified
and analyzed issues concerning a quality
management system that takes into
consideration the “evaluations by patentapplicants and agents” and exploredmeasures to solve those issues.

II

Survey on Foreign Patent Offices

We sent questionnaires to foreign patentoffices that mostly consisted of PCT
International
Preliminary
Examination
Authorities and International Searching
Authorities. The questionnaire containedquestions about a quality management
system that takes into consideration
customers’(patent applicants’ and/or
agents’) evaluations of the quality of patentexamination. As described below, all of thepatent offices that responded to the survey
had already introduced a system to obtaincustomers’ evaluations.
(1) United States: United States Patent andTrademark Office: USPTOThe USPTO issued the 2007-2012
Strategic Plan, which lists “Optimize Patent
Quality and Timeliness” as the mostimportant strategic goal. Under the Plan, the
USPTO
has
been
implementingacomprehensive quality system for patentexamination.
The USPTO conducts a Customer Panel
Quality Survey by sending a quarterly
questionnaire to the users. All technicalfields are subject to the survey. The
respondents are requested to answer, on ascale, detailed questions designed tomeasure their evaluations of patentexamination conducted in the preceding
three months. The survey results, which are
not available to the public, are sent to the
Office of Patent Quality Assurance (OPQA)of the USPTO, which analyzes the results bycomparing them with the results of internal
review and so on.
●(2) Europe: European Patent Office: EPO
In 2005, the EPO reorganized andrestructured itself to improve its quality
management system. The EPO conducts theUser Satisfaction Surveys by calling users
every two years if possible. The survey isconducted on customers who are randomly
selected
from
among
those
whose
applications were examined within a certain
period in the selected technical field (joint
clusters : JC).
Both search procedure andexamination procedure are subject to thesurvey. The survey is conducted on various
aspects of these procedures that the EPO
monitors to determine whether its quality
management system meets certain criteria.
These aspects subject to the survey include
Timeliness, Thoroughness, Clarity, Validity,
and overall customer satisfaction level
observed in the EPO’s handling of a specificapplication in the course of the search
procedure. The survey results are analyzed
statistically and further examined in
combination with other quality management
data (Quality Audit, Operational Quality
Control, Complaints, etc.). The results of theanalysis and examination, which are notavailable to the public, are taken into
consideration by the EPO management inmodifying those procedures.
(3) United Kingdom: UK Intellectual Property
Office: (UKIPO)
The
UKIPO
conducts
Customer
Satisfaction Survey by sending questionnaires.
It also conducts face-to-face interview
surveys and online surveys on its website. In
addition, it carries out these surveys on
patent applicants and agents selected fromamong the non-individual customers whose
applications were examined in the previous
year. The survey is not about specificapplications, but about the UKIPO services
in general (such as customer evaluations of
the quality of patent examination andproposals for improvement). The survey
results, which are publicized through its
website, are used by the UKIPO to identify
problems and explore measures to solve
them.
69 ●
IIP Bulletin 2008

(4) Germany: German Patent and Trade
Mark Office: GPTO
The GPTO conducted its first customer
satisfaction survey in 2005. The next survey
is scheduled for around 4 years later. In thesurvey,
the
GPTO
sent
different
questionnaires to patent applicants andagents respectively, while the two types ofquestionnaires contain almost the samequestions on the quality of patentexamination. The questionnaires containdetailed questions about patent examination.
Many questions require respondents toanswer on a scale. The survey results arecompiled for each question and disclosed
through the website.
(5) Spain: Spanish Patent and Trade Mark
Office: SPTO
The SPTO has an external institution
conduct an annual customer satisfaction
survey on PCT applicants and agents.
Applicants are questioned about specificapplications, while agents are questioned
about patent examination in general. Bothapplicants and agents are requested toanswer questions about the details ofinternational search reports and written
opinions, search examiners’ capabilities, etc.,by evaluating the level of their satisfaction
with and significance on each item.
The survey results are utilized by the
SPTO within the framework of its quality
management system in order to eliminate
the causes of quality defects and problems
pointed out by customers and explore
remedial measures. If necessary, survey
results are directly sent to examiners.
Given the low response rates (Agents:
30.76%, Applicants: 14.07%), the SPTO is
considering ways to raise the rates.
(6) Sweden:
SwedishPatentandRegistration Office: PRV
The PRV conducts the Customer Survey
about every two years by sending
questionnaires in order to measure theoverall level of satisfaction of major
customers. Any customer who filed two or
more patent applications in the preceding 18●

IIP Bulletin 2008

months would be subject to the survey.
Each respondent must evaluate, on ascale, the search reliability, description
clarity, and timeliness of both national
application and PCT application procedures.
The PRV analyzes survey results andidentifies issues that would greatly affect theoverall quality of patent examination. Suchissues may often indicate the areas to focus
on. The survey results, which are available
to the public, are used by the PRV to
improve its patent procedure and develop
training programs.
(7) South Korea: Korean International
Property Office: KIPOThe
KIPO
conductsacustomersatisfaction survey, a part of which isdevoted to measuring customer satisfaction
with the patent examination procedure. The
KIPO conducted this survey twice in 2007 bycalling 1,000 randomly-selected domestic
customers. All technical fields are subject tothe survey. The survey items include theunderstandability of notifications of reasons
for refusal, timeliness of procedures,
accuracy of examination, fairness in thecourse of examination, level of examiner’s
understanding,mannerofhandling
customers, and overall level of satisfaction in
the course of examination. Answers must be
given on a scale. The survey results areanalyzed by an external institution andpublicized through its website.
The results are also used by the KIPO to
improve its system and raise the level ofcustomer satisfaction.
(8) Australia: IP Australia: IPA
The IPA has an external institution
conduct a customer satisfaction survey on
Patent Attorneys, Patent Corporate bodies,and Patent Individuals/Subject Matter
Experts every two years. All technical fields
and the entire examination procedure aresubject to the survey. The purpose of thesurvey is to measure the overall level ofcustomer satisfaction with its search andpatent examination services.
The survey results are fed into an
70 ●improvement log and/or
operational plans for
implementation.

businessarea(9) Finland: National Board of Patents andRegistration of Finland: NBPRThe NBPR conducts a customer
satisfaction survey on major customers every
two years. The examination procedures fordomestically filed patent applications, PCT
applications, and utility model applications,regardless of their technical fields, aresubject to the survey.
The same questions are asked to every
respondent, who is requested to evaluate thetimeliness, realibility, proficiency, clearness,understandability of search reports andwritten opinions.
The survey results are publicized on its
website and also sent to customers forfollow-up discussions.
(10) Other agency (which requests to remain
anonymous)
It conducts a customer satisfaction
survey by sending questionnaires annuallyto those who were selected randomly for thesurvey. Domestically-filed patent applications,regardless of their technical fields, aresubject to the survey, which covers thefirst-time search and examination.
The questionnaire asks the samequestions to all respondents, requestingthem to evaluate the detailed aspects ofpatent examination.
The survey results, which are notavailable to the public, are utilized to
explore implementation plans to resolve
problems.

III Trial and Analysis of a Quality
Management System That Takes
into Consideration the Evaluationsof Patent Applicants and Agents(1) Purpose
As part of this survey research, we
implemented a quality management systemon a trial and used the trial results as abasis to figure out how to reflect user●

evaluations in the JPO’s patent examination
quality management policy.
(2) Response rates
We carried out a survey (trial) tomeasure customer satisfaction with thepatent examination procedure. The survey
was conducted on patent applicants andagents (Applicants: 604 (2,302 applications),
Agents: 360 (360 applications)) who wererandomly selected from among those whose
patentapplications
completedtheexamination procedure (decision to grant apatent, decision of refusal, or disposition ofabandonment or withdrawal of patentapplication was rendered) in August 2007. At
the same time, we conducted an inquiry
survey in order to identify issues concerning
the trial survey. The rates of response fromthe applicants and agents were 62.3% and53.3%, respectively, in terms of the numberof respondents and were 65.6% and 52.8%,
respectively, in terms of the number ofpatent applications subject to the survey.
These high rates indicate their great interestin the quality of patent examination.
(3) Analysis of the customer satisfaction
with the examination procedure
Regarding procedures in general such as
the patent examination procedure and theinternational search procedure, about 80% of
both applicants and agents evaluated theprocedures as “Appropriate.” Regarding
individual cases of patent examination,
about 90% of both applicants and agents
evaluated the way a notification of reasons
for refusal was written as “Appropriate,”
while about 85% of the applicants and about
80% of the agents evaluated the way a
notification of examiner's decision of refusal
was written as “Appropriate.” Regarding thecommunication with examiners in interview
or by telephone or fax and suggestions foramendments, more than 90% of both
applicants and agents evaluated them as“Beneficial,” indicating that they weresatisfied with the JPO’s advice andsuggestions.
In slightly less than 30% of the cases of
71 ●IIP Bulletin 2008

patent examination subject to the survey, anapplicant’s or agent’s evaluation of the way a
notification of reasons for refusal was
written differed from each other even if
those notifications were given for the same
patent application, while in many cases, anapplicantor
agent
evaluated
both
notifications as “Appropriate.”
When asked to list important factorsthat should be taken into account inmeasuring customer satisfaction with patentexamination, both applicants and agentsanswered that the following four factors
were especially important: “Accurate priorart search,” “Uniform decision that does notvary by examiner,” “Notification of reasons
for refusal written in an appropriatemanner,” and “Technical understandings of
examiners.”
These survey results revealed that, inorder to further raise customer satisfaction
with a notification of reasons for refusal, we
need to explain in more detail the reasons
why an examiner found that the invention
does not involved novelty and/or inventive
step and also need to indicate the citedplacesofcited
documents
more
appropriately.
(4) Analysis of the results of the inquiry
survey on the survey method
In this inquiry survey, respondents wereasked whether they were willing to cooperate
if this type of survey is periodicallyconducted in the future. Most respondents,
both applicants and agents, answered
positively, indicating their strong interest in
the quality of patent examination.
When asked whether the questionnaire
contained questions that were difficult tounderstand or answer, most respondents,
both applicants and agents, answered “No,”
while some of them pointed out that theyfound it “difficult to prepare answers” to thequestions that required them to identify theapplication
numbersofthepatentapplications that they considered to havebeen
examined
appropriatelyorinappropriately.
They
attributed
thedifficulty to the trouble of identifying the●

IIP Bulletin 2008

application numbers and to their hesitationin disclosing specific application numbers.
Regarding individual cases of patentexamination, many respondents suggested
that any cases where the applicant failed torespond to a notification of reasons forrefusal and where an examiner decided to
grant a patent without sending a notificationof reasons for refusal should not be subject tothe survey.

IV Interview Survey in Japan
We conducted an interview survey on 19
applicants and 8 agents who were selectedfrom among those subject to the survey
described in Chapter III. This interview
survey was conducted to ask how theyevaluated the examination procedure in
general,individual
casesof
patentexamination, and the system to obtain
applicants’ and agents’ evaluations of patentexamination including the trial survey.
(1) Examination procedure
The interviewees’ evaluations of theexamination procedure were almost thesame as those obtained in the trial survey.
The interviewee pointed out that thefollowing four factors were especially
important in determining their evaluationsof the way a notification of reasons forrefusal was written: whether the notificationrefers to “the specific sources of the cited
documents,” “the reasons for the lack of an
inventive step,” “the reasons of forimperfection of the application,” and “theevidence for the existence of a well-known
art or design.”
Furthermore, the interviewees responded
that the following three factors were
especially important in determining theirevaluations of the JPO’s manner of handlingcustomers in interview or by telephone or fax:
“The reasons for rejecting a request for aninterview,” “An examiner’s attempt to contact
customers,” and “The record of an interview.”
This interview survey allowed us tounderstand in detail what factors would
determine the level of customer satisfaction
72 ●the JPO to obtain applicants’ and agents’evaluations effectively and sufficiently.

with the examination procedure.
(2) Survey method
Most interviewees evaluated this trial
surveyas“Appropriate
ingeneral.”Regardingasurveymethod,mostinterviewees mentioned that they did nothave a strong preference about a survey
method, while some of them preferred adigital-format survey or a web-based survey.
As the results of the trial survey showed,
some interviewees complained that theyfound it “difficult to prepare answers” to thequestions that required them to identify theapplication
numbersofthepatentapplications that they considered to havebeen
examined
appropriatelyorinappropriately.
Furthermore,manyinterviewees suggested that any cases where
the applicant failed to respond to a
notification of reasons for refusal and where
an examiner decided to grant a patent
without sending a notification of reasons forrefusal should not be subject to the survey.
Regarding the survey method, many
interviewees requested more opportunities toexchange opinions directly with the JPO andan option of selecting cases of patentexamination by themselves to state theiropinions.

V

Analysis of Issues Concerning a
Quality Management System that
Takes into Consideration the“Evaluations by Applicants andAgents”

Applicants’ and agents’ evaluations ofpatent examination, which function as the“evaluations
byoutsiders,”wouldsignificantly enable the JPO not only to
directly monitor customer satisfaction with
the quality of patent examination but also tomeasure and improve the effectiveness of thequality management system. Many patentoffices in other countries have recognized theimportance of such a system and have begun
to make efforts to establish and improvetheir quality management systems. The JPO
is also needed to create a system that allows●(1) System to obtain “applicants’ andagents’ evaluations”(i) Survey method
Some foreign patent offices conduct a
telephone survey or a web-based survey.
These survey methods require the patentoffice to contact the target applicants andagents several times, causing additional
work for the patent office. Currently, thequestionnaire-mailing method is considered
to be the simplest way to conduct a survey.
It may be convenient for respondents
that they can select the way to respond to
the survey either by handwriting or byelectronicmeans.
For
example,thequestionnaires mailed to the applicants andagents subjected to a survey indicate a
website address allowed to download thequestionnaire in a digital format.
(ii) The types of people and individual cases
subject to the survey and the scale andthe frequency of the survey
This trial survey was conducted on both
applicants and agents, as is the case withthe surveys conducted by many foreign
patent offices. The survey results revealed
that it was important to check both
applicants’ and agents’ evaluations of various
aspects of patent examination. The morequestions sent to applicants share with thosesent to agents, the easier to compare andanalyze their responses.
The questions about individual cases of
examination were beneficial in that we could
understand the grounds for applicants’ andagents’ evaluations based on a particularcase of examination. However, since therespondents referred to specific patentapplications, they pointed out the need tokeep their responses a secret. Their concerns
deserve careful consideration.
Some respondents mentioned that theyfound it rather difficult to select specificcases by themselves to make a point andthat it would be easier for them to makecomments if the JPO selects specific cases
73 ●IIP Bulletin 2008

for them in advance.
While the questionnaire containedquestions about individual cases of patentexamination, this trial survey was conducted
about four months after the completion of
the examination of the patent applications
subject to the survey. Despite this time lag,most respondents found the timing of thesurvey “Appropriate.” However, we believethat questionnaires should be sent as soon as
patent examination is completed because
applicants and agents tend to dispose of
related documents or lose an accurate
memory as time goes by.
If possible, a response period of one
month or longer should be given to
respondents because some of them might
have rather many individual cases subject tothe survey.
The EPO prepares Joint Cluster(JC)-specificquestionnaires,
insertingtechnology-specific questions into eachquestionnaire. The possibility of introducing
such a survey method should be considered
into the JPO’s future survey.
The scale of the survey should bedetermined in consideration of the expected
response rate as well as the type of analysis,suchastechnology-specificanalysis,scheduled to be conducted based on thesurvey results. The response rate of the trial
survey was extremely high in comparison
with the rates disclosed by some foreign
patent offices. The continuous efforts should
be made to maintain such a high rate.
The frequency of the survey does nothave to be so high because outsiders’evaluations of patent examination do notchange greatly within a short period of time.
The survey should be conducted at a regularinterval determined in consideration of thefrequencies of the surveys conducted byforeign patent offices, requests fromapplicants
andagents,burdensonrespondents, etc.
(iii) Utilization, publication, and feedback of
the survey results
Many applicants and agents subject tothis trial survey and interview survey●

IIP Bulletin 2008

requested feedback of the survey results.
Most of them expressed deep concern over“careless handling of security.” Many foreign
patent offices said that they especially paid
careful attention to security. From theperspective of confidentiality protection, thesurvey results that could lead to theidentification of individual cases of patentexamination should not be publicized.
Feedback should be limited to statistical
data about the survey results. Carefulhandling of data would prevent identificationof individual applications or respondents andencourage applicants and agents to respondto a survey without worrying about any
negative effect on subsequent patentexamination of their applications.
(2) Questionnaire for the trial survey(i) Comparison between overall evaluations
and case-specific evaluations
Respondents’ evaluations of the way the
first notification of reasons for refusal were
written were higher than those of the recent
patent examination as a whole, indicatingthat their overall evaluations were notnecessarily the same as their case-specificevaluations. Their overall evaluations could
be influenced by their evaluations of someindividual cases that left a strong impression
on them. This influence should be taken into
consideration in the analysis of the survey
results.
(ii) Comparison between the evaluations
given in the trial survey and those givenin the interview survey
Respondents’ overall evaluations givenin the trial survey differed from those givenin the interview survey due to the differencein the number of respondents between thetwo surveys. Since an interview survey helps
us understand the real intentions ofrespondents, an interview survey should be
conducted to supplement the data from aquestionnaire-mailing survey.
(iii) Comparison between the evaluations
made by applicants and those made byagents74 ●When asked about a specific case ofpatent examination, both the applicant andagent gave almost the same evaluations on
the way a notification of reasons for refusal
and a notification of examiner's decision ofrefusal were written. There were, however,some areas where their evaluations did notmatch. Since applicants and agents havedifferent perspectives in some respects, thetwo groups might have different needs
concerning patent examination. Therefore, asurvey should be conducted on both groups.
(iv) Questionnairetomeasureoverallevaluations of patent examination,international search, etc.
If the questionnaire contains anyquestion to measure the respondent’s overall
evaluations, a questionnaire may be needed
to state clearly whether the respondent is
expected to give answers on a staff level oron a company-wide level.
If the questionnaire contains anyquestion that requires the respondents toinclude specific patent application numbersin their answers, we should give a priornotice of survey to alleviate the burdens onthem because many respondents of the trial
survey and interview survey expressed theirconcerns about the “trouble of identifying theapplication numbers.”
The responses to the questions tomeasure applicants’ and agents’ satisfaction
with patent examination revealed that they
considered the following four factors
especially important: “Accurate prior art
search,” “Uniform decision that does not varyby examiner,” “Notification of reasons forrefusal written in an appropriate manner,”and“Technicalunderstandingsofexaminers.” It would be beneficial to conduct
a further survey specifically on these factors
in order to analyze in more detail the levelsof customer satisfaction with patentexamination in general and individual casesof patent examination.
(v) Questionnaire to measure customer
satisfaction with individual cases ofpatent examination●

Many respondents of the trial survey
and interview survey suggested that any
cases where the applicant failed to respondto a notification of reasons for refusal andwhere an examiner decided to grant a patent
without sending a notification of reasons forrefusal should not be subject to the survey.
However,somecomments
madebyrespondents about these particular cases
were beneficial. If these cases are subjectedto the survey, a questionnaire will be needed
to contain questions such as respondents will
be able to answer easily.
Regarding the questions to measurecustomer satisfaction with the way a first
notification of reasons for refusal and anotification of examiner's decision of refusal
were written, most respondents who could
not perceive the reasons in the written
notification chose answers from the options
provided, indicating that these options wereappropriate.
Regardingthequestions
aboutsuggestions for amendments, we should havedistinguished suggestions given in interview
or by telephone or fax from those given in a
notification of reasons for refusal anddevised questions specifically for each type ofsuggestion in order to ensure easier andmore effective analysis.
The questions concerning examiner’s
decision to grant patent were free-answer
questions. It might be beneficial to devise
multiple-choice questions based on theopinions presented in the trial survey.
Regarding the questions to compareindividual applications subject to this survey
with the corresponding applications filedwith foreign patent offices, most of theanswers were sent from applicants. This wasprobably because agents who had been
involved only in nationally-filed applications
found it difficult to answer such questions.
Some interviewees subject to the interview
survey pointed out that a simple comparison
between an application filed in Japan andthe corresponding application filed inanother country in terms of the scope ofclaims would be meaningless because thetwo applications do not necessarily contain75 ●IIP Bulletin 2008

the same claims. In order to make anaccurate comparison, the cases where
corresponding patents have been filed withforeign patent offices should be carefully
selected for enough statistic population as
individual cases subjected to the survey.
(3) Others
Since the JPO is expected to makereliable examination results available toforeign patent offices, a survey should be
conducted on foreign applicants as well inorder to measure their satisfaction with JPO
patent examination. To
obtaintheirevaluations, it is an effective way to conduct
a survey on agents who handle applications
from other countries or on applicants via
these agents.
(4) Conclusion
The
trial
surveyenjoyedanexceptionally high response rate despite thefact that the survey was conducted bymailing questionnaires containing freeanswer questions. This high rate suggeststhat applicants and agents recognize theneed for a customer satisfaction survey. It istherefore important to conduct such a survey
on a regular basis, while making efforts tofurther improve the questionnaire based on
the findings from the analysis andexamination carried out in this survey
research.
(Senior Researcher: Naoko KANEKO)●IIP Bulletin 200876 ●