White Paper - Measuring Patent Quality


View as PDF (PDF: 107 KB; 2 pages)

By Cecilia Alperin, Ph.D., Canadian Intellectual Property Office

It is often said that "you can't manage what you can't measure". However, there is no real consensus as to how to measure the quality of a patent. Some IP professionals speak of number of words in the independent claims, the overall number of claims, ratio of independent claims to dependent claims, number of alternative embodiments, number of office actions sent before grant, and number of forward citations. We could come up with countless other metrics. The truth of it though, is that it is extremely difficult to determine a true metric relating to the quality of a patent. Most metrics suggested are highly qualitative or subjective. However, that doesn't stop us from trying to measure the quality of the products we grant.

Different Patent Offices approach the topic in their own way. In addition, a large number of the metrics each office makes available to their stakeholders are not true measures of the quality of the patents they grant but more related to the quality of the service they provide. Below follows a summary of the quality indicators 6 major IP offices (USPTO, EPO, UKIPO, IP Australia, IP Japan, IP Korea) make available to their stakeholders.

The USPTO considers three major types of metrics: product indicators (office actions), process indicators and perception indicators. Product indicators include metrics on the correctness and clarity of work products. Process indicators track the efficiency and consistency of internal processes, mainly focusing on analyzing the reopening of prosecution, and rework of office actions. Lastly, perception indicators focus on both internal and external stakeholder surveysFootnote 1.

The EPO focuses more on customer satisfaction metrics as well as timeliness of the patent grant procedures and timeliness of customer service. The EPO monitors customer satisfaction by performing quantitative customer satisfaction surveys related to search and examination as well as patent administration services. Timeliness metrics focus on search, examination and duration of opposition procedureFootnote 2.

IP Australia publishes metrics relating to conformance with set quality standards. They are divided into three tiers which include ensuring correct search procedures are adopted and search results are correctly communicated, reports are free from invalid objections/considerations and that all objections which could adversely affect the validity of the patent are considered. Acceptable conformance levels along with results for the last quarter and last complete quarters are published. Service level commitments relating to the timeliness of first reports are also publishedFootnote 3.

The Japanese Patent Office publishes data relating to the level of satisfaction on the overall quality of patent examination for both national and international applications. The full survey results along with the survey methodology and sample questions are presented for the last 5 fiscal yearsFootnote 4.

The Korean Patent OfficeFootnote 5 publishes the results of peer review by the Examination Quality Assurance Officer. The EQAO samples completed examination cases and reviews them according to given guidelines in order to provide feedback to the examiner. A complex examination quality index is available to the public. That said, the available results are from 2012.

It should be noted that while all these Offices are designated as International Search Authorities as well as International Preliminary Examining Authorities, only the JPO publishes metrics relating to the quality of these services. Whether other Offices collect these metrics internally is unknown. However, given that most Offices do not actively inform their stakeholders regarding their international obligations leads one to believe that they consider quality of their national patents products to be more important.

CIPO is committed to ensuring a consistent client experience that delivers quality patent products and services in an efficient and timely manner, in order to create certainty in the marketplace and stimulate innovation. We've also committed to develop quality indicators, tools and checklists to improve the services we deliver to clients. At the moment CIPO makes available to its clients the following: service standards, monthly production statistics, the number of applications filed, inventory (RE awaiting 1st action), the number of patents granted, performance targets, along with the annual report, and the results of the Client Satisfaction Survey. It is unclear as to how useful our clients consider these indicators. In order to find out, CIPO is hosting the first CIPO Patent Quality Summit to engage with different groups as to how to best measure the quality of our products. We're hoping this dialogue will ensure that the indicators we will publish in the future are in line with our international colleagues as well as meet clients' expectations.