Valuation of Technology Partnerships Canada Benefits

This page has been archived on the Web

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Prepared for:

Audit and Evaluation Branch Industry Canada

Prepared by:

Logo - Hickling Arthurs Low. Technology Management, Strategy, And Economics

September 16, 2005 HAL Ref: 7553

Executive Summary

Technology Partnerships Canada (TPC) is a special operating agency of Industry Canada with a mandate to provide funding support for strategic research and development, and demonstration projects that will produce economic, social and environmental benefits for Canadians. TPC’s focus is on key technology areas such as environmental technologies, aerospace and defence technologies, and enabling technologies, which include biotechnology, advanced materials, advanced manufacturing, and information and communications technologies.

TPC offers two main delivery mechanisms: TPC R&D, delivered directly by TPC, that is targeted at larger firms, and TPC IRAP, delivered by the National Research Council Industrial Research Assistance Program, that supports small to medium-sized enterprises (SMEs) with projects valued under $3 million. Some 693 projects are reviewed in this study, including 420 projects with SMEs managed by TPC-IRAP.

The objective of this study is the valuation of the economic and non-economic benefits generated by TPC projects. The study uses a socio-economic model to value TPC benefits. Central to the TPC Valuation Model is the TPC Funded Companies that undertake technology development, product development, and eventually production and marketing activities. The TPC Funded Companies receive funding through TPC, and may be influenced and assisted by other government activities, such as the procurement of goods and services, the work of government laboratories, and the efforts of other government programs. The TPC Funded Companies, in turn, influence the rest of Canadian industry through the intentional and unintentional diffusion of skills and knowledge to other companies. The TPC Funded Companies also have economic and non-economic impacts through the Canadian users of their goods and services. In the end, the activities of the TPC Funded Companies, Canadian government, Canadian industry, and Canadian users result in impacts on the Canadian economy in the form of infrastructure improvements, wealth creation, and benefits in the public good.

A number of factors can be taken into account when evaluating the magnitude of the economic impacts:

  • Sales is the value of sales that will accrue to Canadian companies directly from the TPC funding, through spinoffs within the TPC Funded Companies, and through diffusion to other Canadian companies.
  • Attributed Sales is the value of sales adjusted for incrementality and attribution, i.e. the fact that some portion of the impact may have been achieved without TPC assistance and that TPC is not the only input to the achievement of sales and therefore should not be credited with the full value of the impacts.
  • Discounted Attributed Sales adjusts the attributed sales to account for the time-value of money – the preference for impacts that occur sooner rather than later. For example, spinoffs and diffusion impacts can occur many years after the base year. Discounting reduces these impacts by a discount factor.
  • Discounted Impact adjusts the discounted attributed sales to account for leakage from the economy and displacement of domestic sales. Leakage occurs because some of the value of products sold requires imports of component materials, which is a loss to the Canadian economy. Domestic sales that simply displace sales that would have accrued to other (non-TPC) Canadian companies do not provide a net gain for the economy, and are not counted. The sales that are counted as having impacts, therefore, are exports and incremental domestic sales.

The total economic impact of Technology Partnerships Canada is summarized in Table A. The discounted values are for a base year of 2005 and a discount rate of 10%.

Table A: Total Economic Impact Results*

Expenditures Expenditures   Discounted
Expenditures
Discounted
Impact
Program Expenditures $2,797,845,062   $3,741,166,880 $1,909,781,416
Company Expenditures $8,644,407,620   $11,508,389,309 $5,852,158,379
Total Expenditures $11,442,252,682   $15,249,556,189 $7,761,939,794
Impacts Sales Attributed Sales Discounted Attributed Sales Discounted Impact
Direct Sales $281,351,575,336 $84,405,472,601 $40,376,262,087 $14,300,588,608
Spinoff Sales $72,550,148,095 $21,599,348,749 $11,811,043,559 $4,138,198,907
Diffusion Sales $152,078,979,821 $4,223,230,959 $942,876,970 $352,850,660
Domestic User Value       $5,697,375,629
TOTAL $517,422,955,934 $110,228,052,309 $68,379,738,804 $32,250,953,598

* As of June 2005.

When comparing the economic impact of this program with others, it is important to be careful to compare like measures. Total Sales is often the measure quoted for other programs in studies of this type, however discounted impact is a more conservative and reasonable measure, and is the focus of this analysis.

In summary, the combined TPC R&D and TPC IRAP program expenditures of $2.8 billion plus Company Expenditures of $8.6 billion are forecasted to result in $11.4 billion of total expenditures. This is expected to result in $281.2 billion of resulting direct sales. Further, this will develop Canadian experience and capabilities that will be worth $72.6 billion in future sales for TPC Funded Companies (spinoffs), and $152.1 billion in future sales for other Canadian companies (diffusion). Note that the majority of this value is from estimates of direct sales, which are the benefits that are the most certain and which do not depend on the TPC valuation model.

When attribution, discounting, leakage, and displacement are considered, the impacts are as follows. The discounted (to 2005) program cost of $3.7 billion is forecasted to result in $15.2 billion in Canadian company expenditures. These expenditures will have a net impact of $7.8 billion in the Canadian economy and produce direct, spinoff, and diffusion sales representing another $18.8 billion in net impact. Finally it is estimated that these TPC derived products will have an economic benefit worth $5.7 billion to Canadian users. Overall, the net impact on the Canadian economy is estimated to be $32.3 billion, which is 8.6 times the program expenditures.

All forecasts of the future are inherently uncertain. This uncertainty has been addressed by assigning probability distributions to some of the input parameters and generating output distributions using Monte Carlo analysis techniques.

Figure A shows the distribution for Total Sales. The mean1 is $518 billion with a 90% probability range between $448 billion and $597 billion.

Figure A: Total Sales Probability Distribution

Figure A

Figure B shows the distribution for the Total Discounted Impact. The mean is $32 billion with a 90% probability range between $30 billion and $35 billion.

Figure B: Total Discounted Impact Probability Distribution

Figure B

Figure C shows the distribution for the ratio of discounted impact to discounted program expenditures. The mean is 8.7 with a 90% confidence interval between 7.7 and 9.7.

Figure C: Ratio of Discounted Impact to Discounted Program Expenditures Probability Distribution

Figure C

The use of TPC technology brings impacts that include economic benefits, but also extends to other benefits that cannot be expressed in economic terms, such as public health, sovereignty and security, environment, social, physical assets and advancement of knowledge. Some of these impacts accrue to the users of the technology; many also accrue to society in general.

Domestic impacts all accrue to Canada. Foreign impacts accrue to Canada only to the extent that Canadians benefit from impacts in other countries. For example, Canadians may benefit from the advancement of knowledge and contributions to the environment in other countries, but they probably do not benefit from contributions to economics, sovereignty and security, and public policy in other countries. We have only considered domestic non-economic impacts in this analysis.

The magnitude of the impacts depends on the degree of benefit in a particular instance, and the amount of use overall. Figure D shows the distribution of non-economic benefits over time. Interpretation of these numbers must be done with care:

  • The numbers have no absolute meaning2, however comparisons over time do provide an indication of relative impact.
  • Comparisons across impact types are not valid; this is because the impact types are inherently different and cannot be compared directly.

Figure D: Time Profile of Non-Economic Benefits

Figure D

Table of Contents

Figures

Tables

1. Introduction

1.1 Background

Technology Partnerships Canada (TPC) is a special operating agency of Industry Canada with a mandate to provide funding support for strategic research and development, and demonstration projects that will produce economic, social and environmental benefits for Canadians. The program aims to invest in high-risk ventures that are not viable from a strict financial point of view, but are to the benefit of Canada. TPC’s focus is on key technology areas such as environmental technologies, aerospace and defence technologies, and enabling technologies, which include biotechnology, advanced materials, advanced manufacturing, and information and communications technologies. In addition to repayments, the program seeks to achieve other benefits including increased technological capability, creation and maintenance of high quality jobs, and an improved environment.

TPC offers two main delivery mechanisms: TPC R&D, delivered directly by TPC, that is targeted at larger firms, and TPC IRAP, delivered by the National Research Council Industrial Research Assistance Program, that supports small to medium-sized enterprises (SMEs) with projects valued under $3 million. Some 693 projects are reviewed in this study, including 420 projects with SMEs managed by TPC-IRAP.

1.2 Study Objective

The objective of this study is the valuation of the economic and non-economic benefits generated by TPC projects.

The value of TPC projects comes from both the production of the technologies and their subsequent use. Potential benefits are of three types: economic, quality of life, and infrastructure. The production of technologies can result in economic benefits (such as revenue and jobs). The use of technologies can result in economic benefits (such as cost savings and quality improvements) and quality of life benefits (such as health improvements, sovereignty and security contributions, environmental benefits, social improvements, and advancement of knowledge). Infrastructure consists of things such as physical assets, improved skills, and new techniques that will result in economic or quality of life benefits in the future.

Benefits flow directly from the technologies that are developed, through spinoffs into other products and services within the contracted firm, through diffusion to other firms within the Canadian economy, and finally through the utilization of the products and services that embody the technologies by users within the economy.

With the exception of limited infrastructure benefits, there are benefits from projects to the extent that the technologies developed are eventually bought and used. Therefore, sales by the contracted firm are an important indicator of all the downstream benefits.

1.3 Report Structure

The model used for valuing the benefits from TPC is introduced in Chapter 2 and described in more detail in Appendix A.

The implementation of the model and the parameters values used in the analysis are described in Chapter 3.

The results from the model are summarized in Chapter 4, with the details provided in Appendix B.

2. TPC Valuation Model

This chapter introduces a socio-economic model to value TPC benefits. A technical description of the model is contained in Appendix A. Similar models have been used to assess the impact of a number of programs and the model has been calibrated and validated as a result. The model incorporates the following considerations:

  • Stages of development of the R&D process from basic research through market development to commercialization;
  • Assessments of increased R&D activity induced by the supported activity (the budget enhancement effect);
  • Assessments of incrementality whereby the impacts are reduced by what would have happened in absence of the program;
  • Assessments of attribution whereby contributions to the success of R&D are explicitly recognized so that the sponsoring program is not attributed all the benefits;
  • Spin-off activity (broad implementation of R&D results) in organizations that engage in supported R&D activity;
  • Diffusion of technology, innovations and information that extends the impacts of the results of R&D beyond the program participating organizations;
  • Time response of implementation or commercial sales from different stages of R&D and engineering activity; and
  • Assessment of both economic and non-economic impacts.

2.1 Structure

The TPC Valuation Model can be thought of in terms of a framework with five main parts (Figure 1). Central to the framework are the TPC Funded Companies that undertake technology development, product development, production and marketing activities. The TPC Funded Companies receive funding through TPC, and may be influenced and assisted by other government activities, such as the procurement of goods and services, the work of government laboratories, and the efforts of other government programs. The TPC Funded Companies, in turn, influence the rest of Canadian industry through the intentional and unintentional diffusion of skills and knowledge to other companies. The TPC Funded Companies also have economic and non-economic impacts through the Canadian users of their goods and services. In the end, the activities of the TPC Funded Companies, Canadian government, Canadian industry, and Canadian users result in impacts on the Canadian economy in the form of infrastructure improvements, wealth creation, and benefits in the public good.

The following sections examine these five parts of the TPC Valuation Model

Figure 1: The TPC Valuation Model Structure

Figure 1: The TPC Valuation Model

2.1.1 Canadian Government

Government can assist in technology development and commercialization through a number of mechanisms; for example procurement, laboratories, and programs (such as TPC). In the early stages of the technology development cycle, government assistance often plays a major role. The rationale for this assistance is twofold:

  • The private sector typically under-invests in research and development because the benefits that accrue to this kind of activity cannot be adequately captured by the firm (improved understanding cannot be readily marketed or patented). Also, the level of risk and uncertainty about activity costs and likely outcomes is such that one firm often cannot afford to undertake the activity.
  • Barriers to the successful market development of a major new product or system may exist when reliability, performance, and cost are not well enough known or not yet demonstrated. Government assistance or involvement in demonstration projects can effectively address these barriers to the benefit of all firms in the industry and the market segments involved.

The justification for government assistance is that the increased (or accelerated) economic and social benefits that accrue because of the government intervention exceed the economic and social costs.

In the later stages of the development cycle, government assistance often involves more general enterprise development. The reason for government intervention here is generally industrial development, regional development, employment generation, etc. that are consistent with government and social priorities and objectives.

2.1.2 TPC Funded Companies

TPC spending on technology development provides some immediate impact, however economic returns from technology come primarily from the commercialization and application of the technology rather than from its development. When assessing the potential payoff from technology development, it is these downstream impacts that must be identified and estimated.

These downstream impacts are of two types. First, the sale of new goods and services resulting directly from the TPC funded projects, and second, the longer-term spinoffs that result from the increased expertise and experience TPC Funded Companies acquire in undertaking the TPC work. Possession of a skilled and experienced labour force, production facilities, technological and project management expertise, and so on, will make a significant contribution to the future competitiveness of the TPC Funded Company.

Factors that characterize the relationship between contracted activity, direct sales, and spinoffs include:

  • The capability of firms that undertake the contracted activity to pursue commercialization; and
  • A long list of product and market development factors (time lags due to certification, registration, patenting, etc., competition from foreign sources, market conditions, business conditions, etc.) that impact whether the resulting product becomes a profitable venture.

2.1.3 Other Canadian Industry

Through diffusion, other companies, which do not have a relationship with TPC, can gain people, technology, or concepts from TPC Funded Companies that will have a beneficial impact on their own capabilities to develop and commercialize technologies. Diffusion occurs through mechanisms such as strategic partnerships, patenting, published papers, movement of personnel, and reverse engineering.

2.1.4 Canadian Users

There are also economic impacts resulting from the application of TPC technology by users in the form of improved product quality or lower costs of production that improve the competitiveness of these firms. It is recognized, however, that many such new technologies would likely be eventually available from outside of Canada. Therefore the incremental benefit comes mainly from the advanced timeliness of such developments or the application of technology in manners particularly suited to Canadian conditions, resulting in an accelerated adoption rate and greater penetration for Canadian users, as shown in Figure 2.

Figure 2: Technology Use

Figure 2: Technology Use

However, some of the economic benefits to users are negated by the price that users pay for the products and services. The benefit of interest here is the consumer surplus – the value of the product over and above its cost.

In calculating the economic benefit of the use of TPC technology, it has been assumed that users on average will buy technology at a price where their return on investment (ROI) for the goods or services equals the minimum acceptable ROI for the organization. Therefore, the economic benefit to Canadian users is at a minimum the expected ROI for investments in the user industry.

In addition (unlike the production and sale of products), the use of a technology may also have non-economic benefits for the environment, sovereignty & security, social welfare, health, and the advancement of knowledge. Since these are non-economic by nature, it is not practical to try to reduce them to monetary terms. Instead, we have estimated the magnitude of these impacts by assigning a qualitative weight to the relative degree of impact and multiplying it by the value of sales (as a proxy for amount of use).

2.1.5 Canadian Impacts

To summarize, the impacts from TPC projects come from three sources:

  1. Research and development of the technology,
  2. Production and sales of products and services that embody the technology, and
  3. Use of technology (embodied in goods and services).

While perhaps obvious, it is important to note that the benefits from technology production and use are directly proportional to the degree of use – if end users do not buy and use the technologies, there will be no benefits.

To the extent that spending on technology development and production occurs in Canada, there will be benefits to Canada. Less obviously, some portion of the work done in Canada will ‘leak’ out in the form of payments for component parts and services that are not available domestically.

Traditionally, impacts have been thought of in terms of direct wealth creation - stimulating the economy through the production and sale of tangible goods or services in the economy, usually by the private sector. In addition, we know that technology can enhance the social well-being of a country (the public good), and the infrastructure. Infrastructure improvements, such as faster communications systems or improved methods and codes, will ultimately contribute to productivity, wealth creation, and the public good.

2.2 Considerations

When analyzing the results of a program, the following considerations should be kept in mind.

2.2.1 Incrementality

The impacts and effects to be considered are those that are directly due to the government action under review – TPC. These impacts and effects are called incremental, which is defined as the difference between what did happen with the government action (the TPC program), and what would have happened if the action had not been taken. If nothing changes as a result of the action, impacts and effects are the same with and without the action, and incrementality is zero.

2.2.2 Attribution

A concept related to incrementality is that of attribution. Even if TPC makes incremental differences in impacts, some fraction of the impacts may logically be attributable to other programs, funding sources, organizations, or stimulants. Impacts and effects may have benefited from more than one government program or policy. Such incremental activities may give rise to impacts and effects that are not wholly (or fairly) attributable to TPC. In these cases, if the other programs or activities are to be credited with some of the impacts, these impacts must be attributed to the various contributing programs in some way. To the extent these other sources can be identified, they should share in the allocation of impacts and effects associated with TPC.

2.2.3 Time

Time frame plays an important role in the assessment of impacts. The major benefits attributable to R&D and technology transfer will accrue to society long after completion of particular government activities and over many years into the future.

This causes difficulties for identifying and measuring impacts and attributing them to the originating activity. These difficulties involve:

  • The uncertainty as to whether the program-assisted development will actually be implemented commercially to take advantage of the potential benefits;
  • The uncertainty as to whether the technology will perform after implementation as expected based on results from prototype or demonstration experience;
  • The lack of knowledge on the unintended or unexpected effects of introducing new technologies or changing existing processes; and
  • The uncertainty of the level of benefits and costs from any particular implementation of technology.

Also, there is a preference to have benefits sooner rather than later, so that future benefits have less value than present benefits, and should be discounted accordingly.

3. TPC Valuation Model Implementation

3.1 Overview

The TPC Valuation Model that has been outlined in the Chapter 2, and detailed in Appendix A, has been implemented as a computer program in an Excel spreadsheet using the @Risk analysis software. @Risk allows input parameters to be specified as probability distributions. Using a technique known as Monte Carlo analysis, the model is iterated thousands of times for many parameter values, creating output variables that are probability distributions.

The contents of the Excel spreadsheet are contained in Appendix B. The spreadsheet contains 14 worksheets, each concerned with a different aspect of the model.

The choices of input parameters for the model are described in the following sections.

3.2 Data Sources

Table 1 presents an overview of the data requirements for the TPC Valuation Model and the sources for the data. The sources are the project databases kept by TPC-R&D and TPC-IRAP, economic statistics from Statistics Canada and other sources, and interviews with a selection of firms. In some cases, parameter values will be based on multiple sources of information. Where possible, the reporting burden for companies was minimized by obtaining data from other sources.

Table 1: Data Requirements and Sources

Data Requirements TPC-R&D and TPC-IRAP Databases Economic Statistics Company Interviews
Technology Area    
Project Stage of Development    
TPC Contribution    
Company Contribution    
Other Contributions    
Probability of Success    
Incrementality    
Attribution  
Sector Number of Firms    
Sector Size of Firms    
Sector Linkages    
Project Sales  
Sales Delay    
Sales Period    
Exports  
Import Substitution    
Employment    
Contribution Repayment    
Expected ROI    
Discount Factor    

3.3 TPC R&D Data

3.3.1 Program Expenditures

The analysis of the TPC R&D program included 273 projects with a total authorized assistance of $2,641,108,284. The distribution of these projects among the three technology areas (aerospace and defence, enabling technologies, environmental technologies) is shown in Table 2. The H2 Early Adopters technology area, containing five projects worth $18,410,868, was omitted from the analysis because of limited data availability. Note that the full authorized assistance is not necessarily completely distributed to companies; however, in this analysis it has been assumed that it will be.

Table 2: Authorized Assistance

Technology Areas Number of Projects Authorized Assistance Average Assistance
Aerospace and Defence 161 $ 1,577,873,748 $ 9,800,458
Enabling Technologies 63 $ 588,868,270 $ 9,347,115
Environmental Technologies 49 $ 474,366,266 $ 9,680,944
Total 273 $ 2,641,108,284 $ 9,674,389

Information was available from the TPC R&D database on the contract start and end dates, but not on the specific disbursement of money over that period. Therefore, it was assumed that the money was disbursed evenly over the life of each project. The resulting spending profile for the program is shown in Figure 3.

Figure 3: Assumed Authorized Assistance Disbursement

Figure 3: Assumed Authorized Assistance Disbursement

Information was also available on each company’s contribution, which was assumed to be disbursed in the same profile as the program funding. The average company contribution multiplier (the amount the company contributed as a multiple of the program authorized assistance), the company contribution, and the total expenditures for the three technology areas are shown in Table 3.

Table 3: Company Contribution and Total Expenditures

Techonolgy Area Multiplier Company Contributions Total Expeditures
Aerospace and Defence 2.17 $ 4,604,199,211 $ 6,202,848,930
Enabling Technologies 3.25 $ 2,313,006,109 $ 2,902,012,774
Environmental Technologies 2.66 $ 1,425,375,279 $ 1,899,563,784
Total 2.51 $ 8,342,580,599 $ 11,004,425,489

3.3.2 Repayment Terms

TPC R&D contributions are repayable based on terms negotiated separately with each company. The contract files for each project were reviewed to determine the repayment terms. While there were variations among all of the contracts, in general they could be classified into six types:

  • Type A: Conditional Repayment
  • Type B1: Repayment based on the value of product sales
  • Type B2: Repayment based on the value of company sales
  • Type C: Repayment based on quantity sold
  • Type D: Unconditional repayment
  • Type X: Contract cancelled
  • Type NA: Information not available

Descriptions of each of these repayment terms types follow.

Type A: Conditional Repayment – 15% of the Contribution disbursed is repaid in each year of the repayment period, provided that the two conditions are met: 1) the company has achieved more than 10% growth in gross revenues over the previous year, and 2) the proponent’s gross revenues are higher than the ‘base amount’ defined as the revenues the year prior to the first repayment year. These payments continue until 115% of the Contribution disbursed has been paid, or 10 years has elapsed, whichever comes first. These repayment terms only apply to the Supplier Development Initiative (SDI), part of TPC’s Aerospace and Defence program.

Type B1: Repayment based on value of product sales – A percentage of gross project revenues is repaid in each year of the repayment period. If by the end of the repayment period, the total cumulative repayments equal or exceed a multiple of the contribution, no further repayments are due. However, if this amount has not been repaid, the payments continue for a further limited period or until the repayments equal the contribution multiple.

Type B2: Repayment based on value of company sales – A percentage of gross company revenues is repaid in each year of the repayment period. If by the end of the repayment period, the total cumulative repayments equal or exceed a multiple of the contribution, no further repayments are due. However, if this amount has not been repaid, the payments continue for a further limited period or until the repayments equal the contribution multiple.

Type C: Other – The repayment terms do not fit into the other categories. Many of the contracts of this type have repayments terms based on the quantity (rather than value) of a product sold.

Type NA: Information not available

Type U: Unconditional Repayment – The Contribution amount is repaid in 7 to 15 equal and consecutive annual payments, with the first repayment being payable two years after completion of the work.

Type X: Contract Cancelled – The contract has been cancelled and no further repayment is expected. In some cases, some money has been repaid, often in a lump sum.

The distribution, total value, and average value of the different contract types is shown if Table 4. The frequency of contract types among the three technology sectors is shown in Table 5.

Table 4: Value of Contract Types

Contract Type Number Total Contribution Average Contribution
A – Conditional Repayment 31 $ 18,605,667 $ 600,183
B1 – Product Sales 124 $ 1,545,447,380 $ 12,463,285
B2 – Company Sales 56 $ 435,639,317 $ 7,779,274
C – Other 15 $ 350,672,980 $ 23,378,199
NA – Information Not Available 14 $ 246,704,061 $ 17,621,719
U – Unconditional Repayment 22 $ 19,156,989 $ 870,772
X – Contract Cancelled 11 $ 24,881,890 $ 2,261,990
Total 273 $ 2,641,108,284 $ 9,674,389

Table 5: Frequency of Contract Types

Contract Type Aerospace and
Defence
Enabling
Technologies
Environmental
Technologies
Number
A – Conditional Repayment
31
0
0
31
B1 – Product Sales 72 35 17 124
B2 – Company Sales 22 19 15 56
C – Other 8 1 6 15
NA – Information Not Available 2 5 7 14
U – Unconditional Repayment 22 0 0 22
X – Contract Cancelled 4 3 4 11
Total 161 63 49 273

For the purposes of the analysis in this study, we need to determine the actual and forecasted sales of TPC R&D related products. Some of the contract types permit these actual and forecasted sales to be calculated from the company actual and forecasted royalty payments. Table 6 considers the applicability of the contract types for this calculation, and the assumptions that have been made.

Table 6: Contract Type Sales Calculation Applicability

Contract Type Applicability Assumptions
A – Conditional Repayment These repayment terms provide no information on sales. It has been assumed that sales are in the same proportion to repayments as in the average of the Type B1 and B2 cases.
B1 – Product Sales These repayment terms provide information on sales. Sales equal repayment divided by the royalty rate. It has been assumed that the maximum amount will not be reached by the end of the repayment period, and therefore the repayment amount does not under-estimate sales.
B2 – Company Sales These repayment terms provide information on sales. Sales equal repayment divided by the royalty rate. It has been assumed that the majority of company sales are related to the TPC project. Also, it has been assumed that the maximum amount will not be reached by the end of the repayment period, and therefore the repayment amount does not under-estimate sales.
C – Other These repayment terms provide no information on sales. It has been assumed that sales are in the same proportion to repayments as in the average of the Type B1 and B2 cases.
NA – Information Not Available No information. It has been assumed that sales are in the same proportion to repayments as in the average of the Type B1 and B2 cases.
U – Unconditional Repayment These repayment terms provide no information on sales. It has been assumed that sales are in the same proportion to repayments as in the average of the Type B1 and B2 cases.
X – Contract Cancelled Repayment terms are no longer relevant. It has been assumed that no further sales will result in these cases.

As can be seen in the table, only Type B1 and B2 contracts enable the calculation of sales from repayment information. Fortunately, these contract types represent the majority of cases – 66% of the number of contracts, and 74% of the value of contracts.

Type B1 and B2 contract types have associated royalty rates. Table 7 shows the average royalty rates for the three technology sectors for projects with those contract types.

Table 7: Royalty Rates for B1 and B2 Contract Types

Technology Area Number Average Royalty Rate
Aerospace and Defence 94 2.30%
Enabling Technologies 54 2.58%
Environmental Technologies 32 2.21%
Total 180 2.37%

In the case of Type U, NA, C, and A contracts, the average royalty rate for the corresponding sector has been used. It has been assumed that there will be no further sales from Type X contracts.

3.3.3 Repayment Estimates

At the contract stage, estimates are made of the expected repayments. Table 8 contains the aggregate of these estimates and Figure 4 shows the repayment profile. For a variety of reasons, repayments are typically less than originally forecasted. Every year, TPC adjusts their expectations of the repayment amounts based on project and company performance. Table 8 contains these estimates and Figure 4 also shows the current estimate of the repayment profile. At the time of contract, the aggregate of repayment estimates totalled about $4.3 billion, while the current repayment estimates total about $2.4 billion, or 55% of the original aggregated estimates. Figure 4 compares the aggregate of contract estimates and current total repayment profiles, while Figure 5 and Figure 6 show the original and current repayment profiles for each of the technology areas.

Table 8: Aggregate of Contract Estimates and Current Repayment Estimates

Technology Area Aggregate of Contract
Repayment Estimates
Current Repayment
Estimate
Percentage
Aerospace and Defence
$ 2,384,578,011
$ 1,569,378,414
66%
Enabling Technologies $ 1,109,059,854 $ 292,483,974 26%
Environmental Technologies $ 856,058,907 $ 533,712,956 62%
Total $ 4,349,696,772 $ 2,395,575,344 55%

Figure 4: Aggregate of Contract Estimates vs. Current Estimate Repayment Profile

Figure 4: Aggregate of Contract Estimates vs. Current Estimate Repayment Profile

As can be seen in Figure 4, repayments estimates have been both reduced and delayed, although estimates far in the future tend to remain constant.

Figure 5: Contract Repayment Profile

Figure 5: Contract Repayment Profile

Figure 6: Current Repayment Profile

Figure 6: Current Repayment Profile

3.3.4 Sales Estimate

The indicator of interest is the sales that result from TPC R&D funded research. However, sales data is not available directly – the program instead tracks estimates of repayments. For contract types where repayment is tied to sales and the royalty rate is known, sales can be calculated based on the repayment stream (actual and forecasted). For the minority of other contract types, we have made the assumption that, on average, the terms of the contract will reflect the majority of contracts where royalties are tied to sales, and therefore we have used the average royalty rate for each technology area to calculate sales.

Table 9 compares the current repayment estimate with the resulting sales to calculate the implied average royalty rate. Figure 7 shows the sales profile for the three technology areas.

Table 9: Implied Royalty Rates

Contract Type Current Repayment
Estimate
Resulting Sales Implied
Royalty
Aerospace and Defence
$ 1,569,378,414
$ 205,230,421,334
2.3%
Enabling Technologies $ 292,483,974 $ 13,876,328,319 2.6%
Environmental Technologies $ 533,712,956 $ 56,303,474,945 1.8%
Total $ 2,395,575,344 $ 275,410,224,597 2.3%

Figure 7: Sales Estimates

Figure 7: Sales Estimates

3.3.5 Non-Economic Benefits

Each of the projects was reviewed and rated according to its impact on six non-economic benefit types3: public health, sovereignty and security, environment, social, physical assets and advancement of knowledge. The original rating was at three levels: no impact, minor impact, and substantial impact. This was subsequently transformed, in aggregate, to a five-point scale (1 for little impact to 5 for substantial impact) for each of the technology areas, as shown in Table 10.

Table 10: Non-Economic BenefitRatings

Non-Economic Benefit Types Aerospace and Defence Enabling Technologies Environmental Technologies
Public Health 1.19 2.87 2.34
Sovereignty & Security 1.74 1.09 1.20
Environmental 1.46 4.88 1.27
Social 1.25 2.67 2.93
Physical Assets 1.72 1.70 1.51
Advancement of Knowledge 1.71 1.97 2.13

3.4 TPC IRAP Data

3.4.1 Program Expenditures

The analysis of the TPC IRAP program included 420 projects with a total authorized assistance of $156,736,778. The distribution of these projects among the three technology sectors is shown in Table 11. The majority of projects (55%) are in information and communications technology. Note that the full authorized assistance is not necessarily completely distributed to companies; however, in this analysis it has been assumed that it will be.

Table 11: Authorized Assistance

Technology Area Number of Projects Authorized Assistance Average Assistance
Advanced Manufacturing 78 $ 28,537,197 $ 365,862
Advanced Materials 23 $ 10,225,002 $ 444,565
Aerospace and Defence 11 $ 4,290,944 $ 390,086
Biotechnology 34 $ 13,240,466 $ 389,425
Environmental Technologies 44 $ 15,329,292 $ 348,393
Information Technology 230 $ 85,113,877 $ 370,060
Total 420 $ 156,736,778 $ 373,183

The spending profile for the TPC IRAP technology areas was available and is shown in Figure 8.

Figure 8: Authorized Assistance Disbursement

Figure 8: Authorized Assistance Disbursement

Information was also available on each company’s contribution, which was assumed to be disbursed in the same profile as the program funding. The average company contribution multiplier (the amount the company contributed as a multiple of the program authorized assistance), the company contribution, and the total expenditures for the six technology areas are shown in Table 12.

Table 12: Company Contribution and Total Expenditures

Technology Area Multiplier Company Contribution Total Expenditures
Advanced Manufacturing
2.67
$ 63,593,067 $ 92,130,264
Advanced Materials 2.24 $ 22,225,521 $ 32,450,523
Aerospace and Defence 2.10 $ 8,932,061 $ 13,223,005
Biotechnology 2.23 $ 29,227,608 $ 42,468,074
Environmental Technologies 3.50 $ 35,985,851 $ 51,315,143
Information Technology 2.55 $ 191,106,789 $ 276,220,666
Total 2.62 $ 351,070,897 $ 507,807,675

3.4.2 Repayment Terms

TPC IRAP contributions are repayable based on terms negotiated separately with each company. While there are variations among all of the contracts, in general they can be classified into four types:

  • Gross Revenues above baseline amount
  • Gross Revenues
  • Fixed Payments upon success condition
  • Other

Descriptions of each of these contract types follow.

RGR above baseline amountRGR is Gross Revenue. The baseline sales amount is negotiated with the firm on an individual basis and is based on sales history. The amount above baseline is assumed to be the incremental sales due to the TPC investment. The percentage is usually higher for “above baseline” projects and ranges from 8%-12%. The fixed period usually begins at 2-4 years out from first the TPC contribution and does not extend past 10 years, even if no repayment has been made. Usually, total repayment is capped at 150% of the TPC contribution.

RGR : – Repayment is based on total gross revenue of the firm, and not just incremental sales attributable to TPC. However, these firms typically have only the TPC supported product, so the result is that total firm revenues are mostly attributable to TPC. Terms are typically 1%. The fixed period is the same as above. Usually, total repayment is capped at 150% of the TPC contribution.

Fixed Payments upon success condition: – Repayment is fixed once a success condition is met.

Other: – Negotiated separately with each firm. It sometimes involves a fixed payment tied to sales ranges.

The distribution, total value, and average value of the different contract types is shown in Table 13. As can be seen, the majority of contracts (90% in number and 91% in value) are of the RGR type. The frequency of contract types among the six technology sectors is shown in Table 14.

Table 13: Value of Contract Types

Contract Type Number Total Contribution Average Contribution
FP upon success condition
5
$ 2,203,275
$ 440,655
Other 8 $ 3,042,037 $ 380,255
RGR above baseline 29 $ 9,362,830 $ 322,856
RGR 378 $ 142,128,637 $ 376,002
Total 420 $ 156,736,778 $ 373,183

Table 14: Frequency of Contract Types

Contract Type FP upon success condition Other RGR above baseline RGR Total
Advanced Manufacturing 1 1 2 74 78
Advanced Materials 0 1 2 20 23
Aerospace 0 0 2 9 11
Biotechnology 1 2 4 27 34
Environment 1 2 6 35 44
Information Technology 2 2 13 213 230
Total 5 8 29 378 420

Contracts have associated royalty rates. Table 15 shows the average royalty rates, where known, by technology area.

Table 15: Royalty Rates

Technology Area
Number
Average Royalty Rate
Advanced Manufacturing 76
3.12%
Advanced Materials 23
3.20%
Aerospace 11
1.87%
Biotechnology 34
6.20%
Environment 43
4.09%
Information Technology 229
2.84%
Total 416
3.29%

3.4.3 Repayment Estimates

At the contract stage, estimates are made of the expected repayments. Table 16 contains these estimates. For a variety of reasons, repayments are typically less than expected. IRAP does not keep revised estimates of expected repayments. Therefore, the TPC IRAP contract estimates have been reduced by the same factor as has been estimated for the SME firms for the TPC R&D projects. Since the majority (87%) of the TPC IRAP projects are Enabling Technologies, the factor for TPC R&D SME Enabling Technologies has been used (0.4) when estimating future repayments. Actual repayments have been used to date. The total of the repayments after applying this factor is $85.7 million, or 37% of the estimate of $228.9 million at the time of the contracts. Figure 9 shows this repayment profile for the six technology areas.

Table 16: Contract RepaymentEstimates

Technology Area Contract Repayment Estimated Repayment Ratio
Advanced Manufacturing $ 37,514,323 $ 12,406,767 0.33
Advanced Materials $ 14,698,750 $ 5,627,219 0.38
Aerospace $ 5,373,815 $ 2,253,644 0.42
Biotechnology $ 18,807,671 $ 8,437,811 0.45
Environment $ 20,603,633 $ 7,322,320 0.36
Information Technology $ 131,945,712 $ 49,637,807 0.38
Grand Total $ 228,943,904 $ 85,685,569 0.37

Figure 9: Repayment Profile

Figure 9: Repayment Profile

3.4.4 Sales Estimate

The indicator of interest is the sales that result from TPC funded research. At the contract stage, TPC IRAP makes estimates of future project related sales. These were used to calculate an implied royalty rate, which was then used in conjunction with the revised repayment estimates to determine the revised sales estimates.

Table 17 compares the revised repayment estimate with the resulting sales to calculate the implied average royalty rate. Figure 10 shows the resulting sales profile for the six technology areas.

Table 17: Implied Royalty Rate

Contract Type Revised Repayment Estimate Resulting Sales Implied Royalty
Advanced Manufacturing
$ 12,406,767
$ 1,117,958,101
1.11%
Advanced Materials $ 5,627,219 $ 407,608,553 1.38%
Aerospace $ 2,253,644 $ 164,249,853 1.37%
Biotechnology $ 8,437,811 $ 481,374,166 1.75%
Environment $ 7,322,320 $ 543,201,045 1.35%
Information Technology $ 49,637,807 $ 3,226,959,021 1.54%
Total $ 85,685,569 $ 5,941,350,739 1.44%

Figure 10: Sales Estimates

Figure 10: Sales Estimates

3.4.5 Non-Economic Benefits

Based on the review of the TPC R&D projects, the TPC IRAP projects were given ratings of their impact for six non-economic benefit types: public health, sovereignty and security, environment, social, physical assets and advancement of knowledge, as shown in Table 18.

Table 18: Non-Economic Benefit Ratings

Non Economic Benefit Types Advanced Manufacturing Advanced Materials Aerospace Biotechnology Environment Information Technology
Public Health 2.33 1.00 1.19 4.60 2.87 1.17
Sovereignty & Security 1.00 1.00 1.74 1.53 1.09 1.06
Environmental 1.67 2.33 1.46 1.42 4.88 1.06
Social 1.67 1.00 1.25 4.50 2.67 2.31
Physical Assets 1.00 2.33 1.72 2.05 1.70 1.18
Advancement of Knowledge 2.33 1.83 1.71 2.35 1.97 2.01

3.5 Other Parameters

3.5.1 Global Parameters

Many of the economic impacts of TPC accrue well into the future. In order to account for the preference that impacts occur sooner rather than later, a discount factor has been used to calculate the net present value (NPV ) of impacts. The choice of a suitable discount factor is somewhat contentious. We have chosen a rate of 10%, as recommended by Treasury Board4.

In order to calculate the distribution over time of impact, the TPC valuation models allows projects to be segregated into four stages of development, nominally termed research, development, engineering, and market. Each stage has a different lag time until returns begin to be realized. The lag times that have been used are as follows: research – 6.5 years, development – 4.5 years, engineering – 2.5 years, and market – 1 year.

3.5.2 Technology Specific Parameters

The TPC projects have been assigned to a stage of development category based on the anticipated lag between contract end and the start of repayments. Table 19 shows the percentage breakdown of projects in each of the categories.

Table 19: Stage of Development Breakdown by Technology Area

 
Stage of Development
Technology Area Market Engineering Development Reasearch
TPC R&D        
Aerospace and Defence 0.14 0.38 0.29 0.19
Enabling Technologies 0.21 0.40 0.32 0.07
Environmental Technologies 0.17 0.26 0.38 0.19
TPC IRAP        
Advanced Manufacturing 0.30 0.67 0.03 0.00
Advanced Materials 0.45 0.50 0.05 0.00
Aerospace 0.36 0.64 0.00 0.00
Biotechnology 0.24 0.67 0.09 0.00
Environment 0.28 0.65 0.08 0.00
Information Technology 0.43 0.54 0.04 0.00

The number of firms and size of firms participating in TPC projects, and in the total population of Canadian firms, are shown in Table 20 by technology area.

Table 20: Number and Size of Firms byTechnology Area

 
In TPC Projects
In Canada
Technology Area Number of Firms Size of Firms* Number of Firms Size of Firms*
TPC R&D        
Aerospace and Defence 161 3.16 300 0.40
Enabling Technologies 63 1.09 3147 0.09
Environmental Technologies 49 1.13 7474 0.02
TPC IRAP        
Advanced Manufacturing 78 0.09 25592 0.11
Advanced Materials 23 0.13 2402 0.15
Aerospace 11 0.18 300 0.40
Biotechnology 34 0.07 417 0.09
Environment 44 0.01 7474 0.02
Information Technology 230 0.07 2232 0.09

* Firm size is expressed in units of $100,000,000 of sales.

The import content of technology area products, the proportion of products exported and the amount of domestic sales that is assumed to represent import substitution is shown in Table 21 by technology area5.

Table 21: Import Content, Exports, and Import Substitution by Technology Area

Technology Area Import Content* Exports** Import Substitution***
TPC R&D      
Aerospace and Defence 0.51 0.73 0.1
Enabling Technologies 0.55 0.31 0.1
Environmental Technologies 0.30 0.30 0.1
TPC IRAP      
Advanced Manufacturing 0.51 0.49 0.1
Advanced Materials 0.36 0.61 0.1
Aerospace 0.51 0.73 0.1
Biotechnology 0.23 0.07 0.1
Environment 0.30 0.30 0.1
Information Technology 0.75 0.40 0.1

* ‘Import Content’ is the portion of inputs to products that have been imported into Canada.
** ‘Exports’ is the portion of goods and services that have been exported out of Canada.
*** ‘Import Substitution’ is the portion of goods and services that have been sold domestically and that displace goods and services that otherwise would have been imported. Otherwise, domestic sales displace the domestic sales of other Canadian companies.

3.5.3 User Benefits

The expected return on investment (ROI) for all user industries has been assumed to be 20%6.

4. TPC Valuation Model Results

The socio-economic model results presented in this Chapter are based on the model theory introduced in Chapter 2 and the model implementation described in Chapter 3. Quantitative estimates of economic impacts are reported in Sections 4.1 to 4.3, and estimates of non-economic impacts (public health, environment, sovereignty and security, social welfare, physical assets, and advancement of knowledge) are reported in Section 4.4. Finally, a short discussion of result validity is contained in Section 4.5.

4.1 Economic Impacts

4.1.1 Total Impacts

The total economic impact of Technology Partnerships Canada is summarized in Table 22. The results are broken out for the TPC R&D and the TPC IRAP programs in Table 23 and Table 24. The discounted values are for a base year of 2005 and a discount rate of 10%.

Table 22: Total Economic Impact Results

Expenditures Expeditures   Discounted Expeditures Discounted Impact
Program Expenditures $2,797,845,062   $3,741,166,880 $1,909,781,416
Company Expenditures $8,644,407,620   $11,508,389,309 $5,852,158,379
Total Expenditures $11,442,252,682   $15,249,556,189 $7,761,939,794
Impacts Sales Attributed Sales Discounted Attributed Sales Discounted Impact
Direct Sales $281,351,575,336 $84,405,472,601 $40,376,262,087 $14,300,588,608
Spinoff Sales $72,550,148,095 $21,599,348,749 $11,811,043,559 $4,138,198,907
Diffusion Sales $152,078,979,821 $4,223,230,959 $942,876,970 $352,850,660
Domestic User Value       $5,697,375,629
TOTAL $517,422,955,934 $110,228,052,309 $68,379,738,804 $32,250,953,598

Table 23: TPC R&D Economic Impact Results

Expenditures Expeditures   Discounted Expeditures Discounted Impact
Program Expenditures $2,641,108,284   $3,523,551,701 $1,819,470,645
Company Expenditures $8,293,336,724   $11,021,279,554 $5,649,522,409
Total Expenditures $10,934,445,007   $14,544,831,256 $7,468,993,054
Impacts Sales Attributed Sales Discounted Attributed Sales Discounted Impact
Direct Sales $275,410,224,597 $82,623,067,379 $38,727,078,294 $13,974,370,532
Spinoff Sales $68,985,374,440 $20,590,568,792 $11,159,345,077 $3,993,218,893
Diffusion Sales $133,535,842,605 $3,890,006,929 $855,578,397 $328,932,886
Domestic User Value       $5,331,106,929
TOTAL $488,865,886,649 $107,103,643,100 $65,286,833,023 $31,096,622,294

Table 24: TPC IRAP Economic Impact Results

Expenditures Expeditures   Discounted Expeditures Discounted Impact
Program Expenditures $ 156,736,778   $ 217,615,179 $ 90,310,771
Company Expenditures $ 351,070,897   $ 487,109,755 $ 202,635,969
Total Expenditures $ 507,807,675   $ 704,724,933 $ 292,946,741
Impacts Sales Attributed Sales Discounted Attributed Sales Discounted Impact
Direct Sales $ 5,941,350,739 $ 1,782,405,222 $ 1,649,183,793 $ 326,218,076
Spinoff Sales $ 3,564,773,656 $ 1,008,779,957 $ 651,698,482 $ 144,980,014
Diffusion Sales $ 18,543,137,216 $ 333,224,030 $ 87,298,573 $ 23,917,774
Domestic User Value       $ 366,268,700
TOTAL $ 28,557,069,285 $ 3,124,409,209 $ 3,092,905,781 $ 1,154,331,305

Economic impacts come from five sources: project spending, direct sales, spinoffs and diffusion from project spending, and domestic technology use. These are described briefly below:

  • Project Expenditures have an impact if it is assumed that the government spending, and the money it leverages from industry, is incremental to the economy. While this is rarely true, it is common to count program expenditures as an impact.
  • Direct Sales is the net economic activity resulting from the sale of products resulting directly from TPC funded projects.
  • Spinoff Sales is the net economic activity that Canadian companies participating in the TPC program achieve as a direct result of the knowledge and capabilities they acquire from their participation.
  • Diffusion Sales is the net economic activity that Canadian companies not participating in the program achieve as a direct result of the knowledge and capabilities that diffuses through the Canadian economy from the program. Diffusion occurs through mechanisms such as published papers, movement of personnel, and reverse engineering.
  • Domestic User Value is the net economic value to Canadian users of TPC funded technology. An argument can be made that users will continue to purchase the technology until the marginal cost of using the technology7 is equal to its marginal value. In other words, the marginal net value is zero and there is no incremental value to the economy. However, there will still be consumer surplus benefits to the technology used before the margin and these have been considered to be equal to the return on investment (ROI) expected by the user industry on their technology investments (assumed to be 20%).

A number of factors can be taken into account when evaluating the magnitude of the economic impacts:

  • Sales is the value of sales that will accrue to Canadian companies from each of the three sources described above.
  • Attributed Sales is the value of sales adjusted for incrementality and attribution, i.e. the fact that some portion of the impact may have been achieved without TPC and the fact that TPC is not the only input to the achievement of sales and therefore should not be credited with the full value of the impacts.
  • Discounted Attributed Sales adjusts the attributed sales to account for the preference for impacts that occur sooner rather than later. For example, spinoffs and diffusion impacts can occur many years after the base year.
  • Discounted Impact adjusts the discounted attributed sales to account for leakage from the economy and displacement of domestic sales. Leakage occurs because some of the value of products sold requires imports of component materials, which is a loss to the Canadian economy. Domestic sales that simply displace sales that would have accrued to other (non-TPC) Canadian companies do not provide a net gain for the economy, and are not counted.

When comparing the economic impact of this program with others, it is important to be careful to compare like measures. Total Sales is often the measure quoted for other programs in studies of this type, however discounted impact is a more conservative and reasonable measure, and is the focus of this analysis.

When evaluating the economic impact of a program, it is common to quote ratios of impacts to program cost. Ratios are provided here for each of the impact levels. The ratio for Sales has been calculated using the program expenditures in current dollars. The ratio for Discounted Impact has been calculated using the discounted program expenditures.

In summary, the combined TPC R&D and TPC IRAP program expenditures of $2.8 billion plus Company Expenditures of $8.6 billion are forecasted to result in $11.4 billion of total expenditures. This is expected to result in $281.2 billion of resulting direct sales. Further, this will develop Canadian experience and capabilities that will be worth $72.6 billion in future sales for TPC Funded Companies (spinoffs), and $152.1 billion in future sales for other Canadian companies (diffusion). Note that the majority of this value is from estimates of direct sales, which are the benefits that are the most certain and which do not depend on the TPC valuation model.

When attribution, discounting, leakage, and displacement are considered, the impacts are as follows. The discounted (to 2005) program cost of $3.7 billion is forecasted to result in $15.2 billion in Canadian company expenditures. These expenditures will have a net impact of $7.8 billion in the Canadian economy and produce direct, spinoff, and diffusion sales representing another $18.8 billion in net impact. Finally it is estimated that these TPC derived products will have an economic benefit worth $5.7 billion to Canadian users. Overall, the net impact on the Canadian economy is estimated to be $32.3 billion, which is 8.6 times the program expenditures.

All forecasts of the future are inherently uncertain. This uncertainty has been addressed by assigning probability distributions to some of the input parameters (see Chapter 3).

Figure 11 shows the distribution for Total Sales. The mean8 is $518 billion with a 90% probability range between $448 billion and $597 billion.

Figure 11: Total Sales Probability Distribution

Figure 11: Total Sales Probability Distribution

Figure 12 shows the distribution for the Total Discounted Impact. The mean is $32 billion with a 90% probability range between $30 billion and $35 billion.

Figure 12: Total Discounted Impact Probability Distribution

Figure 12: Total Discounted Impact Probability Distribution

Figure 13 shows the distribution for the ratio of discounted impact to discounted program expenditures. The mean is 8.7 with a 90% confidence interval between 7.7 and 9.7.

Figure 13: Ratio of Discounted Impact to Discounted Program Expenditures Probability Distribution

Figure 13: Ratio of Discounted Impact to Discounted Program Expenditures Probability Distribution

Figure 14 breaks down the total results into the TPC R&D and TPC IRAP components.

Figure 14: TPC R&D and TPC IRAP Sales, Discounted Impact, and Benefit Cost Ratio
Probability Distributions

Figure 14: TPC R&D and TPC IRAP Sales, Discounted Impact, and Benefit Cost Ratio Probability Distributions

4.1.2 Impacts by Technology Area

Table 25 to Table 33 and Figure 15 to Figure 23 show the results by technology area.

Table 25: Aerospace & Defence R&D

Expenditures Expeditures   Discounted Expeditures Discounted Impact
Program Expenditures $1,577,873,748   $2,160,168,237 $1,058,482,436
Company Expenditures $4,554,225,014   $6,093,483,533 $2,985,806,931
Total Expenditures $6,132,098,762   $8,253,651,769 $4,044,289,367
Impacts Sales Attributed Sales Discounted Attributed Sales Discounted Impact
Direct Sales $205,230,421,334 $61,569,126,400 $26,980,034,425 $10,972,780,001
Spinoff Sales $46,574,025,273 $14,903,688,087 $8,110,239,332 $3,298,434,337
Diffusion Sales $113,281,983,112 $3,398,459,493 $747,695,785 $304,087,876
TOTAL $371,218,528,480 $79,871,273,981 $44,091,621,312 $18,619,591,580

Figure 15: Aerospace & Defence R&D

Figure 15: Aerospace & Defence R&D

Table 26: Enabling Technologies R&D

Expenditures Expeditures   Discounted Expeditures Discounted Impact
Program Expenditures $588,868,270   $773,520,865 $348,084,389
Company Expenditures $2,313,218,094   $3,142,966,949 $1,414,335,127
Total Expenditures $2,902,086,364   $3,916,487,814 $1,762,419,516
Impacts Sales Attributed Sales Discounted Attributed Sales Discounted Impact
Direct Sales $13,876,328,319 $4,162,898,496 $3,011,327,243 $555,589,876
Spinoff Sales $12,139,452,340 $2,913,468,562 $1,664,555,519 $307,110,493
Diffusion Sales $11,606,834,910 $232,136,698 $56,147,866 $10,359,281
TOTAL $40,524,701,933 $7,308,503,755 $8,648,518,442 $2,635,479,167

Figure 16: Enabling Technologies R&D

Figure 16: Enabling Technologies R&D

Table 27: Environmental Technologies R&D

Expenditures Expeditures   Discounted Expeditures Discounted Impact
Program Expenditures $474,366,266   $589,862,599 $412,903,819
Company Expenditures $1,425,893,616   $1,784,829,073 $1,249,380,351
Total Expenditures $1,900,259,882   $2,374,691,672 $1,662,284,171
Impacts Sales Attributed Sales Discounted Attributed Sales Discounted Impact
Direct Sales $56,303,474,945 $16,891,042,483 $8,735,716,625 $2,446,000,655
Spinoff Sales $10,271,896,827 $2,773,412,143 $1,384,550,226 $387,674,063
Diffusion Sales $8,647,024,583 $259,410,737 $51,734,746 $14,485,729
TOTAL $77,122,656,236 $19,923,865,364 $12,546,693,270 $4,510,444,618

Figure 17: Environmental Technologies R&D

Figure 17: Environmental Technologies R&D

Table 28: Advanced Manufacturing IRAP

Expenditures Expeditures   Discounted Expeditures Discounted Impact
Program Expenditures $28,537,197   $41,738,806 $20,452,015
Company Expenditures $63,593,067   $93,201,028 $45,668,504
Total Expenditures $92,130,264   $134,939,834 $66,120,518
Impacts Sales Attributed Sales Discounted Attributed Sales Discounted Impact
Direct Sales $1,117,958,101 $335,387,430 $319,349,199 $92,323,854
Spinoff Sales $810,475,254 $218,828,319 $147,778,050 $42,722,634
Diffusion Sales $3,677,177,218 $110,315,317 $29,760,642 $8,603,802
TOTAL $5,697,740,837 $664,531,065 $631,827,725 $209,770,808

Figure 18: Advanced Manufacturing IRAP

Figure 18: Advanced Manufacturing IRAP

Table 29: Advanced Materials IRAP

Expenditures Expeditures   Discounted Expeditures Discounted Impact
Program Expenditures $10,225,002   $13,917,109 $8,906,950
Company Expenditures $22,225,521   $29,929,787 $19,155,063
Total Expenditures $32,450,523   $43,846,896 $28,062,013
Impacts Sales Attributed Sales Discounted Attributed Sales Discounted Impact
Direct Sales $407,608,553 $122,282,566 $99,684,564 $45,296,666
Spinoff Sales $388,367,821 $120,394,024 $76,392,374 $34,712,695
Diffusion Sales $1,827,508,993 $54,825,270 $14,727,443 $6,692,150
TOTAL $2,655,935,890 $297,501,860 $234,651,277 $114,763,524

Figure 19: Advanced Materials IRAP

Figure 19: Advanced Materials IRAP

Table 30: Aerospace IRAP

Expenditures Expeditures   Discounted Expeditures Discounted Impact
Program Expenditures $4,290,944   $5,892,616 $2,887,382
Company Expenditures $8,932,061   $11,967,594 $5,864,121
Total Expenditures $13,223,005   $17,860,210 $8,751,503
Impacts Sales Attributed Sales Discounted Attributed Sales Discounted Impact
Direct Sales $164,249,853 $49,274,956 $47,093,907 $19,153,092
Spinoff Sales $169,353,820 $54,193,222 $34,210,283 $13,913,322
Diffusion Sales $1,085,785,546 $32,573,566 $8,313,424 $3,381,069
TOTAL $1,432,612,224 $136,041,745 $107,477,823 $45,198,986

Figure 20: Aerospace IRAP

Figure 20: Aerospace IRAP

Table 31: Biotechnology IRAP

Expenditures Expeditures   Discounted Expeditures Discounted Impact
Program Expenditures $13,240,466   $18,467,765 $14,220,179
Company Expenditures $29,227,608   $41,226,575 $31,744,463
Total Expenditures $42,468,074   $59,694,340 $45,964,642
Impacts Sales Attributed Sales Discounted Attributed Sales Discounted Impact
Direct Sales $481,374,166 $144,412,250 $133,849,524 $17,520,903
Spinoff Sales $384,399,509 $119,163,848 $75,794,651 $9,921,520
Diffusion Sales $1,845,624,806 $55,368,744 $14,068,856 $1,841,613
TOTAL $2,753,866,555 $318,944,842 $283,407,371 $75,248,678


Figure 21: Biotechnology IRAP

Figure 21: Biotechnology IRAP

Table 32: Environment IRAP

Expenditures Expeditures   Discounted Expeditures Discounted Impact
Program Expenditures $15,329,292   $20,987,833 $14,691,483
Company Expenditures $35,985,851   $50,016,946 $35,011,862
Total Expenditures $51,315,143   $71,004,779 $49,703,345
Impacts Sales Attributed Sales Discounted Attributed Sales Discounted Impact
Direct Sales $543,201,045 $162,960,313 $134,017,656 $37,524,944
Spinoff Sales $186,818,112 $41,099,985 $25,931,951 $7,260,946
Diffusion Sales $3,607,410,972 $21,644,466 $5,455,574 $1,527,561
TOTAL $4,388,745,272 $225,704,764 $236,409,961 $96,016,796

Figure 22: Environment IRAP

Figure 22: Environment IRAP

Table 33: ICT IRAP

Expenditures Expeditures   Discounted Expeditures Discounted Impact
Program Expenditures $85,113,877   $116,611,050 $29,152,763
Company Expenditures $191,106,789   $260,767,825 $65,191,956
Total Expenditures $276,220,666   $377,378,875 $94,344,719
Impacts Sales Attributed Sales Discounted Attributed Sales Discounted Impact
Direct Sales $3,226,959,021 $968,087,706 $915,188,943 $114,398,618
Spinoff Sales $1,625,359,141 $455,100,559 $291,591,173 $36,448,897
Diffusion Sales $6,499,629,681 $58,496,667 $14,972,634 $1,871,579
TOTAL $11,628,168,508 $1,481,684,933 $1,599,131,625 $247,063,812

Figure 23: ICT IRAP

Figure 23: ICT IRAP

4.2 Regional Impacts

Regional impacts have been calculated under the assumption that impacts will be proportional to
regional program expenditures. Table 34 and Table 35 show the expected regional distribution of
discounted impacts for the TPC R&D and TPC IRAP programs respectively.


Table 34: TPC R&D Discounted Impact Regional Distribution

  Aerospace & Defence Enabling Technologies Environmental
Technologies
Total 1.00 $18,619,591,580 $2,635,479,167 $4,510,444,618
BC 0.12 $25,220,151 $13,797,694 $5,434,146
Prairies 0.03 $6,408,725 $3,506,150 $1,380,878
Ontario 0.42 $88,511,339 $48,423,674 $19,071,399
Quebec 0.39 $82,406,048 $45,083,530 $17,755,902
East 0.03 $7,224,545 $3,952,477 $1,556,661

Table 35: TPC IRAP Discounted Impact Regional Distribution

  Advanced
Manufacturing
Advanced
Materials
Aerospace Biotech Environment ICT
Total 1.00 $ 209,770,808 $ 114,763,524 $ 45,198,986 $ 75,248,678 $ 96,016,796 $ 247,063,812
BC 0.19 $ 39,987,938 $ 21,877,004 $ 8,616,138 $ 14,344,415 $ 18,303,375 $ 47,096,984
Prairies 0.21 $ 43,710,834 $ 23,913,763 $ 9,418,305 $ 15,679,887 $ 20,007,428 $ 51,481,736
Ontario 0.31 $ 65,896,041 $ 36,051,069 $ 14,198,516 $ 23,638,132 $ 30,162,094 $ 77,611,024
Quebec 0.20 $ 41,737,219 $ 22,834,018 $ 8,993,053 $ 14,971,914 $ 19,104,060 $ 49,157,252
East 0.09 $ 18,438,775 $ 10,087,670 $ 3,972,974 $ 6,614,330 $ 8,439,840 $ 21,716,816

4.3 Employment Impacts

Employment impacts have been calculated under the assumption that employment will be
proportional to sales. Table 36 and Table 37 show the expected regional distribution of
employment impacts, in terms of total person-years, for the TPC R&D and TPC IRAP programs
respectively.

Table 36: TPC R&D Person-Years Employment Regional Distribution

  Aerospace & Defence Enabling Technologies Environmental
Technologies
Total 1.00 67,708 17,570 30,070
BC 0.12 8,140 2,112 3,615
Prairies 0.03 2,069 537 919
Ontario 0.42 28,569 7,413 12,688
Quebec 0.39 26,598 6,902 11,813
East 0.03 2,332 605 1,036

Table 37: TPC IRAP Person-Years Employment Regional Distribution

  Advanced
Manufacturing
Advanced
Materials
Aerospace Biotech Environment ICT
Total 1.00 1,049 574 164 502 640 1,647
BC 0.19 200 109 31 96 122 314
Prairies 0.21 219 120 34 105 133 343
Ontario 0.31 329 180 52 158 201 517
Quebec 0.20 209 114 33 100 127 328
East 0.09 92 50 14 44 56 145

4.4 Non-Economic Impacts

The use of TPC technology brings impacts that include economic benefits, but also extend to other benefits that cannot be expressed in economic terms, such as public health, sovereignty and security, environment, social, physical assets and advancement of knowledge. Some of these impacts accrue to the users of the information; many also accrue to society in general.

Domestic impacts all accrue to Canada. Foreign impacts accrue to Canada only to the extent that Canadians benefit from impacts in other countries. For example, Canadians may benefit from the advancement of knowledge and contributions to the environment in other countries, but they probably do not benefit from contributions to economics, sovereignty and security, and public policy in other countries. We have only considered domestic non-economic impacts in this analysis.

The magnitude of the impacts depends on the degree of benefit in a particular instance, and the amount of use overall. Table 38 and Table 39 give a qualitative indication of the relative impact of TPC technology by impact type and application area. Interpretation of these numbers must be done with care:

  • The numbers have no absolute meaning9, however comparisons among the numbers do provide an indication of relative impact.
  • Comparisons can be made among applications within an impact type (i.e. within rows). Comparisons among impact types within applications (i.e. within columns) are not valid; this is because the impact types are inherently different and cannot be compared directly.

Table 38: Non-Economic Benefits, TPCR&D

  Aerospace and Defence Enabling Technologies Environmental Technologies
Public Health 14.18 25.14 13.99
Sovereignty & Security 20.79 9.52 7.16
Environmental 17.46 42.71 7.56
Social 14.92 23.35 17.51
Physical Assets 20.54 14.85 9.00
Advancement of Knowledge 20.42 17.22 12.72

Aerospace and Defence ranks very highly in all cases. This is because of the amount spent in this sector, more than the degree of benefits for a particular user.

Table 39: Non-Economic Benefits, TPC IRAP

  Advanced Manufacturing Advanced Materials Aerospace Biotechnology Environment Information Technology
Public Health 0.75 0.09 0.03 1.21 0.48 1.12
Sovereignty & Security 0.32 0.09 0.05 0.40 0.18 1.01
Environmental 0.54 0.21 0.04 0.37 0.81 1.01
Social 0.54 0.09 0.04 1.19 0.44 2.22
Physical Assets 0.32 0.21 0.05 0.54 0.28 1.13
Advancement of Knowledge 0.75 0.17 0.05 0.62 0.33 1.93

Figure 24 shows the time profile for the non-economic benefits.

Figure 24: Time Profile of Non-Economic Benefits

Figure 24: Time Profile of Non-Economic Benefits

4.5 Sensitivity

Figure 25 shows a tornado diagram for the major sources of sensitivity for the model input parameters that were assigned probability distributions. The two parameters that have the most impact are both discount rates: 1) the discount factor for the net present value calculations, and 2) the expected ROI of technology users. In the case of the discount factor, a higher value decreases impacts and is therefore more conservative (a triangular distribution with a peak of 10%, a minimum of 8%, and a maximum of 12% was used). In the case of the expected ROI, a higher value increases impacts and is therefore less conservative (a normal distribution with a mean value of 20% and a standard deviation of 5% was used for all technology areas).

Figure 25: Sensitivity for Total Discounted Impact

Figure 25: Sensitivity for Total Discounted Impact

4.6 Validity

Having produced estimates on returns, it is important to ask whether these estimates are, in fact, reasonable. Do they, for example, under-state or over-state the impacts? This is, of course, an empirically difficult proposition given the inherent uncertainty associated with technological innovation.

One approach is to assess the validity of the estimates through comparisons with those from other technology impact studies. In the following sections, impacts have been compared with estimates available in the literature on socio-economic returns to R&D investments, and with the overall benefit/cost ratios in studies evaluating the US Advanced Technology Program (ATP), a program that is similar in objectives and design to Canada’s TPC.

4.6.1 The Diffusion Literature

Diffusion of knowledge and skills results in TPC activities accruing to firms other than TPC Funded Companies. Among the earliest efforts to quantify diffusion was the work of Terleckyj (1974) who examined diffusion embodied in intermediate products including capital equipment. He estimated that the productivity improvements from R&D in downstream industries implied an excess return to industry R&D of 20% to 50%.10 Similar results were found by Mansfield, et al (1977), who identified 17 specific innovations, and attempted to estimate the actual cost and overall social benefits of each. For these technologies, the median private rate of return was about 25% and the median social rate of return was about 50%.11 These figures compare to an additional return of 30% for diffusion sales calculated in this study for TPC.

4.6.2 The Advanced Technology Program

The usefulness in comparing the HAL estimates to those from ATP studies stems from similarity of the program itself and the quality and quantity of quantitative evaluations of the ATP program itself. ATP began operations in 1990 with the goal of funding early-stage development of innovative technologies that show promise of significant commercial payoffs and widespread national benefits. Between 1990 and September 2004, ATP awarded $2.3 billion on a competitive basis, funding over 750 projects, with industry contributing a further $2.1 billion.

Table 40 summarizes the program statistics for ATP and TPC, along with economic impact figures from the literature and the HAL model respectively. Although extensive evaluations have been conducted on the ATP program, few carry out a full-scale quantitative valuation study as has been done here for TPC. Nonetheless, the studies do provide some indication of the type of returns technology programs can generate. One study of the ATP component-based software focused program, found the benefit cost ratio to be 10.5 with a net present value of the investment projected to be $840 million. This estimate, according to the authors, is deemed conservative due to the fact that benefits were calculated from just eight of the projects but included costs of all 24. Also, the authors note that the measures assumed very limited life spans for resulting products relative to their likely potential. In figures prepared by the ATP Economic Assessment Office, a total economic benefit of $17 billion was calculated from an analysis of 41 projects, though it is not clear from the study whether such a figure is fully attributable to ATP. Finally, in a third study, this one of a single project, ATP’s HDTV joint venture project, a more conservative benefit / cost ratio of 4.2 was achieved, with an approximate NPV of net benefits of $US166 million. These results are in reasonable agreement with those of this study that found a benefit cost ratio of 8.6 for TPC.

Table 40: Comparison of TPC and ATP,selected indicators

  TPC R&D TPC IRAP ATP
Objectives • Support for projects in technology areas that will produce economic, social and environmental benefits • Support for SMEs with projects valued under $3 million • Development of high-risk, enabling technologies
Supported sectors • Aerospace & Defence 59%
• Enabling Tech. (IT, Biotech) 22%
• Environmental Tech. 18%
• H2 Early
• Adopters 1%
• Adv. Manufacturing 18%
• Advanced Materials 7%
• Aerospace 3%
• Biotechnology 8%
• Environment 10%
ICT 54%
• Adv. Materials & Chemistry 21%
• Biotechnology 20%
• Electronics and Photonics 25%
IT 23%
• Manufacturing 11%
Cost sharing of project costs • 40% • 38% • Joint-ventures – 50%
• Large companies – 60%
SMEs – indirect costs.
Economic impact • Benefit cost 8.8 • Benefit cost 5.3 • Benefit cost of 10.5 (White et al. 200212) for component-based software
• $17 Billion in net economic benefits from 41 projects to date (ATP 2004 Annual report13)
• Benefit cost of 4.2 (White et al. 2004) for ATP’s HDTV Joint Venture project along with an estimated social rate of return of 28.6%14
Projects • 278 • 420 • 768
Program Contributions • $2,641 million • $157 million • $US 2,269 million
Industry Contributions • $8,343 million • $351 million • $US 2,102 million

A. TPC Valuation Model Description

A.1 Introduction

A.1.1 Modelling Parameters

Inputs to the Model include:

  • Expenditures: Total expenditures of the activity that generate spin-off benefits. This includes both public and private sector investment.
  • Type of activity: Accounts for the nature of the activity funded. Speculative activity with a large research component will have larger spin-offs than a contract for a deliverable (e.g. hardware) which requires little or no new technology development.
  • Incrementality and attribution: Incrementality refers to the difference in impacts and effects in Canada between what would have happened without the supported activity and what did (or will) happen with the activity. An attribution factor determines what portion of the total economic activity is justifiably attributable to the supported activity (i.e. accounts for the other contributing factors).
  • Other model parameters: Includes import content, number, and size of firms in sector, and sponsorship.

A.1.2 Categories of Benefits: Supported Activity, Spin-offs and Diffusion

Economic returns from technology come from the commercialization and application of the technology rather than from its development. Economic activity resulting from a million dollars worth of development of technology is essentially the same as from a million dollars worth of road building (or any other activity). The development of technology creates greater returns from investment because of its commercialization and application potential. When assessing the potential payoff from technology development, these downstream benefits must be identified and estimated. This assignment is concerned with the estimation of these downstream benefits (spin-off and diffusion) from the public support of R&D and the development of technology.

In the Model, we define public support of R&D and technology development as Supported Activity. The basic challenge of this study is the estimation of expected cumulative associated downstream benefits resulting from specific program supported activity.

The categories of Supported, Spin-Off and Diffused Technology impacts are illustrated in Figure 26. The direct output from the technology development program is Supported Activity, which consists of deliverables to the public sector and direct sales from products developed under contract.

Figure 26: Supported Activity, Spin-offActivity, and Diffused Activity

Figure 26: Supported Activity, Spin-off Activity, and Diffused Activity

Supported Activity leads to Spin-Off Activity within the same firms or organizations under contract to the sponsoring agency. This Spin-Off Activity results from the expertise and capabilities attained through their involvement in the Supported Activity. Possession of a skilled and experienced labour force, production facilities, technological and project management expertise, and so on, will make a significant contribution to the competitiveness of the industrial team in other product markets, and in both Canadian and world markets.

Technology that is developed under contract will be diffused throughout the industrial sectors in Canada. Thus, Diffused Technology Activity results in products from technology-producing firms other than firms who have undertaken supported activity.

Benefits from the use and application of spinoff and diffused technologies in Canada, other than for the firms that produce and sell those technologies, are very difficult to estimate. The benefits that accrue are often in the form of improved product quality or lower cost production in the field of application. Such improvements improve the competitiveness of Canadian firms and result in economic development benefits. There are also qualititative benefits that need to be taken into account such as environmental, sovereignty and security, social, health, and advancement of knowledge benefits. It is recognized, however, that many such new technologies would likely be developed even without the technology development program within some timeframe and in some country. The only benefits that can legitimately be counted here are those resulting from the advanced timeliness of such developments for Canadian firms because of the development in Canada of the spinoff technologies.

A.2 Model Concepts

The Model estimates the sales of spin-offs and diffusion of technologies developed under public sector initiatives. The underlying concepts for this Model come from an understanding of the innovation adoption process (Section A.2.1) and incrementality and attribution (Section A.2.2). Section A.2.3 illustrates how supported activity leads to spin-offs and diffusion.

A.2.1 Innovation Adoption Process

From past work, HAL has developed a descriptive model of the innovation development process from basic research to commercialization. We apply this descriptive model to define the stage of development of activity associated with Spin-Offs and Diffused Technology. This stage of development is then used to define the probability of achieving commercial success for a Spin-Off or Diffused Technology Activity, and to determine “downstream” or future expected costs of commercialization. This model is also applied to define the delay and time response of expected activity commercial resulting from R&D activity.

The innovation development process can be characterized as comprising four stages of activity (Research, Development, Engineering, Market) with three major decision points (a Go/No Go decision after each of the first three stages). This four-stage process is illustrated in Figure 27.

Figure 27: Simplified Four-Stage Innovation Development Process Model

Figure 27: Simplified Four-Stage Innovation Development Process Model

As shown, the typical R&D and innovation development process begins with many “ideas” only a few of which survive the inevitable assessments during investigation, research and development. The typical process is characterized by 20 “projects” at the output of the research stage for every project that is fully successful through market development to commercialization.

The typical number of projects at each stage for each successful project, can be interpreted in probabilistic terms to define probabilities of success. Using the average numbers of projects at each stage, the probability of a given idea reaching successful commercialization is one in a 100, or one percent. After successful completion of the research stage, the probability of commercial success is increased to one in 20, or five percent.

Assuming that a successful project would generate one million dollars of sales, the expected sales from any given project at any stage in the process would be the product of the cumulative probability and the sales of $1,000,000. From the (output of the) research stage, the expected sales would be 0.01 times $1,000,000, or $10,000, per project.

Costs of research, development, engineering and market development vary widely depending on the product, the innovation, the technology, the firm, the sector, etc. Surveys and analyses have been undertaken and aggregate average values exist in the literature15. For the high-technology sector, typical values for expenditures by stage of development (as a percentage of revenue or sales) are ten percent for R&D, ten percent for Engineering and 15 percent for marketing. These typical values have been calculated to include average allocations for G&A and profit--that is, the typical cost estimates for G&A and for profit have been allocated proportionally to engineering, marketing and costs of production.

These average cost estimates have been applied in the four-stage innovation development process model shown in Table 41. The appropriate probabilities of success from each stage, the probabilities of a project surviving the decision process, and the cumulative probabilities of success from each stage through all future stages to successful market development are also illustrated.

Table 41: Four-Stage Process Model with Probabilities and Costs

  Research Development Engineering Market
Total Cost of Stage $50,000 $50,000 $100,000 $150,000
  In Out In Out In Out In Out
Number of Projects 100 20 10 6 4 2.5 1.5 1
Cumulative Probability of Sales 0.01 0.05 0.1 0.167 0.25 0.4 0.667 1.0
Cost per Project $500 $2,500 $5,000 $8,333 $25,000 $40,000 $100,000 $150,000
Sales
Project Cost
2,000 400 200 120 40 25 10 6.7
Prob. x Sales
Project Cost
20 20 20 20 10 10 6.7 6.7
Stage of Development Factor 2.9 3.3 4.0 6.7

As illustrated in Table 41, the ten percent cost allocation for R&D has been divided equally between the research stage and the development stage. The average cost per project in each stage, and the ratio of sales to this average project cost, are shown in the table. This ratio of sales to the cost of the specific successful project at a given stage can be very large; i.e., the sales payoff for that one successful project can be as high as 2000 or 400 to one in the research stage. (The problem, of course, is to identify which project at these early stages will be successful.) These large ratios of sales to development costs are typical, and are sometimes quoted in a misleading way.

A more meaningful ratio is the expected sales (sales multiplied by the probability of successfully achieving the sales) divided by the project cost at each stage. These ratios are also shown in Table 41, and are the more typical 20 to one for the R&D stages, ten to one for the engineering stage, and 6.7 to one for market development.

A more useful ratio of sales per unit cost takes into account the costs of the “downstream” stages of development. This ratio is defined by apportioning expected sales between the current stage and all downstream stages in proportion to the expected costs of each stage. These apportioned expected sales are then divided by the current stage average project cost to yield a ratio of apportioned sales per unit project cost. Such a ratio recognizes that to achieve successful sales, there are future project costs to be incurred, and such future stages need to share the expected successful sales to justify their costs.

We define this ratio as the Standard Spinoff Ratio. It is essentially a “Spinoff” ratio - that is, a measure of the expected sales resulting from an R&D activity. The lower values at the earlier stages are consistent with the realization that government support is appropriate at these stages.

A.2.2 Incrementality and Attribution

Economic activity can be associated with supported activity in many ways. In this Model, we identify several factors that help determine what fraction of associated economic activity is justifiably attributable to the supported activity. Only the incremental and attributable activity is justifiably used to estimate the economic impacts of the supported activity. This interpretation of economic impact is stricter, and therefore tends to result in lower impact estimates, than that applied in many economic analyses. Comparisons with the results from other analyses, therefore, should include a comparison of the definitions of associated economic activity.

Incrementality refers to the difference in impacts and effects in Canada, and elsewhere if appropriate, between what would have happened without the supported activity and what did (or will) happen with the activity. If nothing changes as a result of this activity, impacts and effects are the same with and without, and the supported activity is said to have zero incrementality.

Even if the supported activity makes incremental differences in impacts, some fraction of the impacts may logically be attributable to other activity, funding sources, organizations or stimulants. To the extent these other sources can be identified, they should share in the allocation of impacts and effects associated with the supported activity.

Funding sources for activity associated with the supported activity other than from the funding source under review require particular attention. This additional funding has two effects: it increases the amount of activity, but the impacts of the activity must be allocated to their respective funding sources.

For Spin-Off Activity, incrementality is defined as the same as that for the associated Supported Activity. It is recognized, however, that there may be other factors contributing to the success of the Spin-Off Activity, and these factors are assessed as a separate attribution factor.

Diffused Technology Activity is stimulated by both Supported and Spin-Off Activity. The magnitude of the Diffused Technology Activity is related to the incremental and attributed Supported and Spin-Off Activity. In this way, the assessments of incrementality and attribution for the Supported and Spin-Off Activity are already automatically taken into account in the estimation of the size of Diffused Technology Activity. A further attribution to other sources for Diffused Technology Activity is assessed based on the expected nature of the activity, how it is linked to the supported activity, and what other influences there may be on the Diffused Technology Activity.

The assessments of incrementality and attribution are accounted for in the Model by means of multiplicative factors between zero and unity.

A.2.3 Generators and Receptors

An illustration of the process of how supported activity leads to spin-offs, and how these two activities lead to diffused technology, is given in Figure 28. In this figure, the initial Generator is identified as the Supported Activity that is closely linked to Receptor 1 where spin-offs are produced. Receptor 1 is the collection of firms (and other organizations, if appropriate) that are involved in the Supported Activity. By our definition, Spin-Off Activity takes place only in these organizations.

Figure 28: Generators and Receptors in Spin-Offs and Diffusion

Figure 28: Generators and Receptors in Spin-Offs and Diffusion

Once Supported Activity and Spin-Off Activity is underway, the diffusion process transmits certain ideas, information, licences, etc. to a broader set of firms and organizations which are illustrated as Receptor 2 in the figure. These organizations are still producers of technology, but they are once removed from the organizations which are involved in the supported activity. This group (Receptor 2) receives stimuli from both the Supported Activity and the Spin-Off Activity.

The linkage between the (Supported Activity) Generator and Receptor 1 is very close--there is usually a contractual linkage. The linkages between these activities and Receptor 2 is much more tenuous than that between the (Supported Activity) Generator and Receptor 1.

There is a set of organizations called Receptor 3 which represents the Users of New Technology in the Canadian Economy. It is within these firms that the new technologies will be put to use to reduce costs of production, to improve product quality or to develop new products.

A.2.4 Factors Affecting Spin-Offs and Diffusion

Estimating spin-offs and the extent of technology diffusion is not straightforward. Identifying the direct linkage between supported activity and spin-offs is often problematical, even for known spin-offs over historical time periods.

Factors which characterize the relationship between supported activity and spin-offs or the adoption of innovations include:

  • The capability of firms that undertake the supported activity to pursue spin-offs and innovations at the same time as performing the supported activity;
  • The capability and desire of private business to investigate and evaluate new technologies;
  • The number of firms or entrepreneurs that have access to a specific new technology;
  • The degree of linkage between firms that have the interest and desire and the firms that have direct access to the new technologies (usually via the supported activity); and
  • A long list of product and market development factors (time lags due to certification, registration, patenting, etc., competition from foreign sources, market conditions, business conditions, etc.).

These considerations are used to define the functional form of the Model, as discussed in Section A.3.

A.3 The Spinoff Component

A.3.1 Basic Form of the Model

A simple approach to calculating spin-off sales is to relate this value linearly to R&D or supported activity. In these applications, scalar parameters from less than five and up to 20 have been used and justified on the basis of specific examples.

In the Model, we have extended this simple linear model which links spin-offs to supported activity to account for some of the factors discussed in Section A.2. The Model is motivated by the observations that:

  • A given amount of supported activity will likely lead to greater spin-off and/or technology diffusion activity if more firms (or more people) are involved; and
  • For a given firm or set of firms undertaking a common supported activity, it is unlikely that doubling the expenditure on the supported activity will double the resulting spin-off and/or technology diffusion activity.

Consideration of a few hypothetical examples leads to the functional form for the Model outlined below. It seems reasonable to expect that doubling the level of the supported activity while at the same time doubling the number of firms involved in the supported activity would double the expected level of spin-off activity. This is consistent with the situation of having two separate programs sponsoring activity in two separate technology areas. For the Model to operationalize the two observations listed above and preserve the constant returns to scale characteristic illustrated in the example just cited, the following form is suggested:

Formula 1

With this form, a doubling of both the expenditure on supported activity and involving twice as many firms will double the expected spin-offs. The relative importance of the level of supported activity and the number of firms in determining spin-off sales can be adjusted by setting the exponent parameter a to different values between zero and one. Setting the parameter a equal to unity would, of course, let the model degenerate into a simple linear relationship between spin-offs and supported activity.

These observations were the motivation for the development of the model described below.

Formula 2

Based on the form of a model suggested above, we have formulated a descriptive, quantitative model of the Spin-Off and Diffused Technology process as follows:

In this formulation, the terms are defined as follows:

  • The Response in the Receptor refers to spin-offs, or to diffused technology, defined in terms of cumulative (over all future time) sales (in constant dollars). This response occurs in a group of firms which form the receptor associated with the driving force (stimulus).
  • The Factor M represents a set of multiplicative factors to account for incrementality, attribution and a scaling factor.
  • The Standard Spin-Off Ratio specifies the typical response to a stimulus (e.g., an R&D activity) in the innovation development process.
  • The Generator Activity Stimulus refers to the driving force for the spin-off, or for the diffused technology, and is measured in terms of cumulative expenditure (in constant dollars). For spin-offs, the generator activity is the Speculative Portion (as defined in Section A.3.2) of the Supported Activity, and for diffused technology, the generator activity is the sum of the Supported Activity and the Spin-Off Activity.
  • The Innovation Adoption Factor is an expansion of the “number of firms” factor introduced above.

These parameters and variables are described in more detail below.

A.3.2 Spinoff Sales

The Model recognizes two categories of Supported Activity: Speculative and Deliverable.

Speculative supported activity is the true technology development or R&D activity, activity that encompasses a certain degree of risk associated with R&D. Deliverable supported activity is procurement of technology by government for a specific purpose or application. Such activity generates spin-offs and diffusion but in a different manner than supported R&D activity.

Each category of supported activity will generate spin-offs, but under different circumstances and with different multipliers. Our approach to spin-offs based on the innovation development process applies to the speculative category of supported activity. For the deliverable category of supported activity, we apply a simple multiplier type of spin-off model. Such an approach adequately represents the assumed underlying processes linking spin-offs to this kind of supported activity, that is, spin-offs resulting from the increased capability resulting from learning new techniques due to participation in the technology development, and increased exposure and image associated with the specific deliverable.

The spin-off response is estimated separately for each of these categories. It is spin-offs from the Speculative category of Supported Activity that is usually more significant (in magnitude) and more complex to model. It is the Speculative category to which the above described models are applied.

Motivated by the considerations described above, and the model of the innovation development process, the Model Spin-Off component becomes:

Formula 3

where K is a scaling constant, the Incrementality and Attribution Factors account for all aspects of incrementality and attribution for spin-offs related to the Supported Activity, the Standard Spin-Off Ratio is as defined above, the Total Speculative Supported Activity includes the amount of the supported funding from other than sources under review, and the Innovation Adoption Factor is as defined below.

A.3.3 Innovation Adoption Factor

The Innovation Adoption Factor is defined by:

Formula 4

The product of the “number of firms” and the “size of firms” measures the aggregate size of the receptor for the Spin-Off or Diffused Technology Activity. The “size of firms” is measured in terms of average annual sales or revenue for the firms involved. These size factors account for the effect that innovations or opportunities that have been identified have a greater chance of progressing through all stages of development and marketing, and reaching the market place in a larger firm than in a smaller one. This effect is intended to take into account only the business and management risks involved; technical risks associated with the adoption of innovations and generating commercial sales is included separately as part of the Standard Spin-Off Ratio as described above. The incorporation of size in terms of sales in dollars has the desirable benefit that the equation is dimensionally correct; that is: (dollars)a X (dollars)1-a = dollars.

The “degree of linkage to activity” factor accounts for the separation between the driving or lead activity in the application to the situation of the adoption of diffused technology. In this situation, the Supported Activity together with the Spin-Off Activity, is the lead or driving activity for the diffusion of technology. In the situation of linking Spin-Off Activity to the Supported Activity, this linkage is very close because the Spin-Off Activity is assumed to take place within the same firms as the Supported Activity. For diffused technology, the firms adopting the new technology are only loosely linked with the leading activity in the Supported Activity firms.

The linkage factor is defined in relative terms with unity representing a perfect linkage between Sponsoring Activity and a spin-off receptor. This factor is an order of magnitude smaller when representing the linkage between the Spin-Off Activity and the Diffused Technology Activity.

A.3.4 Diffused Technology Sales

The calculation of cumulative diffused technology sales is the same as that for spin-off sales except that the driving force for Diffused Technology Activity includes both Supported Activity and Spin-Off Activity. Given the same form, however, the parameter values must be set to match the environment for diffused technology.

Formula 5

A.4 The Time Response Component

The Spin-offs component of the Model, as discussed in Section A.3, determines the total spin-off and diffused sales resulting from the total supported expenditures. It makes no distinction about when either the expenditures or sales occur. The distribution of benefits over time, however, is important since a benefit is preferred sooner to later. The Time Response component of the Model translates the stream of supported expenditures into a stream of spin-off and diffused sales so that later benefits can be appropriately discounted in value. This Section develops the formula for calculating the present value of the Supported Expenditures, Spin-off Sales, and Diffused Sales streams.

The essential feature of the Time Response component is the concept of a chain of stages of development activities that an innovation process passes through from the early stages of R&D to commercialization. The average time required to proceed from stage to stage, or from any stage to commercialization, has been studied and quantified using this model. As well, the technical risk for the successful development of innovations to commercialization has been defined using the model.

A.4.1 Distribution of the Supported Expenditures

The distribution of supported expenditures is an input to the Time Response component. The amounts are given in current dollars by year. The present value of this expenditure stream to the base year, PVBE, is calculated using the following formula:

Formula 6

Note that present valuing discounts expenditures which occur after the base year, but inflates expenditures which occur before the base year.

A.4.2 Distribution of Spin-off Sales

Spin-off sales are attributed to a particular year in the same proportion as the supported expenditures which occur in that year. For example, if 10% of supported expenditures are to be spent in 1999, then 10% of the spin-off sales are attributed to 1999. These sales are then distributed over time from that year forward.

When the sales will occur depends on the stage of development of the program. The Model considers four stages of development: Research, Development, Engineering, and Market. Typically, a supported program will have technologies in a number of these stages. The time response of each stage is modelled separately and the proportion of the program at each stage is used as an input.

Sales can only occur once a technology is brought to market. Each stage of development will have a delay, during which there will be no sales, and then sales will occur over time. The delay is greatest at the research stage and least at the market stage. The delay for each stage is an input.

Once sales begin, they are assumed to increase quickly, reach a maximum, and then decline slowly. This behaviour is modelled using a Weibull distribution. The shape of the distribution is defined by two parameters that are inputs to the Time Response component. They are determined empirically and are the same for all stages of development.

The total distribution over time of spin-off sales attributed to a particular year’s supported expenditures is calculated as the sum of these four delayed and scaled distributions, one for each stage of development.

The Time Response component of the Model is illustrated in Figure 29. In this example, the Base Year is considered to be year 0. Spin-off Sales of $1M are attributed to year 10.

Figure 29: Spinoff Sales Distribution

Figure 29: Spinoff Sales Distribution

The portfolio of projects responsible for these sales is distributed among the stages of development as follows: Research 0.3, Development 0.4, Engineering 0.2, Market 0.1. Sales from projects in the Market Stage begin immediately. Sales from Engineering Stage projects are delayed three years, Development Stage projects are delayed five years and Research projects are delayed 12 years.

In each case, sales are distributed over time according to a Weibull distribution with parameters 2.0 and 7.5. The Total Sales over time are shown by the dotted line.

Using Research Sales as an example, the Weibull distribution is present valued to the Initial Year of sales using Equation 2 (described below). This amount is then present valued to the Attributed Year using Equation 3. The Attributed Year amount is present valued to the base year using Equation 4.

The following equation calculates PVIS, the present value of a unit Weibull distribution to the initial year of the distribution:

Formula 7

The following equation calculates PVAS, the present value of the PVIS to the Attributed Year (the year of the supported expenditure), scaled according to the proportion of the program at each stage of development:

Formula 8

The following equation calculates PVBS, the present value of the PVAS stream to the base year:

Formula 9

A.4.3 Distribution of Diffused Sales

The distribution of diffused sales is calculated in a manner similar to the calculation of the spin-off sales. Like Spin-off Sales, Diffused Sales are attributed to a particular year in the same proportion as the supported expenditures which occur in that year. And like Spinoff Sales, Diffused Sales follow are assumed to follow a Weibull distribution, delayed and scaled according to the proportion of projects at each stage of development. The difference is that Diffused Sales are considered to be driven by Spinoff Sales, which themselves have a complex distribution over time, as described in the previous section.

The following equation calculates PVID , the present value of a unit Weibull distribution to the initial year of the distribution. It is analogous to PVIS for Spinoff Sales.

Formula 10

The following equation calculates PVAD , the present value of PVID for each stage of development to the year of the driving spinoff sales, scaled according to the proportion of the program at each stage of development. It is analogous to PVAD for Spinoff Sales except that the attributed year is the year of Spinoff Sales, rather than the year of Supported Expenditure.

Formula 11

The following equation calculates PVED , the present value of PVAD to the year of supported expenditure. This calculation is additional to those required by Spinoff Sales.

Formula 12

The following equation calculates PVBD , the present value of the PVAD stream to the base year:

Formula 13

A.4.4 PV of Total Sales

The present value of the total sales to the base year is then calculated as follows:

Formula 14

A.5 Analysis of Uncertainty

An important aspect of this analysis is the requirement to specify the degree of confidence in the results. It is our experience from reviewing many study reports that too seldom is the level of confidence or the range of results specified in the documentation.

The approach applied in this application not only specifies the level of confidence of the results, but incorporates the analysis of uncertainty of key factors. This approach comprises:

  • Identification of the key factors which are uncertain;
  • Quantification of this uncertainty using expert input; and
  • Combination of these assessments of uncertainty within a rigorous framework of applied probability theory.

The analysis of uncertainty process involves four steps.

  1. Development of the structure and logic models. This step establishes the approach, methodology and models, and ascertains which variables and assumptions must be subjected to detailed analysis in the Study.
  2. Estimation of initial parameter values and ranges. In this step, estimates and ranges are developed for each variable and assumption identified in Step 1. These estimates are based on the consulting team’s statistical analysis of survey and other data, and subjective judgement drawn from experience in the field.
  3. Expert Review. The purpose of expert review is to identify factors and issues that may have been overlooked in Step 2. The experts are best qualified to provide quantitative and qualitative information necessary to complete the analysis. Their experience, training, “street-wise” judgement and knowledge of relevant facts and issues provide a database and analytical process which would be impossible to model. Their opinions of the initial estimates and ranges enable the probabilities associated with expected outcomes to be assessed with a broad knowledge base.
  4. Simulation. Once the experts have completed their work, the ranges for each assumption are transformed within the computer model into input probability distributions. And once final distributions are generated for all assumptions and variables, they are combined using probability theory to yield a probability distribution for each output variable of interest.

B. Detailed Model Results

The following pages contain TPC Valuation Model parameters and results.

Chart
Chart
Chart
Chart
Chart
Chart
Chart
Chart
Chart
Chart
Chart
Chart
Chart
Chart
Chart
Chart

Footnotes

1 The Monte Carlo simulation results and the results in Table 4-1 do not agree exactly due to the finite number of simulations (10,000) that were performed. The Monte Carlo results will converge to the results in Table 4-1 as the number of simulations approaches infinity.

2 Numbers are the product of the value of sales (as a proxy for amount of use) and relative qualitative impact.

3 TPC has in the past examined five benefit types: 1. Benefits specific to recipient firms, 2. Benefits specific to the industry and to the Canadian Innovation Network, 3. Environmental benefits, 4. Connectivity and access to the e-world, and 5. Benefits to Canadians. The first of these encompasses a number of infrastructure benefits that will ultimately result in spinoffs. The second also covers infrastructure benefits that will ultimately result in diffusion. The third is identical to one of our non-economic benefits, “environment”. The fourth is one component of one of our non-economic benefits, “physical assets”. The fifth is a catch-all for the other non-economic benefits that we consider explicitly.

4 See, for example, the Treasury Board Secretariat, Benefit Cost Analysis Guide, DRAFT July 1998; www.tbs-sct.gc.ca/fin/sigs/Revolving_Funds/bcag/BCA2_E.asp. On the other hand, the 2001 report “Research at DOE: Was it Worth it?” from the United States National Research Council suggests a discount rate of 0% and the United States Office of Management and Budget specifies 7%.

5 Data was derived from OECD Input Output Tables. See www.oecd.org/document/1/0,2340,en_2825_495684_34062721_1_1_1_1,00.html

6 See for example, Moor, MC; Arent, DJ, and Norland, D (2005) “R&D Advancement, Technology Diffusion, and Impact on Evaluation of Public R&D, p5.

7 The cost of using a technology may be significantly greater than the cost of purchasing the technology. There are significant organizational costs to learning how to use the technology and incorporating it into the operational process of the organization.

8 The Monte Carlo simulation results and the results in Table 22 do not agree exactly due to the finite number of simulations (10,000) that were performed. The Monte Carlo results will converge to the results in Table 22 as the number of simulations approaches infinity.

9 Numbers are the product of the value of sales (as a proxy for amount of use) and relative qualitative impact.

10 This section is drawn from a summary of the literature on knowledge spillovers by A. Jaffe (1996) Economic Analysis Of Research Spillovers Implications For The Advanced Technology Program, Economic Assessment Office, Advanced Technology Program. http://www.atp.nist.gov/eao/gcr708.htm#II.B

11 Ibid.

12 W. J White, M. Gallaher (2002) Benefits and Costs of ATP Investments In Component-Based Software, Economic Assessment Office Advanced Technology Program, National Institute of Standards and Technology Gaithersburg MD 20899-4710. http://www.atp.nist.gov/eao/gcr02-834/gcr02-834.pdf

13 Mearuring ATP Impact, 2004 Report on Economic Progress, ATP Economic Assessment Office, http://www.atp.nist.gov/eao/2004annual/2004annual.pdf

14 W. J. White and A. O’Connor, (2004) ‘Economic Impact of the Advanced Technology Program's HDTV Joint Venture’, ATP Economic Assessment Office, NIST GCR 03-859. http://www.atp.nist.gov/eao/gcr03-859/contents.htm

15 We have based our estimates on data presented in “Technology Venturing in Canada: A Guide to the Commercialization of the Results of Federally Funded Research in Your Community” prepared for the Ministry of State for Science and Technology, by Denzil Doyle, 1986.