h2 Early Adopters Program
Risk Based Audit Framework (RBAF)
Archived
Table of Contents
- 1.0 Introduction
- 1.1 Background
- 1.2 Definition of Risk
- 1.3 Integrated Risk Management
- 2.0 Program and Risk Profile
- 3.0 Risk Assessment
- 3.1 Overall Risk Assessment
- 3.1.1Strategic Risks
- 3.1.2Technological Risks
- 3.1.3Project Level Risks
- 3.2 Approach to Risk Mitigation
- 3.1 Overall Risk Assessment
- 4.0 Risk-Based Audit Plan
- 4.1 Objectives
- 4.2 Risk Based Audit Plan
- 4.2.1Monitoring
- 4.2.2Recipient Audit
- 4.2.3Internal Audit
- 5.0 Reporting
1.0 Introduction
1.1 Background
The h2 Early Adopters (h2EA) program is aimed at addressing the urgent need toaccelerate market adoption of hydrogen technologies and other hydrogen-compatibletechnologies that facilitate the transition to a hydrogen economy and attract world-classtalent and investment to Canada. Through Technology Partnerships Canada (TPC),support will be given to the establishment of integrated hydrogen complexes such as"Hydrogen Villages", "Hydrogen Highways", and other similar partnerships in locationsacross Canada.
This support will be targeted at multiple public- and private-sector partners todemonstrate these technologies and showcase Canadian capabilities. These partnershipswill involve integrating hydrogen compatible technologies and hydrogen production,storage and distribution technologies with fuel cell and related portable, stationary andmobile applications in a microcosm of the hydrogen economy. Results of this initiativewill include real-world experience and expertise; early market adoption of hydrogentechnologies and infrastructure needed to support their wide-spread use; and increasedconsumer and investor awareness.
This Risk Based Audit Framework has been prepared in order to identify major riskswhich exist or may arise during the design, deployment and operation of this programover the five-year period. This framework provides a formal statement of the risks knownor likely at the time of its preparation. It will demonstrate that the h2EA program,Technology Partnerships Canada (TPC) and Industry Canada understand and accept thelevel of risk in the operation of this initiative and that they have measures and strategiesin place to identify and address such risks if they should occur.
In addition to this framework, a Results-based Management and AccountabilityFramework (RMAF) has been developed to identify the appropriate level ofmeasurement, management and reporting of progress in implementation and operation ofthe program. These two documents are complementary and should be considered as acoordinated response to the need to demonstrate accountability, manage and report onperformance, identify risks to objectives achievement and provide appropriatemanagement responses.
1.2 Definition of Risk
The definition of risk found in the Treasury Board Secretariat (TBS) Integrated RiskManagement Framework is the following:
Risk refers to the uncertainties that surround future events and outcomes. It is theexpression of the likelihood and impact of unplanned events with the potential toinfluence the achievement of objectives.
1.3 Integrated Risk Management
The current operating environment in the Government of Canada demands a moreintegrated approach to risk management than has previously been applied. Organizationsare faced with many types of risk, and it is no longer sufficient to manage risk on an adhoc basis. As described by TBS's Integrated Risk Management Guidelines, integratedrisk management is:
a continuous, proactive and systematic process to understand, manage andcommunicate risk from an organization-wide perspective. It is about makingstrategic decisions that contribute to the achievement of an organization's overallcorporate objectives through the monitoring of the program environment andplanned delivery.
2.0 Program and Risk Profile
2.1 h2 Early Adopters Program Background
While Canada is currently a world-leader in the development of hydrogen technologies,its first mover advantage is not secure since other countries are making significant newinvestments toward a hydrogen economy and are attempting to attract Canadian growntechnological leadership. In addition to this international competition, Canadian firmsface many technology and marketplace hurdles which they cannot overcome alone. Withrespect to hydrogen technologies, significant headway must be achieved in reducing costand improving their performance and safety. On the marketplace side, the most urgentneed is to begin testing hydrogen technologies and infrastructure in real-world settings toassess and improve reliability and durability, to support efforts to further reduce costs,and to increase public, consumer and investor awareness and acceptance of hydrogentechnologies. Implicit in a transition are necessary steps to an ultimate objective. Thus,emphasis is also required on technologies that are hydrogen-compatible.
Given the importance of social/behavioural change in the acceptance of newtechnologies, increased consumer and investor awareness and understanding of these newtechnologies and their uses are critical. There are significant technical, social,institutional and market barriers that cannot be overcome without government assistance.
Substantial progress has been and continues to be made through a variety of federalgovernment initiatives in the areas of technology development and partnership activities,however greater collaboration and coordination is required to meet the next strategicobjective - high profile, multiple partner market demonstration projects.
The h2EA program will be managed by Technology Partnerships Canada. However, theprogram has its own separate Terms and Conditions and is distinct in that it includescontributions to coalitions.
The h2EA program will support public and private-sector partnerships that showcasehydrogen complexes, for example "Hydrogen Villages" and "Hydrogen Highways", todemonstrate hydrogen technologies, illustrate their integration and accelerate marketacceptance and take-up. Through demonstration projects, we will learn by doing: lessonslearned from both successes and failures will advance the science and pace oftechnological development in this field. The historical experience has shown that thesedemonstration projects do not allow firms to rapidly realize main revenue streams, butrather help them to pursue further research and development on next generationtechnologies. Pursuant to the Treasury Board Policy on Transfer Payments, thoughcontributions under the h2EA program are repayable, little or no repayments areexpected.
The h2EA program will be implemented by Industry Canada through TechnologyPartnerships Canada (TPC) in conjunction with the regional development agencies andother federal departments and agencies and their respective programs. Memoranda ofunderstanding may be developed between TPC and federal departments and agencies tosupport the implementation of this initiative.
During the period FY2003-04 to FY2007-08, funding will be made available by TPC tothe Canadian Transportation Fuel Cell Alliance (CTFCA) to allow it to strengthen orextend its ability to meet the objectives of the h2EA program.
Regional development agencies will be invited to participate in the building ofpartnership coalitions, considering and monitoring the projects, and communicating thesuccesses of regional initiatives. Other federal departments and their respective programswill also be invited to participate, where their program objectives complement those ofthe h2EA program.
2.2 Roles and Responsibilities
Consistent with the Treasury Board Policy on Transfer Payments, TPC management willbe responsible for determining whether recipients under the h2EA program havecomplied with the terms and conditions of contribution agreements. Audits of recipientswill be undertaken when deemed necessary, and will be risk-based in accordance withthis audit framework.
Program Management
Program Management at TPC is responsible for the ongoing financial and operationalmonitoring of the program. They are responsible for the audit of recipient's complianceto terms and conditions of the contribution agreements and the reliability of results data.
Co-deliverers
Regional Development Agencies (RDAs) will assist with the delivery of the program.
Audit and Evaluation Branch
The Audit and Evaluation Branch (Internal Audit) employs a risk-based approach toplanning and conducting audits. These audits provide assurance on the adequacy ofintegrated risk management practices, management control frameworks and informationused for decision-making and reporting in the achievements of overall programobjectives.
2.3 External and Internal Environment
There are key internal and external factors and risks that will influence implementation ofthe h2EA program.
External factors that have been identified as potential risks include:
- Political - ongoing political support, unexpected changes in relationships between federal and provincial partners
- Economic - impact of the state of economic activity on the demand for hydrogen and fuel cell technologies and the willingness of developers and suppliers to participate
- Social - levels of social interest in and acceptance of development of the hydrogen economy in achieving social and public good objectives
- Technological - the pace of technological change and the ability of the program to keep up with change; rate of commercialization necessary to maintain a viable industry
- International - changes in international relationships that may affect the willingness of foreign research organizations to collaborate with Canadians
Internally, the following factors are considered relevant to the h2EA program's riskprofile:
- overall management approach;
- governance and accountability structures;
- values and ethics;
- operational work environment;
- corporate risk management culture and tolerances;
- existing risk management expertise and practices;
- human resources capacity, and,
- corporate policies, procedures and processes.
These external and internal factors have been given careful consideration in thedevelopment of this RBAF.
3.0 Risk Assessment
3.1 Overall Risk Assessment
The following is a summary of the main risks identified in conjunction with deploymentand implementation of the h2EA program.
3.1.1 Strategic Risks
These are defined as risks outside the control of the program that can have a significantimpact on the overall success of the project.
- Economic: Changes to the Canadian economy could affect either demand for research or the ability of firms to deliver the required services.
- Political: Political changes could affect support for the Hydrogen Initiative or for the h2EA program at the federal or provincial government level.
- Social: Changing attitudes towards and scepticism of hydrogen technology and other non-polluting sources of energy may affect social acceptance of the technology being demonstrated through the h2EA program.
- International: Changes in the international economy, relationships, research environment or investment by foreign governments in hydrogen technology may affect the amount of investment to develop the hydrogen economy in Canada and in other countries - risk that investments by other countries may draw technical expertise and commercial interest away from Canada.
A number of specific risks associated with the technology, the management, design, anddeployment of the h2EA program are discussed in the following sections.
3.1.2 Technological Risks
The h2EA program targets integrated models of hydrogen systems - these are consideredpre-commercial but later stage demonstration. There is a risk Canadian firms' technologyis not ready for this advanced stage of demonstration and program uptake may not be asstrong in early years of the program.
3.1.3 Project Level Risks
Production Capacity
Canadian hydrogen companies are currently demonstrating technologies and systemsmainly abroad and currently have 2-3 years of production of systems planned andcommitted. Given their current limited production capacity, Canadian program will becompeting with foreign jurisdictions for projects and may not get sufficient uptake on theprogram if its Terms and Conditions are not as or more attractive as those otherjurisdictions.
Partnerships
The mandate of the program will require strong partnerships within the industry todeliver an "integrated model of a hydrogen system, such as "hydrogen villages" or"hydrogen highway". The program will more than likely fund one lead organization topartner with other organizations. There is a risk that the program does not receiveapplications for the types of projects it is targeting or that partnerships that are developedare not maintained over the project period.
In addition, there are risks related to liability. The way the project is designed, where acontract may be solely with one lead organization, TPC will not be able to easilyfacilitate both financial and technical liability issues between the lead contractor andpartners which may lead to non-completion of projects.
Program Management
The h2EA program plans to establish a firewall between the h2EA program and TPCbase program. There is a risk that this firewall is not effective and the lines becomeblurred between the two programs causing issues with clients on the "fairness" of Termsand Conditions of TPC and of the h2EA program which has a much higher contributionrate.
3.2 Approach to Risk Mitigation
A number of risk areas have been identified above as potentially affecting the success ofthe h2EA program. With active monitoring, planning and auditing, most of these riskscan be adequately dealt with. A number of the key performance indicators in the RMAFreport are closely related to the risks identified and the Departmental Performance Reportwill contribute to the active monitoring of these risks.
In the following sections, Tables 1 and 2 are general descriptions of risk factors, theireffects and the required management response. They are used as the basis of assessing therisks listed in Table 3, which describes the risk areas, the likelihood of occurrence andlevel of impact, the results of the risk, and the response strategy.
Risk Likelihood
Risk likelihood is classified as:
- low;
- medium; or,
- high.
Risk likelihood depends on the probability of the risk occurring over the period of theinitiative.
Risk Impact
Risk impact is classified as minor, moderate or severe. These are described in moredetail below.
Table 1 - Risk Impact | ||
---|---|---|
Minor | Moderate | Severe |
Causes disruption to specific areas of the program, and potential setback. | Causes disruption to some essential program elements, negative image and media attention, and probable delivery and/or operational setback. | Significant disruption to the overall program, underachievement of objectives, criticism of the initiative and the department. |
Relationship between Risk and Required Management Action
Table 2 outlines the relationship between risk and required management action.
Table 2 - Relationship between Risk and Required Management Action | |||
---|---|---|---|
Impact | Likelihood | ||
Low (L) | Medium (M) | High (H) | |
Severe (S) | Considerable monitoring and management required | Must monitor and manage risks | Extensive management effort essential |
Moderate (Mod) | Risks can be accepted with monitoring | Management effort required | Management effort required |
Minor (M) | Accept risks | Accept, but monitor risks | Monitor and manage risks |
Matrix of Risks, Likelihood, Impact, Effects and Response Measures
The matrix in Table 3 outlines the key risks and describes the likelihood, impact, effectsand response measures.
Table 3 - Matrix of Risks, Likelihood, Impact, Effects and Response Measures | ||||
---|---|---|---|---|
Risk | Severity (Minor, Moderate, Severe) | Likelihood (Low, Medium, High) | Assessment | Response Strategy |
Strategic Risks | ||||
Changing attitudes towards and scepticism of hydrogen technology and other non-polluting sources of energy may affect acceptance | Severe | Low | Lower market acceptance of technology, progress negated by entrenched attitudes | Monitor and adjust communication strategy, activities as necessary |
Changes in the international economy, relationships, research environment or investment by foreign governments in hydrogen technology may affect the amount of investment to develop the hydrogen economy in Canada and in other countries and the level of competition faced by Canadian Firms | Severe | Medium | Other countries move faster than Canada, Canadian capacity committed to foreign projects | Implement program quickly Establish cooperative arrangements with other countries, particularly the U.S. |
Technical Risks | ||||
Technological performance does not meet expectations | Moderate | Low | Technology is new and, in some cases, untried | Program staff will perform systematic review and assessment before decisions are made on subsequent stages and investments |
Project Risks | ||||
Canadian firms' technology is not ready for this advanced stage of demonstration | Moderate | Low | Program uptake may not be as strong in early years of the program; failure of technological integration; failure to receive comprehensive proposal; failure to bring the different players to work together | Involve co-delivers, stakeholders and users in implementation and deployment of program Monitor program delivery and program uptake |
Canadian firms not able to come together to form coalitions (e.g., compete negatively) | Severe | Low | Failure to receive comprehensive proposal; failure to bring the different players to work together | Monitor program delivery and program uptake |
Coalitions not able to manage project for full term, coalitions dissolves | Severe | Low | Failure of financial viability of one of the player, project delayed or terminated | Due diligence on potential recipients. Monitor program delivery closely |
Recipients do not comply with Terms and Conditions | Moderate | Low | Improper payments, requirement to recover funds, loss of confidence in program and TPC | Due diligence on potential recipients, Ts&Cs detail conditions for receipt and audit of expenses. Ts&Cs can be amended by mutual consent to reflect unexpected circumstances, monitoring provisions to review recipient performance and expenditures |
Internal Risks | ||||
Inability to obtain relevant information to measure program's performance and achievement of outcomes | Moderate | Medium | Lack of accountability, loss of confidence by partners and stakeholders | RMAF identifies indicators for required information Report on critical performance indicators annually |
Inadequate capacity to deliver the program | Severe | Low | Particularly a risk in the implementation of the program | Re-allocate human resources to this program. Monitor program delivery |
4.0 Risk-Based Audit Plan
4.1 Objectives
This risk based audit plan has a number of objectives. It is intended to ensure that:
- appropriate diligence is exercised with respect to the expenditure of public funds;
- the program is administered in accordance with the Terms and Conditions in the funding agreement;
- relevant legislation and policy of the Financial Administration Act are respected; and,
- the quality of information used by the program, TPC and Industry Canada to monitor and manage the initiative are relevant and available for decision-making purposes.
This risk based audit plan will identify a number of strategies to manage the risksidentified in the previous section. These include audits of management practices,operational controls, and contribution agreements, as well as performance monitoring andreporting identified in the companion Results-based Management and AccountabilityFramework.
This framework will cover the above issues with regards to the management of the h2EAprogram funding and will also provide the basis for audits of management practices andoperational controls. This will give management the ability to ensure the propermanagement of contributions as well as ensure the proper management of the overallprogram.
4.2 Risk Based Audit Plan
The risk based audit plan will cover all RBAF objectives associated with transfer offunds, and will also examine the need to audit program and financial controls. Thisprovides the ability to ensure that the funds provided to TPC to deliver the h2EAprogram are well managed and that the overall delivery of the initiative is well managedby both Industry Canada and TPC.
TPC has certain tools and techniques such as the risk rating scale, detailed guidelines forrisk rating and a set of procedures for risk assessments to enable identification of risks,assess the impact of risks as well as to monitor risk migration throughout the project's lifecycle - all of which contributes towards accountability and stewardship of public funds.
4.2.1 Monitoring
Program management of TPC will perform monitoring related to the h2EA program inthree areas:
- the external and internal environment;
- the indicators set out in the RMAF; and,
- the operational efficiency and effectiveness control process.
Monitoring, in order to reduce risk, will also be conducted on a routine basis as part ofstandard project management practices in the areas of:
- achievement of expected outcomes;
- threats to achievement of expected outcomes;
- due diligence in the expenditures of funds; and,
- the efficient, effective and economical use of resources.
4.2.2 Recipient Audit
TPC will adopt a risk-based approach to the selection of contributions for audit. Theaudit risk of each contribution agreement will be assessed annually. The intent is to auditcontributions where TPC has identified some particular concerns. In addition, auditselection criteria will be used to help determine the purpose, scope and timing ofcontribution audits. TPC will undertake to coordinate its audit plans with othergovernment organizations at the federal and/or provincial level, where these are involvedin funding the same projects, and will give due consideration to audits of recipientsperformed by other government organizations. The h2EA program will require auditorsto agree that contribution audits will be conducted according to generally acceptedauditing standards.
4.2.3 Internal Audit
Audit and Evaluation Branch is responsible for the internal audit of the administration ofthe h2EA program. The h2EA program will be included in the annual audit universe riskassessment, and will be included in the audit plan based on relative risk.
In its audit of the contribution program, Internal Audit will assess the adequacy ofprogram and financial controls to identify and assess risks over the selection, approval,payment and review of eligible projects or activities and the program's operations: thepropriety of transactions; general compliance with Terms and Conditions; the economy,efficiency and administrative effectiveness of program operations and delivery systems;and, any other requirements stipulated under the Treasury Board's Policy on TransferPayments, the most current version of the Treasury Board publication Guide on the Auditof Federal Contributions, or specific to the Terms and Conditions of the program. Risksto be considered during the audit of the management of the program include: the dollarvalue of projects; a project's public profile, etc.
5.0 Reporting
The h2EA program will rely on the following reporting strategy:
- The Internal Audit Report will be available as required. Internal audit reports are tabled with the Departmental Audit and Evaluation Committee.
- For each project, the funding recipient will provide an annual report on progress achieved during that fiscal year within one hundred and twenty days following the end of each fiscal year. This annual report shall contain the information listed in Schedule C of each contribution agreement.
- Performance information will also be provided through the Industry Canada reporting progresses for the Reports on Plans and Priorities (RPP) and the Departmental Performance Report (DPR).
- The program's annual progress will be included in TPC's Year In Review.
- The formative evaluation report, to be available in fiscal year 2005, will be submitted to the Department's Audit and Evaluation Committee, to TBS, and an executive summary will be posted on the Department's web site. The full evaluation report will be a public document and, as such, accessible to others upon request.
- The summative evaluation report, to be available in fiscal year 2008, will also be submitted to the Department's Audit and Evaluation Committee, to TBS, and an executive summary will be posted on the Department's web site. The full evaluation report will be a public document and, as such, accessible to others upon request.