This publication is available online at https://www.ic.gc.ca/eic/site/ae-ve.nsf/eng/h_03931.html.
To obtain a copy of this publication or an alternate format (Braille, large print, etc.), please fill out the Publication Request Form at https://www.ic.gc.ca/eic/site/iccat.nsf/frm-eng/EABV-9E9HE7 or contact:
Web Services Centre
Innovation, Science and Economic Development Canada
C.D. Howe Building
235 Queen Street
Ottawa, ON K1A 0H5
Telephone (toll-free in Canada): 1-800-328-6189
Telephone (Ottawa): 613-954-5031
TTY (for hearing-impaired): 1-866-694-8389
Business hours: 8:30 a.m. to 5:00 p.m. (Eastern Time)
Permission to Reproduce
Except as otherwise specifically noted, the information in this publication may be reproduced, in part or in whole and by any means, without charge or further permission from Innovation, Science and Economic Development Canada, provided that due diligence is exercised in ensuring the accuracy of the information reproduced; that Innovation, Science and Economic Development Canada is identified as the source institution; and that the reproduction is not represented as an official version of the information reproduced, nor as having been made in affiliation with, or with the endorsement of, Innovation, Science and Economic Development Canada.
For permission to reproduce the information in this publication for commercial purposes, please fill out the Application for Crown Copyright Clearance at www.ic.gc.ca/copyright-request or contact the Web Services Centre mentioned above.
© Her Majesty the Queen in Right of Canada, as represented by the Minister of Innovation, Science and Economic Development Canada, (insert year of publication).
Cat. No. Iu4-399/2021E-PDF
Aussi offert en français sous le titre Vérification du Fonds stratégique pour l'innovation.
Table of Contents
- 1.0 Executive Summary
- 2.0 Background
- 3.0 About the Audit
- 4.0 Findings and Recommendations
- 4.1 Introduction
- 4.2 Governance and Oversight
- 4.3 Initial Intake and Assessment
- 4.4 Project Selection Process
- 4.5 Claims Administration
- 4.6 Information Management
- 4.7 Repayment Process
- 4.8 Recoveries Process
- 4.9 Risk Management
- 4.10 Program Progress and Performance
- 4.11 Management Response and Action Plan
- 5.0 Overall Conclusion
- Appendix A: Audit Criteria
List of Acronyms Used in Report
- Annual Performance Benefits Report
- Audit and Evaluation Branch
- Contribution Agreement
- Investment and Experimentation Oversight Committee
- Investment Oversight Committee
- Investment Resource Committee
- Operational Audit Committee
- Operational Management Committee
- Repayments and Recoveries Directorate
- Strategic Aerospace and Defense Initiative
- Strategic Innovation Fund
- Statement of Interest
1.0 Executive Summary
The Strategic Innovation Fund (SIF) is a key initiative under the Innovation and Skills Plan and was announced in 2017 as a $1.26B grant and contribution fund to support innovation in all sectors of Canada's economy. The program's objective is to spur innovation by funding projects that will provide benefits to Canada across three metrics: economic; innovation; and public benefits.
In the Fall 2018 Economic Statement, an additional $800 million over five years was added to the SIF program funding. Through its funding, SIF aims to transform key sectors and supply chains for long-term, cleaner growth and create new well-paying jobs across Canada.
The program consolidates existing innovation programs focused on the aerospace and automotive industries into a streamlined innovation fund, which is now expanded to support high-growth sectors such as clean technology, information and communications technology and agri-food. The transition to a single program window aims to simplify application processes, accelerate processing, and be generally more responsive and results-oriented.
There are five distinct streams of activities in the program that are designed to cover all sectors of the economy and consolidates previous programming. Streams 1 to 3 target for-profit companies with direct funding to recipients, while streams 4 and 5 target projects with multiple participants and aim to fund collaborative partnerships of various types of organizations
As of September 3rd, 2019, 64 projects had been announced through the SIF program with $2B in committed contributions. In addition to funding new projects, SIF holds the portfolios of seven legacy programs, for a total 163 legacy projects valued at $5.2B in total contributions, including 139 projects in the Repayment and Benefits Phase.
1.2 Audit Background
The objective of the audit was to provide assurance that the Strategic Innovation Fund (SIF) is operating efficiently and effectively.
The audit scope focused on activities and processes related to streams 1 to 3 of the SIF program and its legacy programs between October 1, 2017 and September 30, 2019, including:
- Governance and oversight;
- Risk management;
- Contribution agreements;
- Claims processes; and,
- Monitoring and reporting.
1.3 Overview of Audit Results
The SIF program has established key oversight committees to support project selection and management. These committees provide an effective oversight and challenge function throughout the different phases of the program, which are supported by documented records of decision.
SIF has developed its own governance, project assessment processes, and performance evaluation systems to manage the program and the various streams. The program's Statement of Interest (SOI) assessment and project selection process is supported by defined assessment criteria that are linked to program objectives, resulting in a transparent and well-supported selection process. SIF's claims processes are clearly defined and include effective and functioning controls, supported by verification and approval tools.
For projects in the later stages of the program lifecycle, there are defined repayment processes, which are being applied consistently. Repayment schedules are maintained in a centralized database and communications with recipients are documented and maintained in a centralized GCDocs file.
The program monitors and measures both program and project performance regularly. For program performance, standard information across projects is aggregated and reported on annually. Project performance and benefit commitments are monitored and assessed on each claim submission, with remedies being applied where necessary.
Areas for Improvement
Some opportunities for improvement were identified by the audit. Guidance supporting the use of invoice testing and progress assessment tools for claims could be strengthened to ensure consistency in performing and documenting claim assessments.
Risk management practices could be expanded to include information from project level activities, such as claims and repayments, to ensure risks are being identified, assessed and addressed for all phases of the program lifecycle.
1.4 Audit Opinion and Conclusions
The Strategic Innovation Fund has established an overarching management control framework that integrates strong governance processes, transparent and documented assessment processes, and effective controls for claims verification and program monitoring. There are opportunities to further strengthen claims administration, and risk management practices.
1.5 Management Response
Management has agreed with the findings included in this report and will take action to address all recommendations by July 2021.
1.6 Statement of Conformance
This audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada, as supported by the results of the Audit and Evaluation Branch's quality assurance and improvement program.
Chief Audit Executive
Innovation, Science and Economic Development Canada
2.1 Strategic Innovation Fund Overview
The Strategic Innovation Fund (SIF) was announced in 2017 as a $1.26B grant and contribution fund to support innovation in all sectors of Canada's economy and is a key initiative under the Innovation and Skills Plan.
SIF consolidated existing innovation programs, which were focused on the aerospace and automotive industries, into a streamlined innovation fund, which is now expanded to support high-growth sectors such as clean technology, information and communications technology and agri-food.
Five distinct streams of activities are used by the program to support all sectors of the economy and consolidate previous programming:
- Stream 1: Support Research & Development
- Stream 2: Facilitate Scale Up
- Stream 3: Investment Attraction
- Stream 4: Tackle Grand Innovation Challenges
- Stream 5: Support National Ecosystems
Streams 1 to 3 target for-profit companies with direct funding to recipients, while streams 4 and 5 target projects with multiple participants and aim to fund collaborative partnerships of various types of organizations.
In addition to funding new projects, SIF holds the portfolios of seven legacy programs, which include 163 projects valued at $5.2B in total contributions. Legacy programs include: the Automotive Innovation Fund; the Automotive Supplier Innovation Program; the Strategic Aerospace and Defence Initiative; the Technology Demonstration Program; the Bombardier C Series Program; the Program for Strategic Industrial Projects; and Technology Partnerships Canada. Most of the legacy program are currently in the Repayment and Benefits Phase, with 24 projects still in the Work Phase and submitting claims for reimbursement.
As of September 3rd, 2019, 64 projects had been announced through the SIF program with $2B in committed contributions.
The SIF projects are assessed based on their risks and benefits, which in turn drives the funding agreement structure for each project. Depending on a project's risk profile, the contribution agreement (CA) can include different repayment clauses such as: non-repayable; unconditionally or conditionally repayable; or a hybrid of the above. Approved projects are funded at up to 50% of their eligible costs and can include a variety of defined benefit commitments, including job creation, research and development, gender equity and diversity, and environmental impacts, which will generate benefits to Canada.
As demonstrated in Figure 1, there are three main phases to the SIF program lifecycle. In the Application Phase, proposals are first submitted to the program through an initial Statement of Interest (SOI) for assessment. If approved, a full application is submitted for further assessment. Once due diligence is performed on the completed application, a CA is negotiated and finalized. During the Work Phase, the recipient is responsible for managing the project and submitting claims for applicable expenses. Reimbursement of claims for SIF project expenditures typically last three to five years, followed by a two to three year grace period, defined in the CA.
Once all contributions are paid, the Repayments and Benefits Phase begins, where repayments are scheduled (if applicable) and benefit commitments are expected to be realized. This Phase usually lasts fifteen years for the SIF projects. During this period, the program monitors the recipient through annual financial reports and performance benefit reports. If applicable, recipients will begin repayments, and the project can be subject to recipient auditing on their revenues in order to verify the repayment amounts.
SIF projects span over a twenty to twenty-three year period from the approval process to completing the Repayments and Benefits Phase. At the time of the audit, there were no SIF projects in the Benefits Phase. SIF also administers its legacy project portfolio, including projects, which are still considered in the Work Phase with claims being submitted and others in repayment or recoveries. At the time of the audit, 24 legacy projects were in the Work Phase and 139 were in the Repayments and Benefits Phase.
2.2 Previous Audit Engagements
In 2016, the Audit and Evaluation Branch (AEB) conducted the Audit of Strategic Aerospace Defense Initiative (SADI), a legacy program that was later consolidated with SIF. The objective of this audit was to provide assurance that SADI's Management Control Framework (MCF) was adequately designed and implemented to support the delivery of the program. The scope of the audit was limited to an assessment of control design and implementation in recipient risk management and monitoring, and claims verification procedures.
The audit identified opportunities to enhance the program's existing framework with three recommendations related to: updating program documents to reflect how the compliance risk factor is assessed in the Application Phase; reviewing the recipient progress reporting requirements related to claim submissions to ensure that they reflect the value of funding and risk profile of recipients; and adjusting verification procedures to allow for formal consideration of recipient risk. In June 2017, all recommendations were deemed closed.
3.0 About the Audit
3.1 Audit Objective, Scope and Methodology
In accordance with the approved Innovation, Science and Economic Development (ISED) 2019-2020 Risk-Based Internal Audit Plan, the Audit and Evaluation Branch (AEB) undertook an audit of the Strategic Innovation Fund (SIF).
The objective of the audit was to provide assurance that the SIF is operating efficiently and effectively.
The audit scope focused on activities and processes related to streams 1 to 3 of the SIF program and its legacy programs from October 1, 2017 and September 30, 2019, including:
- Governance and oversight;
- Risk management;
- Contribution agreements;
- Claims processes; and,
- Monitoring and reporting.
The audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada.
Based on the risk assessment, audit criteria and sub-criteria, linked to the overall audit objective, were developed (see Appendix A).
The methodology used for this audit included various procedures to address the engagement's objective. These included interviews, review of documentation, walkthroughs of systems and processes, and file sampling, including contribution agreements in place.
Samples from each program activity throughout the lifecycle of the program were selected to assess controls over project selection, financial and risk management, and documented approval processes throughout the program lifecycle. This included reviewing elements related to claims administration, information management, as well as decision-making, governance and program progress and performance processes.
A debrief meeting was held with the Director General of the Strategic Innovation Fund on October 7th, 2020, to validate the findings that form the basis of this report. This meeting also provided the auditee an opportunity to offer any additional information and clarification regarding the findings.
4.0 Findings and Recommendations
This section presents detailed findings from the audit of the Strategic Innovation Fund. The findings are based on evidence and analysis from both the initial risk assessment and the detailed audit work.
4.2 Governance and Oversight
The Strategic Innovation Fund has established governance and oversight committees to support project selection and management. These committees are effectively fulfilling their mandates and provide a challenge function at key points of the program lifecycle.
The Strategic Innovation Fund (SIF) has the mandate and funding to support innovation in all sectors of Canada's economy and continuously accepts applications for funding projects across all sectors of the economy.
SIF has several key oversight committees, which oversee project selection and investment as well as coordination and use of the organization's resources. These include:
- Investment and Experimentation Oversight Committee (IEOC) [formerly the Investment Oversight Committee (IOC)]: This committee reviews proposed projects and amendments that meet a defined risk threshold, and recommends a course of action.
- Investment Review Committee (IRC): Engaged during the Project Selection Phase, the IRC's mandate includes acting as a key challenge mechanism to ensure that approved projects align with program and departmental objectives.
- Operational Management Committee (OMC): This committee guides general program operational activities, including review of all proposed amendments, repayments, and ongoing alignment with the investment plan, before referral to other committees where applicable.
- Operational Audit Committee (OAC): This committee oversees the recipient audit work led by the SIF audit team, including the multi-year audit plan, monitoring of management action plans, and approval of audit reports.
These committees have defined terms of reference, including defined roles and accountabilities, and hold regular meetings to support timely decision-making and where necessary, secretarial functions are used to address urgent requirements.
IRC meetings are held on a regular basis to review initial applications, make recommendations on next steps, and provide advice on priorities as they relate to investment decisions. Each initial application is presented at IRC for review and challenge and a recommendation is made by the committee for approval, referral, rejection or to be put on hold. Initial applications may be brought back with additional information or to be assessed against other similar applications. Projects recommended for approval are invited to submit a full detailed application.
Approved applications, which have successfully completed the due diligence process and have moved forward to a contribution agreement, are presented at the IEOC for review and approval. For each project presented, the committee reviews a detailed project summary form, which includes a summary of the project proposal with the proposed benefits and risks. Committee members may request additional information prior to making a final recommendation to the Minister for approval.
It was found that records were maintained for key decisions, and that the respective committees effectively fulfilled their role as a challenge function, where project were presented and discussed prior to the decision-making process.
4.3 Initial Intake and Assessment
SIF's initial intake process consists of a statement of interest (SOI) application assessed by the program for completeness and eligibility as well as projected benefits. Initial assessments of SOIs are standardized and completed in a consistent manner.
The SIF program uses a continuous intake approach, and SOIs are received on an ongoing basis. These include general applicant information, a funding request and the projected project benefits. Applications are assessed using standardized forms that streamline the information received by the program and incorporate input from designated industry analysts.
Once the SOI assessment is completed, the investment officer and industry analyst present the assessments at IRC, where SOIs can be rejected, referred, recommended for full application, or be put on hold for further consideration at a later date. In some cases, the IRC requested that an SOI be brought back with more information.
The audit sampled 140 SOI assessments with a variety of assessment results, including approval, rejection, or on hold. An assessment form was on file for all sampled SOIs, which included an assessment of completeness and eligibility, as well as supporting analysis related to projected benefits of the projects. In addition, for all the approved SOIs tested, the assessment forms provided a clear recommendation from the investment officer and included industry input.
In order to manage the large number of SOIs, the program tracks each SOI and analyzes the length of time elapsed from intake through to assessment. One of the tools used for tracking the progress and status of all SOIs received is the Global Status Tracker. This internal tracker tool includes key information on the submissions received, such as intake date, application status, and links to key decision documents. The tracker is also a key reporting tool used to feed reporting to senior management on SOI progress. Of the 140 SOIs tested, the tracker was reflective of the file's status, and included a decision document link for the appropriate document.
Maintaining key project information and decision documents enabled the program to support a transparent and objective assessment process for the initial intake of SOIs.
4.4 Project Selection Process
The SIF project selection process is supported by clearly defined assessment criteria and results in a transparent and objective selection process, which is documented and linked to program objectives.
From the beginning of the program's inception, the SIF program has received hundreds of SOIs for funding. These SOIs spread across the different program streams, business sectors and vary significantly in the level of funding being requested. During the scope of the audit, 55 of 1038 total submitted SOIs were approved by the program and proceeded to the project selection process.
Applicants with successful SOIs are invited to submit a detailed application for consideration, which includes submitting financial statement information and a detailed project proposal with defined expected benefits.
As part of the project selection process, applications are subject to a rigorous due diligence process that includes a risk assessment of financial, technical, market and trade risks. The financial due diligence process is done internally, while the market, technical and trade processes are performed using external parties with the required expertise. Standard templates are used for each due diligence element assessed, which include defined assessment criteria for each element subject to the assessment. For all 55 applications approved during the scope of the audit, the due diligence process was completed, documented, and supported by detailed analysis.
Once the due diligence process is completed, summary information from the proposed project benefits as well as the risk assessments are used to populate the Project Summary Form, which is reviewed by IOC before recommendation to the Minister. The program also developed a terms sheet, which is used to negotiate key elements of the proposed contribution agreement with the recipient prior to inclusion in the final agreement. The terms sheet includes the cost sharing ratio, repayment expectations, and pre-disbursement conditions, if applicable. For all 55 files tested, the terms sheet was used consistently and approved by the recipient prior to the official contribution agreement being developed.
The nature of the SIF program results in diverse applications from all sectors of the economy requiring different expertise and considerations throughout the selection process. The audit found that throughout each phase of the application selection process, the objectives of the program were clearly outlined and linked to the assessment criteria, resulting in a transparent, objective selection process supported by documentation.
Additionally, the due diligence process used for assessing applications is well established and supported by defined assessment criteria, including guidance for assessments and multiple levels of review. This process also leverages other ISED sectors and external federal government expertise to ensure a fulsome risk assessment over the diverse applications.
4.5 Claims Administration
A defined claims verification process is in place, supported by effective controls and documented tools. However, tools used to support claims verification were not always used consistently.
Once a project is approved and a contribution agreement is in place, recipients enter the Work Phase of the program life cycle and submit claims for reimbursement on a schedule defined in the contribution agreements. Claims submissions include a request for reimbursement, which is supported by detailed costing breakdowns and a progress report.
Both a claims verification officer and an investment officer within SIF review claim submissions, and then provide a final recommendation as to the amount of reimbursement to be released. For projects with pre-disbursement conditions, these are evaluated prior to releasing claim funds.
A total of 168 claims were received and approved during the scope of the audit. All 168 claims for approved projects were included in testing, and were found to be compliant, with controls in place for verifying and reconciling claims submissions and only claims for eligible expenses were reimbursed.
It was found that SIF recipients are not always submitting timely claims submissions according to their contribution agreement schedules. The program leverages tools available to manage the late submissions, including, sending frequent communication requesting the submissions and in one instance, deeming a project to be in default based on a lack of claims submissions.
SIF uses standard templates, tools and checklists to perform the analysis on claims submissions. However, some tools were not used consistently, such as the invoice testing sheet and the calculation for progress assessment. Detailed claim assessments require recipients to submit a designated number of invoices, sometimes related to specific cost categories, to support the costs being claimed. These invoices, at the time of the audit were itemized in an invoice sheet within the detailed claim assessment. However, for 47 out of 168 claims tested, the invoice sheet in the detailed assessment was not completed.
Each claim submission also includes a progress assessment, indicating what percent of completion the project has reached, which is documented in the investment officer's assessment form. However, for ten of 168 claims tested, the progress assessment calculations were performed inconsistently. While neither of these tools impacted the review and verification for eligible costs for the files tested, as the information could be located throughout the submission, it could have led to inconsistencies in the claims assessment process.
An additional 116 claims were tested for legacy programs, which included claims from the 26 legacy files still in the Work Phase for the scope of the audit. While these claims are processed using the legacy program's contribution agreement requirements, there were controls in place to verify and reconcile legacy claims submitted.
Recommendation #1 (Low Risk)
The program should strengthen the available guidance supporting the use of invoice testing and progress assessment tools for claims to ensure consistency in project assessment and reporting.
4.6 Information Management
SIF effectively leverages existing systems to support program activities, including a fully electronic records management process. Claims are supported by key decision and approval documents maintained in centralized electronic or paper-based files.
Information Management Systems
SIF maintains electronic files for each project, which are structured according to the project lifecycle. The files include dedicated folders for the initial intake process, as well as the application process, the contribution agreement approval, claims administration and for repayment and monitoring activities. These project files are maintained centrally on GCDocs.
The program also uses the departmental grants and contributions system, CMIS, to log financial commitments, claims payments and repayment schedules. Specific project information in the CMIS systems includes:
- Project tombstone information;
- Number of claims submitted, amounts paid and claim status;
- Summary of financial information, including amount funded and amount remaining; and,
- Repayment schedules, including logs of each payment and interest charge.
Repayments and recovery files use an internal access database to log communication between applicants and the program. This database tracks the status of repayments, and leverages information from CMIS to schedule touch points with recipients for ongoing monitoring of payments and late payments.
All 55 approved project files tested were electronically accessible and followed the standard file structure, with key decision documents for each phase generally saved within the designated files. This central electronic storage of information across project files and program activities supported key decision-making documents to be readily accessible.
Project File Management
For project claims, at the time of the audit, submissions could be retained both electronically or paper-based, and approval signatures were recorded on hard copies. The SIF claims team would then scan key documents and maintain them either electronically on GCdocs or in a centralized paper file.
At the time of the program's inception, claims were processed using a paper-based system. During the scope of the audit, the program was transitioning from paper-based files to electronic files. Since the time of the audit, the SIF program has transitioned to working remotely. As a result, claims are now processed electronically, and no paper files are maintained separately. Having these centralized files for key documents ensures that the support documentation for each claim is maintained and readily accessible.
4.7 Repayment Process
There is a defined process for monitoring and recording repayments that is applied consistently, is supported by documentation, and leverages internal and departmental systems effectively.
Once the Work Phase is completed and the recipients are no longer submitting claims for reimbursement, projects enter the Repayments and Benefits Phase. SIF contributions can be repayable, non-repayable or a combination of the two. For contributions that are repayable in part or in full, a repayment schedule is defined within the contribution agreement. Based on the Terms and Conditions of the program, these schedules can be:
- Unconditional, where set payments are made regardless of the recipient's earnings, or
- Conditionally repayable, where the payments are calculated based on the performance or earnings of the recipient for each period.
The Repayments and Recoveries Directorate (RRD), within the Corporate Management Sector, is responsible for the monitoring and recording of the projects in the Repayment Phase. Communications with recipients are documented in an internal access database, while repayment forecasts, payments and interest charges are tracked in the ISED departmental Cognos system. Invoices and key communication documents are maintained in the project's central GCDocs file.
During the scope of the audit, there were 139 legacy projects in the Repayment Phase, from which a sample of 39 projects were selected for testing. For all 39 projects, it was found the defined process for contacting recipients and monitoring late payments was followed, and key documents and communications were documented in the appropriate systems.
It should be noted that there were no SIF files in the Repayment Phase during the scope of the audit, but this is expected to shift in the next five years as early SIF projects are expected to move from the Work Phase to the Repayment and Benefits Phases.
4.8 Recoveries Process
The Repayment and Recoveries Directorate maintains up-to-date guidance outlining the recoveries process and defines exceptions to the standard process.
Once a Contribution Agreement (CA) has been finalized for a given project, it moves through the various project lifecycle stages, and is monitored to ensure compliance with the signed CA. In cases where there is a breach of conditions that cannot be resolved at the program level, such as a breach in repayment terms, projects are transitioned into the recoveries process.
The recoveries process is a complex, multi-phased process that begins when contribution agreements have been placed in default, and ends when a portion of the contribution funds granted has been recovered or written off. The Repayment and Recoveries Directorate (RRD) has developed procedural documents that provide guidance on the recoveries process, including standard templates and a flow chart that clearly outlines the steps to be taken from default to recovery of funds.
The RRD has also developed a detailed process map for the recoveries process. However, in practice, the process is often not linear due to the complexity of the files. As a result, there are often circumstances under which the team may need to shift from the documented process due to the dynamic nature of companies in financial distress.
For the recoveries files tested, 13 of 23 files were identified to have deviations to the standard process due to the complex nature of the companies in question. These instances were mitigated by the fact that the RRD team maintained an up-to-date status tracker to help clarify any exceptions to the process, and provided the most recent status of all files to ensure that the timeline of events is defined and documented.
4.9 Risk Management
At the program level, SIF has developed a comprehensive risk framework, which includes both risk principles and procedures. There is an opportunity to improve risk management practices at the project level, where the risk framework could be updated to provide further guidance to support the assessment of project-level risks.
The risk environment for the SIF program comprises of risks at both the program and project level. The program level risk management framework includes controls and measures that have an impact on the program as a whole, while the project level framework includes controls and measures that relate to the project lifecycle.
Program-Level Risk Management
As part of the SIF program, projects enter a lifecycle that begins with initial selection and moves through full assessment, approval, payment of claims (claims, repayment, and recoveries), and monitoring phases. At the program level, the SIF team has established a comprehensive framework for risk management that is supported by procedural documents that provide guidance to staff. These documents include the Management Control Framework and Risk Register, which outline the risk identification, assessment, and monitoring process at the program level.
The SIF program has its own internal recipient audit group who perform regular cost, revenue and lobbyist audits on SIF recipients. The recipient audit group conducts an annual audit planning exercise to determine which audits to perform in a given year and this process leverages risk information identified through claims verification for inclusion in the risk-based audit plan.
Risk information, including repayment or claims issues related to specific projects or recipients, is presented at the Operations Management Committee for further escalation. However, project-level risk information is presented on a case-by-case basis, when issues arise, and is not the result of a defined risk assessment process with established risk thresholds.
Project-Level Risk Management
At the project level, during the selection phase, each recipient undergoes a rigorous due diligence process to assess different types of project risks. This includes; financial, technical, market, and trade risks. There is a defined process to assess these risks, which includes guidance to support the implementation of this process.
It was found that the risk assessment process was regularly followed, including assessments being completed consistently and documented. These assessments are used to inform contribution agreement requirements, and higher risk recipients have additional pre-disbursement conditions, which have to be passed prior to claims being funded, to help mitigate risks during the Work Phase.
Once projects are in the Work Phase and submitting claims for reimbursement, there is an expectation that risk assessments be performed for each claim submitted by analyzing the recipient's progress report. This expectation is highlighted further as an element of the program's Master Project Checklist, which links claims assessments to the risk assessment process for the work phase.
However, for 85 of the 168 claims tested, performance of the risk assessment by the investment officer was incomplete or inconsistently performed. Without a consistent methodology for performing these risk assessments, the program's ability to identify and mitigate new or evolving risks throughout the program lifecycle could be reduced.
Risk Management for Projects in Repayment Phase
The Repayments and Benefits Phase for projects can last a significant amount of time, and in some cases over ten years. For the repayments element of this phase, the Repayments and Recoveries Directorate (RRD) works with recipients and is often informed of issues and concerns directly by the recipients. The RRD and SIF program meet regularly to discuss potential issues and late payments.
However, there are limited tools and guidance available for documenting, assessing, and reviewing risks in the repayment phase. The risk management process is reliant on the recipients bringing the issues forward, with analysis done on an individual project basis. As this analysis is not aggregated, it could cause the program to miss an opportunity to identify and leverage information on risk trends.
Recommendation #2 (Medium Risk)
The program should review their project risk management during the work and benefit phases and update the claims and repayments processes to ensure recipients are periodically assessed against defined risk thresholds.
4.10 Program Progress and Performance
SIF has effective processes in place to monitor project progress and service standards for SOIs and claims, and the program leverages data to monitor overall program performance.
The SIF program has a mandate to support innovation in all sectors of Canada's economy and to select projects that provide the highest benefits possible to Canada across three metrics: economic benefits; innovation benefits; and public benefits. To achieve this, SIF has multiple program streams focusing on different areas of innovation, approves projects based on proposed benefits and includes specific benefit criteria with each contribution agreement.
The SIF program monitors individual project progress, as well as the performance of the program as a whole. The program uses a survey template for each recipient to gather annual data on key performance indicators such as R&D, employment data and revenues. This standard performance information collected across all projects is reported on an aggregate basis to demonstrate the performance of the program in an Annual Performance Benefits Report (APBR). During the scope of the audit, the APBR process was in the early stages of data analysis and the report had not been issued.
For individual projects, recipients submit a progress assessment for each claim and their progress against objectives is assessed. If a project is found to be significantly behind schedule in the claims assessment, this concern is escalated and can result in a proposed amendment or additional monitoring activities. For the sampled claims, all recipients submitted a progress report, which was assessed and included in the final recommendation for approval.
For each project, there are defined commitments in the CA, such as number of jobs created. These commitments are expected to be maintained throughout the project's lifecycle and are monitored on an ongoing basis. If these are not met, an amendment to the contribution agreement can be made or a decision to default the contribution agreement. The program applies the remedies available to them in the contribution agreements if progress or commitments are not being met. For the sampled projects, one file out of 55 was found to breach a commitment during a progress assessment, and termination was on mutual consent with a repayment by the company.
4.11 Management Response and Action Plan
The findings and recommendations of this audit were presented to the Director General of SIF, and members of the SIF senior management team. Management has agreed with the findings included in this report and will take action to address all recommendations by July 2021.
5.0 Overall Conclusion
The Strategic Innovation Fund has established an overarching management control framework that integrates strong governance processes, transparent and documented assessment processes, and effective controls for claims verification. There are opportunities to further strengthen claims administration and risk management practices to ensure the program maintains a strong control environment throughout the program lifecycle.
Appendix A: Audit Criteria
|Governance and Oversight|
|1. The program is supported by effective governance and oversight processes.|| |
1.1 Governance structures, including roles, responsibilities and accountabilities are clearly defined and communicated, and include key stakeholders involved with the SIF program.
1.2 Management of program resources is effective and aligns with legislative and Departmental requirements.
|2. There are effective processes and controls in place to support the administration of the program's objectives.|| |
2.1 Projects are assessed and selected in line with program objectives.
2.2 Contributions and repayments are managed according to contribution agreement requirements.
2.3 Information is managed effectively and allows for timely reporting to support decision-making.
2.4 Processes are in place to identify, assess and respond to program and project risks.
|Monitoring and Reporting|
|3. Monitoring and reporting of program activities are timely and effective.|| |
3.1 Recipient performance and progress against program objectives is regularly measured and reported.