BAI National Dialogue Summary

March 2017

Table of Contents

1. 1. Summary
2. 2. Underlying Questions, Drivers and Target Outcomes
   1. 2.1 Framing the Discussion: Key Questions and Target Outcomes from the Session
   2. 2.2 Drivers and Incentives for Creating a National Framework
3. 3. Challenges in Measuring Performance
4. 4. Building a National Framework
   1. 4.1 Considerations when Developing a National Measurement Framework
   2. 4.2 National Framework: A Working Concept
   3. 4.3 Questions and Challenges for further Discussion
5. 5. Conclusions and Recommendations
6. Schedule A: Meeting Agenda
7. Schedule B: Attendees

---

1. Summary

Over the past number of years, public sector organizations have increasingly recognized the need to develop and deploy performance measurement systems to ensure that they have timely, strategically focused, objective and evidence-based information on their performance, in order to produce better results and remain high-performance organizations. With this goal in mind, the Department of Innovation, Science and Economic Development (ISED) met with representatives from Canada`s entrepreneur support ecosystem at three informal roundtables during the Autumn 2016 to discuss how government, entrepreneurs, investors and industry could work together to establish a national performance measurement framework for Business Accelerators and Incubators (BAIs). During the consultations, stakeholders expressed a willingness to collaborate on a national scale and, more specifically, to advance a national discussion on best practices in performance measurement.

With the support of ISED, an industry-led Working Group of Business Accelerators and Incubators took the first step toward framing a national solution for data collection and performance reporting by hosting a national discussion on February 10^th in Toronto. Leaders from 18 organizations spent the day discussing BAI activities, opportunities and challenges, and exploring the benefits and challenges of creating a national performance measurement framework.

Prior to the session, participants were asked to provide information about their organizations (including affiliation, client composition, technology sectors supported, etc.) and to describe their current performance reporting activities. The survey provided a baseline understanding of what metrics are currently being collected and for what reasons (drivers), and helped to facilitate a discussion about how a national performance measurement framework could provide value to the community and its funders.

Over the course of the discussion on February 10^th there was broad agreement on a number of key issues and a collective resolve to work toward the following outcomes:

1. Building a national measurement framework. While participants expressed differing visions for how they would like to see a national performance measurement solution take shape, there was virtual unanimity in the desire to continue the efforts to forge agreement on a standardized reporting framework that establishes consistent definitions for job creation, revenue generation, firm survival rates and other outcome-related metrics. There was broad agreement that a national performance measurement framework will provide BAIs with reliable and comparable data on which to make sound decisions, as well as timely information on the relevance, success and cost-effectiveness of their programs and activities. It was also noted that a common evaluation framework will provide governments and other funders with a rigorous and objective evidence base with which to assess the performance of BAIs and make informed resource allocation decisions. As a summary of the discussions, this document outlines a working concept for a national performance measurement framework. However, subsequent dialogues among BAIs and policymakers will invite further input and inevitably lead to many significant refinements and improvements along the way.
2. Fostering collaboration around shared objectives. Although the discussions focused on building a national collaboration around performance measurement, there was recognition that the efforts to work together at a national level will result in many other tangible benefits for BAIs, their clients and the broader economy. These benefits include the ability to share best practices across institutions and jurisdictions, establish relevant performance benchmarks for different regions and sectors, enhance input into public policy, position and promote Canada as a destination for start-up activity and reduce the administrative burden associated with data collection and reporting. In other words, the collaborative efforts to build a common system for measuring the performance of BAIs could herald the beginning of an exciting new chapter in the growth and evolution of Canada's start-up ecosystem.
3. Working together to address key measurement challenges. The discussions surfaced a wide range of challenges associated with forging a standardized approach to performance measurement. Among other things, these challenges included the need to craft a measurement framework that can be customized to the unique services and circumstances of different BAIs. Specifically, participants expressed a desire to ensure that nationally standardized metrics capture the value created by entities that operate in different sectors and regions and with clients of varying levels of maturity. Other key issues include the challenges associated with attributing economic impacts to individual institutions and capturing intangibles such as the contributions BAIs make to developing client skills and building new clusters of economic activity.
4. Piloting standardized reporting with a small, but representative group of BAI leaders and policymakers. Taking advantage of the leadership of a select number of BAIs would provide an opportunity to test and refine the framework with a smaller group before rolling out it on a national basis. It was agreed that the pilot phase should include representation from policy leaders and funding organizations and reflect the diversity of programming models and services offered by BAIs, along with the key economic sectors and regions of Canada. In addition to piloting a performance measurement solution, it was suggested that a number of working groups should be tasked with developing solutions for some of the key measurement and ecosystem challenges identified during the dialogue.

The BAI national dialogue on February 10^th represents the first step toward increased collaboration among BAIs in Canada and the creation of a national framework for performance measurement. There will be subsequent opportunities for BAIs, policymakers and other stakeholders to participate in the ongoing dialogue and to provide leadership in crafting a national performance measurement solution that works for the BAI community and its partners in government.

This summary document provides an overview of the key discussion points and findings from the Feb 10^th session. It is intended to help guide the BAI community and its partners in government as they proceed with the next phase of building a national performance measurement solution.

2. Underlying Questions, Drivers and Target Outcomes

2.1 Framing the Discussion: Key Questions and Target Outcomes from the Session

For many in the room on February 10^th, the national discussion presented the first opportunity for BAI leaders from across Canada to come together with their peers. Although the 18 BAI organizations present in the room constitute only a fraction of the total number of incubators and accelerators across Canada, the diversity of Canada's BAI community in terms of regions, sectors and service offerings was generally well-represented. Indeed, the diversity of expectations, experiences and perspectives on the issues at hand was evident from the very outset of the discussion.

As a collective, the group was asked to reflect on the following key questions:

• What problems can a national collaboration on performance measurement help solve for BAIs and the broader start-up ecosystem?
• How should the industry collaborate on performance measurement and what are the most valuable features in a national collaboration?
• What are the key metrics around which to build a national standard?
• How can a national framework reconcile the need for a simple, instructive and cost effective reporting solution with the need to capture the diversity of BAI models and sectors in Canada?
• What challenges/obstacles will need to be overcome to make industry collaboration successful?
• Are BAIs willing to move forward with a pilot project to validate the potential for a national performance measurement solution?

The five target outcomes for the day were to:

• Gain a greater understanding of the different operating models among participants in terms of funding, service delivery and types of clients.
• Establish a preliminary list of common metrics and data collection priorities to assess current level of alignment.
• Gain a greater understanding of the incentives for collaboration on performance measurement, including the wider uses for collecting performance data using a national framework.
• Develop a shared appreciation of the value others perceive in a national framework and collaboration on performance measurement.
• Identify potential challenges that will need to be addressed in implementing a national framework for performance measurement.

The remainder of this documents highlights the key findings for each of the five target outcomes, particularly goals 2 to 5. For a more in-depth overview of the different operating models among BAI participants, please consult the accompanying BAI survey analysis document. This report on the national discussion concludes with a set of recommendations for next steps to be taken by the industry-led working group and its partners in government.

2.2 Drivers and Incentives for Creating a National Framework

The results of the pre-meeting survey indicated that the majority of business accelerators and incubators (BAIs) participating in the national dialogue already collect performance data and issue annual reports to their funders, partners and stakeholders. However, the survey and subsequent dialogue also revealed that BAIs are currently measuring their performance using a diverse and (often) inconsistent range of metrics and with widely varying levels of success in obtaining data from their clients. As the national dialogue unfolded, a broad (though not universal) consensus emerged that a national framework for performance reporting would generate several key benefits for the BAIs, their clients and the start-up ecosystem as a whole. The benefits envisioned by the group include:

• Improving economic impact analysis: A consistent, national performance measurement approach would allow BAIs to better evaluate their impact on client performance and on the broader economy, both locally and nationally. As an industry-led effort, a national framework would also ensure that the metrics used to evaluate performance are measuring the right outcomes (i.e., the outcomes that create the most value for BAI clients and stakeholders) and are appropriately calibrated to the services BAIs deliver and the context in which they deliver them.
• Boosting data collection: A partnership with ISED, Canada Revenue Agency and Statistics Canada on the national framework could create an opportunity to use official tax data to supply information about client performance. While not replacing the need to survey clients annually, such an approach would vastly improve the reliability and comprehensiveness of the economic data collected and help solve other challenges related to double counting of economic outcomes such as job creation.
• Increasing transparency: Improved data collection and reporting of economic impacts, in turn, would better inform firms in their search for support, provide the transparency that public and private funders require to allocate resource efficiently, and allow BAIs themselves to benchmark their own performance against their peers. 
• Enabling collaboration: BAIs would also be able to showcase their areas of comparative strength—be it by sector, growth stage or connections—which could help facilitate collaboration among BAIs and other ecosystem participants, including investors and government service providers.
• Facilitating learning and continuous improvement: Simultaneously, public reporting on outcomes will allow BAIs to share best practices and benchmark their performance against organizational leaders (domestically and internationally), thereby facilitating a process of learning and continuous improvement.
• Positioning and marketing of Canada as a destination for start-up activity: A national dashboard highlighting the activities and achievements of Canada's BAIs could help tell a powerful story to the world and thereby attract international participants to the ecosystem, including founders, investors and corporate partners.
• Enhancing input into public policy: A national collaboration could also strengthen the ability of the BAI community to shape the country's innovation and entrepreneurial support system by engaging collectively with public policymakers.
• Reducing administrative burdens: Finally, the Canadian ecosystem is complicated by its multiplicity of funders, and a subsequent proliferation of different demands for different data. This reporting burden puts a significant strain on the resources of Canadian BAIs. A national framework holds the potential to simplify and streamline performance reporting for BAIs by creating a consistent set of metrics against which to report and (in some circumstances) a unified reporting solution that would deliver one set of annual results to all relevant stakeholders.

3. Challenges in Measuring Performance

While participants in the national dialogue were enthusiastic about the potential benefits of a national framework, they also urged considerable caution in addressing some of the perceived challenges associated with forging a standardized approach to performance measurement. The following were among the top challenges noted by participants:

• Flexibility. The top challenge noted by participants is the need for a flexible approach to performance measurement that can be customized to the unique services and circumstances of different BAIs. In other words, there is no one-size-fits-all framework that can be applied generically to all accelerators and incubators across the country. Indeed, one of the most striking takeaways from the national dialogue was the considerable diversity represented in the room in terms of the services BAIs offer and the context in which they deliver them (e.g., the types of firms and sectors served, ecosystem maturity, etc.). As discussed further in the next section, these differences must inform the types of metrics that are used to evaluate performance and the interpretation of the outcomes BAIs report. Performance indicators that are relevant for some stakeholders are not relevant for others, or could be relevant over very different time spans and to varying degrees. Therefore, a national performance measurement framework will need to specify broad categories of performance metrics that can be customized to fit the unique programming offered by different BAIs.
• Autonomy. BAIs leaders were equally unequivocal in wanting to protect their autonomy to make strategic management decisions about how they invest their resources and deliver their services to clients. In other words, they want metrics that are aligned with their success factors and do not want a national measurement framework to intentionally or unintentionally dictate the choices they make about how best to serve their clients in an environment where technologies, markets and client needs are changing rapidly. For example, concerns were raised about the potential for overly prescriptive metrics to require specific types of interventions and services that are not necessarily aligned with evolving markets and client's needs. It was suggested that the best way to safeguard autonomy is for the national framework to specify broad economic outcomes for BAIs to achieve but then allow considerable discretion in how to meet and measure those outcomes.
• Attribution. In addition to flexibility and autonomy, a recurring theme throughout the session was the issue how to attribute economic impact to BAIs. There is often not a clear link between the services provided by one BAI and the ultimate activity and outcomes of its client companies (and individuals). The outcomes are inevitably affected by many factors outside of the control or influence of individual BAIs. This holds true for the broader innovation ecosystem, where the BAI is just one of many players that help companies in the growth process from the ideation stage onward. Due to the many partners and players typically involved in this process, it is currently a challenge to clearly attribute the success of companies solely to a BAI. Participants did note, however, that qualitative survey research can help shed light on the attribution question to the extent firm founders can subjectively assess the degree to which an accelerator experience was or was not pivotal in their firm's development. Use of client satisfaction, net promoter scores and related metrics were posed as potential solutions.
• Double or triple counting success. Participants noted that another attribution issue occurs when (as is often the case) firms interact with more than one BAI over a short time period. For example, a founder team may initiate product development in an incubator and, six months later, join an acceleration program to receive help in raising financing and commercializing its product. Each of the organizations may have had a hand in the firm's success, and each may be entitled to claim some of the associated economic benefits. However, the overall economic impact of these organizations would be inflated if they each placed 100% the jobs, revenue growth and investment the firm experienced over that period on their own economic impact statements. In the absence of reliable and identifiable firm-level data, it is difficult to know when such instances of double or triple accounting have occurred or to assess how prevalent the problem of double-counting is in the data collected by incubator and accelerator organizations.
• Ecosystem analysis. In the conversation about attribution and causality, there was considerable reflection on the fact that measurement frameworks can place too much emphasis on assessing the performance of individual institutions rather than the performance of the broader ecosystem in which they operate. As one participant put it, "it takes an ecosystem to raise a successful company." While not replacing the need for individual BAIs to account for their outcomes, measuring ecosystem productivity would illuminate the total value creation associated with BAI activities in a given region and help promote collaboration among co-located institutions. It would also place some focus on identifying factors that underpin the productivity of unique ecosystems and allow policymakers and stakeholders to better identify potential ecosystem gaps. Several participants also suggested that some of the issues related to double counting can be addressed by moving to an ecosystem approach to measurement whereby stakeholders evaluate the cumulative economic impact of several BAIs in a regional context rather than measure each organization in isolation.
• Successful failure. A further noteworthy shortcoming of conventional measurement frameworks, and quantitative evaluations in general, is that they are not able to account for the scenario of a "successful failure." As one participant argued, a start-up may not last more than a few months; however, if its experience in an accelerator quickly demonstrates that its idea was not viable, it conserves resources for other ventures. This phenomenon is not captured in the current metrics, but the accelerator and the start-up may consider such a scenario to be a valuable outcome. From the accelerator's perspective, it helped the start-up and investors avoid going down an unprofitable path, and the accelerator could devote resources to helping the start-up redesign the venture or develop a new venture that would be more successful. From the start-up's perspective, the accelerator helped identify areas of weakness that could have resulted in a failed enterprise with potentially severe financial consequences.
• Contributions to skills development. Numerous participants suggested that accelerator and incubator organizations play a role in building skills and human capital—outcomes that are no doubt valuable but not easily quantifiable. As classrooms for entrepreneurs, frequent direct contact with experienced founders, investors and other relevant professionals is a core aspect of most incubator and accelerator programs. In addition, most accelerators, and many incubators, also provide structured programming that includes everything from tax and legal advice to practicing the art of the perfect business pitch. Such experiences typically leave a positive impact on founders, helping them learn rapidly, create powerful networks and become better entrepreneurs. Even if graduates of an accelerator or business incubation program ultimately fail as entrepreneurs, they may go on deploy their newly honed skills and personal networks in other ways that are valuable to society. Leading a successful innovation program at a large company or becoming a valuable employee in someone else's start-up are not necessarily negative outcomes for an accelerator graduate, especially for the individual in question. However, they are not the outcomes for which incubators and accelerators are typically measured or rewarded.
• Establishing new clusters of economic activity and a culture of entrepreneurship. Accelerators and incubators also act as hubs around which entrepreneurial networks form—networks that attract diverse, but complementary stakeholders that can catalyze outcomes together that are more powerful than they could if acting alone. The formation of clusters that ignite new entrepreneurial possibilities could serve as an economic lifeline for regions within Canada that are seeing their traditional economic base deteriorate. Accelerators and incubators also create success stories that help convince more people to start businesses. In other words, they contribute to creating a 'culture of entrepreneurship' that investors and governments so covet in a region. In an economy where more young people will be required to create their own jobs, fostering the courage and aptitude for entrepreneurship is a necessary, albeit insufficient ingredient for economic success.
• Measuring velocity. Finally, the issue of speed (or velocity) was another top measurement challenge posed by the group. It was noted that the speed at which milestones are reached by companies (e.g., the time required to reach a certain revenue threshold or obtain series A financing) is a key indicator tracked by several of the BAIs convened for the dialogue. In other words, they measure their own success based on their ability to accelerate their clients' achievement of key milestones. It was suggested that the measurement framework account for velocity in addition to static indicators such as the amount of revenue generated or capital raised in a given year. However, participants also argued that one's interpretation of what constitutes success when measuring the speed at which firms mature or achieve certain milestones will depend on many factors such as sector and geographic location (discussed in the next section). Ideas for solutions included tools for tracking the economic performance of client cohorts over time and analyzing the time required to meet particular targets for revenue, investment attraction and job creation.

Despite the challenges associated with establishing of a national framework, the overwhelming majority of participants agreed that the potential benefits of a national collaboration are significant for all stakeholders and, as a result, they expressed a willingness, even eagerness, to work together to find appropriate responses to the challenges posed above.

4. Building a National Framework

What qualifies as success for start-up assistance organizations? And how should a national framework measure this success? Most participants in the national dialogue agreed that the essential measures of success for BAIs are linked to the growth and competitiveness of incubated/accelerated firms. If incubators and accelerators are successful in selecting and nurturing promising business ideas, incubated firms, on average, should enjoy higher survival rates, grow faster, employ more people and attract more capital than a comparable cohort of non-incubated firms.

In designing the measurement framework to capture these outcomes, however, it became clear from the dialogue that the various stakeholders that participate in and contribute to the entrepreneurial support ecosystem have differing objectives and will therefore prioritize different outcomes and measures of these outcomes.

Investors, for example, are principally interested in deal flow and high-value exits. Successful accelerators are accordingly those where a significant proportion of firms obtain follow-on investments and/or become attractive acquisition targets upon graduation. Government funding bodies are largely concerned with broader economic impacts such as investment attraction and job creation. Successful accelerators, by these measures, are those whose graduates who not only attract private sector investment, but also reinvest in R&D and generate sustainable, high-quality jobs in their jurisdictions. Accelerators and incubators themselves are naturally invested in the success of their clients, but also value operational goals such as the competitiveness of their selection process, the quality of their programming and their long-term financial viability. Among other things, successful incubators and accelerators must therefore attract a large number of high potential applicants, offer high quality programming to their participants, build a roster of top mentors and generate sufficient revenues to cover their costs.

Taken together, these diverse outcomes provide a reasonable starting point for defining of success. Most of these measures are in alignment, though not always. There is a potential tension, for example, between the desire of investors for a quick exit, which often means an acquisition, and the desire of elected officials and policymakers to retain high-growth companies within their jurisdictions and drive long-term job creation. There is no methodological reason, however, why these diverse outcomes cannot be incorporated into a coherent measurement framework.

What follows is an attempt to distill the input from the national dialogue into a working concept for a national performance measurement framework. The suggestions below offer a starting point for subsequent conversations which will inevitably lead to many significant refinements and improvements along the way.

4.1 Considerations when Developing a National Measurement Framework

In addition to the broad measurement challenges outlined above, there were some specific considerations discussed in the context of identifying metrics that could be standardized at a national level. The key considerations include the following:

1. Comparison vs. context. While recognizing the performance benchmarks and comparisons across the ecosystem are useful, participants urged caution to ensure that data is collected and interpreted using a sophisticated and nuanced approach that takes context into account. The relative weighting and interpretation of performance metrics – and the subsequent comparison of BAI performance across the country – should depend on distinctions such as:
   1. Regional differences, including population (and entrepreneur) density, funding models, and proximity to complementary business support services.
   2. Size-based differences
   3. Stage-of-growth (firm maturity)-based differences
   4. Sector-based differences
2. Activities vs. outcomes. Participants noted a clear preference for metrics that focused on outcomes (e.g., investment, revenue and jobs) rather than activities (e.g., # of firms supported, # of products developed, # of investors at demo days, # of customer demonstrations, etc.), noting that activities are generally not a good indicator of meaningful economic impacts and that metrics based on activities tend to be overly prescriptive, while also leaving insufficient room for innovation in service delivery.
3. Volume vs. impact. Participants noted that current and past reporting requirements have often emphasized volume over ultimate impact. For example, BAIs have been required to report on the overall number of firms supported rather than more meaningful measures of the quality and economic impact attributable to a given cohort or portfolio of firms. Participants suggested that the national framework emphasize metrics that provide clear indicators of impact rather than volume.
4. Static vs. longitudinal measurement. Additionally, there was widespread agreement that it is important to build appropriate timeframes into the measurement framework, with a preference for the use of longitudinal rather than static measures of success. Most, if not all, of the metrics listed below can be measured longitudinally, which means that indicators of client performance will be tracked over time (ideally 5-7 years). Longitudinal measurement is particularly important when the meaningful economic impacts associated with BAI interventions take several years to manifest, which is the case in most sectors but certainly some sectors more so than others. Longitudinal measurement of the same firms using the same indicators over time will also allow BAIs to assess the speed / velocity at which their clients are meeting defined targets for investment, revenue and job growth.
5. Prescription vs. customization. Above all, there was a strong desire expressed throughout the dialogue for the national measurement framework to avoid becoming overly prescriptive and to respect the principles of flexibility and autonomy expressed earlier. In other words, BAI leaders want a national framework that balances the desire for standardization with the reality that a useful framework must inevitably allow for the measurement approach to be customized to the unique circumstances and offerings of different BAIs across the country.

4.2 National Framework: A Working Concept

With the above-mentioned considerations in mind, the working concept elaborated below poses a number of broad categories of economic outcomes that BAIs would be encouraged, and perhaps required, to track. Each economic outcome category features a number of suggested metrics—a list that will no doubt be refined and improved over time. BAIs would adopt the metrics that provide the best fit with their current positioning within the ecosystem, the programming and services they offer and the clients they serve. This approach places the focus on outcomes rather than activities; is amenable to longitudinal analysis; and allows context to guide the measurement process through customization of metrics that BAIs will adopt to analyze and report on their economic impact.

The specific economic outcome categories that participants identified include the following:

• Firm Creation and Survival: Measures of the number of firms created during a given reporting period and the operational status of alumni firms over time.
• Investment: Measures of the value and kind of follow-on funding and investment capital raised by client firms, as well as the return on invested capital.
• Revenues: Measures of any increase in customers and revenues, including a breakdown of domestic and international sources.
• Job Creation: Measures of the total number of jobs created and retained, including more specific measures of the types of jobs that have been created (e.g., full-time vs. temporary or contract positions)
• Markets, Products and Intellectual Property: Measures of the market potential of client firms, the products developed and the number of patent applications filed and granted.
• Capability Creation (people and firms): Qualitative measures of entrepreneurial competencies developed by individuals and founder teams participating in BAI programs.
• Social Impact: Measures of the social and environmental impacts associated with the products and processes commercialized by client firms.

4.3 Questions and Challenges for further Discussion

During the metrics discussion, many questions were raised that remain unresolved and will require further discussion. Among the top questions and challenges for further discussion were the following:

• Early (leading) indicators vs. lagging (trailing) indicators and rates of return. Of the many metrics identified by the group, which are best early indicators of success and which should be considered lagging indicators? Are there metrics for identifying and measuring early indicators of disruption or game-changing products and companies? What are the most effective measures for assessing the rate of return on investments in BAIs and client companies?
• Modelling, comparison and interpretation of data. How should BAIs and stakeholders interpret performance data in a way that takes regional and sector-based differences into account? Can the community develop baselines that would permit fair and effective benchmarking of BAI impact? What is the appropriate level of granularity for measurement: the individual, company, cohort, institution, ecosystem, or all of the above?
• Data synthesis, validation and reporting. If BAIs agree to contribute their data to a shared national platform, who will hold the responsibility for validating, synthesizing, packaging and reporting the data? Will this responsibility be delegated to an industry body or a trusted 3rd party? Will shared data analysis tools be developed to enable more sophisticated data modelling and reporting across the BAI community?
• Attribution, causation and correlation. How can the industry best untangle the manifold issues related to attribution, causation and correlation when it comes to representing the impact of BAI services on the economic performance of their clients and on local, regional and national GDP growth? How should these issues be addressed in the context of the national framework?
• Client response rates. How have BAIs that have generated high response rates sustained a high level of engagement with their alumni companies? How can other BAIs increase their response rates? Can BAI clients be contractually obligated to report, or could economic outcome data be collected from official government records?
• Administrative costs of data collection and reporting. Given the increased emphasis on high quality data collection and reporting, should funds for data collection and reporting be built into government funding contracts with BAIs? To what degree will funding be required to support an ongoing national collaboration, including the development and maintenance of shared assets such as a national dashboard with tools for data analysis and visualization?

Some of these questions will defy easy answers. For others, there may never be a clear consensus across the community. For example, there was considerable debate over which metrics provide the early (leading) indicators of success. For some, capital raised is the best early indicator because capital provides 3rd party validation and often precedes revenue generation. Citing the fact that many profitable, high-growth companies never seek outside capital, others argued that revenue generation provides the earliest genuine indicator of company viability because revenues (and sustained revenue growth) provide clear evidence traction in the market. There will never be one right answer in this debate as the question of which data points provide the best indicator of early success will inevitably depend on the types of firms and sectors a given BAI supports.

This one (simplified) example provides an indication of the nuance and complexity of some of the questions that the BAI community will confront in forging a national framework. Nevertheless, there is a clear eagerness to debate and discuss all of these questions further and one of next steps for the industry-led working group is to establish smaller working groups that can explore these issues in more detail and report back to the broader community.

5. Conclusions and Recommendations

The fact that policymakers, investors and program leaders are coming together to the build a common system for measuring the performance of BAIs heralds the beginning of an exciting new chapter in the growth and evolution of Canada's start-up ecosystem. As they do so, the following key principles and actions will be key:

1. Forge agreement on standardized metrics, measurement tools and platforms. The first step is to continue the dialogue among industry stakeholders in order to forge agreement on a standardized reporting framework that establishes consistent definitions for job creation, revenue generation, firm survival rates and other outcome-related metrics. While the framework must be flexible enough to accommodate the diversity of programming models in Canada (including diversity in sector and growth stage focus), such standardization is a prerequisite for enabling reliable data aggregation and comparison across the ecosystem.
2. Pilot standardized reporting with a small, but representative group of BAI leaders and policymakers. Taking advantage of the leadership of a select number of BAIs would provide an opportunity to test and refine the framework with a smaller group before rolling out it on a national basis. This pilot phase should include representation from policy leaders and funding organizations to help ensure the viability of the solution. The pilot group should also represent the key economic sectors and regions of Canada and reflect the diversity of programming models and services offered by BAIs.
3. Calibrate performance measures to the various stages of firm maturity. The pilot phase will be helpful in ensuring that nationally standardized metrics capture the value created by entities that operate in different sectors and regions and with clients of varying levels of maturity. Entities or programs that focus on later stage companies, for example, should be evaluated according to the key company growth metrics identified above, including revenue, investment and job growth. Entities and programs that deliver support services for early-stage companies, on the other hand, should focus on measurable progress towards specific milestones agreed to by funders and program participants.
4. Centralize collection and reporting. Once out of the pilot phase, BAIs and government can work together to streamline performance measurement, by centralizing annual data collection and reporting through a single online platform. The creation of a single, shared platform for publicly reporting outcomes would further ease the reporting burden on BAIs and vastly improve the ability of policymakers, researchers, investors, firms and other stakeholders to access and interpret the data.
5. Build data reporting obligations into contracts for funding and support. Adding data sharing requirements into service contracts between accelerators and supported firms could improve the ability to collect longitudinal data across the various metrics identified in the measurement framework. Likewise, Canadian funding agencies should include reporting and public disclosure requirements in their contracts with BAIs, and these requirements should adhere to national standards set by the industry working group.
6. Bolster the rigor of performance measurement with authoritative statistical data. First and foremost, greater rigor can be achieved by leveraging authoritative firm-level data on employment, revenue growth and profitability in partnership with Statistics Canada and Canada Revenue Agency. Doing so will not only boost reliability, but also help avoid the problem of double or triple counting. In addition, creating a control group of non-incubated firms would enable evaluators to more definitively estimate the differential in economic performance between firms that receive support and those that don't.
7. When the time comes to evaluate investments in BAIs, recognize that different types of start-up assistance organizations do different jobs. As noted, there is considerable diversity in the structures and approaches used to support start-ups and SMEs in Canada. Moreover, the diverse economic realities present within different sectors and regions means that it is only fair to judge outcomes against targets that can be reasonably achieved in a given context. Policymakers and evaluators must take this diversity into account when benchmarking performance and when formulating the targets against which success for BAIs will be evaluated. The more systematic the collection of data across this diverse pool of organizations, the better informed the expectations for success are likely to be in different sectors and regions, and for the various organizational types that exist in Canada.
8. Identify key success factors and share best practices. While not directly captured within framework above, is it worth thinking about how the national collaboration could shed light on the success factors that drive high-performing organizations. Indeed, as innovative, high-potential models continue to evolve (often with hybrid models and/or a more specialized niche sector focus), it is important to seek a deeper understanding of the factors that distinguish the most successful BAIs organizations. Is success attributable to management capabilities, location, a competitive client selection process, partnerships with investors, research universities and corporate tenants or some combination of other factors? As part of the national collaboration, BAI leaders should seek to determine whether there are common characteristics or design choices that positively influence success and/or characteristics or design choices are correlated with poor performance. While not an exact science, a combination of data modelling and qualitative interviews could generate valuable insights about key success factors and therefore the structures and best practices that ought to be replicated across the ecosystem.
9. Include qualitative research as part of the measurement process. While quantitative approaches are best for assessing the impact of BAIs on jobs, investment and growth, qualitative research provides a necessary subtlety to the analysis of performance that can shed light key trends, opportunities and challenges. Surveys, focus groups and one-to-one interviews will not only help BAI managers and policymakers make better sense of the data, they are essential tools for identifying intangible contributions to economic performance and for formulating recommendations for strengthening Canada's entrepreneurial ecosystem.

Schedule A: Meeting Agenda

• TRACK 1: Introductions & Landscape
  • Opening Remarks
  • Round Table Introductions
  • Review of Survey Results
  • What the Results Tell Us: Group Discussion
• TRACK 2: Assessing the Potential for Collaboration on a National Performance Measurement Framework
  • Group Discussion Questions
    • What problems can a national collaboration on performance measurement help solve for BAIs and the broader start-up ecosystem?
    • What new value could collaboration unlock for BAIs in Canada?
    • How should the industry collaborate on performance measurement?
    • What are the most valuable features in a national collaboration?
    • What challenges/obstacles will need to be overcome to make industry collaboration successful?
• TRACK 3: Building a National Performance Measurement Solution
  • Group Discussion Questions
    • What are most desirable/feasible options for creating a national performance measurement solution?
    • What are the key metrics around which to build a national standard?
    • How can a national solution balance the desire for simplicity with the need to capture the diversity of BAI models and sectors in Canada?
    • Are BAIs willing to move forward with a pilot project to validate the potential for a national performance measurement solution?
• TRACK 4: Remarks by The Honourable Bardish Chagger, Minister of Small Business and Tourism
• TRACK 5: Concepts into Action & Next Steps
  • Group Discussion Questions
    • What are the logical next steps?
    • Who will do what and by when?
    • How would the group like to continue the dialogue?
• Summary & Wrap Up

Schedule B: Attendees