Evaluation of the Digital Literacy Exchange Program (DLEP)

December 2021

Key Definitions

Digital literacy:
The ability to use technological tools to solve problems, underpinned by the ability to critically understand digital content and tools. This can include the more advanced ability to create new technological tools, products and services.Footnote 1
Digital skills:

The ability to use digital technology and tools to find, manage, apply, create and share information and content. For example, creating spreadsheets, safely using social media, and securely making online purchases. Digital technology has changed the way information is found and shared, as well as how problems are solved and communicated with others. Most jobs now use digital skills, which are needed when applying other skills such as reading, writing and numeracy. Digital skills help individuals keep up with changing demands in the modern workplace and in daily life.Footnote 2

Basic digital literacy skills are needed to engage with computers and the Internet and include: the ability to use computer programs such as word processors, web browsers, email and other communication tools; the ability to access and use knowledge resources such as search engines and online databases; and the ability to access emerging technologies such as cloud computing.Footnote 3

It is important to couple these basic technical skills with analytical skills to comprehend, contextualize and critically evaluate digital media. With these skills, users can make informed decisions about the actions they take online (e.g., sharing personal information), the information they encounter online, and understand how networked technology affects users' behaviour.Footnote 4

The digital transition emphasizes the importance of digital and other relevant skills and access to digital technologies. This creates a new risk of inequalities between people who have access to the right education or training and to digital technologies (the haves), and those who have not (the have-nots). Such inequalities – often referred to as the 'digital divide' – can create new forms of exclusion.Footnote 5

Background

Overview of the DLEP

Digital Literacy Exchange Program (DLEP) funding of $29.5 million from 2018-19 to 2021-22 supports projects that teach fundamental digital literacy skills to Canadians who would benefit from participating in the digital economy. The program aims to equip Canadians with the necessary skills to engage with computers, mobile devices and the Internet safely, securely and effectively.

Eligible Recipients

An applicant must be: (1) A not-for-profit Canadian organization that has successful experience in the delivery of digital literacy initiatives; (2) Targeting underrepresented groups in the proposed initiative; and (3) Delivering the initiative at no cost to participants.

Delivery Approach

Partnerships are integral to the delivery approach as training is delivered virtually or at pre-existing facilities, such as public libraries, refugee housing complexes, seniors' homes, community centres, schools, friendship centres, meeting halls or other facilities, or locations in the community where groups can gather.

Target Population

The underrepresented groups which are targeted by the DLEP include: seniors (aged 65 and older); persons with disabilities; Indigenous People; newcomers to Canada; language minorities; low-income individuals; individuals with low education levels; and those living in rural and remote communities.

March 2017
Program announced
November 2018
36 projects selected
February 2019
Projects begin
April 2021
Program evaluation begins
March 2022
Project funding ends

DLEP Funded Projects

Through the DLEP, a total of 36 projects have been funded, with total assistance amounting to $26.2 million (the full list of projects is included in Appendix A). A total of $3.3 million was allocated to administer the program. Authorized assistance ranged from $19,127 to $3.1 million and averaged $727,146. The majority of projects (58%) received contributions of less than $500,000, while projects ranging between $500,000 and $1 million accounted for the second largest share (22%). However, 7 projects (19%) with contributions over $1 million accounted for 62% of the authorized assistance through the DLEP.

Figure 1 : Distribution of DLEP projects by range of funding assistance (in %)

Libraries and library services accounted for the greatest number of projects (8), however community organizations and technology organizations received the greatest share of funding (28% each). The majority of projects offered training in multiple languages, with 29 (or 80%) delivering training in at least two languages. Most of the projects provided training in English (31 or 86%) and French (28 or 78%), while half of projects (18 or 50%) also offered training in other languages.

DLEP Funding Approved as of March 31, 2019

Recipient Type # of projects Total DLEP $ Approved % of DLEP $

Library/Library Services

8

$4,414,072

17%

Technology

6

$7,382,639

28%

Adult Learning/Literacy

6

$2,329,404

9%

Community

4

$7,400,663

28%

Immigrants/ Refugees

4

$2,123,641

8%

OLMC

2

$219,259

1%

Other

6

$2,307,582

9%

TOTAL

36

$26,177,260

100.0%

The training offered through DLEP funded projects using various delivery methods. Based on the project applications, all approved projects employed some measure of group classes and workshops, however 50% of all projects also utilized one-on-one training, followed by drop-in sessions (31%) and train-the-trainer (17%). Online options accounted for 11% at the application stage, although this figure was much higher in practice (27 projects or 75%), as many projects adapted their delivery as a result of COVID-19. The majority of projects used multiple delivery methods, with about one-third of projects (36%) using a single delivery method and close to half of projects (44%) employing three delivery methods.

Delivery methods # of projects % of projects

Group classes/ workshops

36

100%

One-on-one

18

50%

Drop-in sessions

11

31%

Train-the-trainer

6

17%

Online

4

11%

Other

2

6%

Note: categories are not mutually exclusive

The majority of approved projects (81%) had activities concentrated within a single province or territory, while three projects were more national in scope with activities spread across 10 provinces and territories.

# of PTs with project activity # of projects % of projects

1

29

81%

4

2

6%

5

1

3%

6

1

3%

10

3

8%

TOTAL

36

100.0%

Based on the respective project application forms, seniors, low-income individuals and persons who had not completed high school were the most common participants, with more than 80% of projects targeting participants from these groups.

The number of groups targeted by project ranged from 1 to 9, with an average of 6 groups being targeted per project (target groups are not mutually exclusive). There were only two projects which targeted a single group and both were geared to seniors.

Target Group # of Projects % of Projects

Seniors

31

86%

Low-income individuals

30

83%

Not completed high school

29

81%

Persons with disabilities

25

69%

Newcomers to Canada

25

69%

Indigenous People

24

67%

Language minorities (official language)

24

67%

Rural and remote

23

64%

Do not speak English at home

17

47%

The number of learners per project ranged from 84 to 43,060, with an average of 3,429. Three-quarters of the funded projects (75%) also included a learning component for trainers. Combined, these projects are estimated to have reached 7,657 trainers, with the number of trainers for projects ranging from 1 to 4,850, with an average of 284.

Figure 2: Number of target groups per DLEP project

Methodology

About the Evaluation

An evaluation of the Digital Literacy Exchange Program is required in accordance with the Financial Administration Act and the Treasury Board Policy on Results.

The objectives of the evaluation are to examine the relevance, performance, and efficiency of the DLEP in accordance with the Treasury Board Secretariat Policy on Results.

The scope of the evaluation included all projects approved under the DLEP. The evaluation covered the period from April 1, 2017 to March 31, 2021, but more recently available data was also analyzed.

The evaluation was conducted in-house by ISED's Audit and Evaluation Branch. The evaluation used a results-based approach, examining the achievement of expected outcomes, as identified in the DLEP logic model in Appendix B.

All evaluation findings and recommendations are supported by at least three lines of evidence.

Evaluation Areas and Questions

The evaluation examined the following areas and questions.

Relevance

Is there a continued need to invest in developing and supporting the digital skills of Canadians who are under-represented in the digital economy?

Performance

To what extent is the DLEP contributing to:

  • Partnerships being formed to deliver targeted training?
  • Increased access to digital literacy training?
  • Improved understanding of the digital environment?
  • Increased confidence and improved skills in using the internet?

To what extent:

  • Do Canadians have the skills to participate in the digital economy?
  • Is Canada a connected and engaged society?

Efficiency

To what extent is the DLEP funding and delivery model an efficient approach for encouraging the participation of underrepresented groups in the digital economy in Canada?

Data Collection Methods

Four data collection methods were used to support the evaluation.

Document and Literature Review

The document review was comprised of key program and reporting documents to provide insights into the relevance, performance and efficiency of the program. The literature review was comprised of pertinent literature to gain a thorough understanding of the need to invest in developing and supporting the digital skills of Canadians who are underrepresented in the digital economy.

Performance, Administrative and Financial Data Review

The DLEP's performance data was reviewed in order to assess the extent to which progress has been made towards achieving the expected outcomes outlined in the DLEP logic model (Appendix B). An analysis of the administrative and financial data from the DLEP was also performed to assess efficiency.

Virtual Interviews

A total of 44 virtual interviews were conducted (27 by AEB and 17 by a consultant hired by the Program) with the following stakeholders to gather perspectives on the relevance, performance and efficiency of the DLEP:

  • Program recipients (non-profit organizations delivering training);
  • Partner organizations delivering training;
  • ISED officials;
  • Provincial training program officials; and
  • Representatives of other federal departments who deliver similar training.

Case Studies

Case studies focusing on five of the funded projects were conducted to provide a more detailed perspective on the incremental impacts and provide a more in-depth assessment of potential improvements to enhance program effectiveness. Case studies were selected by AEB in consultation with the Program and represented a broad range of funded projects by region, type of organization, as well as by target audience.

Challenges for the Evaluation

Three challenges were identified during the conduct of the evaluation and the evaluation team implemented mitigation strategies for these challenges.

Attribution

The presence of other similar programs makes isolating and measuring the direct impact of the DLEP challenging. To alleviate this challenge, interview questions were designed and articulated in a way that respondents could answer, to the extent possible, the incremental impact of the DLEP.

COVID-19 Impacts

Due to the onset of COVID-19, some funded projects were interrupted in 2020-21, making it difficult to measure the impacts for this fiscal year. Consequently, where appropriate, more emphasis was placed on the first three fiscal years of the evaluation period.

Respondent Bias

Many interview participants are either involved in program delivery or are direct beneficiaries and, as a result, responses may have been positively biased. To mitigate this, the purpose of the interview and strict confidentiality was communicated to participants and responses were validated across stakeholder groups and other lines of evidence.

Evaluation Findings

Findings – Essential Skills: Relevance

Finding 1: Basic digital literacy skills are essential to participate in Canadian society and the emergence of the COVID-19 pandemic has heightened their need. Digital technologies have become pervasive in many aspects of life and digital literacy skills are increasingly necessary to access education, employment and government services. However, not all Canadians have basic digital literacy skills and gaps in Internet access exist.

Digital literacy skills are essential for everyone and are key to successfully participating in society.

Multiple sources report a need to develop and support the digital skills of Canadians who are underrepresented in the digital economy, with digital inclusion being a priority across the world. Digital inclusion is important from both an education and workforce perspective. In fact, ESDC has identified digital skills as one of the nine skills for success which "are the skills needed to participate and thrive in learning, work and life."Footnote 6 However, a recent study shows that close to one-in-four Canadians (24%) had either no engagement or very limited engagement with the Internet and digital technologies.Footnote 7

Interviews with DLEP recipients and partners highlighted that the need for digital skills training has evolved over time as newer technologies are developed. Technology is only going to continue to evolve and tablets were referenced as one technology that has been important for seniors to learn to navigate – particularly during the pandemic. Having digital skills training was further suggested as being more essential than ever for job seekers, as many employers do not provide this type of training.

The expansion of digital technologies across multiple facets of everyday life presents multiple benefits, but also risks exacerbating existing socio-economic divides. With the pace of technological change, there is a need for people to become "resilient lifelong learners."Footnote 8

The COVID-19 pandemic has led to increased Internet use for various activities, with many being done for the first time.

According to findings from the 2020 Canadian Internet Use Survey, many Canadians are turning to the Internet for their purchases. More than four in five Canadians (82%) shopped online in 2020, up from 73% in 2018. Just over one in five Canadians (21%) said they shopped online for groceries more frequently than before the pandemic, and 45% shopped more frequently for other physical goods. Notably, 13% of Canadians surveyed ordered groceries online for the first time during the pandemic and 9% bought other physical goods for the first time.

Working from home was the only option for some Canadians during the pandemic, with 43% of working Canadians saying they used the Internet more often to work from home than before the pandemic and 15% indicating they used the Internet to work from home for the first time.

Figure 3: Activities done by Canadians for the first time during COVID-19

Source: Statistics Canada, Canadian Internet Use Survey 2020

Without the ability to connect in person for much of 2020, most Canadians used the Internet to communicate with others through instant messaging apps (76%) or social networking websites (71%). Almost two-thirds of Canadians (64%) used the Internet to make online voice or video calls, while 28% of Canadians used video conferencing services to communicate with family or friends for the first time in 2020.

During interviews, DLEP funded recipients noted that developing the digital skills of underrepresented Canadians has been critical during the pandemic (particularly for seniors) in helping them connect with family and friends, schedule medical appointments, conduct online banking and shopping, and avoid online misinformation and fraud.

Case Study Highlight: "While there was growing demand for CNIB's workshops before COVID-19, during the pandemic CNIB staff said demand exploded as people wanted to learn how to use Zoom to connect with friends and family."

Findings – Access Gaps: Relevance

Significant gaps exist in Internet use and access among various groups across Canada.

Digital access is crucial for digital literacy and large gaps remain in Canada despite broadband being declared a basic service by the CRTC.Footnote 9 Recent surveys show that underrepresented groups in Canada fall well below the overall national average for Internet use. A greater proportion of respondents from each of these groups did have access to the Internet at home, pointing to gaps in usage versus access. Those closest to the national averages overall were newcomers, whose gap between use and access was also among the smallest (3.9 percentage points). While persons with disabilities had the highest rate of Internet access (94.9%), their use versus access gap was among the highest (14.4 percentage points) – most likely due to accessibility issues, such as requiring specialized equipment or software.Footnote 10 Low-income individuals had the lowest rates on both dimensions (65.8% and 71.4%), although the gap between access and use was smaller (5.6 percentage points).

Figure 4: % of respondents reporting Internet use and Internet access at home (2018)

"The digital literacy agenda is complicated by the pace of technological change, socioeconomic barriers to participation in the digital economy, and a fragmented digital literacy policy and education landscape. In other words, who you are and where you live will have significant impacts on your exposure to and ability to develop digital literacy skills and competencies."

Brookfield Institute. Digital Literacy in a Digital Age: A Discussion Paper, 2017

Findings – Access Gaps: Relevance

Finding 2: Low digital literacy leads to adverse social and economic consequences for individuals, and therefore the overall economy, necessitating support for basic digital literacy training. There was strong demand for DLEP funding, which filled a gap in the fragmented training landscape by enabling free training on various topics, providing flexibility to adjust content to suit the needs of target audiences, and supporting access to online services during the pandemic.

Lack of digital skills has significant social and economic consequences and can exacerbate socio-economic disparities.

"The consequence of not closing the gap between the demand and supply of Information and Communications Technology (ICT) talent and skills will result in a $27.3 billion direct loss in GDP. In addition, the associated losses from multiplier effects, lost tax revenues, lower productivity and human resources costs by Canadian businesses trying to fill these positions will be billions of dollars."

Information and Communications Technology Council (ICTC). Digital Talent: Road to 2020 and Beyond – A national strategy to develop Canada's Talent in a global digital economy, 2017

Digital inclusion presents benefits from many angles, including economic, social and access to government services, with the need for digital inclusion being highlighted during the COVID-19 pandemic.Footnote 11 Lack of capacity and knowledge are common barriers to participation in the digital economy, as are language barriers due to the predominance of English content.Footnote 12 The digital divide primarily affects groups that are most vulnerable and would benefit from participation in the digital economy. Basic digital skills are needed by everyone and the DLEP provides free training to key underrepresented groups in the digital economy, working closely with partners to serve clients in settings where they are familiar.

Interviewees noted that the need for digital literacy training is particularly important for newcomers to Canada, those with lower incomes who cannot afford the cost of upgrading their digital skills, and those in rural areas where there is a lack of access to dependable technology or the Internet.

Findings – Availability of Training: Relevance

There are gaps in the availability and use of free training resources, with most turning to friends or family for instruction.

The 2018 Canadian Internet Use Survey had a series of questions pertaining to the use of training resources, where respondents were asked what kinds of learning activities they undertook to improve digital skills within the last 12 months. A significant share (44%) reported that they did not undertake any training at all. For those who did undertake training, the gaps in the availability and use of free and publicly accessible resources captures the need to invest more in developing and supporting the digital skills of Canadians, especially for underrepresented groups that face more barriers at the onset.

Data shows that respondents overall rely most on interpersonal instruction from friends or family, followed by free online or self-guided training. Across all three categories, respondents from underrepresented groups tended to be below the Canadian averages for training uptake. Non-English speakers and seniors had gaps that were comparatively much greater than most groups. Seniors benefitted the least from free public programs or organizations. This points to possible gaps in publicly available programming or accessibility issues.

Figure 5: % of respondents reporting use of training resources (2018)

Source: Canadian Internet Use Survey 2018

Case Study Highlight: "The CNIB's DLEP-supported workshops are one of the few places where those with vision loss can be introduced to the assistive technology and high-tech adaptive equipment that allow them to participate in the online world. CNIB staff found that those in rural areas were the least aware of the digital technologies available to those with vision loss and in greatest need of CNIB assistance."

The digital literacy training landscape in Canada is fragmented with various providers targeting different demographics.

There is a growing number of digital literacy training providers, with third-party delivery organizations being well placed to provide responsive curriculums that can adjust to changing technologies. Digital technologies themselves also present opportunities for building digital skills, for example through Massive Open Online Courses (MOOCs), which are particularly well suited to supporting lifelong learning as there are many free options available, with no limits on the number of participants, and they can be completed any time.Footnote 13

The digital literacy training landscape is however fragmented, with different actors filling different needs. The DLEP helped increased training under the Library/Community Access Programs and Grassroots Community / Mentorship Programs. In some cases, DLEP funding allowed recipients to expand their existing programs to reach more participants or broader client groups. Nevertheless, gaps exists. The literature shows that despite the growth in programs offered, access to training continues to be a challenge particularly for low-income individuals and those outside of urban centres.Footnote 14

Figure 6: Digital Literacy Training Landscape

Adapted from: Brookfield Institute, Levelling Up: The quest for digital literacy, 2018

There was strong demand for funding through the DLEP, with applications totalling more than five times the DLEP budget.

There was strong demand for the DLEP, with 151 applications received requesting $136 million against a budget of $26.2 million. The amount of DLEP funding requested ranged from $6,000 to $10.2 million, with an average of $964,495 and a median of $401,576. Most applicants were either social/community services, groups or advocacy organizations (33%), or learning/literacy-related organizations (32%). A total of 143 applicants were identified as eligible.

The delivery organizations were selected through a competitive process and were assessed against criteria such as their success in providing similar training in the past, the quality of the proposed training and how well it meets the needs of the target population. A total of 36 applications were ultimately approved (24% of all applications).

Literature shows that community and library programs like those funded through the DLEP are filling a gap, and their flexibility in responding to needs of various learners is important. Some sources call for increased government funding to support equal access to training across the country and cite short term funding as a challenge to developing stable, long-term programs.Footnote 15 This was also raised in the interviews with DLEP funding recipients.

"Community programs, including libraries, immigrant service organizations, seniors' organizations, and other social service programs are filling a distinct gap in the digital literacy landscape, providing accessible, low-cost or free, beginner and intermediate programming, as well as drop-in access to equipment and internet for practicing and self-study."

The DLEP enable training on various topics to meet the needs of different groups and provide flexibility to adjust content.

The DLEP provided flexibility to adjust content to meet the needs of various groups. Although curriculums varied, the majority of organizations offered online/device safety, security, and privacy (86%) and internet basics (83%) in their courses. Computer basics and computer programs/productivity apps (78%) were also popular, followed by social media (75%), online goods and services (69%), and communication tools and mobile device basics (67%) – all of which have become key activities in participating in the digital economy and social environment. The inclusion of online research (56%), cloud-based technologies (50%), and digital multimedia (44%) points to the growing relevance of these skills in digital literacy, while foundational keyboarding skills have become less imperative and only offered by 17% of organizations. Over one-third of organizations (36%) also covered "other" skills and topics in emerging technologies – such as coding, AI and automation, 3D modelling and printing, cultural/language resources, accessibility functions, bookkeeping, and ethical use/conduct.

Figure 7: Most popular topics covered in DLEP-funded projects(%)

With the DLEP providing flexibility to adjust content, it enabled many recipients to adjust their content as a result of the COVID-19 pandemic. Interviews and case studies showed that many funding recipients added new topics such as using Zoom, MS-Teams and Google Hangout/Meet/Classroom; fraud prevention; and recognizing misinformation online. Content for booking vaccines and downloading vaccine passports was also introduced.

Findings – Complementary Initiatives: Relevance

Finding 3: The DLEP complements other government programs aimed at improving broadband access and affordability of Internet and devices. There is a need for ongoing access to digital literacy training to ensure that all Canadians can keep up with the pace of technological change.

Supporting digital inclusion requires governments to address multiple issues with different policy levers.

Internet access and ownership of a digital device are important precursors to digital inclusion. However, the need for programs that teach basic digital literacy skills is also important to support digital adoption.Footnote 16 Interviewees and case studies pointed to the need to do more to increase access to broadband, as well as affordability of Internet and digital devices over and above support for digital literacy skills, in order to ensure that Canada is a connected and digitally engaged society.

The DLEP is part of the Innovation and Skills Plan, an ambitious effort to make Canada a world-leading centre for innovation, help create more well-paying jobs, and help strengthen and grow the middle class. One of the key pillars of this plan is inclusiveness – ensuring all Canadians have an opportunity to participate in the digital economy, especially for those to whom evolving technology can deliver life-changing benefits but who may encounter barriers to accessing it, such as income, literacy, geography and ability.Footnote 17

The DLEP is complemented by initiatives such as the Universal Broadband Fund, Computers for Schools, Technical Work Experience Program, Connecting Canadians, Connect to Innovate, Affordable Access and Accessible Technology Development. The DLEP also complements existing efforts underway at the provincial level to support the integration of digital literacy skills in curriculums and to increase the online delivery of government programs and services.

Findings – Partnerships: Performance – DLEP

Finding 4: Partnerships were important for the successful delivery of the DLEP. Although partner involvement varied at the project level, they were seen as effective in delivering the training and were key to accessing the underrepresented groups targeted by DLEP projects.

Partnerships were an integral part of delivering training for DLEP funded projects.

Given the DLEP approach of delivering training at pre-existing facilities, partnerships were an essential component to the program. Interviews and case studies showed that partners were important for providing resources such as facilities and volunteers, and also for recruiting participants. Based on annual reports submitted for April 2019 to March 2020, there were a total of 1,541 partnerships leveraged or created to deliver DLEP training. The number of partnerships per project ranged from 0 to 1,020, with an average of 50 and a median of 10. From April 2020 to March 2021, there were a total of 1,075 partnerships leveraged or created to deliver DLEP training, with the number of partnerships per project ranging from 0 to 794, with an average of 60 and a median of 8. Five projects reported no partnerships across both years.

Number of Partnerships to deliver DLEP 2019-2020 2020-2021

Mean

50

60

Minimum

0

0

Maximum

1,020

794

Total

1,541

1,075

Percentiles

25

4

5

50

10

8

75

26

35

Through interviews and case studies, funded recipients provided numerous examples of partnerships including with municipalities, Indigenous communities, ethnic communities, newcomer associations, community organizations, school boards, social clubs, libraries, and retirement homes. Certain partners are needed to reach specific types of participants (e.g., immigrant/refugee organizations for newcomers to Canada). However, because some funded projects are located in rural/remote locations, they may not have access to as many partners or specific types of partners.

It is difficult to assess the success of the DLEP on this indicator (number of partnerships) as no target was established. However, all funding recipients and partners interviewed agreed that the value of the partnerships created was exceptional. Although there were some challenges identified (e.g., changes to how the training would be delivered), they were all resolved in a timely fashion.

Two case studies showed that as the number of partnerships grew, the coordination work also grew and ensuring adequate human resources to support the partnerships was therefore seen as important. In one of the case studies, partners were recruited to participate on a Steering Committee which was used to inform the development of workshop content. Volunteers were also recruited to help during the training, with many of them coming from the partner organizations. One of the partners interviewed for the case study appreciated having somewhere to refer clients with digital literacy needs and the flexibility to develop workshop content to meet specific needs.

Partnerships were an effective means for recruiting training participants.

Case studies demonstrated that partnerships were particularly useful for recruiting training participants. Evidence from two case studies showed that once a partner was enlisted, they would take care of recruiting participants for the sessions. Funded recipients also noted that the partnerships enabled access to target groups that they otherwise might not have been able to reach. Often, the partners provide the equipment and facilities for training, while the funded recipient only has to commit to training people at the location provided. Partners noted that the partnerships were essential for outreach, as the benefits of advertising during the pandemic (e.g., via posters at libraries) have been significantly lower.

Often, new partnerships can lead to additional collaboration and service opportunities that wouldn't otherwise have been possible, and essential to helping reach target groups. Using the existing resources of libraries and library personnel, for example, was paramount to leveraging the network and broadening the reach of the program. In addition, having partnerships allowed access to free physical space for the training. Partnerships allowed training providers to quickly establish where the needs existed and match clients to the program. Further, partnerships with community organizations were also deemed to be critical in delivering the training.

One of the case studies noted that other literacy centers across Quebec would also like to offer similar training programs to their DLEP funded project. This was echoed in an interview with a representative from a provincial association representing a network of community-based Literacy and Basic Skills agencies, who indicated they would also be interested in leveraging the resources created through the DLEP funded projects.

Training library staff in rural communities so that they can answer clients' technical questions was also identified as another opportunity to expand partnerships. It was also noted that this approach has a more sustained impact than training individual participants.

Case Study Highlight: "According to staff one funded recipient, once a partner was recruited to provide the location, they would take care of recruiting participants, often through word of mouth. This approach worked very well as trainers simply showed up to deliver the training."

Findings – DLEP Reach: Performance – DLEP

Finding 5: The DLEP has increased access to digital literacy training for Canadians who are under-represented in the digital economy. The program has been effective in reaching seniors, which was critical in light of the growing demand stemming from COVID-19 confinement measures, with less progress for some target groups. Funded recipients who transitioned to online delivery greatly increased their reach, although there were challenges for performance measurement.

The overall DLEP targets were exceeded, however not all targets for the various underrepresented groups were met.

Based on the initial targets for the 36 funded projects, the DLEP anticipated reaching 123,457 participants by March 2022. As of June 2021, data shows the DLEP had exceeded these initial targets with 185,394 participants reached to date. Interviews indicated that there were two primary factors which resulted in greater uptake – the first was the COVID-19 pandemic, which increased training needs for people to use digital skills for work, school and to connect with family and friends. The second was the transition to online training, which demonstrated how online delivery can significantly increase the reach for training providers.

Targets for five underrepresented groups were developed for the DLEP. Data shows the program greatly exceeded its target for seniors (424%). The DLEP is also close to meeting its target for persons with disabilities (76% of target reached) and those who do not speak English at home (71%), but may attain these targets given the 9 months of project activity remaining. Progress towards the target for Indigenous people (52%) and people who have not completed high school (33%) is however low. Some interviewees suggest that some groups may be under-reported as a result of participants preferring not to disclose personal information. The data and document review also identified potential issues due to inconsistent definitions (e.g., language minorities, low-income and rural) and lack of consistency on how to report participants belonging to multiple underrepresented groups.

Figure 8: # of DLEP participants reached vs. target

Note: categories are not mutually exclusive

The transition to online delivery increased reach but impacted who could access training and posed challenges for data collection.

With respect to online delivery, not all projects were able to pivot in response to COVID. Based on quarterly data, there were six projects where there were at least two quarters (6 months) with no activity in 2020-21. Some of these were eventually able to resume in-person training once public health measures allowed, however these often resulted in smaller groups or even individual training which hampered their ability to meet their targets. The shortfall was, however, more than made up for by the projects that pivoted to online delivery. Based on revised estimates, the DLEP is now forecast to reach 347,474 participants by March 2022.

Although some interviewees noted benefits for virtual training sessions (e.g., enabling access for immobile or isolated people, not having to travel in winter, etc.), some people could not make use of online training due to a lack of access to technology or due to having no computer skills to be able to get online. Some training providers were able to loan laptops in certain cases, but this was not the norm. Interviewees cautioned that this shift to online delivery often meant that those most in need may be left behind due to financial barriers and access issues. This quick shift also meant that many training providers were unable to collect the required performance measurement information, resulting in data gaps. Paper surveys that had been used to collect data were no longer an option with online classes, and with many providers offering self-guided training there was no mechanism to ensure the required information was collected. Some interviewees who offered facilitated online sessions, noted that training sessions were shorter, which meant that participants did not always have time to complete the surveys.

Case Study Highlight: "With the closure of the library branches in March 2020 due to COVID-19, the Palliser Regional Library System transitioned to virtual courses. Since then, it has created a Digital Academy with 60 short courses for participants on Internet basics and other topics of interest, and 38 courses to assist librarians in answering technical questions. The Digital Academy has allowed the library to vastly expand its reach and to provide courses on a wider array of topics where people can move at their own pace. As of March 2021, those courses had been used by more than 7,000 participants." – (Initial target to March 2022 was 1,225).

DLEP increased the capacity of recipient organizations to deliver training by also supporting training for instructors.

In addition to training for participants, DLEP also supported the development and delivery of training for instructors needed to deliver digital literacy training. As of June 2021, a total of 6,265 instructors had been trained through the DLEP funded projects, which represents 82% of the original forecast (7,675) at the application stage. While three-quarters of the funded projects included an instructor component in their application, nearly all projects (34 or 94%) reported training had been provided for instructors with only two projects reporting no training for instructors. One-third of projects (12 or 33%) reported 1 to 10 instructors trained. Two projects reported over 1,000 instructors trained, and together they accounted for the majority of the instructors trained (3,724 or 59%). Both projects were among the largest funded through DLEP, each with contributions over $1 million.

Figure 9: Number of Instructors trained by DLEP projects

Case Study Highlight: "Training library staff in rural communities so that they can answer clients' technical questions has a more sustained impact than training individual participants."

The type of training provided to instructors is reported through the quarterly claim forms when eligible costs are reviewed. The case studies provided examples with four out of the five recipients indicating that instructors were trained walking through the DLEP curriculum. One case study noted hiring an IT technician to provide training to the instructors in advance of delivering the DLEP training. The literature and case studies also show that training instructors may lead to longer-term benefits as a result of increase capacity to deliver training on a continued basis.Footnote 18

Finding 6: The DLEP was successful in providing training in each province and territory. The number of projects varied greatly by region, as did the number of participants.

On a per capita basis, reach was generally in line with population, although there was a higher reach in Nunavut and Yukon.

As of June 2021, the DLEP was successful in meeting its target to provide training in each province and territory (PT) by March 2022. However, the number of projects providing training in each PT varied greatly. Prince Edward Island had one project with activity, while all other regions were served by multiple projects. Quebec had the most activity (15), followed by Ontario (13), Alberta (10) and British Columbia (9).

Figure 10: Number of DLEP projects with activities by PTs (June 2021)

The program saw a greater number of participants in the more populous provinces, with the largest number in British Columbia (65,616), followed by Ontario (54,646) and Quebec (12,411). Looking at program reach relative to the population overall, the DLEP was able to reach the greatest share of the population in Nunavut (5.9%), followed by Yukon (2.4%), BC (1.3%) and NWT (1.1%). The successful reach in Nunavut, Yukon and BC was attributable to greater than expected demand for digital literacy training.

Geographic Location Participants Reached
June 2021
Population
(Q2 2021)
Population Reached (%)

British Columbia

65,616

5,174,724

1.3%

Alberta

8,612

4,444,277

0.2%

Saskatchewan

9,892

1,179,906

0.8%

Manitoba

4,822

1,382,904

0.3%

Ontario

54,646

14,789,778

0.4%

Quebec

12,411

8,585,523

0.1%

New Brunswick

715

783,721

0.1%

Prince Edward Island

8

160,536

0.0%

Newfoundland & Labrador

3,790

520,286

0.7%

Nova Scotia

2,517

982,326

0.3%

Yukon

1,007

42,596

2.4%

Northwest Territories

475

44,991

1.1%

Nunavut

2,348

39,536

5.9%

Total

166,859

38,131,104

0.4%

Note: Total is lower than participants reached (185,394) due to missing data.

Findings – Participant Impact: Performance – DLEP

Finding 7: The DLEP has contributed to increasing digital literacy skills among Canadians who are underrepresented in the digital economy. Although there were gaps in performance measurement data, the available data and stakeholder feedback demonstrated that DLEP participants increased their knowledge, skills, confidence and Internet use.

Overall, the DLEP exceeded its targets related to increasing confidence, skills and Internet use.

The DLEP experienced data collection challenges. Performance data related to skills, confidence and Internet use was not available by underrepresented group, making it difficult to gauge the effectiveness by group. Some projects also did not provide data or provided incomplete data and some challenges were observed with the consistency/categorization of information being reported. The DLEP also experienced low response rates for participant surveys relative to the number of participants reached (185,394). Caution should be used in attributing the same the level of effectiveness to online and in-person training given that the results from the virtual training are under-reported.

Despite the data collection challenges, the available performance measurement data submitted through annual reports suggests that the DLEP was effective at increasing the skills, confidence and Internet use of participants and improving their ability to understand the digital environment and make informed decisions when online. The program exceeded its targets on all four measures.

Figure 11: % of DLEP participants reporting 'yes'

Recommendation 1: ISED's Connected Canada Branch should provide additional guidance to recipients to ensure data is collected consistently across projects, including developing common questions and definitions and exploring the use of an online survey tool, to improve data collection efficiency and provide an anonymous self-identification option which may improve the under-reporting for some groups.

Various sources showed that the DLEP is having a strong impact on the digital literacy skills of participants.

Program documents, key informant interviews and case studies demonstrated various ways in which DLEP training has benefited participants. Testimonials obtained through quarterly reports for the case studies showed that by the end of the training, participants were better able to use their devices and had acquired a variety of skills and the ability to use different tools and applications. Participants were also better able to resolve their own issues and demonstrated increased confidence as a result. The learners were grateful and appreciative of the training, particularly during the pandemic, as their new digital skills enabled them to overcome the isolation they would have experienced otherwise.

Interviewees noted that, among other things, the training has helped participants secure employment, connect with family members virtually for the first time, use devices to help navigate the transit system, place online orders for grocery delivery, and use social media. In addition, the training has helped with fundamental digital skills such as connecting to Wi-Fi, setting up passwords, keeping accounts secure, dealing with online harassment, and keeping children safe online. A couple of interviewees indicated that participants receive a certificate at the conclusion of their training that they can then use when applying for jobs, which was noted as being especially important for newcomers to Canada.

All funded recipients remarked that the DLEP has greatly impacted the confidence and skills of participants in using the Internet. For some groups like seniors, it was noted that the Internet can be overwhelming. Further, many individuals are afraid of falling into online scams or conducting online banking transactions or purchases. The training helped participants become more aware of what to do to avoid falling into traps.

Many participants with prior anxieties realized that after receiving basic computer training that it was not as complicated as they had thought. As a result, participants often remarked how they appreciated having increased independence and less reliance on others for assistance.

Case Study Highlight (participant testimonial): "One elderly woman mentioned that the DLEP training has changed her life. Before, she had no idea how to operate her tablet (a gift from her grandson). When she experienced difficulties with her device, her grandson was the only source of support. At the same time, they cannot communicate effectively due to a language barrier. Therefore, her grandson would only be able to solve the problem for her, but not teach her how to do it herself. Often, she would give up when she encountered a challenge because she did not want to bother her family.

After completing the DLEP training in her mother tongue, the program not only gave her a strong fundamental background in the digital world, but it also allowed her to solve her own problems step by step. All of her family members were amazed by how she can use email for communication, search for information online and use her Apple ID for downloading. She is now enrolled in our Digital Ambassador training and hopes to volunteer as a teaching assistant in the future."

Findings – Promising Practices: Performance – DLEP

Finding 8: Interviewees reported numerous promising practices and lessons learned to support the delivery of digital literacy training to Canadians who are underrepresented in the digital economy. Opportunities for ISED to leverage these, along with the digital literacy resources created with the support of the DLEP, should be explored.

As a result of delivering training over a sustained period of time, DLEP recipients identified many promising practices.

Key informant interviews and case studies highlighted a number of lessons learned and promising practices to support the delivery of digital literacy training to underrepresented groups. Numerous interviewees suggested that ISED should explore opportunities for DLEP funded recipients to connect with each other in order to share and implement promising practices and lessons learned.

General Lessons Learned and Promising Practices

  • Teaching participants to take screenshots to facilitate tailored support for individual issues.
  • Having volunteers attend training sessions to help with providing direct assistance during the workshops.
  • Combining in-person and online classes, with in-person sessions to get comfortable going online and then using online sessions to further develop the skills of training participants.
  • Using one-on-one sessions to tailor support to a person's current skill level.
  • Establishing peer support groups and providing additional training to participants who wish to help train their peers on their newly-learned digital knowledge.

Virtual Delivery

  • The virtual learning model was effective in reaching participants that are physically disabled.
  • For the delivery of online training, using live demonstration of the trainer's screen was effective.
  • Using QR codes (a two-dimensional bar code) to join online workshops was helpful, as it eliminated the need to type in an address.

Resources and Equipment

  • Having a booklet to follow along and being able to take it home contributed greatly to improving confidence and skills.
  • Having videos embedded within tutorials to provide a visual aid, as well as using simple language and teaching proper terminology, were noted by interviewees as being valuable.
  • Lending equipment to allow participants to determine which device best suited their needs before purchasing their own.
  • Working with local retailers to stay up-to-date on tablet sales so that special offers and sales could be shared with participants.

Workshop Content

  • Adjusting content based on feedback and emerging needs.
  • Involving partners in the identification of needs and the development of content.
  • Including training content on available phone carriers and their fee structures so people are less worried about hidden fees and overage charges.
  • Having the workshops tailored to a specific device type (e.g., smartphone, tablet, laptop or computer).
  • Making course content available online after class to allow trainees to practice at home and/or refresh their memories.

Recommendation 2: ISED's Connected Canada Branch should explore opportunities to share some of the digital literacy resources that have been created through the DLEP to allow partners to leverage these resources. ISED should also explore opportunities for DLEP recipients to connect in order to share promising practices and lessons learned.

Findings – Global Comparison: Performance – Canada

Finding 9: Globally, Canada ranks high in terms of digital skills. Despite this, certain gaps persist among underrepresented groups with respect to Internet use. This suggests that more needs to be done to address the gap in digital literacy skills, including improving access and affordability to the Internet for all Canadians.

- Ranking Overall Score Accessibility
2020 2017 Change 2020 2017 Change 2020 2017 Change

Singapore

1

1

Unchanged

86

83

Improved

86

80

Improved

Sweden

2

2

Unchanged

85

80

Improved

76

68

Improved

Denmark

3

5

Improved

85

79

Improved

80

71

Improved

Netherlands

4

3

Degraded

84

79

Improved

79

70

Improved

United States

5

4

Degraded

84

79

Improved

77

68

Improved

Australia

6

6

Unchanged

84

78

Improved

78

67

Improved

South Korea

7

7

Unchanged

84

78

Improved

82

70

Improved

Qatar

8

13

Improved

83

75

Improved

75

63

Improved

Canada

9

14

Improved

82

75

Improved

77

67

Improved

United Kingdom

10

8

Degraded

81

77

Improved

75

68

Improved

International comparison shows that Canada ranks high across four digital inclusion levers.

(Continued) Affordability Ability Attitude
2020 2017 Change 2020 2017 Change 2020 2017 Change

Singapore

98

87

Improved

84

83

Improved

92

85

Degraded

Sweden

96

93

Improved

86

84

Improved

85

84

Improved

Denmark

94

94

Unchanged

84

77

Improved

81

66

Improved

Netherlands

92

89

Improved

86

85

Improved

79

76

Improved

United States

95

92

Improved

88

86

Improved

73

73

Unchanged

Australia

96

92

Improved

86

83

Improved

72

66

Improved

South Korea

87

86

Improved

84

80

Improved

80

78

Improved

Qatar

98

95

Improved

73

67

Improved

85

84

Improved

Canada

90

87

Improved

84

78

Improved

74

70

Improved

United Kingdom

91

89

Improved

82

81

Improved

76

74

Improved

Adapted from: Berger, Roland (2021), Bridging the digital divide: Improving digital inclusion in Southeast Asia.

Canada ranks high on the Digital Inclusion Index (9th), and has shown improvement between 2017 and 2020. Canada ranked 9th on the Index in 2020, up from 14th in 2017. Across the four dimensions, Canada scored highest on affordability (90/100), while its score on ability increased from 78 to 84 between 2017 and 2020.

Canada ranks high on various global measures of digital readiness, but gaps in Internet use persist.

According to the OECD, Canada's workforce is among the five highest in the world for generic digital skills.Footnote 19 It is ranked 11th in another survey, the 2019 IMD Digital Competitiveness Ranking, from a pool of 63 countries.Footnote 20 Despite this, earlier studies from 2016 and 2017 found that certain groups in Canada lack the required skills to participate in the digital economy. Over and above structural barriers related to access and affordability, barriers to online participation included lack of capacity or knowledge, lack of motivation, not seeing value and security concerns. Barriers also exist for French-speaking Canadians and newcomers stemming from the predominate use of English on the Internet.Footnote 21

Country Institute for Management Development (IMD) World Digital Competitiveness Ranking 2019 Network Readiness Index 2019

Canada

11

14

Germany

17

9

Singapore

2

2

Sweden

3

1

UK

15

10

Australia

14

13

Source: Gekara, Victor et al. RMIT University (2020). International scan of approaches taken by select economies to build the digital skills of the existing workforce in response to the digital transformation of industry.

Preliminary data released from the 2020 Canadian Internet Use Survey shows that between 2018 and 2020 the share of Canadians (92%) using the Internet increased slightly from 2018 (91%). Despite the high level of Internet use among the Canadian population, gaps remain. For example, only about 62% of Canadians 75 years of age and older used the Internet in 2020. Other groups that registered Internet use below the Canadian average were: people with a disability (84%), those not employed (85%), those living outside of a Census Metropolitan Area or Census Agglomeration (87%) and Indigenous people (88%). Almost all Canadians living in a family household with children under 18 years of age (98%) used the Internet in 2020, compared with 80% of single-person households.

Figure 12: % of respondents using the Internet (2020)

Findings – Connected Society: Performance – Canada

Canadians reported greater use of the Internet for social and leisure activities, although gaps persist between groups.

The 2018 Canadian Internet Use Survey provides data on a broad range of activities that are key to participating in the digital economy. Participation in these activities would denote that respondents have the necessary skills or training or use the Internet on various devices. In general, Internet use was higher for communication and information related activities, suggesting greater use for social and leisure activities, although gaps between groups persisted.

Communication-related activities:
uploading content on blogs/content-sharing platforms, dating platforms, online voice/video calls, social networking, instant messaging, emails
Information-related activities:
online research, mapping locations/searching up directions, weather-checking, reading the news

Figure 13: % of respondents by group reporting communication or information-related activities in the last 3 months

Source: Statistics Canada, Canadian Internet Use Survey, 2018

Participation in device-related and internet-related activities were lower, suggesting larger gaps in terms of technical skills.

Device-related activities:
automatic updates, Bluetooth, location privacy settings, GPS, router security, photo/video transfer, connecting to network
Internet-related activities:
backing up/sharing via online data storage, profile privacy settings, file downloads, blocking unwanted emails and messages, erasing browser history

When compared to the national averages, all underrepresented groups had lower rates of participation across all these activities. Low income individuals and seniors had the lowest participation rates with a gap of close to 30%, followed by those who had not completed high school and non-English speakers (where the gaps were close to 20%).

Figure 14: % of respondents by group reporting device or internet-related activities in the last 12 months

Source: Statistics Canada, Canadian Internet Use Survey, 2018

Stakeholders indicated that more needs to be done to ensure all Canadians have the necessary digital literacy skills.

Funded recipients interviewed indicated that in some respects, for instance online shopping, Canadians are by and large digitally literate. However, in terms of being able to effectively telework from home and learn new software, the digital literacy rate is not as high. The general consensus is that skills vary significantly based on age, location and income – with disadvantaged groups having lower digital literacy skill levels. For example, Canadian residents in the North have challenges in accessing technology and the Internet and thereby have lower skill levels. Similarly, one funded recipient noted that close to 50% of newcomers to Canada do not have any digital literacy skills. Three common suggestions (not necessarily all within the purview of the DLEP) were put forth by funded recipients and partners:

  1. Continue delivering targeted training to develop basic digital literacy skills for disadvantaged groups;
  2. Provide additional support to community groups, as they are better suited to reach certain segments of the population; and
  3. Find ways to make high quality Internet access more readily available in all areas of Canada – particularly in rural areas.

During interviews, funded recipients generally agreed that Canada is a connected and digitally engaged society compared to other countries. However, it was noted that there are still certain pockets within Canada, particularly in poorer areas, where Canadians have little to no digital skills. In addition, it was remarked that many businesses aren't as digitally engaged as they could be (e.g., not having a functional website). Suggestions for improving this included the following (again, not all under the purview of the DLEP):

  1. Having accessible Internet for all via improved infrastructure;
  2. Teaching facilitators in marginalized communities;
  3. Putting programs in place to donate recycled computers to communities in need; and
  4. Implementing programs to help small businesses adopt to digital practices.

Findings – Administrative Costs: Efficiency

Finding 10: The DLEP funding model was perceived to be efficient by stakeholders in reaching the various underrepresented groups targeted by the program. Some of the features that contributed to the efficiency were the targeted approach, flexibility and funding duration.

The funding model was generally perceived to be efficient in reaching target stakeholder groups.

Overall, interviewees reported that the delivery approach was efficient and was a good model, as it helped reach many clients who otherwise wouldn't have been reached. From a training perspective, it was said that having adjustable training content that was free and specifically targeted to underrepresented groups was ideal. The project timeframes (up to 5 years) also allowed more creativity for training and enabled the measurement of impacts. Interviewees also indicated that having support for travel, staff and equipment were the main features of the DLEP that were working well. In addition, one funding recipient noted that allowing flexibility within a project was beneficial as they were able to re-allocate some unused funding from one cost category towards another to support delivering more virtual classes.

All interviewees noted a good working relationship and communication with ISED staff. However, a few respondents mentioned challenges related to program administration. For instance, seven funded recipients noted challenges with reporting, either due to the frequency or level of detail required (e.g., some of the questions being asked in the quarterly reporting were noted as being repetitive).

At the program level, total ISED administrative costs were forecasted to be 11.3%. As of March 2021, the administrative costs to date were 10.8%. The lower ratio of administrative costs is attributable to underspending in direct salary and O&M, largely due to early delays in implementing the DLEP. G&C spending was lower than expected in 2018-19, as the DLEP received a large number of applications that required additional time for review and delayed approvals. G&C spending was also lower in 2020-21 due to COVID-19 impacting projects' ability to deliver activities as planned.

Figure 15: DLEP O&M Budget vs Actual

Figure 16: DLEP G&C Budget vs Actual

Findings – Cost per Participant: Efficiency

Finding 11: At the project level, the cost per participant varied greatly. Recipients who were able to pivot to online training during the COVID-19 pandemic demonstrated improvements to operational efficiency by increasing their project reach and decreasing their cost per participant. The pivot to online learning, as well as the leveraging of reallocated funding to successful projects, has helped the DLEP reach almost three times the number of projected participants.

The reallocation of unused G&C funding to projects with strong results allowed the program to increase its reach by nearly three times.

In early 2020, approximately $2.5 million in DLEP funding from previously uncommitted funding and lapsed funding was reprofiled into future years for recipients which had reliably delivered their projects. The Connecting Canadians Branch reached out to potential recipients to determine which organizations could achieve or further advance their project milestones and targets.

The proposed amounts per recipient were determined based on their proposal for increased funding, as well as their participant and geographical reach. In considering which organizations to select for additional funding, key factors were:

  • Meeting their Contribution Agreement obligations and objectives;
  • Ability to significantly reach additional participants;
  • Proven delivery track record and diligent administration; and
  • National and regional coverage.

The additional funding was meant to increase training, increase targets and to ensure geographic reach across Canada, with a focus on under-served areas as a result of COVID-19 impacts on deliver partners.

As a result, the DLEP is now estimated to have reached a total of 357,035 learners and instructors (up from 123,457). This results in a revised cost per participant of $73, which is 39% of what had originally been forecasted for the program ($185).

Figure 17: DLEP Participant Reach – Application vs Amendment

Although the cost per participant varied, the transition to online delivery improved efficiency and increased DLEP reach.

At the project level, the difference in how DLEP funded projects are delivered varies greatly, as does the resulting cost per participant. Based on results as of June 2021, the cost per participant ranged from $3 to $3,165, with an average of $394.

There were a number of variables that contributed to this high variance in cost per participant, with the two key factors being location (urban versus rural) and delivery method (in-person versus online). The majority of projects had a lower cost per participant ($394 or less) since they were delivered in larger urban areas where more participants could be reached per session and because instructor travel costs were lower. DLEP initiatives which were delivered in rural and remote areas had much higher costs for the opposite reasons.

For example, the project with the highest cost per participant ($3,165) was serving the needs of Indigenous people in rural and remote communities within the Northwest Territories and involved high travel costs. The project used a train-the-trainer model focused on capacity building and also involved significant costs to support the co-development of digital literacy resources with the Indigenous communities (including research to inform the curriculum, consultants to develop online content, tools, resources and campaigns). Although the cost per participant was the highest, it was lower than the forecast at approval ($4,373) due to the addition of online training and resulting lower travel costs. Other higher cost initiatives also included costs incurred for website development, instructor training, and laptop purchases.

There was consensus among funded recipients that a hybrid model would be best going forward, with both online and in-person delivery. For the true novice, it was noted that there is some need for hands on demonstrations and support before they can go online to further their training.

Figure 18: DLEP Cost per participant – actual (June 2021)

Case Study Highlight: "The transition to virtual learning through the library's Digital Academy has led to a significant decrease in the cost of reaching participants from more than $150 originally to less than $25 a participant."

Conclusions

Summary of the Evaluation

Three findings presented the relevance for investing in developing and supporting the digital skills of Canadians who are underrepresented in the digital economy.

Finding 1: Basic digital literacy skills are essential to participate in Canadian society and the emergence of the COVID-19 pandemic has heightened their need. Digital technologies have become pervasive in many aspects of life and digital literacy skills are increasingly necessary to access education, employment and government services. However, not all Canadians have basic digital literacy skills and gaps in Internet access exist.

Finding 2: Low digital literacy leads to adverse social and economic consequences for individuals, and therefore the overall economy, necessitating support for basic digital literacy training. There was strong demand for DLEP funding, which filled a gap in the fragmented training landscape by enabling free training on various topics, providing flexibility to adjust content to suit the needs of target audiences, and supporting access to online services during the pandemic.

Finding 3: The DLEP complements other government programs aimed at improving broadband access and affordability of Internet and devices. There is a need for ongoing access to digital literacy training to ensure that all Canadians can keep up with the pace of technological change.

Six findings demonstrated the effectiveness of the DLEP in forming partnerships, increasing access to digital literacy training for underrepresented groups, and improving the understanding, skills and confidence of training participants.

Finding 4: Partnerships were important for the successful delivery of the DLEP. Although partner involvement varied at the project level, they were seen as effective in delivering the training and were key to accessing the underrepresented groups targeted by DLEP projects.

Finding 5: The DLEP has increased access to digital literacy training for Canadians who are underrepresented in the digital economy. The program has been effective in reaching seniors, which was critical in light of the growing demand stemming from COVID-19 confinement measures, with less progress for some target groups. Funded recipients who transitioned to online delivery greatly increased their reach, although there were challenges for performance measurement.

Finding 6: The DLEP was successful in providing training in each province and territory. The number of projects varied greatly by region, as did the number of participants.

Finding 7: The DLEP has contributed to increasing digital literacy skills among Canadians who are underrepresented in the digital economy. Although there were gaps in performance measurement data, the available data and stakeholder feedback demonstrated that DLEP participants increased their knowledge, skills, confidence and Internet use.

Finding 8: Interviewees reported numerous promising practices and lessons learned to support the delivery of digital literacy training to Canadians who are underrepresented in the digital economy. Opportunities for ISED to leverage these, along with the digital literacy resources created with the support of the DLEP, should be explored.

Finding 9: Globally, Canada ranks high in terms of digital skills. Despite this, certain gaps persist among underrepresented groups with respect to Internet use. This suggests that more needs to be done to address the gap in digital literacy skills, including improving access and affordability to the Internet for all Canadians.

Two findings highlighted the operational efficiency of the DLEP funding model.

Finding 10: The DLEP funding model was perceived to be efficient by stakeholders in reaching the various underrepresented groups targeted by the program. Some of the features that contributed to the efficiency were the targeted approach, flexibility and funding duration.

Finding 11: At the project level, the cost per participant varied greatly. Recipients who were able to pivot to online training during the COVID-19 pandemic demonstrated improvements to operational efficiency by increasing their project reach and decreasing their cost per participant. The pivot to online learning, as well as the leveraging of reallocated funding to successful projects, has helped the DLEP reach almost three times the number of projected participants.

Two recommendations were produced in the evaluation, stemming from the assessment of performance, and supported by at least three data collections methods.

Performance

Recommendation 1: ISED's Connected Canada Branch should provide additional guidance to recipients to ensure data is collected consistently across projects, including developing common questions and definitions and exploring the use of an online survey tool, to improve data collection efficiency and provide an anonymous self-identification option which may improve the under-reporting for some groups.

Recommendation 2: ISED's Connected Canada Branch should explore opportunities to share some of the digital literacy resources that have been created through the DLEP to allow partners to leverage these resources. ISED should also explore opportunities for DLEP recipients to connect in order to share promising practices and lessons learned.

Appendix A: DLEP Funded Projects

Funding Recipient Funded up to

ABC Life Literacy Canada

$ 660,000

ACORN Canada

$ 104,000

Alphabétisation IOTA

$ 110,000

Altered Minds Inc.

$ 356,000

Atelier 850

$ 112,000

Atikokan Literacy Inc.

$ 31,000

Atwater Library and Computer Centre

$ 236,000

BIBLIOPRESTO.CA

$ 2,930,000

Burnaby Neighbourhood House

$ 400,000

Calgary Chinese Community Service Association (CCCSA)

$ 530,000

Le Carrefour l'éducation populaire de Pointe-Saint Charles

$ 212,000

Centre alpha Le Tracé Inc.

$ 299,000

Chinook Arch Library System

$ 439,000

Canadian National Institute for the Blind (CNIB)

$ 1,270,000

Community Sector Council of Newfoundland and Labrador (CSC NL)

$ 885,000

Digital Nova Scotia

$ 528,000

Fredericton Area Network Inc. (the FAN)

$ 50,000

Gluu Technology Society

$ 207,000

Guelph Public Library (GPL)

$ 20,000

Information and Communication Technologies Association of Manitoba (ICTAM)

$ 1,330,000

Literacy Link South Central (LLSC)

$ 355,000

MediaSmarts

$ 1,100,000

Palliser Regional Library

$ 192,000

Parkland Country Library Board

$ 47,000

Peel Multicultural Council

$ 750,000

Pinnguaq

$ 2,860,000

Réseau Internet de Brossard

$ 847,000

S.U.C.C.E.S.S.

$ 332,000

Société d'aide au développement des collectivités (SADC) Centre-de-la-Maurice

$ 138,000

Skills Canada Saskatchewan

$ 466,000

Stony Plain Public Library

$ 300,000

Tides Canada Initiatives Society

$ 2,700,000

Townshippers' Association

$ 41,000

Vision Gaspé-Percé Now

$ 151,000

YMCA of the Greater Toronto Area

$ 2,650,000

Yukon Learning Society

$ 544,000

Appendix B: DLEP Logic Model

Text version

Appendix B depicts a logic model for the Digital Literacy Exchange Program (DLEP). A logic model shows how program activities are expected to produce outputs and in turn how these outputs are expected to lead to different levels of results or outcomes.

The left section of the logic model describes the DLEP's activities and outputs. The activities of the program include:

ISED

  • Program promotion and selection of program recipients
  • Administer Contribution Agreement
  • Monitor plans and progress

Delivery Partners

  • Design training programs for learners and instructors
  • Implement in-person training programs for instructors and learners
  • Measurement of training impacts

These two sets of activities lead to five outputs:

  • Partnerships
  • Contribution agreements
  • Training programs
  • Workshops/ events
  • Program reports

The short term outcomes of the program are:

  • Recipients form active partnerships with communities, other NPOs and support organizations to deliver targeted training across Canada
  • Canadians who are underrepresented in the digital economy have access to digital literacy training programs through the Digital Literacy Exchange

The medium term outcomes of the program are:

  • Canadians who are underrepresented have an improved understanding of the digital environment and are able to make informed decisions
  • Canadians who are underrepresented in the digital economy have increased confidence and improved skills in using the Internet

The long-term outcomes of the program are:

  • Canadians have the skills to participate in the digital economy
  • Canada is a connected and digitally engaged society

At the bottom, an arrow stretches from the far right to the far left showing how these activities and outputs lead to the short, medium, and long-term outcomes.

Appendix C: End Notes