Evaluation of the Access to Documentary Heritage Program, 2011-2012 to 2015-2016

Evaluation Function
Corporate Planning and Accountability

December 2017

Table of contents

download pdf 

 

Print version of Evaluation of the Access to Documentary Heritage Program (PDF, 1.67 MB )

Catalogue No.: SB4-49/2017E-PDF

ISBN 978-0-660-20427-7 – Evaluation of the Access to Documentary Heritage Program, 2011-2012 to 2015-2016 (Library and Archives Canada) 

Summary

Introduction

This report presents the findings, conclusions and recommendations stemming from the evaluation of Library and Archives Canada (LAC)’s Access to Documentary Heritage Program. The evaluation was conducted by the Program Evaluation Division of the Corporate Planning and Accountability Directorate in accordance with the directives of the federal government’s Policy on Evaluation. Its main objective was to examine the program’s relevance and performance. The evaluation covered a five-year period, from 2011–12 to 2015–16.

Program profile

The purpose of the Access to Documentary Heritage Program is to promote awareness of Canadian documentary resources and to make them readily available to Canadians and to anyone with an interest in Canada, its society or its history. The program had a budget of $25.7 million and 311 full-time equivalents (FTEs) in 2015-16. The program consists of two main areas of activity. The first entails the description and contextualization of our documentary heritage. This process includes activities through which our continuing memory is digitized, described, organized, structured, inventoried and interconnected to facilitate access and meet client needs and expectations. The second consists of reference, information, consultation, reprography and research services for a diverse range of clients.

Methodology

To complete this evaluation, a review of administrative and financial documents, performance statistics and other internal program documents was completed. Interviews were conducted with managers and employees involved in the management and delivery of the program. Three case studies were carried out to illustrate certain aspects of the program and to answer specific evaluation questions. Finally, the results of an internal review endnote1 and an external survey endnote2 were also added to the data analysis. This methodology is consistent with the mandate with regard to the evaluation of the Access to Documentary Heritage Program, as approved by LAC’s Departmental Program Evaluation Committee (DPEC).endnote3

Evaluation results

Relevance

The information from the documentary review and interviews indicated that the program is still relevant and remains a core pillar of LAC’s mandate. Moreover, the program plays an active part in LAC’s and the Government of Canada’s priorities. Managers recognize the need to adapt to the evolving needs of clients.

Performance

The evaluation indicated that a number of Access Program activities are working well. However, an effort should continue to be made to improve access to the LAC collection.

While a number of indicators are useful, others need to be revised to ensure that they adequately measure expected results and outputs and that they meet the need for performance-related information. Because of a lack of ongoing data, there has not been a solid analysis of the performance of certain program activities or their progress towards the attainment of expected results.

The results show that, over the past five years, reference services and services provided under the Access to Information Act have worked very well and have produced convincing results. Moreover, through block review, LAC has demonstrated progress with regard to the availability of government records. Partnerships with Canadiana and Ancestry have also facilitated access to our documentary heritage through the digitization and indexing of LAC documents.

While some activities have made good progress towards the attainment of medium-term results, others are moving more slowly. For example, second-level description, which makes items in the collection easier to find, has been neglected due to a lack of resources and conflicting priorities within the institution. Because such activities have not been prioritized, second-level descriptions have not always been created for archival holdings and rarely for government archives. Digital reprography of LAC documents has been implemented and has improved service, but after such documents have been digitized they are rarely available online. In addition, finding aids that facilitate the discovery of documents are not standardized and are mostly in paper format, and few are available online. Therefore, they are not accessible to clients who live outside the national capital. Finally, improving navigation and search tools on the website would facilitate access to LAC collections by enabling clients to find what they are looking for on their own.

Resource allocation

LAC’s financial resources declined significantly between 2011–12 and 2015–16 (from $112 million to $91 million), largely because of the federal government’s Deficit Reduction Action Plan. The Access Program’s financial resources also declined significantly from 2011–12 to 2015–16, from $36.8 million in 2011–12 to $25.7 million in 2015–16: a decrease of $11 million. At the same time, the Access Program’s human resources, which consisted of 376 FTEs in 2011–12, fell to 311 in 2015–16, a decrease of 65 FTEs.

Conclusion

Despite a significant drop in resources during the period under review, a number of good practices were successfully implemented to improve client service, such as the front-line client approach implemented by Reference Services, services offered under the Access to Information Act, block review of government records, and partnerships intended to facilitate digitization and indexing of the collection. However, coordination of priorities, better descriptions, online access to search tools and better navigation and search tools on the website would make it easier for the public to use LAC’s available documentary heritage and foster public engagement. An effort should also be made to digitize finding aids in order to make items in the LAC collection easier to find online. Finally,  some indicators need to be revised to ensure they adequately measure outputs and expected results and meet ongoing needs for performance-related information.

Recommendations

The following four recommendations are being made in response to the findings from the evaluation of the Access to Documentary Heritage Program:

Recommendation 1: Better coordination of activities and prioritization of tasks among branches is needed to clarify the governance of the Access Program and the role of staff involved, regardless of the shape the program takes in the future.

Recommendation 2: Program managers should undertake a review of output and outcome indicators to ensure that they are collected on an ongoing basis, that the indicators identified are useful in decision making, and that data collection is possible and practical so the program’s progress and outcomes can be measured.

Recommendation 3: Efforts should be made to complete the digitization of finding aids.

Recommendation 4: To facilitate access to the collection on its website, LAC should improve the search tools found there as well as navigation. 

1. Introduction

The report presents the results, findings and recommendations for the evaluation of Library and Archives Canada (LAC)’s Access to Documentary Heritage Program. This evaluation was carried out between January and December 2016 and meets the requirements of the Treasury Board Secretariat (TBS)’s 2009 Policy on Evaluation. endnote4

1.1 Objectives of the evaluation

The main objective of this evaluation was to review the relevance and the performance of the Access to Documentary Heritage Program. The evaluation was also intended to assess the following:

  • how easy it is to navigate through LAC’s reference services;
  • the effectiveness of services provided to clients under the Access to Information Act; and
  • findability of the LAC collection.

2. Profiles of Library and Archives Canada and of the Access Program

2.1 Brief description of Library and Archives Canada (LAC)

Library and Archives Canada is a federal institution tasked with acquiring and preserving Canada’s documentary heritage and making it accessible. The Dominion Archives, founded in 1872 as a division of the Department of Agriculture, was transformed into the stand-alone Public Archives of Canada in 1912. In 1987, the organization was renamed the National Archives of Canada. LAC was created in 2004 when the functions of the National Archives of Canada were combined with those of the National Library of Canada (founded in 1953). The Library and Archives of Canada Act endnote5 (“the Act”) came into force in 2004. It sets out the mandate of the institution responsible for:

  • preserving the documentary heritage of Canada for the benefit of present and future generations;
  • being a source of enduring knowledge accessible to all, contributing to the cultural, social and economic advancement of Canada as a free and democratic society;
  • facilitating in Canada cooperation among the communities involved in the acquisition, preservation and diffusion of knowledge; and
  • serving as the continuing memory of the government of Canada and its institutions.

2.2 Access to Documentary Heritage Program

The purpose of the Access to Documentary Heritage Program is to promote Canadian documentary resources and to make them readily available to Canadians. The program consists of two main areas of activity, as shown in the logic model presented in Appendix E. The first relates to organization of the collection, which includes description and contextualization of the documentary heritage. This process includes activities through which our continuing memory is described, organized, structured, inventoried and interconnected in order to facilitate access and meet clients’ needs and expectations.

The second area of activity consists of reference, information, consultation, reprography and research services for a diverse range of clients. The institution’s services can be obtained through multiple channels, including in person, by telephone, by mail or email and via the Internet.

The expected results for the program are as follows:

Immediate outcome:
Improved access to Canada’s documentary heritage
Intermediate outcome:
Improved use and engagement  with Canada’s documentary heritage among the general public
Ultimate outcome:
Canada’s continuing memory is documented and accessible for current and future generations

As a memory institution, LAC must apply the standards and practices in effect in archival and library science environments in the context of the applicable federal legislation, policies and regulations, such as the following:

In addition, the following LAC policies underpin its work:

LAC's responsibilities with regard to access to our documentary heritage entail facilitating the identification and availability of and access to documentary resources in analog or digital format. Those resources include published and unpublished documents, textual records, maps, photos, audio and audio-visual documents, artifacts and stamps.

In Canada and around the world, governments endeavour to provide open access to their collections, including through the use of technology and open data projects. LAC wants to take advantage of the opportunities that are arising; in a rapidly evolving digital and networked environment. To that end, the institution must strategically select activities that will facilitate access to its analog and digital collections for its clients across Canada, including government agencies, private donors, universities, researchers, historians, students, librarians, archivists, genealogists and the general public.

To fulfill its responsibilities, LAC uses advanced technologies and also provides information on its collections through its website and social media. LAC provides access to its documentary resources by:

  • making documentary resources available to the public in analog and digital format;
  • providing on-site services at 395 Wellington Street in Ottawa and in Winnipeg, Halifax and Vancouver; and
  • contributing to exhibitions that give the public an opportunity to discover LAC’s collection at museums and cultural sites across Canada.

2.3 LAC program priorities for 2011–12 to 2015–16

LAC identified a number of priority activities in its Reports on Plans and Priorities (RPPs) for the Access to Documentary Heritage Program from 2011–12 to 2015–16. As can be seen in Appendix F, those priorities are wide ranging and are intended to improve the Access Program. As part of this evaluation, the various priorities were examined to determine whether these commitments were fulfilled through the program. The section of this report that discusses performance-related findings connects these priorities to the results achieved through the program.

2.4 Program governance

The Access Program is under the responsibility of the Public Services Branch endnote6 , which falls within the purview of the Chief Operating Officer.

Under the Branch, there are three directorates whose mandates entail description, content distribution and reference services. A number of internal and external committees and working groups have also been established to foster information sharing and discussions on various issues.

The Public Services Branch is responsible for services delivered directly to the public, such as reference services, access to information and online content available through the website. The Branch is also responsible for regional service points, the Copyright Bureau and exhibitions.

During the evaluation period, description-related activities were allocated to a number of different branches, along with activities related to the LAC website.

The Public Services Branch was also responsible for online content made available through podcasts, Flickr and LAC blogs. The Communications Branch took care of social media (Facebook, Twitter, Pinterest, YouTube), while the Information Technology Office managed databases and dynamic pages on the website. Those activities are carried out in close collaboration with the Private Archives, Government Records, Published Heritage and Preservation Branches.

2.5 Partnerships

The program has established partnerships with entities such as Canadiana and Ancestry in recent years to facilitate the digitization and indexing of a portion of LAC’s collection. This evaluation does not provide an in-depth analysis of the agreements that bind LAC and these two partners. However, a number of observations are made in section 6.1, which deals with LAC partnerships, regarding these two partners given that a question in the evaluation interview focused on LAC partners.

3. Methodology

3.1 Evaluation period

The evaluation of the Access to Documentary Heritage Program covered the five-year period from April 2011 to March 2016.

3.2 Evaluation questions

The evaluation questions pertained to the program’s relevance and performance, including its effectiveness and efficiency. Specifically, the following questions were asked:

  • Is the program still relevant, and does it continue to meet the evolving needs of clients?
  • Are the program’s priorities aligned with those of LAC and the Government of Canada?
  • Are the roles and responsibilities of the Access Program clearly defined and understood?
  • Has the performance measurement strategy been implemented? and
  • Is the program making progress toward achieving its expected results?

Further details regarding the evaluation questions can be found in Appendix H.

3.3 Evaluation methods

A review of administrative and financial documents, performance-related statistics and other internal documents relating to the program was carried out. Interviews were conducted with managers and employees involved in the management and delivery of the program. A total of 29 interviews were conducted with employees of Public Services and other LAC branches. Three case studies were carried out so that specific aspects of the Access Program could be evaluated. The case studies (Appendices B, C and D) cover the following topics:

  • Case study #1: Access to the Public Accounts of Canada (concepts of discoverability, availability and access and how they apply to access to published heritage);
  • Case study #2: Access to files on the ground-breaking ceremony for the St. Lawrence Seaway held on June 25, 1959 (access to government records); and
  • Case study #3: Access to the Burton Cummings fonds (access to private archives).

The evaluators also considered the findings from the review of access methods endnote7 conducted internally. The review highlights the 2014–15 financial data that is used to demonstrate the efficiency of the Access Program. A public opinion survey conducted by Nanos, endnote8 which was published in December 2015, endnote9 was also used.

The use of different survey methods and triangulation of the data (See Evaluation Questions and Methods below) helped corroborate the findings. This methodology is consistent with the mandate for the evaluation of the Access to Documentary Heritage Program, which was approved by LAC’s Departmental Program Evaluation Committee (DPEC) on March 2, 2016. endnote10

Evaluation Questions and Methods:

Relevance: Evaluation questions concerning relevance will be covered by:

  • Documentary Review
  • Interviews

Outcomes: Evaluation questions concerning measurement of results will be covered by:

  • Documentary Review
  • Interviews
  • Case study
  • Nanos survey

Efficiency: Evaluation questions concerning efficiency will be covered by:

  • Documentary Review
  • Interviews
  • Review of Access Methods

3.4 Limitations of the evaluation

  1. Performance-related data needed to evaluate program results was limited, for both the performance measurement strategy and performance measurement itself. The evaluation team therefore used other data sources such as interviews, administrative documents and past audit results to mitigate this limitation and to better support its analysis.
  2. Because detailed financial data by activity was not available, a cost-benefit analysis was not conducted as part of this evaluation. To mitigate this limitation, the evaluation team used the results of the 2016 internal review of access methods, one of the aims of which was to determine the costs associated with the different methods of accessing the LAC collection.
  3. The evaluation did not cover the following activities (although we did make a few references to social media since some of the participants raised this topic during the interviews):
  • Activities intended to promote visibility (Facebook, Twitter, Flickr, YouTube, blogs, podcasts);
  • TD Reading Club;
  • Portrait Gallery; and
  • Documentary Heritage Communities Program.

3.5 Coding of findings

The evaluation findings were categorized by colour to highlight improvements requiring special attention:

  •  Green – no improvement needed
  •  Yellow – some improvements would be needed
  •  Red – improvements needed / recommendations

4. Findings – Relevance

4.1 Program relevance

Finding 1

The Access to Documentary Heritage Program is still relevant and remains a core pillar of LAC’s mandate.

The legal basis for the program is clearly set out in Parts 7 and 8 of the Library and Archives of Canada Act (2004). In addition, information from the documentary review and the interviews shows that the program is still relevant and remains a core pillar of LAC’s mandate. Access Program activities are part of the institution’s six priorities as identified in the 2011–12 to 2015–16 RPPs. The program’s relevance is also demonstrated in LAC’s business plans and annual reports.

Upon his arrival in 2014, the Librarian and Archivist of Canada identified four commitments that subsequently became LAC’s priorities. Of those commitments, three relate to the Access to Documentary Heritage Program:

  • LAC is an institution dedicated to serving its clients, all its clients: government institutions, donors, universities, researchers, archivists, librarians, students, genealogists and the general public.
  • LAC is at the leading edge of new technologies and delivers quality services to Canadians, disseminating a maximum amount of content using digital technologies. 
  • LAC is an institution with greater public visibility, highlighting the value on its collections and services.

The Access Program is continually improving its services in order to satisfy its clients and meet their ever-changing needs. Therefore, an effort is being made to ensure that Canadians outside the National Capital Region have better access to services. To that end, the program is continuing to increase digital content and works collaboratively with partners (including Canadiana and Ancestry) to draw from the strengths and capacities of each.

In its 2013–16 business plan, LAC undertakes to meet the current and future needs of Canadians by:

  • implementing a digital-by-default approach to consulting suitable content in the LAC collection;
  • systematically digitizing documents that are requested frequently;
  • implementing enhanced reference services to better serve Canadians; and
  • adopting a more collaborative approach to better meet the evolving needs of Canadians.

Interview participants were given the opportunity to provide their opinions (see graph below) on program activities that promote access to the collection. According to the results that were collected, 55% of respondents believed that digitization facilitates access to the LAC collection. Another 41% felt that the website and online content promote access, along with block review (23%). In addition, according to 23% of respondents, providing good service to clients helps them access the LAC collection. Finally, partnerships (14%) and social media (14%) promote access to LAC’s collection. According to 32% of respondents, enabling clients to find what they need on their own also improves access to the collection.

However, for 27% of respondents, the program is not doing enough to facilitate access to its collection. Respondents suggested ways to help the program better meet the evolving needs of its clients. Among other things, they said that the institution should invest more in digitization rather than simply rely on partnerships in this area; develop better research tools; and improve descriptions.

Graph 1: Access to LAC Collection endnote11

Graph 1: Access to LAC Collection

Source: Interview participants, June to September 2016.

  • Graph 1: text version

    The graph above describes the participant’s opinions on program activities that promote access to LAC’s collection. Specifically:

    • 55% of respondents believed that digitization facilitates access to LAC’s collection;
    • another 41% felt that the website and online content promoted access;
    • according to 32% of respondents, enabling clients to find what they need on their own also improves access to the collection;
    • 23% of respondents believe that providing good service to clients along with block review,  promotes access to LAC’s collection;
    • 14% of respondents mentioned that the social media and partnerships, respectively, promote access to LAC’s collection; and,
    • for 27% of respondents, the program is not doing enough to facilitate access to its collection.

The information needs of Canadians are evolving, and the program is endeavouring to meet those needs. The institution recognizes that some aspects of its activities need to change in order to adapt to its clients’ needs. The program aims to ensure that Canadians have more timely and convenient access to our documentary heritage. To that end, the program is working to promote access to its collection through a dynamic, interactive and user-friendly website. The program is also endeavouring to increase awareness of its collection among Canadians through the development of digital networks.

4.2 Alignment with LAC and Government of Canada priorities

Finding 2

The Access Program actively participates in the Government of Canada’s priorities, in particular the federal Open Government and Diversity is Canada’s Strength initiatives.

As shown in Section 4.1, the program is clearly aligned with LAC’s priorities. It also contributes to the Government of Canada’s priorities. In its December 2015 Speech from the Throne, endnote12 the federal government set out five priorities. The Access Program specifically contributes to two of those priorities: Open and Transparent Government and Diversity is Canada’s Strength.

The Access Program is an active contributor to the federal Open Government initiative. It also provides access to its documents by responding to thousands of information requests annually (see table below).

Table 1: Number of Requests

This table shows the number of information requests received by LAC between 2011-2012 and 2015-2016.
Number of requests Years
143,716 2011–2012
120,081 2012–2013
100,167 2013–2014
95,874 2014–2015
96,619 2015–2016

Source: Performance Measurement Strategy, Access to Documentary Heritage Program, 2011–12 to 2015–16.

The program has also undertaken block reviews of access conditions to improve access, and each year many government records become available for consultation by Canadians. 

The program supports and promotes the diversity and importance of Canada’s indigenous communities. It was called upon to play a large role in supporting the Truth and Reconciliation Commission on residential schools by providing archival documents that were used in the Commission’s investigation.

 

4.3 Roles and responsibilities of various branches/divisions involved

Finding 3

Some access-related activities are supported by resources from other branches. The Public Services Branch actually controls only 54% of the salary resources allocated to delivery of the Access Program, thus creating governance challenges.

Roles and responsibilities are clearly understood by employees and managers within each branch. However, some access-related activities are under the administrative responsibility of different branches, resulting in multiple requests assigned to staff. This situation creates extra work and makes it difficult to respond to all requests. All of the branches have their own priorities, but they are not always shared with the other branches, resulting in a lack of coordination of tasks at the operational level.

Some access-related activities (see chart below) are shared with other LAC branches. The corresponding percentages have shown a fairly consistent average since 2012–13. It should be noted that the ADM’s Office referred to in the Chart is the Office of the Chief Operating Officer (COO), while the Central Fonds (TRC) refers to the Truth and Reconciliation Commission of Canada.

Chart 1: Distribution of Salary Expenditures Among Branches for the Access Program in 2015–16

Chart 1: Distribution of Salary Expenditures Among  Branches for the Access Program in 2015–16

Source: Library and Archives Canada, Finance Branch.

  • Chart 1: text version

    This chart shows the distribution of salary expenditures among LAC’s branches for the Access Program in 2015–16. Among them, the Services Program spent 54% of its salary expenditures; Preservation spent 12 % ; Information Technology 10 % ; Acquisitions 8 % ; the ADM’s Office and the Central Fonds (TRC) 6 % respectively; and other branches 4 %.

The various branches do in fact work together through committees that give Directors General and Directors an opportunity to discuss and share information about various aspects of their work. However, while coordinating committees do exist, they have a limited impact on how the tasks to be performed are organized.

During the interviews (see graph below), 56% of respondents endnote13 indicated that collaboration needed to be improved, while 12% felt that collaboration was good/excellent. For 16% of respondents, collaboration was good in their own work unit, while another 12% felt that collaboration was good only at the senior management level (executive or higher).

Graph 2: Internal Collaboration among Branches as Perceived by Respondents

Graph 2: Internal Collaboration among Branches as Perceived by Respondents

Source: Interview participants, June to September 2016.

  • Graph 2: text version

    This graph shows how participants perceived collaboration among LAC’s branches. Specifically:

    • 56% of respondents indicated that the collaboration needed to be improved;
    • 16% of respondents indicated that the collaboration was good in their own work unit;
    • 12% of respondents felt that collaboration was good/excellent;
    • while another 12% of respondents felt that collaboration was good only at the senior management level (executive or higher).

Recommendation 1: Better coordination of activities and prioritization of tasks among branches is needed to clarify the governance of the Access Program and the role of the staff involved, regardless of the shape the program takes in the future.

5. Findings - Performance

5.1 Performance measurement strategy

Finding 4

Although the program collects different data, for outputs in particular, a lack of ongoing data limits performance analysis for some program activities and their progress toward expected outcomes.

To meet its needs for performance-related information, the Access Program has developed a number of indicators, which are identified in the performance measurement strategy endnote14 and in its performance measurement framework.endnote15 Although data for some of the output indicators for the strategy was in fact collected (e.g., the number of requests to Reference Services through different channels: in person, by telephone, letter/fax or online), other data had not been collected for the past five years. During the five-year period in question, some indicators were in fact not collected or were dropped. The same observation applies with regard to the indicators associated with the performance measurement framework, for which data is to be reported annually, but they were not. As a result, the review of the available data did not permit sufficient analysis to develop solid performance-related findings for certain program activities, as outlined in section 5.2.

During the interviews, endnote16 participants noted that the systems used for data collection do not allow for effective and reliable collection. Those systems also limit the program’s ability to collect data on outputs and outcomes in support of decision making. Some respondents also indicated that they would like to have an opportunity to express themselves about the indicators, but the Public Service Performance Committee has no longer been in place as of 2012.

Some of those interviewed also reported that in some cases, the indicators identified through the program are difficult to understand and interpret. While 42% of respondents indicated that the data was reliable, consistent and useful (see chart below), the same percentage raised doubts about the reliability, consistency and usefulness of the data collected. Finally, 16% of respondents reported that the data was not reliable, consistent or useful.

Chart 2: Respondents’ Perception of Reliability, Consistency and Usefulness of Data

Chart 2: Respondents’ Perception of Reliability, Consistency and Usefulness of Data

Source: Interview participants, June to September 2016.

  • Chart 2: text version

    This chart shows the respondent’s perception of the reliability, consistency and usefulness of the data collected by the program. Specifically:

    • 42% of respondents indicated that the data was reliable, consistent and useful;
    • 16% of respondents reported that the data was not reliable, consistent or useful;
    • 42 % of respondents raised doubts about the reliability, consistency and usefulness of the data collected by the program.

The graph below also shows that 63% of respondents believed that the indicators should be revised, while 37% indicated that outcome indicators should be added. Another 42% of respondents indicated that qualitative indicators should be added. Finally, 26% of respondents felt that the indicators were good, while another 26% considered the systems endnote17 used to collect the data were inefficient.

Graph 3: Respondents’ Perceptions of Indicators’ quality

Graph 3: Respondents’ Perceptions of Indicators’ quality

Source: Interview participants, June to September 2016.

  • Graph 3: text version

    The graph above shows respondents’ perception about the quality of indicators. Specifically:

    • 63% of respondents believed that the indicators should be revised;
    • 42% and 37% of respondents indicated that qualitative and outcomes indicators should be added;
    • 26% of respondents felt that the indicators were good;
    • while another 26% considered the systems used to collect the data were inefficient.

During the interviews it was reported that reports for senior management are produced regularly for information and decision-making purposes. As shown in chart below, 53% of respondents indicated that some data is discussed and used, 35% did not know if the data is used, and another 12% of respondents had never seen a report.

Chart 3: Respondents’ Perceptions of Data Used by Senior Management

Chart 3: Respondents’ Perceptions of Data Used by Senior  Management

Source: Interview participants, June to September 2016.

  • Chart 3: text version

    This chart shows respondents’ perception about data used by senior management. Specifically:

    • 53% of respondents indicated that they believe some data is discussed and used by senior management;
    • 35% of respondents mentioned that data was reported to senior management but did not know if the data was discussed or used by them; and,
    • 12% of respondents mentioned that they never seen a report for senior management.

A review of quarterly performance reports submitted to senior management was completed. Although the reports differed between 2012–13 and 2015–16, they all shared information on the number of new descriptions (government, private collections and published archives). In 2014–15 and 2015–16, more data on the Access to Information Act was added to the quarterly performance reports. Over the years, reports have become more complete and they provide senior management with more information on the outputs of the Access Program.

The information from the review of access methods conducted in April 2016 endnote18 supports that observation. The review reported that data collection carried out to measure efficiency was difficult between 2012–13 and 2014–15, giving rise to a recommendation to improve the collection of this type of data. In addition to collecting output data for efficiency, the Access Program evaluation also reported that the program had difficulty collecting the outcomes-related data needed to measure the program’s efficiency and economy.

Recommendation 2: Program managers should undertake a review of output and outcome indicators to ensure that they are collected on an ongoing basis, that the indicators identified are useful for decision making, and that data collection is possible and practical so that the program’s progress and outcomes can be measured.

5.2 Short- and medium-term outcomes

Finding 5

The evaluation found that, while some results had been achieved, it was difficult to assess all expected outcomes because of a lack of data.

Findings on short- and medium-term outcomes through the program evaluation are as the following:

Activities that reached their outcomes:

  • Indexing
  • Block review
  • Reference Services
  • Access to information service

Activities that made progress toward reaching their outcomes:

  • Description
  • Digitization
  • Reprography

Activities that did not reach their outcomes and need improvement:

  • Finding Aids
  • LAC website

While LAC is making every effort to improve access to its collection, in practice a certain portion of LAC’s collection is not accessible, since LAC is obliged to comply with policies and abide by administrative or legal restrictions, with regard to copyrighted materials in particular. endnote19 Therefore, an important distinction needs to be made between a document being available endnote20and having access endnote21 to a document. Case study #1 (Appendix B) demonstrates the importance of differentiating between the concepts of discoverability, availability and accessibility. Those three concepts will also be addressed in the different sections of the analysis below and in the three case studies (Appendices B, C and D).

In the interviews with Access Program managers and staff, a number of examples were provided with regard to ways that service delivery could be improved. For example, digitization saves time and money, since digitized reprographies can be sent to customers easily and quickly by electronic means, which are faster and cheaper than regular mail. Moreover, when the most in-demand documents are digitized, they can be consulted directly online without any intervention by the institution’s staff. The following sub-sections give some examples to illustrate the progress that has been made in achieving results.

5.2.1 Description and contextualization of documentary heritage

The process of describing and contextualizing documentary heritage is the first element analyzed in this evaluation. This process includes activities through which documentary heritage is described, organized, structured, inventoried, digitized and interconnected to facilitate access and meet client needs and expectations. The main sub-activities include description itself, finding aids, indexing, block review and digitization.

5.2.1.1 Description

Finding 6

The descriptions that are currently being done facilitate internal management by making it possible to find document. This minimal level of description is not intended to make it easier for LAC clients to find items in the collection.

Descriptions consist of a set of fields (e.g., main entry, title, physical scope, notes) created and organized in accordance with the standards endnote22 to describe the acquisition of documentary resources. A first level of description created at the time of acquisition briefly describes an item so that it can be discoverable in LAC’s systems. This first level of description focuses on management of the collection and not on making it easier for clients to access the documentary heritage. 

This first level of description therefore enables an item in the LAC collection to be discoverable. Preparation of this first level of description (descriptive metadata) is the preferred tool for ensuring discoverability.

Discovery-related activities are therefore the first steps to be taken so that anyone can identify a document, photo or other item in the LAC collection. The items in the LAC collection become discoverable when a first level of description is created in LAC’s systems (such as Amicus or MIKAN) at the time of acquisition.endnote23 

Interview participants confirmed that a first level of description is provided and that, due to a lack of resources and conflicting priorities, second-level descriptions are not always created for private archival holdings and are rarely created for government archives. 

Accordingly, and in order to improve its descriptions, LAC indicated in 2011 endnote24 that it would have a single descriptive structure for publications, private archives and government archives. In relation to that commitment, the institution reported in 2012–13 that it had developed a new approach to describing its content that was more adapted to user needs. LAC also reported that tens of thousands endnote25 of links between bibliographic and archival notes (images and PDFs) had been created in order to enhance the discoverability of the items in its collection. According to the data collected through the program, nearly 34,000 descriptions endnote26 of published documents were created in 2014–15 and close to 150,000 descriptions of archival items were created. endnote27

Case study #3 on access to the Burton Cummings fonds (Appendix D) demonstrated that it is possible to find material in private archives that are part of the LAC collection. This case study also shows the importance of having a detailed and complete (second level) description to enhance the discoverability of items in the collection. Without a complete and detailed description, it is difficult for clients to find precisely what they are looking for, as there may be a large number of search results.

Over 66% of those interviewed provided comments relating to description. Of that number,

  • 41% indicated that description is a key function of access to the collection;
  • 34% felt that descriptions need to be improved.

Respondents also identified a number of current challenges:

  • Backlogs of published documents and archives (24% of respondents);
  • Lack of links between digital copies and descriptions (10% of respondents)

In addition, some of the interviewees confirmed that there were backlogs in second level description for private archives and in cataloguing for published documents.

According to the Metadata Framework for Resource Discovery, endnote28 second-level descriptions would provide more information and context while making items in the LAC collection easier to find. Second-level descriptions would allow clients to find what they are looking for in the collection and help them be better equipped to find what they need on their own, as per the institution’s commitments. endnote29 

5.2.1.2 Finding aids

Finding 7

Finding aids make documents easier to locate. However, the format is not standardized. In addition, finding aids are mostly paper based (analog), so few are accessible online.

Finding aids are created to facilitate searching for documents and items within a particular archival group or on a specific topic. When a file contains a large number of items, a finding aid may have been created to help clients find what they are looking for. Finding aids consist of a list of items or links contained in a particular archival group and are therefore essential in locating material in large archival holdings. However, LAC does not have a standard template for its finding aids. Therefore, the format, content and level of detail differ from one finding aid to the next.

In a public opinion survey conducted in December 2015, endnote30 respondents ranked digitization of finding aids second in importance, thereby pointing to an interest in having access to finding aids.

However, the finding aids found for the case studies were in paper format and could be accessed only at Reference Services in Ottawa. This is a significant barrier to access for clients who do not reside in the National Capital Region.

Efforts to digitize finding aids have been made over the years. For example, more than 120 finding aids for photos have been converted to online search tools, allowing access to some 3,000 photos. In its 2012–2013 and 2014–2015 RPPs, the institution reported its intention to modernize its services by digitizing its finding tools in order to provide Canadians with expanded access and facilitate identification of documents or items in LAC’s documentary resources. No information was found in the corresponding DPRs regarding this commitment. Digitization of finding aids in paper format is under way, but according to respondents it is progressing as resources are available.

In 2015–16 the institution also reported that, endnote31 in order to facilitate identification of documents or items in its collection, it would create new finding aids and tools and update existing ones. Interviews with Access Program management and staff did not confirm whether that commitment had been achieved. None of the documents analyzed provided conclusive information on this subject.

For case study #2, which deals with the ground-breaking ceremony for the St. Lawrence Seaway (Appendix C), paper-based finding aids were consulted at Reference Services, located at 395 Wellington Street in Ottawa. The case study revealed that those aids are not always easy to understand and that some relevant information may be missing. In fact, evaluators determined that the finding aids that were used did not indicate the access conditions for the documents in question. Since that information did not appear in the finding aid or in MIKAN, endnote32 an access to information request had to be made in order to validate the access conditions. If the latter had been indicated in the finding tool, an access to information request would not have been necessary since none of the requested documents was restricted. A follow-up was done in January 2017 to determine whether the information had been updated on the LAC website, which was in fact the case. If another client wants to see these documents, it will be possible to do so without using the Access to Information Act.

In his fall 2014 report, the Auditor General of Canada endnote33 also identified a number of deficiencies with regard to the quality of finding aids relating to the residential school system. Some finding aids were incomplete, not comprehensive or contained inaccuracies.

Finding aids that are accessible online would facilitate the identification of documents or items in the LAC collection for all of its clients.

Recommendation 3: Efforts should be made to complete the digitization of finding aids.

5.2.1.3 Indexing

Finding 8

Indexing of archives helps facilitate access to LAC’s documentary heritage. This activity is also carried out in collaboration with other partners.

In addition to description and digitization, another important aspect of facilitating access to the collection is indexing. An index is a bank of key words used to find information within a document. For example, indexing censuses and electoral lists enables clients to find the name of a specific person more easily than if they had to go through thousands of pages. LAC does not currently have the necessary resources (human or financial) to index all or part of its collection. Respondents acknowledged the importance of partnerships and initiatives that can help meet client needs. Partnerships with Canadiana and Ancestry play a positive and key role in this regard. Other initiatives involving collaboration with the public, such as the Coltman report initiative endnote34 (in which the public is involved in transcribing handwritten documents), also contribute to indexing of the collection and improved access.

5.2.1.4 Block review

Finding 9

The block review approach used to eliminate restrictions on access to government records is in line with Canada’s Action Plan on Open Government.

Since 2011–2012, LAC has undertaken to remove restrictions, where possible, endnote35 in order to facilitate access to government records through the practice known as block review. This practice involves evaluating documents on the basis of sensitivity, age and subject matter with a view to increasing the availability of archived government records. The institution made 18 million pages of government records endnote36 available for consultation from 2011–12 to 2015–16 without a need for further review under the Access to Information Act and the Privacy Act.endnote37 Block review made it possible to process 4,168,133 pages in 2014–2015 and 6,437,499 pages in 2015–2016. Interview respondents confirmed that the institution had undertaken this work and it was perceived positively.

5.2.1.5 Digitization

Finding 10

Along with the digitization of Canadian Expeditionary Force documents from World War I, digitized documents are being posted online on a priority basis and in accordance with available resources.

Digitization is the process of converting an analog item into a digital one. This technique can be used to preserve documents, whatever the original medium (document, photo, portrait, etc.), in electronic format. The items in the LAC collection are digitized through the Preservation Program (PAA 2.2). The official documents consulted demonstrated that some digitization activities are being carried out under the responsibility of the Access Program. LAC reports that since 2013–14 it has been developing and implementing a content digitization strategy that reflects the topics of interest to its clients. endnote38 A multi-year plan to digitize the most popular collections, including those dealing with military heritage and indigenous issues, has been developed. The institution has undertaken to digitize all of the documents in its possession that relate to the Canadian Expeditionary Force along with some 80,000 portraits from various collections of photographs and heritage art.

In order to increase access to its collection, LAC, in collaboration with its partners Canadiana and Ancestry, has also successfully completed the digitization of a large number of documents, photos, films and documentaries endnote39 pertaining to genealogy, government records, and military and indigenous documentary heritage.

However, due to the use of different measurement units, it is difficult to evaluate how digitization of the collection is progressing. LAC sometimes reports the number of images and sometimes the number of pages. The fact that a page can have more than one image makes comparison difficult. As a result, LAC reported in 2011–2012 that more than 4.5 million images, including electoral lists, microfilm and frequently requested portraits, had been added. In 2013–14, over 17 million pages of the LAC collection were digitized by the institution and its partners Canadiana and Ancestry, as compared with 2 million the previous year. The number of images for 2015–16 was 12 million. This increase is mainly due to the implementation of the microfilm digitization initiative, in partnership with Canadiana. Canadians now have online access, through either the LAC or the Canadiana website, to a larger number of documents through this project. However, considering the vast amount of material in the institution's collection, the portion that can be accessed online is still low.

It should be noted that respondents in the December 2015 Nanos survey ranked digitization as the institution’s most important priority over the next three years. Digitization responds to a need that clients have and therefore remains a relevant goal for both the institution and its clients.

5.2.2 Services

The second main area of activity under the Access Program logic model is services. LAC provides reference, information, consultation, reprography and research services to a diverse range of clients. The institution’s services can be obtained through multiple channels, including in person, by telephone, by mail or email and via the Internet. LAC also contributes to Canada’s continuing memory through partnerships with Canadiana and Ancestry and supports them in their programming and interpretation efforts. The main outputs are reference services, access to information, reprography and services via the website.

According to a public opinion survey, endnote40 the main reasons for a visit to LAC (not just Reference Services) are the following, in order of importance: genealogy (51%); research (44%); work-related reasons (38%); personal interest (28%); education (22%); miscellaneous (6%); and other (2%).

5.2.2.1 Reference Services

Finding 11

Reference Services offers effective front-line service and facilitates access to documentary heritage for Canadians.

Reference Services provides front-line access services and a range of guidance and consultation services on the web and elsewhere. Other services are available by appointment to provide support for research in library and archive collections, and specialized services are also available in support of genealogical research. Reference Services professionals manage requests for access to the collection that are filed under the Access to Information Act and the Privacy Act, including access to records of civilian personnel and other federal records under the custody of LAC.

According to the table below, the number of contacts at Reference Services has decreased since 2011–12 but stabilized starting in 2014–15. The review endnote41 report supports this data, with more in-person contact followed by email or Internet requests. It can in fact be concluded that client interaction with LAC through Reference Services has been fairly stable since 2014.

Table 2: Type and Number of Contacts at Reference Services

This table shows the number of contacts at Reference Services between 2011-2012 and 2015-2016 by type of contact: in person, by email or Internet, by telephone or by letter/fax.
Type of contact 2011–2012 2012–2013 2013–2014 2014–2015 2015–2016
In person 45,989 40,584 36,788 33,398 33,862
Email or Internet 55,198 41,928 28,910 31,067 30,558
Telephone 25,901 21,493 20,608 17,991 19,249
Letter/Fax 16,628 16,076 13,861 13,418 12,950
Total 143,716 120,081 100,167 95,874 96,619

Source: Performance Measurement Strategy, Access to Documentary Heritage Program, 2011–12 to 2015–16.

As stated in the internal review report,endnote42 expenditures for Reference Services for 2014–2015 were $2,066,505.94. This means that in 2014–2015 the cost to LAC for each request to Reference Services was $83.78, whatever the type of contact. This is an appropriate reference year to demonstrate efficiency, since the number of contacts between 2014–15 and 2015–16 remained fairly stable.

Of those interviewed, 67% (chart below) indicated that the service offered by Reference Services was good or very good. In contrast, 11% of respondents were more equivocal in their response, stating that service quality varied depending on the type of client and the nature of the request. Finally, 22% did not know whether the service provided by Reference Services was of good quality.

Chart 4: Respondents’ Perceptions of Quality of Service Provided by Reference Services

Chart 4:  Respondents’ Perceptions of Quality of Service Provided by Reference Services

Source: Interview participants, June to September 2016.

  • Chart 4: text version

    The chart above shows respondents’ perception of the quality of services provided by Reference Services. Specifically:

    • 67% of respondents perceived that the services was good or very good;
    • 22% did not know whether the service provided by Reference Services was of good quality; and
    • 11% of respondents stated that the service quality varied depending on the type of client and the nature of the request.

For case study #2, which deals with the ground-breaking ceremony for the St. Lawrence Seaway, the evaluation team used Reference Services and was able to assess the quality of the services provided. Service by email and in person was quick and courteous and gave clients an opportunity to speak with an expert archivist. Reference Services staff also arranged a meeting with a librarian, and it was successful as well. The professionals interviewed had done some research in advance and were able to answer questions in the language of the choice of the client. It can be concluded that Reference Services provides an effective front-line service that facilitates access to documentary heritage for Canadians.

5.2.2.2 Access to Information service endnote43

Finding 12

Service under the Access to Information Act meets service standards, with the exception of 2015–16 due to a larger volume of pages sent and external consultations.

Through the Access to Information Act, Canadian citizens, permanent residents and all individuals and corporations present in Canada have the right to access records under the responsibility of a government institution subject to the Act. Each year LAC receives requests for access to records in its possession that originate from other federal institutions or that are created by LAC. endnote44 LAC deals with formal requests for access to its restricted operational documents under its responsibility. Of the total of requests received, it is estimated that only 5% are considered formal requests, whether they relate to LAC’s operational documents, archived operational documents from other government institutions or restricted personnel files. endnote45 Accordingly, LAC’s access to information services are in fact used as a method of access to documentary heritage. As shown in table below, these requests have been falling gradually since 2013–14. In addition, LAC has undertaken to improve access to information through mechanisms such as block review and more extensive digitization of its collection, two activities discussed earlier in the report.

Table 3: Number of Annual Requests Processed Under the Access to Information Act

This table shows the annual number of requests processed (formal and informal) by the Reference Services between 2011-2012 and 2015-2016 under the Access to Information Act.
Year Number of formal
requests processed
Number of informal
requests processed
Total
2011–2012 821 7,075 7,896
2012–2013 874 5,361 6,235
2013–2014 924 6,922 7,846
2014–2015 821 6,671 7,492
2015–2016 758 5,422 6,180

Source: Annual Report: Access to Information Act, 2011–12 to 2015–16, Library and Archives Canada.

Total expenditures endnote46 on access to information services were $3,411,324.90 for 2014–2015. Each request for service under the Access to Information Act therefore cost an average of $149.17 in 2014–15. That amount could serve as a baseline for an analysis of efficiency at some point in the future.

According to the data collected through the Access Program, over the years the services provided under the Access to Information Act have met and exceeded the service standards, endnote47 with the exception of 2015–16, for which the figure was 88%.endnote48 Program officials have explained this decline endnote49 by the larger number of information pages to be sent and the higher volume of external consultations.

In our interviews, 56% of respondents (see chart below) said that in their opinion the service provided under the Access to Information Act was good or very good.

Chart 5: Respondents’ Perceptions of Quality of Access to Information Services

Chart 5:  Respondents’ Perceptions of Quality of Access to Information Services

Source: Interview participants, June to September 2016.

  • Chart 5: text version

    This chart shows respondents’ perception of quality of Access to Information Services. Specifically:

    • 56% of respondents said that the services provided under the Access to Information Act was very good or good;
    • 38% of respondents did not know whether the service was of good quality; and
    • 6% of respondents were not sure about the quality of the service that was offered.

As part of the case study on the ground-breaking ceremony for the St. Lawrence Seaway (Appendix C), an access to information request was made under the Access to Information Act to obtain documents that had been identified as restricted, thus providing an opportunity to evaluate the service. An electronic form was completed to that end. The 12 requested documents were available for consultation in accordance with the standards (response within 30 days) and the response was highly satisfactory. The requested documents (representing a total of 661 pages) were digitized and recorded on a CD-ROM.

However, the case study showed that a request under the Access to Information Act would not have been needed if the information had been updated in the LAC systems and the finding aid at Reference Services in Ottawa. All 12 documents could have been accessed directly because they were no longer restricted under the Act. Moreover, the use of a CD-ROM may be problematic, and even more so in the future, for clients who no longer have access to this type of aging technology.

It should be noted that a follow-up in the MIKAN system that was done during the period when this report was being written revealed that the information about these documents had in fact been updated in the system. This means that if another client searches for those same documents, he will find that they are available for consultation because access is no longer restricted.

5.2.2.3 Reprography endnote50

Finding 13

Digital reproduction of LAC documents has improved client service. However, digital reprographies are rarely available online.

Through reprography, it is possible to create a copy of an item in the collection (paper or digital format) without altering the original item. This is usually done in response to a request from a client. In the 2011–2012 RPP the institution stated that it would change its practice of reproducing documentary resources and sending copies to clients and shift towards digital reproduction and storage, thereby facilitating the publication of online content.

Access Program employees who participated in the evaluation confirmed that this change in practice was indeed under way. LAC makes approximately 750,000 copies endnote51 per year in response to requests from its clients. The institution is proposing to extend digital reproduction to include requests under the Access to Information Act and the Privacy Act. According to the data collected through the program, the number of pages/images delivered to clients fluctuated significantly between 2011–12 and 2015–16 (table below).

Table 4: Number of Digital Reproductions (Reprography)

This table shows the number of digital reproductions requested to the Reference Services between 2011-2012 and 2015-2016 and the variation in percentage, year over year.
Year Number of pages/images delivered to clients Variation (in percentage)
2011–2012 750,000 --
2012–2013 1,060,916 41
2013–2014 1,623,221 53
2014–2015 1,031,176 57
2015–2016 1,523,634 48

Source: Departmental Performance Report, 2011–2012, Program 2.3: Describe and contextualize documentary heritage, Library and Archives Canada. Performance Measurement Strategy: Access to Documentary Heritage Program, from 2012–2013 to 2015–2016.

This includes pages/images delivered in response to formal and informal access to information requests. On the basis of data collected through the Access Program, it is assumed that the standards for the reproduction service were fully met. endnote52 

Total expenditures endnote53 for reprography in 2014–2015 were $1,243,408.64, representing $1,174,200.90 in salaries and $69,207.74 in operations. Each request for reprography cost LAC an average of $25.50 in 2014–15. That amount could also be used as a baseline for a subsequent analysis of the efficiency of the reprography service.

The more digital reprographies that LAC performs, the more these should be available online. However, the information that was collected indicates that it is not possible to make digital reprographies accessible online. A number of respondents from different branches confirmed that few digital reproductions actually become accessible online. Once digital reproductions have been completed they are stored on Shared Services Canada servers, and there are no processes currently in place to enable them to be posted online. According to respondents, this is due in part to the limitations of computer systems that do not allow for linking more than one copy to a description.

In addition, if a new client requests a copy of an item of which a copy has already been made, the document will almost always be reproduced again. According to respondents, it is quicker to reproduce the document again than to ask our Shared Services Canada partner to locate it on their servers.

5.2.2.4 LAC website

Finding 14

LAC’s website remains popular despite a decrease in use since 2013–14. However, it is not easy for clients to find what they are looking for.

The LAC website is huge and contains thousands of documents, maps, photos, videos, etc. The evaluation included an analysis of the LAC site to determine what the navigation structure looks like from a client’s point of view. The site provides information and research guides to support clients in their exploration and their research.

In addition to its site located at bac-lac.gc.ca, LAC also has a website named collectionscanada.gc.ca, which has been in place since before 2012. Some portions of the LAC collection can be found at collectionscanada.gc.ca. It was not possible to transfer everything to the new bac-lac.gc.ca website given that some outdated technologies make it very complicated to transfer all of the information there. That information is therefore archived under collectionscanada.gc.ca.

Total expenditures endnote54 in 2014–15 for the LAC website were $2,163,567.11, including licensing fees for different systems (such as Amicus, MIKAN, MISACS, WEB and other systems). The estimated cost of each visit made to the LAC website was $1.99 in 2014–15. That amount could serve as a baseline for an analysis of efficiency at some point in the future.

The public opinion survey endnote55 conducted in 2015 indicates that 78% of respondents reported having visited the LAC website to access its services. In addition, according to the Monitoring and Audit Liaison Directorate’s April 2016 study, there were 17,073,646 visits to the LAC website in 2012–2013. Traffic on the institution’s website was higher in 2013–14, at 24,344,772 visits, but decreased to 22,011,883 in 2014–15 and to 20,250,929 in 2015–16. This data confirms the popularity of the LAC website despite the decrease it has experienced since 2013–14.

While it is possible to find a fair amount of information on the website, searching in the LAC collection remains a complex process. Although navigational links such as “Discover the Collection” and “Search Online” are useful, they do not always enable clients to find what they are looking for because they need to know how to navigate through the website.

According to the data collected through the Access Program, the percentage of clients who reported finding what they were looking for on the LAC website was 84% in 2011-2012, 87% in 2012–2013, 86% in 2013–14 and 83% in 2014–2015. It can thus be seen that the level of satisfaction has decreased since 2012–13. There is no data available for this indicator in 2015-16, as data is no longer collected through the program.

The interviews results (chart below) indicate that 19% of respondents believe that clients find what they are looking for on the LAC website and another 19% believe that clients often or usually find what they are looking for there. However, 19% of respondents were more equivocal in their comments; that is, they doubted that clients find what they are looking for, while 10% of respondents believed that clients do not find what they are looking for. The percentage of respondents who had no opinion on this subject was 33%.

Chart 6: Respondents’ Perceptions of Client Satisfaction in Terms of Finding What They are Looking for on the LAC Website

Chart 6: Respondents’ Perceptions of Client Satisfaction  in Terms of Finding What They are Looking for on the LAC Website

Source: Interview participants, June to September 2016.

  • Chart 6: text version

    This chart shows respondents’ perception of client satisfaction in terms of finding what they are looking for on the LAC website:

    • 19% of respondents believe that clients find what they are looking for on the LAC website;
    • 19% of respondents believe that clients often or usually find what they are looking for;
    • 19% doubted that clients find what they are looking for on the LAC website;
    • 10% of respondents believed that clients do not find what they are looking for on the LAC website; and
    • 33% of respondents had no opinion on this subject.

From the case studies conducted as part of this evaluation, we determined that searching on the LAC website in one of Canada’s official languages (French or English) does not yield the same results. It was difficult to obtain specific results on the ground-breaking ceremony for the St. Lawrence Seaway, for example; most of the results we obtained pertained to the Seaway itself but not necessarily the ground-breaking ceremony.

Another finding pertains to the difficulty in understanding some of the search results obtained from the LAC website. For example, the meaning of “Restrictions vary” is not obvious without further explanation. The following message could also be found: “Warning: Descriptive record is in process. These materials may not yet be available for consultation.” To carry out a successful search on the LAC website, it is important to identify what is being looked for in order to select the correct items from the search results. The user must think of doing so in both French and English because the results are not the same. endnote56

In addition, searching through “A-Z Index”, “Browse by Type” and “Browse by Topic” is not the same as making a Google search, since an index rather than a search engine is involved. The “Search BAC-LAC.gc.ca” button may be confusing, as it is not a search button for the institution’s collection but rather for Library and Archives Canada. If clients are not familiar with Amicus, which is used to search published documents, or MIKAN, which is used to search for items in the archives found on the LAC website, it is very difficult for them to find what they are looking for. These search tools are not intuitive and it is not easy for a novice to understand the systems used by LAC.

While LAC has increased the accessibility of its collection, it would be advised for the institution to better explain the concepts of discoverability, availability and accessibility to its clients, partners and staff. endnote57 According to the review conducted by LAC’s Monitoring and Audit Liaison Directorate, the greater the extent to which access services are available online, the lower the cost per user.

Recommendation 4: To facilitate access to the collection on its website, LAC should improve its search tools and navigation.

5.2.3 Barriers to access

In interviews with Access Program managers and staff, a number of examples were given regarding barriers (graph below) that currently restrict access to LAC’s documentary heritage. The main barriers to access appear to be its computer systems, limited resources, priority management, the LAC website and the quality of descriptions.

Graph 4: Barriers to Access as Perceived by Respondents

Graph 4: Barriers to Access as Perceived by Respondents

Source: Interview participants, June to September 2016.

  • Graph 4: text version

    The graph above shows respondents’ perception about barriers to access. Specifically:

    • for 46% of respondents, computer systems are a barrier to access;
    • 42% of respondents believe that LAC’s limited ressources represent a barrier to access;
    • 35% of respondents believe that a bad priority management represent a barrier to access;
    • 31% of respondents believe that LAC’s website itself is a barrier to access;
    • 12% of respondents said a lack of expertise and client training are barriers to access; a same percentage believe that client’s expectations not reach restraint the access;
    • 8% of respondents said not enough digitization is a barrier to access;
    • 4% of respondents said that copyrights, Access to Information Act and Privacy Act are barriers to access;
    • another 4 % of respondents said that a lack of LAC partnership awareness about their obligations is a barrier to access;
    • 4% of respondents said that not putting the clients in front of the services is a barrier; and
    • 4% believe that not providing the same access to publish heritage and archives is a barrier to access.

5.2.4 Client satisfaction

According to the Access Program data, the client satisfaction percentages with on-line services were 74% endnote58 in 2011–12, 80% in 2012–13, 83% in 2013–14 and 75% in 2014–15. It is possible to notice that the client satisfaction increased between 2011–12 and 2013–14 but decrease in 2014–15. The indicator was dropped in 2015–16.

Among the public at large, in a 2015 public opinion survey endnote59 37% of respondents indicated that they were satisfied or somewhat satisfied with how LAC manages access to its collection. A similar proportion of respondents (36%) was somewhat dissatisfied or dissatisfied, and 28% had no opinion. According to the same survey, the most satisfied clients were those who used the services for genealogical research (47.7%); out of personal interest (42.5%); for research (41.1%); or for educational purposes (34.9%).

Researchers were the most satisfied with the access provided by LAC (48.3% satisfaction rate), followed by historians (47.8%), writers (45.0%), government employees (37.9%), and educators and teachers (30.5%). In contrast, the rate of satisfaction for librarians was 24.1% and for archivists it was 23.8%.

5.3 Efficiency: Use of resources

LAC’s financial resources declined significantly between 2011–12 and 2015–16 (see table below), from $112 million to $91 million. That decrease was largely due to the implementation of the federal government’s Deficit Reduction Action Plan at LAC in 2012–2013.

At the same time, the Access Program’s financial resources declined significantly between 2011–12 and 2015–16 (see table below). From $36.8 million in 2011–12, they fell to $25.7 million in 2015–16, a decrease of $11 million. As a percentage of LAC’s budget, the Access Program’s financial resources (which accounted for 33% of LAC’s total budget in 2011–2012) decreased to 28% in 2015–2016. 

Table 5: Financial Resources

The table shows Library and Archives Canada’s and the Access to Documentary Heritage Program financial resources between 2011-2012 and 2015-2016. It also represent the program’s financial resources as a percentage of LAC’s budget.
Fiscal year LAC financial resources
(salaries and operating expenses)

Access to Documentary Heritage Program
(PAA 2.3)

Financial resources

Access to Documentary Heritage Program
(PAA 2.3)

As a percentage of LAC’s budget

2011–2012 $112,021,400 $36,826,100 33%
2012–2013 $118,923,232 $35,649,500 30%
2013–2014 $100,803,692 $31,959,088 32%
2014–2015 $102,593,650 $33,220,247 32%
2015–2016 $91,451,613 $25,694,773 28%

Source: Departmental Performance Reports, 2011–2012 to 2015–2016.

LAC’s human resources (see table below) were also affected by a decrease of 199 FTEs, from 1,112 to 913 during the five-year period covered by this evaluation. The program’s human resource levels were 376 FTES in 2011–12 and 311 in 2015–16: a decrease of 65 FTEs over five years.

Table 6: Human Resources

The table shows Library and Archives Canada’s as well as the Access to Documentary Heritage Program human resources between 2011-2012 and 2015-2016. It also represents the program’s human resources as a percentage of LAC’s human resources.
Fiscal year LAC human resources (FTEs) 

Access to Documentary Heritage Program
(PAA 2.3)

Human resources (FTEs)

Access to Documentary Heritage Program
(PAA 2.3)

As a percentage of LAC’s human resources (FTEs)

2011–2012 1,112 376 34%
2012–2013 961 298 31%
2013–2014 885 329 37%
2014–2015 951 360 38%
2015–2016 913 311 33%

Source: Departmental Performance Reports, 2011–2012 to 2015–2016.

In particular, with respect to operating expenses, the level of flexibility that the Access Program had to invest in non-salary expenditures (graph below) was 13% and 16% respectively for 2013–14 and 2014–15. In 2015–16, that flexibility decreased to 7% of total program spending, limiting the organization’s ability to improve systems or to invest in other access-related services.

Graph 5: Comparison of Salary Expenses as Compared with Operations

Graph 5: Comparison of Salary Expenses as Compared with  Operations

Source: Library and Archives Canada, Finance Branch.

  • Graph 5: text version

    This graph shows a comparison between salary and operation expenses for the Access program between 2011-12 to 2015-16. Salary expenses varied from 92% in 2011-12 to 93% in 2015-16 with a peak of 98% and a depth of 84% in 2014-15. Regarding the Program’s operation expenses, they varied from 8% in 2011-12 to 7% in 2015-16 with a peak of 16% in 2014-15 and a depth of 2% in 2012-13.

6. Other observations

6.1 LAC partnerships

According to the documents reviewed, agreements with partners endnote60 made it possible to digitize and index a number of collections. For example, the agreement with Canadiana led to the digitization of 78 collections, representing 6 million pages that were made available on line. The partnership with Ancestry.ca made it possible to digitize the 1921 Census files and other collections. These agreements therefore allow for greater access to information by speeding up digitization and indexing.

LAC also continued the digitization projects endnote61 conducted in collaboration with Canadiana.org and Ancestry.ca. As of March 31, 2015, 35 of the 40 million images had been digitized by LAC and Canadiana.org, and 22 million images had been posted on the Canadiana.org website. The digitization of 1.3 million images took place in collaboration with Ancestry.ca, and those images were available online in 2015–16.

The partnership agreements with Canadiana and Ancestry were perceived positively by 42% of respondents (Canadiana) and 27% (Ancestry) (chart below). However, a good percentage of respondents were more equivocal in their comments, i.e. they were not sure if those agreements were a good thing for LAC clients (33% and 55% respectively). Further analysis seems to be needed in order to measure the benefits and advantages of these partnerships for LAC clients.

Chart 7: Respondents’ Perceptions of Partnerships

Chart 7: Respondents’ Perceptions of Partnerships

Source: Interview participants, June to September 2016.

  • Chart 7: text version

    Description: This chart shows respondents’ perception of LAC’s partnership agreements with Canadiana on one side and Ancestry on the other side. Specifically:

    • 42% and 27% of respondents provided positive comment on agreements with Canadiana and Ancestry respectively;
    • 9% of respondents provided negative comment on agreement with Ancestry;
    • 33% and 55% of respondents doubted that the agreements with Canadiana and Ancestry were positive for LAC’s clients;
    • 25% and 9% of respondents didn’t know about those agreements with Canadiana and Ancestry respectively.

6.2 Exhibitions and social media

LAC began a major shift in 2014–15 with regard to the development of exhibitions and the use of social media (Facebook, Twitter, Flickr, YouTube, blogs and podcasts) to promote its collection across Canada and to maximize the number of clients it reaches through a wide variety of channels. 

Although the Flickr page averaged 425,000 monthly visits in 2014–2015, the blog posts generated 150,000 visits. The 10 podcasts were downloaded 150,000 times; it is unclear whether this traffic generated more visits to the LAC website or whether there was more interest in the LAC collection, as this type of information is not collected.

LAC’s review of access methods endnote62 indicates that social media is not a method of access but consists instead of promotional tools with three objectives: outreach, engagement and collaboration.  

In 2014–15, $855,875.80 were spent on exhibitions, endnote63 while social media expenditures were $794,307.70, representing a total cost of $1,650,183.50 to promote LAC’s visibility.

Finally, while social media and exhibitions increase the institution’s visibility, LAC should ensure that their impact on access to the LAC collection is measured.

7. Conclusions and recommendations

7.1 Conclusions

The relevance of the Access to Documentary Heritage Program has been confirmed, as it has shown itself to be a core pillar of LAC’s mandate. Access to documentary heritage is recognized in both the legislation and the institution’s priorities. It is also a part of the Government of Canada’s commitments in a number of ways.

With regard to performance, a number of Access Program activities, such as indexing, Reference Services and services offered under the Access to Information Act, have shown that they are working well and progress is being made towards the attainment of medium-term results. Continuing with block review will enable clients to obtain more and more up-to-date information on files open for consultation and will avoid the need to make a request under the Access to Information Act.

However, other activities call for further attention and improvement. It appears that the institution’s large number of priorities impedes smooth operations, given the lack of coordination among the branches that support activities designed to facilitate access to the collection. The Public Services Branch controls only 54% of the resources allocated to delivery of the Access Program, a fact that gives rise to governance challenges.

The Access Program’s objectives are numerous, making it difficult to achieve results. In the case of digital reprography, content was not made available online in accordance with the plans and commitments. Efforts should be made to facilitate access to the documents in question at lower cost and to a larger number of Canadians. Other steps could be taken to improve access to documentary heritage, such as digitizing finding aids to enable the institution’s clients to find what they need on their own, as they would then have access to links to the information they are seeking in the collection. At the same time, improving the search and navigation tools on the LAC website would also make it easier for clients to search for items in the collection.

The Access Program must also improve its data collection to make it easier to evaluate how its activities are performing. Beyond data collection, the program must also ensure that the right indicators are measured on an ongoing basis.

7.2 Recommendations

The evaluation of the Access to Documentary Heritage Program has led us to make the following recommendations.

Recommendation 1: Better coordination of activities and a prioritization of tasks among branches are needed to clarify the governance of the Access Program and the role of the staff involved, regardless of the shape the program takes in the future.

Recommendation 2: Program managers should undertake a review of output and outcome indicators to ensure that they are collected on an ongoing basis, that the indicators identified are useful for decision making, and that data collection is possible and practical so that the program’s progress and outcomes can be measured.

Recommendation 3: Efforts should be made to complete the digitization of finding aids.

Recommendation 4: To facilitate access to the collection on its website, LAC should improve the search tools found there as well as navigation. 

Date modified: