Independent Summative Evaluation of the OPC’s Toronto Office

April 2016
Final Report

List of Abbreviations and Acronyms

Abbreviation	Meaning
CCTV	Closed Circuit Television
Council of CPO	Council of Chief Privacy Officers
CRM	Customer Relationship Management (software)
DPR	Departmental Performance Report
ER	Early Resolution
GTA	Greater Toronto Area
HR	Human Resources
IAPP	International Associations Privacy Professionals
ICT	Information and Communications Technology
I & I	Office of the Privacy Commissioner’s Investigations and Inquiry Branch
MoU	Memorandum of Understanding
NCR	National Capital Region
OCOL	Office of the Commissioner of Official Languages
OIPC	Ontario Information and Privacy Commissioner
OPC	Office of the Privacy Commissioner
PAA	Program Activity Architecture
PHIPA	Personal Health Information Protection Act
PIPEDA	Personal Information Protection and Electronic Documents Act
PMF	Performance Measurement Framework
PWGSC	Public Works and Government Services Canada
RPP	Report on Plans and Priorities
SME	Small and Medium Enterprises
Toronto Office	Office of the Privacy Commissioner’s Toronto Office

Executive Summary

This report presents findings of the Independent Summative Evaluation of the Office of the Privacy Commissioner’s (OPC) Toronto Office. Evaluation supports accountability to Parliament and Canadians by helping the Government of Canada to credibly report on the results achieved with resources invested in programs. Evaluation helps determine the extent to which programs are producing the outcomes that they were designed to achieve at an affordable cost and supports policy and program improvements by helping to identify lessons learned and best practices.

What we examined

The OPC Toronto Office was established in 2010 to create a robust presence for the OPC in the Greater Toronto Area (GTA) primarily through the conduct of PIPEDA investigations and stakeholder outreach activities in order to improve compliance and enhance awareness of privacy obligations in private sector organizations. The scope of this evaluation covers the period from June 2010 to June 2015. The evaluation was conducted in accordance with the TBS Directive on the Evaluation Function, focusing on the Toronto Office’s relevance and performance.

What we found

Relevance

The presence of the Toronto Office remains relevant. Complaint volumes involving companies headquartered or located in the GTA remained fairly consistent over the evaluation period and continue to account for almost half of all PIPEDA complaints received by OPC.

Promoting and enforcing compliance with PIPEDA requires collaboration. The findings from this evaluation indicate that the OPC Toronto Office’s engagement with stakeholders and the Toronto privacy community continues to support a collaborative approach to PIPEDA, consistent with the non-prescriptive, principles-based intent of PIPEDA and the role of the OPC. Furthermore, external stakeholders noted that the rapidly evolving nature and impact of technology and the digital economy continues to require close collaboration and engagement with the commercial sector in the GTA. This rapidly evolving landscape translates into an ongoing need for OPC to systematically engage, monitor and learn from the broader privacy community through outreach, research and intelligence gathering activity to support policy and compliance goals.

The physical presence of the Toronto Office continues to promote stakeholder access and engagement with OPC on privacy issues. This occurs through a varied mix of activities including in-person events such as meetings, conferences, speaking venues and other interactions which provide opportunities for information exchange and relationship building. The physical proximity of the OPC Toronto Office to major stakeholders has played a role in facilitating compliance by enabling the investigations process.

The findings also indicate that the local presence of investigators in Toronto is useful and effective in meeting investigative needs and that the Toronto Office has played a role in balancing investigation workloads and in resolving investigations in an expeditious manner through the leveraging of established relationships in the GTA. In this respect, the ability of OPC Toronto staff to meet face-to-face with respondent representatives has proved beneficial. The findings also indicate, however, that OPC’s investigative capacity in Toronto has varied considerably over the years covered by this evaluation.

The proximity of the Toronto Office to major stakeholders has facilitated relationship building and face-to-face interactions. Face-to-face meetings and site visits have played (and continue to play) a significant role in the work conducted by OPC Toronto Investigators with regards to resolving logjams that can delay the course of an investigation.

OPC and PIPEDA Branch plans and reporting documents indicate that the work of the Toronto Office has, for the most part, been aligned with OPC and government priorities.

Performance (Effectiveness, Efficiency and Economy)

The findings of this evaluation support the view that the Toronto Office has achieved the three program outcomes (outreach, compliance and internal services support) expected of it when it was established. The Office has successfully established a presence in the GTA, conducted significant engagement and outreach activity within the privacy community and performed or supported numerous investigations and compliance related activities. The Office’s efforts have focused primarily on large organizations, associations and intermediaries.

Evaluation findings indicate that the Office has been effective in helping private sector organizations understand their PIPEDA obligations. The Office has been effective in reaching out to key stakeholders through a mix of instruments and has also served as a conduit for knowledge dissemination internally within OPC and online via the OPC web site. The Office has played an instrumental role in developing several innovative communications products for this purpose.

The evaluation finds that the Toronto Office has been supported by appropriate management structure and practice and that mechanisms were in place to ensure accountability and prudent use of resources. Budgets have been managed efficiently and appropriate processes and ICT infrastructure have been in place to support information exchange and communications between the Office and Gatineau headquarters.

Since its inception, however, the Toronto Office has experienced significant variation in staffing levels including periods of time when there was no permanent Director leading the Office. This instability within the human resourcing aspects of the Office has contributed to some uncertainty internally as well as externally regarding the long-term commitment to the Office. Recent steps taken to stabilize resources, including the recent hiring of a permanent Director, should help to resolve these concerns.

What we recommend

This report contains seven (7) recommendations for consideration by OPC. These recommendations are formative in nature and point to areas that, if addressed, could or should help to ensure that the Toronto Office continues to be relevant in meeting the expectations of OPC and stakeholders as well as efficient and effective in achieving its performance goals. These recommendations are contained in section six (6) of this report.

1. Introduction

This report presents findings of the Independent Summative Evaluation of the Office of the Privacy Commissioner’s (OPC) Toronto Office.

The OPC is responsible for overseeing compliance with both the Privacy Act, which covers the personal information-handling practices of federal government departments and agencies, and the Personal Information Protection and Electronic Documents Act (PIPEDA), Canada’s private sector privacy law. The OPC’s main office is located in Gatineau, Quebec.

The Toronto Office was established in 2010 to create a robust presence for the OPC in the Greater Toronto Area (GTA) through the conduct of PIPEDA investigations and stakeholder outreach activities in order to improve compliance and enhance awareness of privacy obligations in private sector organizations^{Footnote 1}.

When the OPC created the Toronto Office, it committed to conducting an independent evaluation of the Office in 2015-16 to determine whether it was having the desired impact. Accordingly, the scope of this evaluation covers the period from June 2010 to June 2015. The Ottawa-based consulting firm of Gelder, Gingras & Associates was engaged to conduct this independent summative evaluation to assess the extent to which the OPC’s Toronto Office is achieving its desired impact. The evaluation was conducted in accordance with the TBS Directive on the Evaluation Function, focusing on the Toronto Office’s relevance and performance.

2. Profile of the OPC Toronto Office

2.1 Background

The OPC is a public advocate for the privacy rights of Canadians. Thus, the OPC places a critical focus on protecting the privacy rights of individual Canadians (OPC’s Strategic Outcome). However, protecting the privacy rights of individuals requires working closely with stakeholders to build trust and ensure compliance. These stakeholders include major enterprises across many sectors (e.g. financial, insurance, communications) as well as small and medium sized enterprises, associations, communities of interest and intermediaries (i.e. legal, privacy, academic, international bodies), consumer groups, and Canadians.

The Toronto Office was officially opened in the fall of 2010 to bolster engagement with local and regional stakeholders. The Toronto Office responded to a need for a stronger regional presence; OPC survey data^{Footnote 2} at that time indicated that stakeholders supported the presence of the office and perceived it as a means to promote privacy discourse and inform industry about privacy regulation and compliance. At the time, the opening of the Toronto Office was, in large part, meant to address the high percentage of complaints (roughly half of all complaints in 2010) against organizations headquartered/located in the GTA. The Toronto Office was expected to support these requirements by proactively engaging in relationship building with stakeholders and related communities.

The Toronto Office conducts an average of 70 stakeholder outreach and engagement activities a year (including one-on-one and large group meetings, conference attendance and exhibiting, speaking engagements, etc.) and produces outreach products such as the Ten Tips series of fact sheets. In addition, the Office conducts compliance activities including approximately 30 investigations annually. Many of these investigations involve face-to-face interactions with large organizations and are complex in nature. Toronto Office investigative staff also oversee, coordinate and contribute to the publication of case summary reports designed to inform organizations and the privacy community regarding PIPEDA findings and decisions^{Footnote 3}. These case summary reports are posted to the OPC website.

The focus of OPC’s Toronto Office is on PIPEDA. The Act^{Footnote 4} sets out ground rules for how private sector organizations may collect, use or disclose personal information in the course of commercial activities. PIPEDA also applies to federal works, undertakings and businesses in respect of employee personal information. The law gives individuals the right to access and request correction of the personal information these organizations may have collected about them.

PIPEDA is intended to support and promote electronic commerce by ensuring privacy rights of Canadians are protected. In general, PIPEDA applies to organizations’ commercial activities in all provinces, except organizations that collect, use or disclose personal information entirely within provinces that have their own privacy laws, where such laws have been declared substantially similar to the federal law.^{Footnote 5} In such cases, it is the substantially similar provincial law that will apply instead of PIPEDA, although PIPEDA continues to apply to federal work, undertakings or businesses and to interprovincial or international transfers of personal information.

2.2 Governance and Resources

The Toronto Office is led by a Director (EX-01). The Office is part of the PIPEDA Investigations Branch, headed by the Director General, based in Gatineau, Quebec. As of April 2016, the Office houses eight indeterminate positions (including two staff seconded from other departments) comprised of the Director; Administrative Assistant; Senior Advisor, Investigations, PIPEDA (2); Senior Analyst, Stakeholder Relations, PIPEDA; and Senior Privacy Investigator (2 seconded). The Office also typically hosts a contract employee, generally a co-op or FSWEP student. It should be noted that the resource capacity of the Office has varied somewhat over the years covered by this evaluation. The implications of this variance are discussed later in the findings section of this report.

Toronto Office staff are expected to coordinate their work with OPC employees in Gatineau, in particular the staff within PIPEDA Investigations Branch, the Legal Services, Policy, Research and Technology Analysis Branch and the Communications Branch, while conducting investigations and direct stakeholder outreach activities in the GTA.

The Toronto Office had a total budget of $609,000 in 2014-15 (see Annex 8) to cover salaries and other direct operational costs. Some indirect costs are associated with the services shared between the Toronto and Gatineau Offices such as IT support, legal counsel, complaint registration, and policy and research support.

The Toronto Office supports the internal services functions of OPC headquarters in Gatineau through ongoing participation in meetings, teleconferences, and working groups and through its various planning and reporting functions.

2.3 Logic Model and Performance Measurement Framework

The purpose of a logic model is to help managers verify that the theory linking activities, outputs, and outcomes is realistic and reasonable; to help produce information that is meaningful for monitoring, evaluation, and decision making; to help managers interpret data collected on the Office; to serve as a reference point for evaluations; and to facilitate communication about the Office to staff and stakeholders.

The Logic Model for the OPC Toronto Office is presented in Annex 1. Originally developed in 2010, the Logic Model provides a roadmap to the expected activities, outputs and outcomes of the Toronto Office. The Logic Model is articulated for three programs: compliance activities, public outreach, and internal services. Under each program, a set of expected outcomes, outputs, and activities have been developed. Note that this model was developed prospectively at the office’s outset; real-time operational adjustments were made upon the observation of additional efficiencies and functions. However, no revisions were made to the 2010 version of the Logic Model as part of this evaluation, as any modifications would have been outside the scope of this evaluation.

The Performance Measurement Framework (PMF) for this evaluation is presented in Annex 2. This framework contains the evaluation issues, evaluation questions, key indicators, data sources, and data collection and timing elements that were expected to guide the evaluation effort. In consultation with the OPC evaluation project authority, minor revisions were made by the consulting team to the original 2010 version of this evaluation framework in order to facilitate the evaluation process^{Footnote 6}.

3. About the Evaluation

3.1 Purpose of the Evaluation

The main purpose of this summative evaluation is to assess the extent to which the OPC’s Toronto Office is achieving its desired impact. At the time of its creation, it was noted that a significant proportion of respondent addresses and privacy officer addresses for PIPEDA complaints originated in the GTA.^{Footnote 7} Furthermore, many of these complaints originated from the financial and insurance sectors headquartered in the GTA. It was thought that OPC and PIPEDA investigations would benefit from a thorough understanding of the industries and sectors located in the GTA, allowing issues to be addressed in a more direct and effective manner. In announcing the establishment of the Toronto Office, then Commissioner Jennifer Stoddart noted that the establishment of the Toronto Office responded to a need for a stronger regional presence for OPC in the GTA, where much of Canada’s business takes place.^{Footnote 8}

As previously noted, at the time the Toronto Office was established, it was decided that an evaluation of the Office would be conducted after five years of operation. This evaluation responds to that commitment.

3.2 Methodology

It should be noted that the OPC is considered a small organization for the purposes of the Treasury Board Policy of Evaluation. As such, it is not required to conduct program evaluations other than an evaluation of its Contributions Program every five years. However, OPC recognizes the value of evaluation in supporting management decision-making and conducts evaluations as needed, in accordance with Treasury Board Policy. Although it is not a program per se, the Toronto Office shares many of the operational attributes of a program and lends itself to the use of similar methodologies. Thus, the methodological framework developed and used for this evaluation of the Toronto Office adheres to the Treasury Board approach.

In conformity with the TBS Directive on the Evaluation Function, five core issues were addressed in the evaluation, with regards to the Toronto Office’s relevance and performance. These issues are shown in Table 1, following.

Table 1: Core Evaluation Issues, TBS Directive on the Evaluation Function

Core Evaluation Issues (source: TBS Directive on the Evaluation Function)

Relevance
Issue 1: Continued need for program	Assessment of the extent to which the program continues to address a demonstrable need and is responsive to the needs of Canadians
Issue 2: Alignment with OPC priorities	Assessment of the linkages between program objectives and (i) federal government priorities and (ii) departmental strategic outcomes
Issue 3: Alignment with federal roles and responsibilities	Assessment of the role and responsibilities of the federal government in delivering the program
Performance (effectiveness, efficiency and economy)
Issue 4: Achievement of expected outcomes	Assessment of progress toward expected outcomes (including immediate, intermediate and ultimate outcomes) with reference to performance targets and program reach, program design, including the linkage and contribution of outputs to outcomes
Issue 5: Demonstration of efficiency and economy	Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes

Table 2 below provides the overarching question framework for the evaluation of the Toronto Office. These questions are linked to the key issues for relevance and performance which originate from the TBS Evaluation Framework. The evaluation questions are also aligned with the Logic Model for the Toronto Office, including the expected outcomes established for the Office. It is important to note that these evaluation questions are high-level and represent the issues to be assessed by the evaluation. More detailed questions are contained in the Performance Measurement Framework and in the interview/survey guides used in conducting the evaluation (Annexes 4, 5, and 7). The expected outcomes from the OPC Logic Model are integrated into issue #4.

Table 2: Question Framework for the Evaluation of the Toronto Office

Question Framework for the Evaluation of the Toronto Office

Evaluation Issue	Evaluation Question
Relevance
Issue 1: Need for the Toronto Office	Does the Toronto Office address a genuine need for the OPC to deliver its mandate?
	Are there impacts to the OPC not having a Toronto Office?
Issue 2: Alignment with OPC priorities and Government direction	Is the Toronto Office in line with OPC priorities?
Issue 3: Alignment with government/OPC roles and responsibilities	Is the Toronto Office aligned with the OPC and government’s roles and responsibilities?
Performance (effectiveness, efficiency and economy)
Issue 4: Achievement of expected outcomes	To what extent has the Toronto Office achieved its expected outcomes? (Ref. to objectives and LM outputs and outcomes for compliance, outreach and internal services)
Compliance (Program activity 1)	To what extent have private sector organizations (particularly in the GTA/Ontario) met their obligations under PIPEDA?
Outreach (Program activity 3)	To what extent have private sector organizations (particularly in the GTA/Ontario) understood their obligations under PIPEDA?
Internal Services (Program activity 4)	Is the Toronto Office integrated into the functioning of OPC and applying sound business principles?
Ultimate outcome	To what extent has the Toronto Office contributed to OPC’s lead role with respect to privacy rights and protection of personal information?
Issue 5: Demonstration of efficiency and economy	Are there alternative, more cost-effective ways to achieve the same outcomes? (Resource utilization, cost-benefits etc.)

3.3 Data Collection Methods

The evaluation team pursued different lines of evidence from multiple perspectives in order to gather evidence to support the evaluation. Annex 3 presents a detailed question bank summarizing the questions used to collect data for this evaluation. The data were subsequently integrated and synthesized to arrive at key findings, conclusions and recommendations.

The approach consisted of interviews (individual and group), a stakeholder survey, and an extensive document and data file review. The feasibility of the approach was validated through a round of preliminary interviews with key internal OPC officials. These preliminary interviews also yielded valuable contextual information to support the evaluation process and subsequent analysis and findings. Details of these lines of evidence follow.

Interviews

Preliminary Phase

Six (6) preliminary interviews (four one-on-one and two group interviews) were conducted with OPC officials with a view to validating the methodological approach and obtaining contextual information and background to support the evaluation.

Data Collection Phase

Sixteen (16) interviews were conducted during the data collection phase. Fourteen (14) one-on-one interviews were conducted in person or by telephone/teleconference, four (4) with internal OPC staff in the GTA and in Gatineau and ten (10) with external stakeholders in the National Capital Region (2) and in the GTA (8). Three (3) group interviews were also conducted with internal OPC staff, two (2) in Gatineau and one (1) in Toronto.

In all, 25 separate interviews (20 individual and five group) were conducted for the evaluation involving a total of 31 informants. Interview guides were developed to support the interview process and ensure consistency and continuity across interviews. The interview guides are appended to this report (Annexes 4 and 5).

Table 3: Completed Interviews

Completed Interviews

Interviews	Type		Total Interviews	Total Informants
	Individual	Group
Preliminary Interviews (OPC staff)	4	2	6	8
Internal Interviews (OPC staff)	6	3	9	13
External Stakeholder Interviews	10	-	10	10
Total	20	5	25	31

Survey

An electronic survey using the FluidSurvey platform was developed and distributed during late December 2015 and mid-January 2016 to a list of 89 key external stakeholders (of which 84 were valid contacts) who were known to have had some engagement with the Toronto Office within the evaluation timeframe and who, therefore, were capable of speaking informatively about the nature, quality and benefit of that interaction based on first-hand knowledge and experience. These informants included recipients, users and beneficiaries of the outputs of the Toronto Office. In total, three (3) reminders were sent to bolster participation in the survey^{Footnote 9}.

Forty-four (44) respondents completed the survey for an overall response rate of 56%. As many recipients of the survey were themselves privacy officials actively engaged in the privacy community, this relatively high response rate probably indicates a strong motivation on their part to participate in the process of assessing the OPC’s Toronto Office, possibly with a view to helping it to improve future operations. The survey questionnaire is appended to this report (Annex 7). Details of the survey response follow.

Table 4: Survey Completion and Response Rate

Survey Completion and Response Rate

Category	Number	%
Stakeholders contacted	89	N/A
Bounced back	5	N/A
Valid contacts	84	100%
Partially Completed	10	12%
Fully Completed	37	44%
Response Rate	(47/84)	56%

Document/File Review

Relevant background documents were reviewed, including various OPC reports as well as Toronto Office plans, reports and scorecards and additional documents that were provided by interview respondents and the project authority. Annex 6 (Document/Data/File Review Evidence) presents the lines of evidence resulting from the document/file review. The list of documents can be found in the bibliography appended to this report (Annex 9). The following table summarizes the nature and type of documents/files reviewed.

Table 5: Sources for Document/File Review

Sources for Document/File Review

Primary Sources: Category Nature and Type	Number
Annual Report	6
Branch Business Plan	5
Budget Report	5
Case Summaries/Report of Findings	11
DPR/RPP	2
HR Structure Document	7
Legislative Document	1
OPC Website	5
Quarterly Outreach Report	13
Research/Outreach Document	13
Scorecard	4
Stakeholder Website	14
Strategy Document	6
Toronto Office Statistics	5
Total	97

3.4 Methodological Limitations

Note about contact lists and limitations

The list of informants used for both the stakeholder interviews and the survey was compiled for the evaluators by the Toronto Office. The list includes only those who have had prior contact with OPC Toronto and who could therefore speak from first-hand experience about their interactions with the Office. In other words, it excludes contacts from organizations who have not had any interaction with the Office and who therefore could have only speculated about its operations. Nevertheless, the evaluators took measures to control for this potential bias. First, the contact list was reviewed to ensure that it contained a fair representation of the commercial sectors across the GTA; second, respondents were promised anonymity so that they would be encouraged to speak openly and freely about their perceptions and experiences.

During the course of the evaluation, the evaluators found the interviewees to be very cooperative, frank and forthright in providing their input and appraisal of the Toronto Office. The same can be said of survey respondents. We detected no obvious intent on the part of respondents to sugar-coat or, alternatively, to diminish the performance of the Office. Indeed, many respondents offered concrete opinions regarding how the Office could enhance or build upon its activities.

During the early stages of the evaluation, several attempts were made by the evaluators to obtain a valid stakeholder contact list. The fact that the Toronto Office does not systematically maintain a stakeholder contact list or utilize a CRM system to track its activity is a limitation with respect to assessing the coverage of the office [see Recommendation 3.1]. This limitation is partly a function of historical concerns within OPC regarding privacy restrictions around the compiling of contact information. In this respect, the breadth and depth of outreach activity relative to the overall potential represented by the GTA remains somewhat of an open question.^{Footnote 10}

Three other limitations are worth noting. First, external stakeholders were not always able to make a clear distinction between OPC Toronto and OPC headquarters in Gatineau with respect to certain experiences and interactions (e.g. information referenced by OPC Toronto but accessed from the OPC web site). Second, although the main focus of the Toronto Office is PIPEDA, stakeholders sometimes speak of broader aspects, issues and concerns related to privacy, both in Canada and Internationally. When appropriate, the evaluators reminded stakeholders to focus their input specifically on the Toronto Office. We did not, however, ask stakeholders to restrict their comments to privacy matters solely related to PIPEDA. As a result, some input provided by stakeholders may go beyond the current mandate of OPC and/or the Toronto Office^{Footnote 11}. Third, during the course of this evaluation, some gaps and discrepancies in data between various OPC reports and systems as well as gaps in data for certain time periods were noted [see Recommendation 2.2]. Where possible, the evaluators took steps to reconcile these gaps and discrepancies. Efforts to enhance data integrity may be helpful to support future evaluations.

4. Findings

4.1 Relevance

4.1.1 Continued Need for the Toronto Office

Extent to which the Toronto Office addresses a genuine need

Initially, the need for the Toronto Office was justified, in large part, by the volume of complaints against organizations headquartered in the GTA. OPC data indicate that in 2010 roughly half of all complaints against private-sector organizations involved companies headquartered or located in the GTA. Furthermore, data indicate that these complaint volumes remained fairly consistent over the evaluation period, ranging from a high of 58% in 2012 to 46% in 2014 (the last year complete data were available). Furthermore, industry sectors concentrated in the GTA continue to account for significant volume of overall PIPEDA complaints. For example, in the year 2014-15, 31% of PIPEDA complaints were in the Internet/Telecom sector and 20% were in the financial industry. These comparatively high volumes continue to reflect the concentration of commercial activity represented by organizations in the GTA. Most of these organizations are large employers with a national and often international presence. As a result, they have been and remain the focus of much of the Toronto Office’s investigation and stakeholder outreach activity.

Due to PIPEDA’s significant and far-reaching implications for commercial activity, promoting and enforcing compliance with the Act requires collaboration. The evidence from both interviewees and survey respondents indicates that the OPC Toronto Office’s engagement with stakeholders and the Toronto Privacy community continues to support a collaborative approach to PIPEDA voluntary compliance, consistent with the non-prescriptive (principles-based) intent of PIPEDA and the role of the OPC.

Furthermore, external stakeholders noted that the rapidly evolving nature and impact of technology and the digital economy continues to require close collaboration and engagement with the commercial sector in the GTA. This rapidly evolving landscape translates into an ongoing need for OPC to systematically engage, monitor and learn from the broader privacy community through outreach, research and intelligence gathering activity to support policy and compliance goals. OPC Toronto is seen by external stakeholders as a vehicle to promote understanding and awareness related to emerging privacy issues. Indeed, external stakeholders (80%) see potential for industry to provide even greater input into the policy-making process (in-reach) through proactive dialogue and engagement with OPC [see Recommendation 6.1]. They also value opportunities to be kept informed in a timely manner about OPC interpretations on privacy related matters and about where OPC is focusing its energies. In this regard, the evaluators noted that the Toronto Office has contributed substantively to the development of PIPEDA case summaries, which are published and accessible on the OPC website. OPC data indicate that from 2010-2014, 84 case summaries were developed and published.

The evidence from a majority of external stakeholders (60%), all internal stakeholders, as well as from survey feedback and outreach reports indicates that the physical presence of the Toronto Office continues to promote stakeholder access and engagement with OPC on privacy issues. This occurs through a varied mix of activity including in-person events such as meetings, conferences, speaking venues and other interactions which provide opportunities for information exchange and relationship building. These activities have benefited audiences while also reducing the operational burden (travel, time, cost, etc.) on OPC headquarters. Survey data indicate that stakeholders find these outreach activities to be relevant to their needs. For example, 89.6% of stakeholders surveyed for this evaluation either agree (54.5%) or strongly agree (34.1%) that the information received and accessed from OPC’s Toronto Office was relevant.

In addition to supporting access and outreach, interview and survey data from external stakeholders suggest that the physical proximity of the OPC Toronto Office to major stakeholders has played a role in facilitating compliance by enabling the investigations process. This was attributed to reduced travel time, knowledge or sensitivity to commercial concerns and the development of trusted (non-adversarial) relationships^{Footnote 12}.

The evidence from both informants and file reviews indicates that the local presence of investigators in Toronto is useful and effective in meeting investigative needs. The Toronto Office has played a role in balancing investigation workloads and in resolving investigations in an expeditious manner through the leveraging of established relationships in the GTA and the ability to meet face-to-face with respondent representatives. These benefits, however, have been constrained somewhat by the fact that OPC’s investigative capacity in Toronto has varied considerably over the years covered by this evaluation^{Footnote 13}.

OPC’s PIPEDA intake function and early resolution (ER) function reside in Gatineau as per a former executive decision^{Footnote 14}. The intake function handles approximately 1200 complaints per year, most (70% -80%) of which are submitted via the OPCs online complaint form. The intake function typically screens out over 50% of these complaints for a variety of reasons. OPC data indicate that the early resolution (ER) process triages a further 38% of PIPEDA complaints. OPC data for the years 2011-2014 indicate that approximately 10% of investigations (complaints closed) were completed in Toronto; this figure rises to 15% when early resolution cases are excluded.

Investigation managers in Gatineau are responsible for assigning cases to investigators in Gatineau and the GTA. Decisions on GTA investigation assignments are made in consultation with the Director of the Toronto Office. When doing so several factors are considered, including capacity, fit, workload, expertise (e.g. recent work on a similar case), and sector knowledge or relationships. Internal interviewees stated that these factors have all played a role in the assignment of cases to the GTA and their eventual resolution.

Impacts of the Toronto Office

Awareness (outreach/stakeholder relations)

As part of its outreach effort, the OPC Toronto Office engages stakeholders through a variety of activities designed to promote awareness and understanding of PIPEDA obligations and related privacy matters. OPC reported 48 Toronto outreach activities in 2011. Quarterly Outreach Reports summarizing outreach activities have been prepared by the Toronto Office since June 2012. Data from these reports indicate that the Office conducted 81 outreach activities from Q2 to Q4 in 2012-13, 70 activities in 2013-14, 72 activities in 2014-15 and 30 activities in Q1 of 2014-15^{Footnote 15}.

Toronto Outreach activities by type are summarized in the table below for the years indicated.

Table 6: Types of Engagement

Types of Engagement

Type of Engagement	2012-13 (Q2-Q4)	2013-14 (Q1-Q4)	2014-15 (Q1-Q4)	2015-16 (Q1)
Hosted Events	0	0	3	5
Events Attended	13	12	6	6
Exhibits	13	15	14	3
Meetings (Associations, Regular, Individual Stakeholders)	33	21	24	10
Speeches/Speaking EngagementsFootnote 16	13	18	24	6
Other (blog posts, mail-outs, invited speakers, etc.)	9	4	1	0
Totals	81	70	72	30

Details of the Office’s current approach to outreach are contained in two planning documents: the Office’s Outreach Strategy (Sept. 2014) and the OPC Toronto Outreach Plan 2015-16. The documents do not contain specific performance measures designed to assess impact and no similar plans exist for the investigations component of the Office. However, some performance measures for investigations may be encapsulated in separate performance agreements. No formal Toronto Office business or outreach plans could be obtained for periods prior to August 2012^{Footnote 17} [see Recommendation 2.1]. The key topics covered by Toronto Office Outreach activity demonstrate an evolving and dynamic nature of privacy issues ranging from mobile applications, notice and consent, breach prevention, know-your-client and anti-money laundering, de-identification of personal information, individual control of information as a means of building trust and anti-spam legislation.

Evidence from selected Toronto Office Outreach reports indicates that the Office engaged with a broad cross-section of stakeholders including, for example, associations (e.g. Canadian Marketing Association, Chartered Professional Accountants, Canadian Bar Association and Interactive Advertising Bureau); corporations (e.g. Walmart, Equifax, TD Bank and Microsoft); and government officials. The Office also engaged stakeholders through venues and events (e.g. Mobile Enterprise Strategy Summit, Privacy Perspective, Canadian Chambers of Commerce, Advertising Association AGM, and the Bell Relevant Advertising Program discussion). Due to insufficient data regarding target audience demographics and limitations of scope, it is not possible to determine the actual reach (i.e. breadth and depth of market penetration) of the Office based on current reports for this activity [see Recommendation 5.1].

Survey evidence indicates that external stakeholders have found their interactions with the Office to be useful. Most (84.1%) survey respondents either agree (59.1%) or strongly agree (25%) that the information they received as a result of their interaction/engagement with the Toronto Office was useful. When questioned about PIPEDA specifically, 51% agreed or strongly agreed that the information accessed/received made them more aware of their PIPEDA obligations. (The gap between usefulness and awareness may suggest that the perceived usefulness of the Office extends beyond PIPEDA or that the Toronto Office may need to focus more on conveying organisations’ PIPEDA obligations). External stakeholders interviewed and surveyed for this evaluation unanimously express appreciation for the engagement and outreach activity conducted by the Toronto Office, but a few explicitly suggest that the potential of this Office has yet to be realized. For example, 80% of external stakeholders suggested that more could be done by the Toronto Office to promote dialogue and ensure that the expertise of industry is fully leveraged by OPC and that voice of Industry is fully heard. They suggest that the Office could be more engaged in shedding light on emerging trends, contributing to dialogue and understanding while serving as an avenue for industry to inform research and policy. Some stakeholders also suggested that the Office (and OPC in general) could play a greater role in reaching out to small and medium size enterprises (SMEs) to promote awareness and compliance with PIPEDA^{Footnote 18}. Our review of OPC documentation and media (online and offline) indicates that the OPC already engages the privacy community in some fundamental and significant ways (i.e. speeches, conferences, consultations, etc.) and makes its findings widely accessible via the OPC web site. Consequently, these perceptions may reflect the fact that some stakeholders are not as aware as they could be of the full scope of OPC information resources and opportunities [see Recommendations 4.1 and 4.2].

Service Quality Impact (defined by access, accuracy and timeliness^{Footnote 19}

The OPC Toronto Office is located in downtown Toronto and is easily accessible by public transit. The headquarters of many major stakeholders are located within relatively close physical proximity to the Office. External stakeholders (60%) suggest that the location of the Office facilitates engagement (outreach and in-reach) by offering stakeholders and OPC staff ready access to one another, while reducing travel time and associated costs.

With regard to service quality impacts of investigation work, evidence from both internal and external interviews suggests that the expertise of the Toronto Office does likely contribute to the quality of investigations. Over the years, OPC Toronto investigators have handled numerous complex or longstanding case files and, as previously noted, Toronto investigations staff have provided capacity and specialized sector and relationship knowledge that are important assets to the expeditious resolution of cases. However, as will be noted later in this report, the full advantage of this investigative potential was also constrained by uneven investigative resource capacity in the Office over the years covered by this evaluation.

With regard to timeliness, OPC data indicate that overall treatment times for complaint resolution have declined significantly over the evaluation period from 15.6 months in 2010 to 7.5 months in 2015^{Footnote 20}). Data indicate that this improvement in treatment times can be largely attributed to the increasing use of early resolution, centralized in Gatineau, to resolve investigations. Conversely, the Toronto Office has been involved in closing some longstanding and complex files that typically take longer. As a result, average treatment times for investigations are substantially higher in Toronto than in Gatineau.

Treatment times for Toronto are shown in the table below:

Table 7: Toronto Office Average Treatment Times

Toronto Office Average Treatment Times

Year	2011	2012	2013	2014	2015
Average Treatment Time (months)	9.1	11.4	15.3	10.3	13.7

The volume of complaints with or without early resolution has an impact on treatment times. The following table presents data for completed investigations (dispositions) for calendar years 2010-2015 (June) for OPC overall and for the Toronto Office. It is important to note that the percentage of Toronto dispositions rises significantly when ER cases are excluded.

Table 8: Total Dispositions

Total Dispositions

Dispositions	2010	2011	2012	2013	2014	June 2015
OPC (Incl. ER)	306	238	260	424	373	139
OPC (Excl. ER)	248	120	144	291	196	65
OPC Toronto	N/A	23	29	28	33	36
% Toronto (Incl. ER)	N/A	9.7%	11.0%	6.6%	8.8%	25.9%
% Toronto (Excl. ER)	N/A	15.8%	13.2%	9.6%	16.8%	55.5%

Site Visits – Investigative Consultations

Face-to-face meetings and site visits have played (and continue to play) a significant role in the work conducted by OPC Toronto Investigators. Our review of 30 investigation related site visits reported by OPC Toronto over the period 2013 to 2015 indicate that these site visit consultations serve a variety of purposes involving a wide range of topics. Internal and external interviewees indicated that these site visits have had a positive impact on service quality and effectiveness of disposition by enabling more timely and accurate communication and information exchange in support of investigations. Internal interviewees stated that these site visits have played a role in resolving logjams that can delay the course of an investigation. Several case outcomes lend support to this contention.

In the Google OBA case (PIPEDA #2014-001), a face-to-face meeting with Google officials in the OPC Toronto Office proved instrumental in fostering a mutual understanding of the issues, pinning down the problem and arriving at a resolution. According to a Senior Advisor familiar with the investigation, the meeting allowed for inconsistencies in understanding of the issue to be addressed on the spot rather than through a typical lengthy chain of e-mail correspondence resulting in an expeditious handling of the case. In another example, a site visit to a large education savings plan provider and a subsequent face-to-face meeting in the OPC Toronto Office with representatives from the company were key to identifying the issues, causes and repercussions involved in hospital health information (confidential information of new mothers) being disclosed by nursing staff to a third party for marketing purposes, as well as arriving at a resolution. The meeting was integral to a process that avoided potentially costly legal action. In a third example (PIPEDA #2013-003), visits to the OPC Toronto Office by the officials of a major bank (including the bank’s CPO) helped to resolve complaints regarding employee snooping, resulting in changes to the Bank’s internal privacy policies and practices.

In all these cases, the existence of the OPC Toronto Office and its proximity to the organizations in question likely facilitated the resolution of the cases faster and more efficiently than would have occurred had the Office not existed. In this regard, it was suggested that the Office provides the OPC with a kind of “home court advantage” that makes it easier to size-up the organization including in some instances obtaining useful visual cues (e.g. physical layout) that could be beneficial to an investigation.

The site visits by type for the period 2013 to 2015 are summarized in the table below.

Table 9: Site Visits

OPC Toronto Office: face-to-face site visits (2013-2015)

Purpose of meeting	Totals
Collecting information	13
OPC Office	5
Respondent's Office	6
Complainant's location	1
Respondent's lawyer's office	1
Negotiation	5
OPC Office	3
Respondent's Office	2
Physical layout and collecting information	2
Respondent's Office	2
Providing recommendations	2
Respondent's Office	2
Reviewing evidence	5
OPC Office	1
Respondent's Office	4
Video surveillance	1
Respondent's Office	1
Facilitate settlement between the parties	1
Respondent's Office	1
Providing update on investigation	1
OPC Office	1
Grand Total	30

Impacts of not having a Toronto Office

Stakeholders that participated in surveys and external interviews almost unanimously support the presence of the OPC Toronto Office. Aside from the obvious impact of the loss of proximity (i.e. increase travel time and cost), most participants stated that without their interaction with the Office locally, their relationship with OPC would be less frequent and more formal. Some external stakeholders (40%) felt that without the Office, there would be a risk that the OPC would become too Ottawa-centric and that there would be a loss of sensitivity and awareness regarding business or commercial concerns, resulting in less effective policy, more formal investigations and more litigation ultimately leading to higher costs and compliance burden. Conversely, a few stakeholders also say that the absence of the Office would not have a significant impact on them as they would shift their privacy advocacy and compliance efforts to Ottawa (Gatineau), as they do with other matters of public policy.

One of the main disadvantages of not having the Toronto Office was expressed in terms of lost opportunity. Some external stakeholders (40%) are of the view that the Office can play an important role in engaging to an even greater extent with stakeholders to explore and mitigate the many emerging issues surrounding privacy protection. In this regard, the impact of not having the Office is viewed in terms of an opportunity cost. Many external stakeholders (survey and interviews) commented that interaction with the Office is valuable for fostering dialogue and a spirit of cooperation. In the view of the evaluators, this can potentially lead to higher levels of voluntary compliance and eventually lower costs (i.e. less litigation). Comments from external stakeholder interviews and the survey suggest that the Office is also seen as a potential hub for privacy issues and an enabler for supporting new initiatives aimed at reaching additional segments of the business community such as SMEs who may be less aware of their privacy obligations [see Recommendation 4.2].

Internally, it was felt that the absence of the Office would represent a certain “loss of connectedness” to the external privacy community. It was also suggested that without the Office, an important “nexus” for the cross-functional integration of policy, communications and compliance efforts within OPC would disappear.

4.1.2 Alignment with OPC Priorities

Extent to which the Toronto Office is in line with OPC priorities

The overall strategic outcome of the OPC is that “the privacy rights of individuals are protected.” These rights are upheld via programs surrounding compliance activities, research and policy development, public outreach, and support from internal services, where the OPC can deliver its privacy business^{Footnote 21}.

The Toronto Office is tasked with supporting both the PIPEDA compliance activities program and the public outreach program as well as OPC internal services programs. The Toronto Office’s plans and activities are meant to support OPC corporate priorities through broad and targeted stakeholder engagement, collaboration, and relationship building in the GTA.

Evidence from PIPEDA Investigations Branch Plans as well as from OPC’s RPP and DPR documentation indicates that from 2010 to 2015, Toronto Office activities have been aligned and integrated with OPC priorities and key commitments. The evolution of the Branch activities and the Toronto Office reflect the evolving nature of the organisation and the various ways in which compliance with PIPEDA legislation is encouraged via non-prescriptive and collaborative approaches.

Planning and reporting documents show that at the inception of the Toronto Office, there were three key OPC priorities^{Footnote 22} relating to the PIPEDA Branch:

• To redefine service delivery through innovation to maximize results;
• To enhance and sustain organizational capacity by employing innovative human resources management approaches to support and develop the organisation and to improve service delivery;
• To develop a long-term accommodation strategy.

PIPEDA Investigations Branch activities and plans aligned initially with these key OPC priorities by focusing on establishing a presence in Toronto and by building its foundation through renting physical space/accommodations and developing a staffing/human resources plan. The Toronto Office was successfully established in downtown Toronto. The Office is currently co-located with the Office of the Commissioner of Official Languages (OCOL).

After the initial establishment of the Toronto Office, Branch activities and plans evolved to reflect a commitment to meeting OPC priorities via the Toronto Office’s outreach and compliance activities and through service program integration within OPC. Corporate priorities in 2012-13 were centered on fully implementing new service delivery models to maximize results for Canadians and on providing leadership to advance the OPC’s priority privacy issues related to information technology, public safety, identity integrity and protection, and genetic information. Priorities also included research and policy development (i.e. via the integration of public and private-sector stakeholders) and advancing knowledge about privacy issues^{Footnote 23}. PIPEDA branch plans and activities for the Toronto Office reflected these priorities.

Between 2013 to 2014-15, the Toronto Office continued to direct its efforts towards supporting OPC key priorities by increasing outreach efforts, increasing its knowledge and intelligence gathering capacities, and building stakeholder relationships. Corporate priorities were aimed at continuing to enhance services to Canadians by refining processes and implementing new legislative authorities by integrating best practices and by consolidating results achieved in the priority privacy issues. Priorities continued to focus on research and policy development by engaging key stakeholders and advancing knowledge about privacy and compliance issues^{Footnote 24}.

From 2014 to present, PIPEDA Branch Plans and Toronto Office activities have been focused on building relationships, understanding rights, promoting voluntary compliance, and showcasing good practices.^{Footnote 25} OPC corporate priorities have focused on enhancing partnerships and collaboration opportunities in order to increase and promote voluntary compliance via public outreach and provision of timely, accurate, and relevant information. Priorities continue to build on research, policy development, and knowledge about privacy and compliance issues^{Footnote 26} while key initiatives were developed or improved to support this direction (e.g. BOOST, PIPEDA Eco-System, the Toronto Outreach Strategy, and the Top Ten series). These plans, priorities and initiatives have been further informed by the Commissioner’s 2014-15 national consultations aimed at finding out what Canadians think about privacy and how privacy experts view the role of the OPC^{Footnote 27}.

4.1.3 Alignment with Government/OPC Roles and Responsibilities

Extent to which the Toronto Office is aligned with OPC’s federal role and mandate

The OPC’s mission is to protect and promote the privacy rights of individuals. The Commissioner works independently of any other part of government in investigating complaints from individuals with respect to the federal public sector and the private sector and is mandated by Parliament to act as the ombudsman and guardian of privacy rights in Canada.

PIPEDA recognizes both the need to protect individual privacy as well as the need for organizations to use information for commercial purposes. This dual function is evident from the stated purpose of the Act:

The purpose […] is to establish, in an era in which technology increasingly facilitates the circulation and exchange of information, rules to govern the collection, use and disclosure of personal information in a manner that recognizes the right of privacy of individuals with respect to their personal information and the need of organizations to collect, use or disclose personal information for purposes that a reasonable person would consider appropriate in the circumstances.

Given that many large commercial enterprises are headquartered in the GTA, OPC’s objective to establish a stronger presence in the region with the Toronto Office can be seen to align with the intent of PIPEDA. The decision to establish the Office in Canada’s most populous region — which does not have equivalent provincial legislation — also aligns with the mission of the Office of the Privacy Commissioner of Canada (OPC) to protect and promote the privacy rights of individuals.^{Footnote 28}

As an agent of Parliament, the OPC has certain reporting and advisory requirements. The OPC issues reports with recommendations to federal government institutions, publicly reports on findings from independent audit and review activities, advises on and reviews privacy impact assessments of new and existing government initiatives; provides legal and policy analysis and expertise to Parliament; responds to inquiries from Parliamentarians and individual Canadians; promotes public awareness and compliance; provides legal opinions and litigates court cases; monitors trends in privacy practices; and works with privacy stakeholders across Canada (OPC, Mandate and Mission, 2012).

The work of the OPC Toronto Office is consistent with certain OPC roles and responsibilities as outlined by its mandate (above in bold). Combined evidence from documents, interviews and survey respondents demonstrates that the Toronto Office promotes and protects the privacy rights of individuals and organisations by conducting investigations and engaging in outreach activities to encourage compliance with PIPEDA.

PIPEDA legislation allows the OPC considerable discretion in enforcing the Act. For instance, article 12.1 of PIPEDA specifies the Commissioner’s role to investigate complaints or to discontinue, as per the situation^{Footnote 29}. The non-prescriptive application of PIPEDA is evident in the Toronto Office’s collaborative efforts towards building relationships with stakeholders. Interviewees stated that the Toronto Office is “focusing less on strict big stick punishment and more on education and cooperation” and spoke to the collaborative and common-sense approach taken by the Toronto Office. Survey respondents mentioned that the “OPC Toronto has been available and helpful” and that OPC Toronto Office is “a great resource for our organisation.” Respondents stated in both the survey and in interviews that the OPC Toronto Office is uniquely placed to communicate with stakeholders, gain knowledge about emerging issues and challenges, undertake research and meet on-site with organisations in the context of an investigation.

Interviewees had somewhat varying viewing of the Toronto Office’s role in investigations. Some spoke to the fact that they have collaborated with the Toronto Office in investigations and were aware of the Office’s lead on certain cases. However, a few external stakeholders (30%) expressed some confusion about the Toronto Office’s role in investigations vis-à-vis the Gatineau Office and several external stakeholders (60%) questioned the logic of the Toronto Office’s dual role in conducting both investigations and outreach. However, others saw benefit in the Office doing both investigations and outreach.

While evidence shows that the Toronto Office contributes through its activities to the OPC’s role as privacy guardian, external stakeholders (70%) see additional opportunity for the Office. For example, given the rapidly evolving global challenges posed by advancing technology, it was mentioned that there is an opportunity for the OPC to explore broader privacy issues beyond Canada’s borders. In this regard, the evaluators note that the OPC does already engage in international fora^{Footnote 30}. For example, the Toronto Office has had significant involvement in a number a number of international files, including certain coordination responsibilities in the global Privacy Sweep.

4.2 Performance

4.2.1 Achievement of Expected Outcomes

Extent to which the Toronto Office has achieved its expected outcomes

Expected outcomes for the OPC’s Toronto Office are found in the Logic Model developed to support this evaluation. These outcomes include compliance activity, public outreach and internal services in support of programs 1, 3 and 4 respectively of the OPC’s Program Activity Architecture (PAA) and PIPEDA Branch Plans. For the purposes of evaluating the performance of the Office, it is important that these outcomes be viewed in the context of the objectives originally envisioned for the Toronto Office and the OPC and PIPEDA Branch priorities which subsequently guided its efforts. It should also be noted that some outcomes are interrelated and different lines of evidence may relate to more than one outcome.

Compliance Activity - Private sector organizations meet their obligations under PIPEDA.

The survey data from this evaluation supports the view that the Toronto Office has helped private sector organizations understand (74% agree or strongly agree) and to a lesser extent comply (50.1% agree or strongly agree) with their obligations under PIPEDA. This has occurred through targeted outreach activity and through investigation and related compliance activities conducted from the Toronto Office. OPC news releases and case summaries indicate that the Toronto Office has carried out a number of investigations with significant privacy implications. These investigations also contributed to outreach efforts by drawing attention to important issues. For example, the Google Online Behavioural Advertising (OBA) investigation (2014) which was carried out in collaboration with the U.S. Federal Trade Commission (FTC) lasted about eight months resulted in adoption of OPC recommendations including changes to Google’s monitoring systems. This complex and high profile investigation was followed up by multiple speeches, panel presentations, and outreach references, and was featured in OPC’s Annual Report. In addition to the above, the Toronto Office has had significant involvement, including certain coordination responsibilities, in the Global Privacy Sweep, an initiative developed by the Office and now involving Privacy Authorities in approximately 30 countries around the globe.

While communicating the results of high profile case work has a potentially broad reach, the more direct outreach activities of the Toronto Office have been focused on a limited number of large employer organizations and related associations, as well as several government bodies. As a rough indicator, the stakeholder list provided for this evaluation by the Office contained 89 separate organizational contacts including 66 organizations, 3 government bodies and 21 industry sector associations. Some organizations (e.g. law firms) and all associations serve as intermediaries that extend the reach of OPC to a broader community of organizations and members. This approach is efficient and cost effective, as the multiplier effect allows OPC Toronto to leverage its resources.

Stakeholders interviewed for this evaluation support the efforts of the Toronto Office and most feel they have benefited from those efforts. Sixty-one percent (61%) of those surveyed for this evaluation say that the Toronto Office has had a least some impact in facilitating compliance with PIPEDA obligations compared to only 16% who say the Office has had little impact. Furthermore, three-quarters (75%) of survey respondents are either satisfied or very satisfied with the service provided by the office compared to 2.3% who are dissatisfied.

Quantitatively speaking, it is not possible to assess from existing data the extent to which OPC Toronto is impacting the broader population of private sector organizations in the GTA (expected outcomes for both compliance and outreach activities, as per the Logic Model presented in Annex 1). For example, until recently (June 2015, the last month covered by this evaluation), SMEs had not been directly targeted by OPC Toronto, although they have been indirectly targeted through national associations. Because it is difficult to gauge the full impact of these indirect interventions, there has been some concern among both internal OPC staff and external stakeholders that SMEs are not as aware of their obligations as they should be. Many larger organizations have fairly mature privacy functions sometimes headed by chief privacy officers well versed in privacy law. Others make use of outside legal counsel as required. Most SMEs are unlikely to have similar resources and are therefore at greater risk of non-compliance, voluntarily or not [see Recommendation 5.1]. Following the Commissioner’s national consultations, the Office announced in its 2015-2020 Privacy Priorities report that it would focus some of its outreach work on SMEs and other vulnerable groups. As part of the PIPEDA Investigations Branch, the Toronto Office is involved in this strategy through its involvement with national associations and in delivering products such as the Ten Tips series.

Complaints and Investigations

Through its complaint resolution and investigations activity, the Toronto Office contributes to ensuring that private sector organizations meet their PIPEDA obligations. The Toronto Office collaborates and supports OPC in conducting a variety of investigations, including many with domestic and global implications (e.g. Google OBA, Globe24h, Employee Snooping). As has been noted elsewhere in this report, complaints against organizations headquartered in the GTA account for roughly half of all PIPEDA complaints received by OPC. Actual data for the period 2010 through 2015 (June) are presented in the table below.

Table 10: Respondent Location

Respondent Location

Respondent location	2010	2011	2012	2013	2013 (a)	2014	2014 (b)	2015 (Jan – Jun)
2013 (a): Result if 166 related complaints against same respondent (respondent located outside GTA – Bell) removed from total.
2014 (b): Result if 55 related complaints against same respondent (respondent located outside GTA – an airline) removed from total.
NB: Approximately 5-10% of complainants did not identify the location of the respondent
All	306	238	260	424	258	373	318	139
Not GTA	148	107	109	280	114	226	171	95
GTA	158	131	151	144	144	147	147	44
% GTA	51%	55%	58%	33%	55%	39%	46%	31%

The majority of OPC complaints and investigations are conducted from Gatineau. All intake and early resolution of cases takes place there. Investigation capacity within the Toronto Office has varied somewhat over the evaluation period, with contracted resources used at times to compensate for lack of permanent investigative staff. Internal stakeholders involved in investigation matters suggest that the presence of experienced, full-time investigators based in Toronto has contributed capacity to the disposition of investigation work conducted there. Several complex and longstanding files have been assigned there for resolution. These cases have benefitted from knowledge and relationships specific to the Office. As summarized earlier in Table 8 - Total Dispositions in section 4.1.1, roughly 10% of OPC cases are disposed in Toronto. This figure rises to approximately 15% when early resolution cases are excluded. Data from the 2015 PIPEDA Investigations organisational charts^{Footnote 31} indicate that Toronto investigation work is more than proportionate to resource allocations and that there is some value to be gained by investigators who have a local presence. This finding is also reflected in Issue #5 pertaining to efficiency and economy.

Public Outreach - Private sector organizations understand their obligations under PIPEDA.

PIPEDA Investigation Branch Plans as well as activity reports and more recent OPC Toronto Office plans demonstrate clearly that outreach has been a cornerstone of the Office’s activity over the course of the evaluation period. As stated in section 4.1.1, the Office has conducted an average of roughly 70 engagement events per year.^{Footnote 32} For the most part, external and internal stakeholders interviewed for this evaluation recognize the Office’s role and contribution. Roughly three-quarters (74%) of survey respondents agreed that information received or accessed from the Office helped them to understand their PIPEDA obligations.

From its inception, the Toronto Office was expected to engage stakeholders and the privacy community in the GTA with a view to promoting awareness of PIPEDA and fostering compliance with PIPEDA obligations. The evidence (from outreach plans and internal documents^{Footnote 33} and 50% of external stakeholders) indicates that over the five year period covered by this evaluation, stakeholder engagement and outreach has been the main focus of the Office, although investigations have also been conducted from the Office along with intelligence gathering and communications activity, albeit not as consistently due to resource fluctuations.

Stakeholder outreach and education has been identified by the OPC Investigations Branch Ecosystem as a “lower resource cost’ means of increasing voluntary compliance with PIPEDA. By encouraging organisations to adopt compliant practices; addressing emerging privacy issues at their inception; and building positive relationships with stakeholders to improve the complaint resolution process, the costs of protracted investigation and potentially litigation can therefore be avoided. Outreach and education is also used by the OPC to increase its internal knowledge of “on-the-ground’ privacy practices and challenges; concerns and trends that are being tracked by industry groups; and key privacy issues that Canadians consumers face.

The four pillars contained in the current OPC Toronto Office Outreach Strategy (2014-15)^{Footnote 34} reflect an ongoing approach aimed at promoting compliance by building and maintaining relationships with stakeholders. In implementing this approach, the Office has focused its energies on building relationships and engaging selectively with major enterprises and stakeholder groups in the GTA where due to their size and reach, a greater return on investment and effort might be realized.

The Toronto Office has also undertaken certain communication-related initiatives to inform both the broader public and internal OPC employees of privacy-related processes and practices. These have contributed to information sharing internally and among external stakeholders. Tangible evidence of the Offices communications and outreach contributions can be found in three initiatives lead by the Office: the 5X5 Key Topics, the Top 10 Tips initiatives, and case summaries. The 5X5 series is a means for internal OPC employees to gather information on key, emerging privacy issues facing Canadian organisations. The process intends to create a foundation of information upon which future communication materials can be developed and is expected to help with making decisions about processes within PIPEDA Investigations and understanding the operational challenges faced by privacy officers. The “Top 10 Tips’ series consists of informational material/documents about privacy-related topics of interest to external and potentially internal stakeholders. The Top 10 Tips documents are available and accessible online on the OPC website as are the case summaries of OPC findings. These products reflect an important intersection between OPC Toronto investigation and outreach roles and indicate how these roles are mutually supportive. In addition to web accessibility, case findings may, for example, form a topic of discussion at privacy meetings and conferences and help to promote awareness and understanding of OPC PIPEDA work and decisions. The case summary team is led by a Toronto Office Senior Advisor.

While the OPC website contains valuable PIPEDA related content, currently, there is no separate profile or contact information for the Toronto Office on the OPC website. This is somewhat inconsistent with the outreach role played by this Office and may be an impediment to promoting broader access within the GTA [see Recommendation 4.1]. Survey data indicate that the role and function of this Office is not as well understood as it could be. Over half (54%) of stakeholders surveyed indicated that their understanding of the role and function of the Office was moderate to limited.

For the most part, external stakeholders interviewed and surveyed for this evaluation almost unanimously want the Toronto Office to continue its outreach activity but many would like to see it do more (40%) while others (30%) would prefer a more defined, prominent or committed role. For example, it was suggested by external stakeholders (50%) that the Toronto Office could engage in more direct contact with stakeholders, invite more practitioners to events and meetings, facilitate more engagement with OPC senior leadership, provide a voice for industry on emerging issues and serve as a channel for feedback on compliance and best practice. Stakeholders also see a role for the Toronto Office in fostering more frequent communications with the privacy community (i.e. regarding cases, interpretations, decisions taken, and issues being tracked by OPC) including greater use of electronic communication and social media [see Recommendation 5.2].

Meanwhile, in addition to OPC, stakeholders frequently turn to other sources to obtain information regarding PIPEDA obligations including for example IAPP, provincial bodies, legal bulletins, the Conference Board (Council of CPOs) and Nymity.

Extent to which the Toronto Office has engaged in dialogue with OIPC

Roles

The work of the OPC takes place within a broader framework of privacy legislation that includes various provincial acts and statutes. Within the GTA in particular, this framework includes the Information and Privacy Commissioner of Ontario (OIPC). The role of OIPC is set out in three statutes: The Freedom of Information and Protection of Privacy Act, the Municipal Freedom of Information and Protection of Privacy Act and the Personal Health Information Protection Act (PHIPA). The OIPC exists independently of government to uphold and promote open government and the protection of personal privacy and has jurisdiction over public sector institutions in Ontario (provincial, municipal, colleges and universities) and the health care sector. The OPC has jurisdiction over federal public sector institutions and the private sector.

Despite the clear statutory distinction between the roles of OPC and OIPC, there has existed some role confusion about the difference between provincial and federal legislation among members of the public and private sector companies. Although the privacy professionals in larger organisations and associations are aware of the distinctions between the OPC and OIPC, interviewees noted that there is lingering confusion among SMEs and members of the public. This confusion arises due to the similar names of the two privacy bodies but also because of perceptions of overlapping jurisdiction in the health care sector in Ontario, where health information custodians have been involved in transferring personal health information across a border to another province or the United States. Indeed, one of the expectations on its creation was that the OPC Toronto Office would help reduce role confusion between OPC and OIPC.

While the areas of existing intersection between the mandates and roles of the OIPC and the OPC are quite narrow, stakeholders pointed out that it is possible to see new areas of overlap arise, especially in new and emerging technologies (e.g. CCTV and other new technologies). As a result, it was suggested by several stakeholders that there could be potential benefits from having regular meetings for updates (e.g. for policy development) and increase in collaborative efforts.

Dialogue

The OIPC and the OPC Toronto Office engage in collaborative dialogue and information sharing about privacy issues and may communicate in a variety of ways. Often this occurs through participation in local privacy related events (hosted by the Toronto Office, by the OIPC, or by a third party such as the IAPP Symposium events) or through third-party referrals where the OIPC might redirect public inquiries involving the mandate of the federal office to the Toronto Office. Additionally, the OIPC and the OPC have engaged in working level communications about matters involving general privacy issues.

In one recent example of a collaboration between the OIPC and the OPC on a privacy/investigation matter (in 2014), the OIPC investigated a privacy breach under PHIPA where information gathered raised questions about a possible breach under PIPEDA by agents/companies selling registered education savings plans. This information was communicated by the OIPC to OPC, who then conducted an investigation of those agents/companies.

Additionally, the OPC recently entered into a Memorandum of Understanding (MOU) with the OIPC regarding consultation, coordination, and information sharing about matters under PIPEDA and PHIPA. The MOU was concluded in consultation with the OPC’s Toronto and Gatineau Offices and was further used as a basis to exchange information regarding areas of overlap. With respect to using OPC produced material, the OIPC routinely monitors the publications of privacy oversight authorities, including the work of the OPC, which was reported in interviews as being excellent, timely, thorough, relevant and useful.

Extent to which the Toronto Office is supported by the appropriate management structure and practice

When it was established in 2010, the Director of the Toronto Office reported to the Investigations and Inquiry Branch (I&I) in Gatineau. Since 2011, the Director of the Toronto Office has reported to the Director General for the PIPEDA Investigations Branch.

The Director and other Toronto Office staff participate regularly in meetings, conferences and working group sessions with their Gatineau counterparts. Much of this interaction takes place by videoconference, teleconference and e-mail. Evidence from the evaluators’ site visit to the Toronto Office indicates that appropriate structures are in place to support effective communication and integration of operations between Toronto and Gatineau, including effective space, location and ICT infrastructure.

With respect to management practice, the Toronto Office has had an Outreach Strategy and Plan in place since 2012. Reporting and feedback processes and mechanisms to Gatineau include quarterly reports, event-specific reporting, summary reporting, working groups, meetings, video conferences and teleconferences. The evaluators found no evidence of a separate formalized business plan or strategy for the Office prior to 2014, although an indicator to this effect appears in the PMF for the Toronto Office. Toronto Office activities and expected outputs were, however, integrated into PIPEDA Investigation Branch plans [see Recommendations 2.1 and 2.2].

Feedback from both internal stakeholders (83%) and external stakeholders (50%) interviewed for this evaluation also points to some uncertainty regarding the governance model for the Office. For example, it is unclear in the minds of some informants whether the Office is intended to be an extension of the Gatineau office, a branch, a satellite, a sub-office or some other configuration. This lack of clarity around the roles and responsibilities has likely had an impact on the commitment to the Office and, if unresolved, might pose a constraint to its evolution. In conducting outreach, communications and stakeholder relationship building^{Footnote 35}, the role of the Office extends to PIPEDA Investigations and the broader context of privacy issues related to PIPEDA. The Office’s activities have a significant impact on other Branches within OPC including Communications, Legal Policy and Research and Corporate Services. The lack of clarity around the role of the Office poses a potential challenge with respect to ensuring cross-functional integration, information flow and accountability [see Recommendation 1.1]. Notably, some steps are already being taken to enhance integration. For example, it was reported that this year the Toronto Office will integrate its exhibit plans into the Communications Branch national exhibiting plan. The Toronto Office is also playing a role in the OPC’s SME strategy.

Extent of any unintended impacts from the Toronto Office

During the course of the evaluation, the evaluators examined evidence of unintended impacts of the Toronto Office, both positive and negative.

On the plus side, collaborative, principles-based approaches centred on relationship building and outreach save time, money and resources otherwise tied up in legal process and litigation. In addition, it was suggested that intelligence gathered from interactions with major stakeholders in the GTA helps to mitigate the risk of policy and law being seen as overly Ottawa-centric. In this regard, outreach also includes in-reach i.e. with the Office providing industry with a channel to policy makers on emerging privacy issues. External stakeholders (70%) indicate that this approach encourages open and constructive dialogue leading to high levels of trust and voluntary compliance while reducing the risk of costly adversarial investigations with damaging economic consequences. Internally, it was suggested that because it is a small Office, OPC Toronto Office does not suffer from the silo effects that sometimes afflict larger organizations. It was noted, for example that because the Office has been flexible in its use of resources, the Office has generalist expertise that can be synergistic with respect to bridging outreach, investigations, policy and communications functions.

Conversely, the positive impacts are offset somewhat by a minority view that the real power brokers and policy decision-makers for PIPEDA reside in Gatineau. In this regard, the lobbying efforts of many major corporations and industry associations are already focused on Gatineau and so a Toronto sub-office serves little net advantage over a centralized office in the NCR and is even perceived by some stakeholders as a potential deterrent to the extent that it acts as a gatekeeper. Moreover, a few internal stakeholders expressed concern that the Toronto Office may be diluting OPC efforts and resources and that other regional (non-GTA) groups and stakeholders could be comparatively underserved. Conversely, others held opposite views stating that by focusing resources on the areas of greatest need, the Office contributes to OPC’s overall performance and actually frees up resources to be spent in other regions. These views are indicative of varying levels of commitment and support for the Toronto Office within OPC.

4.2.2 Demonstration of Efficiency and Economy

Extent to which alternative or more cost-effective ways to achieve outcomes have been considered (assessment of resource utilization)

Annex 8 (Toronto Office Budgets Summarized 2010-2015) presents a summary of the budget and actual expenditure data for the Toronto Office for the fiscal years 2010-11 to 2014-15. The overall allocated budget has ranged from a high of $758,000 (2011-12) to a low of $564,819.00 in 2012-13 when the budget was reduced to help create an OPC reserve fund. The average budget for the five years was $642,830.60. The data indicate that the overall budget has not increased since the Office was established in 2010.

With regards to actual expenditures, the office underspent its budget in each year with the exception of 2012-13, where it exceeded its budget by approximately $12,000. Notably, the budget for salary and wages was underspent for each of the five years covered by the evaluation. The difference is offset to some extent by consulting and other services (which includes contracted resources – consultants, communications support, translation, and temporary help). The disparity between budgeted and actual salary and wage costs reflects the variability in permanent staffing over the past five years.^{Footnote 36}

Over the five years in question, travel expenditures have declined significantly from $95,301.14 in 2010-11 to $23,741.29 in 2014-15. Some travel expenses are allocated to headquarters as corporate expenses.

The space for the Toronto Office is managed by Public Works and Government Services Canada (PWGSC) and is therefore considered a “service without charge’. The space occupied by the OPC Toronto Office is 205 m² with an estimated value of $64,811.60 in 2014-15. The OPC Toronto Office is currently co-located with the OCOL (Office of the Commissioner of Official Languages in Toronto). Prior to OCOL’s arrival in November 2013, the OPC occupied the entire space currently split by the two departments. Co-location coincided with a renewal of the lease and represents a cost saving for government. Our review of cost of commercial office space in the GTA downtown core shows current average cost of approximately $37.80 per ft² (gross). Based on a requirement of approximately 2,000 ft², the rent would be approximately $75,600.

These financial data indicate that the Toronto Office represents and overall investment of approximately $750,000 annually, when all factors are considered.

The financial data provided by OPC and discussed above indicate that the budget has been managed prudently and that standard financial and budgetary practices are in place (in Gatineau) to ensure accountability. Costs have been saved through co-location, contracting for resources where appropriate and use of teleconferencing to minimize travel.

Human Resources

The Toronto Office has experienced considerable variability in staffing over the five-year period covered by this evaluation. Two (2) different sources were used to assess the HR situation over the period covered by the evaluation: official documentation (organisational charts) as well as a more comprehensive historical perspective provided by PIPEDA Investigations Branch.

A review of the organizational charts for the period indicates that the Director position, which is responsible for the leadership of the office, was vacant at least partially during three of the five years in question. In the last two years covered by this evaluation, the position of Senior Advisor, Public Education and Outreach, PIPEDA was also shown as being vacant^{Footnote 37}. With regard to vacancies over the five-year period, the data indicate that the vacancy rates have ranged from none (0%) in 2013 to a high of 67% in 2011. Furthermore, in the last two years, the vacancy rate has been approximately 40%.

Investigation capacity has also varied. Organisational charts indicate that occupied investigative positions ranged from zero in 2011 to one in 2012, two in 2013, three in 2014 and two in 2015. Administratively, organizational charts also indicate variability. From 2010 (the inception of the Toronto Office) to 2013, a Regional Executive Assistant reported to the Director of the Toronto Office. In 2014, the Regional Executive Assistant position was vacant and an Administrative Assistant position (occupied) was added to the organisational structure. The new position was intended to be a replacement for the Regional Executive Assistant position. In May of 2015, the Regional Executive Assistant position was removed and replaced by an additional Administrative Assistant position for a total of two Administrative Assistants positions, although one was vacant^{Footnote 38}.

It should be noted that OPC organization charts are static in nature and serve only as a rough proxy for actual staffing levels of the Toronto Office. The evaluators were advised that the OPC Toronto organization charts at times contained positions that were intentionally left vacant for future planning considerations. The evaluators were also advised that, in addition to these intentional vacancies, gaps in staffing levels arose due to difficulties in replacing resources in a timely manner. As a result, contract staff and temporary help were engaged when needed to supplement permanent staff.

The following table summarizes actual human resource data for the Toronto Office for permanent and contracted positions. These data were provided to the evaluators by the PIPEDA Investigations Branch, for the years in question.

Table 11: OPC Toronto Office Human Resources, 2010 - 2015^{Footnote 39}

OPC Toronto Office Human Resources, 2010 - 2015

Position	Level	2014-15		2013-14		2012-13		2011-12		2010-11
		Perm	Contract	Perm	Contract	Perm	Contract	Perm	Contract	Perm	Contract
*Interchange; **Branch DG operating from TO; ***Partial year (coverage during leave); ****Rotational acting
Director/DG	EX	1****		1		1		1**		1*
Reg. EA	AS				1	0.5	0.5	1		1
Admin. Assistant	AS	0.25	0.75
Senior Advisor Investigations	PM	1		1		0.25				1
Senior Advisor Outreach	IS	0.5		1		1		1			1
Senior Privacy Investigator	PM	1.5		1		1***		1 (sec)	3		1
Senior Reg. Analyst	EC	1		1		1		0.75	0.25	1
Co-op	NA								0.25
Totals	NA	5.25	0.75	5.0	1.0	4.75	0.5	4.75	3.5	4	2

Evolution of Staffing in the OPC Toronto Office

In terms of evolution, a few points are in order. In 2010/11, there was a full time director for the Toronto Office that was hired on executive interchange (also acting as DG for PIPEDA branch-wide). One AS was hired but left and a staffing process was launched to fill that vacancy. One contractor was used for investigations (pending staffing completion for investigation positions). At least one contractor was used for communication/outreach assistance (until staffing for the IS position could be completed).

In 2011/12 the DG for PIPEDA (overseeing all Branch) was operating out of the TO Office for the majority of this time (to end of Q3 before it was decided that her interchange term would not be renewed). At that point both a DG for PIPEDA was sought along with a permanent Director for Toronto. An Acting Director (from Ottawa) was in place in Toronto for the latter part of the fiscal year, and an Acting DG was in place operating out of Ottawa while a permanent DG was sought.

In 2012/13 a permanent DG for PIPEDA (overseeing the entire Branch) was appointed at the beginning of Q1 (in Ottawa). The Acting Director (from Ottawa) remained for part of Q1, before a permanent Director was appointed. Five permanent staff were in place along with one seconded investigator (PM) and two contracted investigators. Subsequently, a decision was made to phase out the use of contractors in favour of permanent staff.

In 2013/14, there were five permanent staff in place and one temporary administrator. For investigations, the remaining contractors were fully phased-out during the year and a transition was made to install permanent in-house investigative capacity. In late 2013-14 (Q4), the Director ceased working for the Office. This was a sudden departure.

In 2014/15 the Director position was filled by rotational acting opportunities within the Toronto Office (2PMs and the EC). The decision to rotate within Toronto was based on several factors including (i) stabilization of work environment/culture following a disruption and (ii) the provision of professional development and experience in Toronto at EX level. Meanwhile, with the permanent Director position vacant, the DG for PIPEDA assumed an increased presence in Toronto to alleviate pressures – e.g. more frequent visits (every 2-3 months), working out of Toronto for up to a week each visit. As of June 2015 (the end of this evaluation period), there was, in addition, a Senior Advisor, Investigations; a Senior Analyst, Stakeholder Relations, a Senior Privacy Investigator (on leave); an Administrative Assistant (1) and, a student.

In September of 2015/16, a permanent, full-time Director was hired for the Toronto Office. There are now (April 2016) six permanent staff in place, two term staff and a student. Although not working specifically for the Toronto Director, an OPC lawyer is currently working out of Toronto, as well as an Executive Assistant for the Director General, Communications Branch.

While some growing pains were to be expected in the first five years of the Toronto Office’s operation, this evolution with its accompanying staffing fluctuations have resulted in variable capacity levels which have posed a risk to the Office’s ability to fully and consistently carry out its work [see Recommendation 7.1].

5. Conclusion

The weight of evidence from this evaluation indicates that the OPC Toronto Office continues to be relevant to supporting OPC’s mission while meeting the evolving needs of stakeholders in the GTA. OPC compliance activity continues to focus on major commercial organizations, many of which are headquartered in the GTA. The presence of the Toronto Office has facilitated investigation work, and stakeholder relationships developed by Toronto staff through engagement and outreach have contributed to a collaborative and informed approach to PIPEDA compliance.

Stakeholders in the GTA continue to express support for the presence of the Office. However, some believe that the Office is not realizing its full potential and that more can be done to engage the privacy community, including reaching out to new segments, proactively communicating PIPEDA interpretations and decisions, exploring emerging issues, serving as a hub for privacy knowledge and a conduit for policy input from the commercial sector. However, these perceptions must be weighed against the efforts and initiatives that the Office has taken to engage the privacy community and may reflect a need for fostering greater awareness among stakeholders of the breadth and depth of OPC PIPEDA information resources and opportunities available to them. The findings from this evaluation support this view.

While the Office has made progress towards most of its originally intended objectives, the evidence indicates that performance of the Office over the past five years has been mixed. In terms of effectiveness, the Office’s activities and outputs have contributed substantively but unevenly to achieving outcomes in the three designated primary program areas (compliance, outreach and internal services) and to influencing private sector organizations with regard to respecting the privacy rights of individuals and protecting their personal information. The evidence indicates that performance effectiveness might have benefitted from a more defined role for the Office, including more intensive communication of the Office’s profile, role and function.

With respect to performance efficiency and economy, the financial evidence presented earlier in section 4.2.2 indicates that the Office has managed its financial resources prudently; however, the variability in staffing experienced during much of the first five years of the Office’s operation has somewhat constrained the capacity of the Office and impeded its ability to fully capitalize on the opportunities to impact the broader community of stakeholders in the GTA. Turnover at the Director level has also impacted the leadership of the Office, causing some uncertainty (both internally and externally) regarding OPC’s long-term commitment to the Office. Recent steps taken by OPC to address staffing issues and leadership of the Office should have a stabilizing effect on performance while signalling a reinforcement of OPC’s presence in the GTA.

6. Recommendations

1. Role of the Toronto Office
   • 1.1. Assess and confirm the role and responsibilities of the Toronto Office to ensure that the Office fully and effectively supports OPC priorities and cross-functional integration, information flow and accountability.
     Reference: Section 4.2.1.
2. Planning
   • 2.1. As a sub-plan component of the PIPEDA Investigations Branch Plan, develop an integrated business plan and strategy for the Toronto Office that takes into account the full scope of its activity.
     Reference 1: Section 4.1.1.
     Reference 2: Section 4.2.1.
   • 2.2. Include, in the business plan, key performance indicators for performance measurement; support these with appropriate tools to assess results (outputs and outcomes) in a timely manner.
     Reference 1: Section 3.4.
     Reference 2: Section 4.2.1.
3. Relationship Management
   • 3.1. Explore opportunities for improving contact information and relationship management to support proactive outreach efforts in the GTA (e.g. CRM System).
     Reference: Section 3.4.
4. Profile of the Toronto Office
   • 4.1. Consider ways of showcasing the presence and contributions of the Office within the OPC and with current and potential external stakeholders.
     Reference 1: Section 4.1.1.
     Reference 2: Section 4.2.1.
   • 4.2. Consider positioning the Office as a community hub for promoting PIPEDA compliance and privacy excellence in the GTA.
     Reference 1: Section 4.1.1.
     Reference 2: Section 4.1.1.
5. Communications, Outreach & Engagement
   • 5.1. Assess the depth and breadth of existing outreach in the GTA to determine more precisely the extent to which the Office is impacting stakeholders, including at-risk and vulnerable segments.
     Reference 1: Section 4.1.1.
     Reference 2: Section 4.2.1.
   • 5.2. Continue to expand reach and frequency of communications and outreach activity through the use of innovative approaches and technologies.
     Reference: Section 4.2.1.
6. Policy and Research
   • 6.1. Enhance awareness of the mechanisms to reflect stakeholder input into the policy-making process.
     Reference: Section 4.1.1.
7. Resource Capacity
   • 7.1. Reconfirm resource needs in line with a Toronto Office business strategy and continue to stabilize human resources requirements.
     Reference: Section 4.2.2.

Annex 1 – Management Response and Action Plan

Management Response and Action Plan

Evaluation Recommendations (Text taken integrally from Final Draft Report)	Management Response	Action Plan		Responsibility
	Agreed, with any relevant context Partially agreed, with rationale Disagreed, with rationale	Action Items	Approx. due date: Month/Year
1.1 Assess and confirm the role and responsibilities of the Toronto Office to ensure that the Office fully and effectively supports OPC priorities and cross-functional integration, information flow and accountability.	Agreed	The OPC will re-affirm and formalize a comprehensive Governance Model for the Toronto Office to clearly state its role, functions and integrating linkages. The Model will account for existing cross-functional collaborations and linkages, e.g. the Toronto Office’s coordinated work and shared responsibility for feeding into the OPC’s SME Strategy, exhibits/speaking engagement plans, stakeholder advisory functions etc. The Model will be developed through consultation with key internal OPC stakeholders (specifically Communications and the Legal Services, Policy, Research and Technology Analysis Branch (LSPRTA)) and broadly communicated in its full form within the organization, and described externally as appropriate.	January 31st, 2017	DG PIPEDA/Director – Development with consultation as described in Action Items SMC – OPC Senior Management Committee – approval of finalized product
2.1 As a sub-plan component of the PIPEDA Investigations Branch Plan, develop an integrated business plan and strategy for the Toronto Office that takes into account the full scope of its activity.	Accepted (Partially agreed with rationale)	The OPC already fully integrates Toronto Office activities in the PIPEDA Branch Business Plan, along with having dedicated annual outreach strategy documents. These will now be formalized and referenced in the overarching Branch Plan as a sub-component.	March 31st, 2017	DG PIPEDA/Director
2.2 Include, in the business plan, key performance indicators for performance measurement; support these with appropriate tools to assess results (outputs and outcomes) in a timely manner.	Accepted (Partially agreed with rationale)	The measures for key Outputs and Outcomes for the Toronto Office will be harmonized and further expanded in the sub and branch plans with OPC and Branch KPIs for programs/operations. Investigation performance measures already exist, so the emphasis will be on further developing indicators/measures for stakeholder relations and outreach. The performance measures for the Toronto Office will be integrated with the OPC’s new Reporting Framework to be developed on the lines of Treasury Board Policy on Results. It is noted that OPC as a whole is already assessing data gaps and renewing reporting tools (Scorecard Renewal Project).	March 31st, 2017	DG PIPEDA/Director DG Communications DG Corporate Services
3.1 Explore opportunities for improving contact information and relationship management to support proactive outreach efforts in the GTA (e.g. CRM System).	Agreed (in principle)	The OPC will explore and consider a suitable contact information and relationship management tool to track activities and assess outreach coverage. The Panorama (a planning blueprint) developed by the Toronto Office already identifies Outreach Mapping as a distinct deliverable for 2016-17. Note that this recommendation is equally of relevance and benefit to OPC as a whole, so it may be beneficial to promote an organization-wide contact/relationship support framework.	January 31st, 2017	DG PIPEDA/Director
4.1 Consider ways of showcasing the presence and contributions of the Office within the OPC and with current and potential external stakeholders.	Accepted (Partially agreed with rationale)	The Toronto Office already takes steps to keep other OPC units abreast of its activities, by information transference means including: P&P briefings, HIF, Market Intelligence Reports, Top Files, Monthly Branch Activity Summaries, ongoing participation in other branch meetings, Conference Notes etc. The OPC’s Annual, Departmental Performance and other Reports already showcase the results. The Toronto Office will explore ways in which such real-time intelligence flows can be better integrated/considered across the OPC to inform actions from both a beneficiary and benefactor perspective.	Established / Ongoing	DG PIPEDA/Director
4.2 Consider positioning the Office as a community hub for promoting PIPEDA compliance and privacy excellence in the GTA.	Agreed	The OPC will consider adding contact and/or location information, and will look at improving online information, as appropriate, regarding the Toronto office’s role in promoting compliance on the OPC website which is currently under redevelopment. The OPC will not add an online “hub” for Toronto office work/activity/outputs, given that this would not be consistent with the new approach to presenting information on the website (topics/tasks/audiences).	Sept. 30, 2016	DG PIPEDA DG Communications DG PIPEDA/Director DG Communications DG LSPRTA DG Audit
5.1 Assess the depth and breadth of existing outreach in the GTA to determine more precisely the extent to which the Office is impacting stakeholders, including at-risk and vulnerable segments.	Accepted (Partially agreed, with rationale)	The OPC will assess, as set out for the actions identified for recommendations 2.2 and 3, the methodology to consistently measure the actual reach, depth, breadth and impact of its Outreach & Stakeholder Relations (O&SR) and Compliance activities. With respect to market and at risk groups (e.g. consumer groups), the Toronto Office’s role will have to be considered/developed in support of the National Strategy in this area (overseen by Communications).	March 31st, 2017	DG PIPEDA/Director DG Communications DG Corporate Services
5.2 Continue to expand reach and frequency of communications and outreach activity through the use of innovative approaches and technologies.	Accepted (Partially agreed, with rationale)	The OPC will, within the scope of the Model, explore the potential for defined, prominent and committed roles for the Toronto Office. The Toronto Office will, building on its current plans and commitments (e.g. SME Strategy, OPC Exhibit and Speaking Engagement Plans, Market Intelligence Reports, 5x5, Fact Sheets, Tip Series, Case Summaries etc.), explore and cultivate any opportunities for: more direct OPC contact with the stakeholders; and better use of existing technology to effectively promote compliance.	March 31st, 2017	DG PIPEDA/Director DG Communications
6.1 Enhance awareness of the mechanisms to reflect stakeholder input into the policy-making process.	Accepted (Partially agreed, with rationale)	The Toronto Office conducts a range of in-reach and engagement activities (see 4.1) but through cross branch discussions (with LSPRTA) will assess and adjust these in-reach tools to ensure optimal intelligence flows into the research and policy making process. As referenced in 5.2, the OPC and Toronto Office conduct various outreach activities that, among other things, enhance stakeholder awareness to input mechanisms.	January 31st, 2017	DG PIPEDA/Director DG DG LSPRTA
7.1 Reconfirm resource needs in line with a Toronto Office business strategy and continue to stabilize human resources requirements.	Agreed	The Toronto Office is fully staffed at this time. The OPC will review the Toronto Office resource plans after the Model is finalized and Business Plan and Strategy confirmed/reset accordingly. The resource plans; primarily the HR Plan and the Budget, will be confirmed or adjusted per the capacity and business needs based on the type, scope and scale of activities, as well as the range of geographic area served by the Toronto Office.	March 31st, 2017	DG PIPEDA/Director

Annex 2 – Toronto Office Logic Model

Toronto Office Logic Model

Strategic Outcome for the OPC: The privacy rights of individuals are protected.
Ultimate Outcome for the OPC: The OPC plays a lead role in influencing Canadian federal government institutions and private sector organizations to respect the privacy rights of individuals and protect their personal information.
Program 1: Compliance Activities	Program 3: Public Outreach	Program 4: Internal Services
Expected Outcomes of the Toronto OfficeFootnote 40
Private sector organizations (particularly in the GTA/Ontario) meet their obligations under PIPEDA.	Private sector organizations (particularly in the GTA) understand their obligations under PIPEDA.	The Toronto Office is integrated to the functioning of the OPC and the Toronto Office applies sound business management practices.
Outputs
Early resolutions Investigation reports Litigation proceedings	Dialogue with OIPC Ontario Public outreach activities and materials Speeches, consultations and other communications	Business plan for Toronto Office (and Performance Agreement for Director) and report card against plan Management infrastructure (processes, practices, systems, internal controls)
Activities
Investigate PIPEDA complaints and issue reports with recommendations to private sector organizations to remedy situations, as appropriate Enforce the rights and obligations contained in PIPEDA, including court litigation when warranted	Maintain a positive dialogue with OIPC Ontario Engage with industry associations, legal community, academia, professional associations and other stakeholders to foster a better understanding of rights and obligations under PIPEDA Carry out public outreach activities in Ontario (particularly in the GTA) to foster better understanding of privacy issues and privacy obligations	Build and maintain the required organizational capacity in the Toronto Office, including functional support (IM/IT, Finance, HR) Establish a plan, aligned with OPC corporate priorities, to deliver on the expected outcomes and report performance against the plan Apply internal controls to manage operations in Toronto in line with corporate standards and Management Accountability Framework (MAF) expectations

Annex 3 – Toronto Office Performance Measurement Framework

Evaluation Methodology

The following table outlines the methodology for the evaluation of the Toronto Office: evaluation issues, questions and corresponding indicators and data source/collection methods, as well as the timing for the collection of the evaluation data.

Question Framework for the Evaluation of the Toronto Office

Evaluation Issue	Evaluation Question	Indicator	Data Source / Collection Method	Timing of Collection
RELEVANCE
Issue 1: Need for the Toronto Office (i.e., Assessment of extent to which the Toronto Office addresses a demonstrable need and is responsive to the needs of Canadians)	Does the Toronto Office still address a genuine need for the OPC to deliver its mandate?	Proportion of PIPEDA respondents that are based or headquartered in the Greater Toronto Area.	Review of OPC statistics, and Statistics Canada OPC’s Annual Reports	Statistics collected continually; review at time of evaluation
		Evidence as to whether the context related to the Toronto Office has changed	Interviews/group interviews with OPC officials	During the evaluation/ data gathering phase
	Are there impacts to the OPC of “not’ having a Toronto Office?	Perceived impact on awareness (media and public opportunities for stakeholder relations) Perceived impact on service quality (timeliness; access; accuracy) Number of direct consultations on current and emerging business and technology practices with industry and other privacy stakeholders. Number and type of face-to-face interactions with respondents from the Toronto Office Number and type of complaint dispositions	OPC’s Annual Reports Review of 2005 Business Case I and II. Review of Communications files for event attendance (for speeches, participation to public events, other) Interviews/group interviews with OPC officials and stakeholders Survey with stakeholders Review of OPC statistics on early resolution	Notes to files and statistics maintained continually; review and interviews at time of evaluation
Issue 2: Alignment with OPC priorities and Government direction (i.e., Assessment of linkages between objectives of the Toronto Office and the OPC priorities)	Is the Toronto Office in line with OPC priorities?	Evidence of alignment between the Toronto Office and the OPC corporate priorities and key commitments	OPC’s Annual Reports Analysis of OPC Reports on Plans and Priorities to determine how/where well the Toronto Office fits	Review at time of evaluation
Issue 3: Alignment with federal roles and responsibilities	Is the Toronto Office aligned with the OPC’s federal role and mandate?	Evidence of the Alignment of the Toronto Office with OPC’s federal role and mandate	Review of documentation about federal direction and analysis against key objectives of the Toronto Office	Review at time of evaluation
PERFORMANCE (Effectiveness, Efficiency and Economy)
Issue 4: Achievement of Expected Outcomes (i.e., Assessment of progress toward expected outcomes with reference to performance targets and reach, design of the Toronto Office, including the linkage and contribution of outputs to outcomes) The OPC plays a lead role in influencing Canadian federal government institutions and private sector organizations to respect the rights of individuals and protect their private information. [LM 1.0] Private sector organizations particularly in the GTA/Ontario meet their obligations under PIPEDA [LM 2.0] Private sector organizations particularly in the GTA/Ontario understand their obligations under PIPEDA [LM 3.0] The Toronto Office is integrated to the functioning of the OPC and applies sound business management principles [LM 4.0]	Has the Toronto Office started to achieve its expected outcomes? i.e.: Make use of “early resolution” for those complaints where appropriate [LM 2.1] More face-to-face mediation and site visits with respondents and investigations [LM 2.1] Better management of turnaround time to close PIPEDA investigations. [LM 2.2] Use of new, innovative investigative solutions/ methodology [LM 2.2, 2.3] More direct contacts with industry and privacy stakeholders to discuss emerging privacy issues and practices. [LM 3.2, 3.3] Outreach activities in GTA/Southern Ontario [LM 3.2, 3.3]	Total number of complaints sent to OPC Number and duration of investigations facilitated by the Toronto Office Proportion of complaints (where appropriate) that were handled by considering early resolution procedures in Toronto Number of face-to-face mediations with respondents from the Toronto Office Number of face-to-face site visits with respondents from the Toronto Office Proportion of face-to-face investigations in Toronto (since the start of the Toronto Office compared to prior) Proportion of intakes, inquiries and complaints from GTA Number of direct meetings/consultations/discussions with industry and private sector stakeholders Number of public events actually attended Evidence of increased visibility of the OPC with privacy stakeholders (industry associations, CPOs, academics, etc.) in GTA/Southern Ontario Evidence of net benefits of proximity in value equation (e.g. business intelligence)	OPC’s Annual Reports Unit specific statistics on intake, inquiries and complaints Statistics on face-to-face versus desk investigations Statistics on turnaround time for investigations Review of sample of investigation files to compare methodology used prior to 2010 Statistics on demand for OPC services and on event attendance Interviews/group interviews with OPC officials and stakeholders (series of executive interviews with stakeholders in GTA re: awareness of OPC, of their corporate obligations) Survey with stakeholders Case studies	Statistics collected continually; review at time of evaluation Comparative analysis at time of evaluation During the evaluation/ data gathering phase
	Has more dialogue with OIPC Ontario materialized with the Toronto Office? [LM 3.1]	Evidence of interaction with OIPC Ontario over issues of joint concern	Review of files relating to OIPC Ontario in Toronto Office Interviews/group interviews with OPC and OIPC Ontario officials	Review at time of evaluation
	Is the Toronto Office set up and supported with the appropriate management structure (IM/IT, HR, Finance, planning, etc.)? [LM 4.1, 4.2]	Evidence of the clarity of roles, responsibilities and reporting relationships of all positions affected by the change Existence of a business plan for Toronto Office, which is aligned to OPC corporate priorities (yes or no); Evidence of effective delivery of the plan Evidence of HR practices (e.g. Values and ethics principles applied through staffing actions, performance appraisals completed yearly) Evidence of integration of the Director, PIPEDA to the SMC team (meetings and discussions)	Interviews/group interviews with OPC officials Review of Toronto Office files against OPC Reports on Plans and Priorities Review of Toronto Office files against HR standards; interview with Director, HR Review of SMC minutes Review of minutes from the PNP meetings and meetings with the Commissioner	File review and interviews at time of evaluation
	Have there been any unwanted impacts from the Toronto Office? [LM all]	Evidence of indirect impacts on the OPC Evidence of indirect impacts on the Toronto Office	Interviews/group interviews with OPC officials Review of SMC minutes and key files from Toronto Office	File review and interviews at time of evaluation
Issue 5: Demonstration of Efficiency and Economy (i.e., Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes)	Are there alternative, more cost-effective ways to achieve the same outcomes, and if so, have these been given due consideration?	Evidence of the cost-benefit of the Toronto Office Extent to which the (relevant) demands on the Toronto Office have been met with allocated financial and human resources? Evidence that the Toronto Office budget allocated in a manner to maximize results Evidence of net benefits of proximity in value equation (e.g. business intelligence) [Note: a new office necessarily generates costs that were not there before; therefore, it is key to consider the “benefits” and estimate their value in terms of outcomes even if the benefits cannot translate in cost savings.] Statistics on PIPEDA complaint backlog files (through using new creative tools and approaches to resolution)	Review of relevant financial reports/data on the Toronto Office and reports on accomplishments (e.g. report cards, progress reports) Interview with OPC officials	File review and interviews at time of evaluation

Annex 4 – Question Bank (inventory for reference)

The following question bank was developed in order to facilitate the development of the various data collection guides. Questions have been cross-referenced to both the OPC Toronto Office Logic Model and the Performance Measurement Framework. Please note that not all questions have been used in the respective guides and some questions have been consolidated for interview duration consideration.

Relevance Issues

Issue #1: Need for the Toronto Office

Need

1. To what extent does the Toronto Office specifically play a role in influencing private sector organizations (audiences and stakeholders) in the GTA/Ontario with respect to PIPEDA obligations? Does any confusion exist? [LM 1.0]
2. To what extent do private sector organizations (audiences and stakeholders) in the GTA/Ontario comply with their PIPEDA obligations? [LM 2.0]
3. To what extent do private sector organizations (audiences and stakeholders) in the GTA/Ontario understand their PIPEDA obligations? [LM 3.0]
4. What changes have you noted in recent years that have influenced the work (outreach and investigation) of the Toronto Office? Have the demands and expectations changed and, if so, how? [1.1]

Impact

5. How has the presence of the Toronto Office impacted the quality ofOPC operations in theGTA in terms of:
   1. timeliness/responsiveness?
   2. access?
   3. accuracy? [LM 4.0]
6. How has the presence of the Toronto Office impacted awareness and understanding of PIPEDA obligations among target audiences and stakeholders? [LM 3.0]
7. How has the presence of the Toronto Office impacted compliance with PIPEDA among audiences and stakeholders in the GTA? [LM 2.0]
8. In your opinion, what would be the net impact if the Toronto Office did not exist? [1.2]

Issue #2: Alignment with OPC priorities and Government Direction

1. What mechanisms are in place to ensure coordination and alignment between OPC HQ and the Toronto Office? How well do these mechanisms work in practice? [LM 4.0, 4.1, 4.2]
2. What gaps or opportunities do you see with respect to coordinating, aligning or integrating the work of OPC HQ and the Toronto Office? [2.1]
3. How would you characterize the working model that exists between OPC HQ and the Toronto Office? (e.g. microcosm, extension, satellite, branch, sub-office etc.) [LM 4.0, 4.1, 4.2] [2.1]

Issue #3: Alignment with federal roles and responsibilities

• What mechanisms are in place to ensure coordination and alignment between the OPC's federal role and mandate and the Toronto Office? How well do these mechanisms work in practice? [Issue 3 TBA]
• What gaps or opportunities do you see with respect to coordinating, aligning or integrating the federal role of the OPC and the Toronto Office? [Issue 3 TBA]
• To what extent are the roles of the OPC, the Toronto Office and the Ontario IPC clearly defined, communicated and understood by target audiences and stakeholders? [Issue 3 TBA]

Performance Issues

Issue #4: Achievement of Expected Outcomes

1. How has the Toronto Office made use of early resolution processes to handle complaints? [LM 2.1]
2. How has the Toronto Office made use of mediation (and site visits) with respondents /investigations? [LM 2.1]
3. What has been the trend in the type and volume of inquiries, complaints and investigations over the past 5 years? How do type and volume compare to national trend? [LM 2.2]
4. What has been the trend in turnaround time for responding to inquiries, complaints and investigations? How does this compare to national trends? [LM 2.2]
5. What new or innovative solutions or approaches have been used by the Toronto Office to achieve outcomes? [LM 2.2, 2.3]
6. What evidence exists that private sector organizations in the GTA are meeting their obligations under PIPEDA? [LM 2.0]
7. How has the Toronto Office engaged in dialogue with the OIPC? [LM 3.1, 4.2]
8. What public outreach activity and material has the Toronto Office been involved in over the past 5years? [LM 3.2, LM 3.3]
9. What contact has there been with industry and private stakeholders to discuss emerging privacy issues and practices? What has been the impact of this activity on awareness and understanding of PIPEDA obligations? [LM 3.2, LM 3.3]
10. What has been the impact on relationships with stakeholders in the GTA/Ontario? [LM 2.0, LM 3.0]

Issue #5: Demonstration of Efficiency and Economy

1. How clearly defined and understood are the roles and responsibilities in the Toronto Office? [LM 4.0, 4.3]
2. How well aligned and integrated are these roles and responsibilities with OPC overall? [LM 4.0, 4.3]
3. Does the Toronto Office have the appropriate plans, structure, processes, tools and controls to efficiently and effectively manage its operations? [LM 4.1, LM 4.2, 4.3]
4. What management practices exist to ensure that the Toronto Office manages its resources (HR, financial, IT etc.) efficiently and effectively? [LM 4.2, 4.3]
5. What "net" benefits can be attributed to the existence of the Toronto Office? [5.1]
6. Are there alternative, more cost effective ways to achieve the same benefits and have these been given due consideration? [5.1]
7. Are there any areas of duplication or overlap between the Toronto Office and OPC HQ? [5.1]

Annex 5 – Interview Guide for Internal Staff

Need and Impact

1. What contextual changes have you noted in recent years that have influenced the work of the Toronto Office? Have the demands and expectations changed and, if so, how?
2. How has the presence of the Toronto Office impacted the quality of OPC operations in the GTA in terms of timeliness/responsiveness, access, accuracy?
3. How do you gauge the impact (awareness, understanding and compliance) of the Toronto Office of PIPEDA obligations among target audiences and stakeholders?
4. In your opinion, what would be the net impact if the Toronto Office did not exist?

Alignment

5. What mechanisms are in place to ensure coordination and alignment between OPC HQ and the Toronto Office? How well do these mechanisms work in practice? [Probe: any gaps or opportunities?]
6. To what extent are the roles of the OPC, the Toronto Office and the Ontario IPC clearly defined, communicated and understood by target audiences and stakeholders? How well aligned and integrated are these roles and responsibilities with OPC overall?

Achievement of Expected Outcomes

7. How has the Toronto Office made use of early resolution and mediation processes to handle complaints?
8. What new or innovative solutions or approaches have been used by the Toronto Office to achieve outcomes?
9. What evidence exists to demonstrate that private sector organizations in the GTA are meeting their obligations under PIPEDA?
10. What contact has there been with industry and private stakeholders to discuss emerging privacy issues and practices? What has been the impact on relationships with stakeholders in the GTA/Ontario?

Efficiency and Economy

11. What systems, tools and management practices exist to ensure that the Toronto Office manages its resources (HR, financial, IT etc.) efficiently and effectively?
12. What “net” benefits can be attributed to the existence of the Toronto Office, including benefits attributed to proximity?
13. Are there alternative, more cost effective ways to achieve the same benefits and have these been given due consideration?
14. Are there any areas of duplication or overlap between the Toronto Office and OPCHQ?

Annex 6 – Interview Guide for External Stakeholders

1. Could you describe the nature of your interaction/engagement with the OPC’s Toronto office?
2. What challenges, if any, have you faced with respect to understanding and complying with PIPEDA requirements?
3. How has the presence of the OPC Toronto Office helped to facilitate your understanding and compliance with PIPEDA obligations?
4. How would you rate the quality of your interaction/engagement with the Toronto Office in terms of timeliness/responsiveness, access, accuracy? [Scale of 1 to 5]
5. In your opinion, what would be the impact if the Toronto Office did not exist?
6. How clear is the role of the OPC relative to that of the Ontario IPC or other privacy bodies?
7. How do you inform yourself and others in your organization about your obligations under PIPEDA? Do you have access to the information resources you need? What gaps or opportunities do you see with respect to better informing yourself/your organization regarding PIPEDA obligations?
8. Have you/your organization had an opportunity to access or review PIPEDA information or material from the Toronto Office? If so, how would you rate the quality of that information/material? [Scale of 1 to 5]
9. How has the presence of the OPC Toronto Office benefitted you/your organization?
10. What efforts do you/your organization take to inform yourself of emerging privacy issues and practices?
11. Do you have any final comments or observations regarding the OPC Toronto Office?

Annex 7 – Document/Data/File Review Evidence

Relevance Issues

Issue #1: Need for the Toronto Office

Need:

1. Evidence that the role of OPC generally and the Toronto Office specifically is understood by audiences and stakeholders in the GTA and if any confusion exists.
2. Evidence that audiences and stakeholders in the GTA understand and comply with their PIPEDA obligations.
3. Evidence of changes in recent years that have impacted the work of the Toronto Office. Evidence of changing demands and expectations.

Impact:

4. Evidence of how the presence of the Toronto Office has impacted the quality of OPC operations in the GTA in terms of timeliness/responsiveness, access, accuracy.
5. Evidence of how the presence of the Toronto Office has impacted awareness and understanding of PIPEDA obligations among target audiences and stakeholders.
6. Evidence of how the presence of the Toronto Office has impacted compliance with PIPEDA among audiences and stakeholders in the GTA.
7. Evidence of the net impact if the Toronto Office did not exist.

Issue #2: Alignment with OPC priorities and Government Direction

1. Evidence of mechanisms in place to ensure coordination and alignment between OPC HQ and the Toronto Office and how well these mechanisms work in practice.
2. Evidence of gaps or opportunities with respect to coordinating, aligning or integrating the work of OPC HQ and the Toronto Office.
3. Illustration of the working model that exists between OPC HQ and the Toronto Office. (e.g. microcosm, extension, satellite, branch, sub-office etc.)

Issue #3: Alignment with federal roles and responsibilities

1. Evidence of mechanisms in place to ensure coordination and alignment between the OPC’s federal role and mandate and the Toronto Office and how well these mechanisms work in practice.
2. Evidence of gaps or opportunities with respect to coordinating, aligning or integrating the federal role of the OPC and the Toronto Office.
3. Illustration of the working model/assumptions that exist between the OPC’s role federally and the Toronto Office.
4. Evidence as to what extent the roles of the OPC, the Toronto Office and the Ontario IPC are clearly defined, communicated and understood by target audiences and stakeholders.

Performance Issues

Issue #4: Achievement of Expected Outcomes

1. Evidence (data) that the Toronto Office made use of early resolution processes to handle complaints.
2. Evidence (data) that the Toronto Office made use of mediation and site visits with respondents/investigations.
3. Evidence (data) of the trend in the type and volume of inquiries, complaints and investigations over the past 5 years and how do the type and volume compare to national trend.
4. Evidence (data) of the trend in turnaround time for responding to inquiries, complaints and investigations? How does this compare to national trends?
5. Evidence that new or innovative solutions or approaches have been used by the Toronto Office to achieve outcomes.
6. Evidence (data) that private sector organizations in the GTA are meeting their obligations under PIPEDA.
7. Evidence that the Toronto Office has engaged in dialogue with the OPC.
8. Evidence (data) of public outreach activity and material that the Toronto Office been involved in over the past 5years.
9. Evidence (data) of contact with industry and private stakeholders to discuss emerging privacy issues and practices and of the impact of this activity on awareness and understanding of PIPEDA obligations.
10. Evidence of the impact on relationships with stakeholders in the GTA.

Issue #5: Demonstration of Efficiency and Economy

1. Evidence of how clearly defined and understood the roles and responsibilities in the Toronto Office are.
2. Evidence of how well aligned and integrated these roles and responsibilities with OPC are overall.
3. Evidence that the Toronto Office has the appropriate plans, systems, tools, controls and infrastructure to efficiently and effectively manage its operations.
4. Evidence that sound management practices exist to ensure that the Toronto Office manages its resources (HR, financial, IT etc.) efficiently and effectively.
5. Evidence (financial/ROI) of "net" benefits that can be attributed to the existence of the Toronto Office.
6. Evidence of alternative, more cost effective ways to achieve the same benefits and whether these have been given due consideration.
7. Evidence of any areas of duplication or overlap between the Toronto Office and OPC HQ.

Annex 8 – Online Survey Questionnaire

1. Could you indicate the nature of your interaction/engagement with the OPC's Toronto Office?

• Visited website
• Attended meeting
• Participated in event/conference
• Submitted question/inquiry
• Consulted OPC Toronto Office staff
• Used OPC information/material
• Complaint/investigation resolution
• Other, please specify…

2. On a scale of 1 to 5, where 1 means strongly disagree and 5 means strongly agree, please indicate the extent to which you agree or disagree with the following statements regarding your interaction/engagement with OPC's Toronto Office:

1. The information I received/accessed was relevant
2. The information I received/accessed was useful
3. The information I received/accessed was timely
4. The information i received/assessed was accurate
5. The information I received/accessed helped me understand my PIPEDA obligations
6. The information I received/accessed made me more aware of my PIPEDA obligations
7. The information I received/accessed helped me comply with my PIPEDA obligations
8. The information I received/accessed made me more aware of privacy issues
9. I feel more informed about my PIPEDA obligations
10. I have access to the information I need to inform myself about my PIPEDA obligations
11. The OPC Toronto Office has helped me/my organisation to comply with my PIPEDA obligations

Please provide any additional comments or clarifications regarding your selections.

3. a) On a scale of 1-5, please rate the impact of the OPC Toronto Office with respect to facilitating compliance with PIPEDA obligations.

• 1 (little impact)
• 2 (some impact)
• 3 (moderate impact)
• 4 (high impact)
• 5 (very satisfied)
• Don't know/ N/A

3. b) On a scale of 1-5, how satisfied would you say you are with the service provided by the OPC Toronto Office?

• 1 (very dissatisfied)
• 2 (dissatisfied)
• 3 (neither satisfied nor dissatisfied)
• 4 (satisfied)
• 5 (very satisfied)
• Don't know/N/A

Please clarify/comment on your selections.

 

4. a) On a scale of 1-5, how well do you understand the role and the function of the OPC Toronto Office?

• 1 (limited)
• 2 (somewhat)
• 3 (moderate)
• 4 (well)
• 5 (very well)

Please explain your answer.

4. b) Do you have any suggestions for how the OPC Toronto Office could be more effective in its operations?

5. a) How do you typically inform yourself and others in your organisation about your PIPEDA obligations?
Please indicate any information sources typically used (I.e. conferences, website, publications, organisations, etc.).

5. b) What opportunities do you see, if any, to better inform yourself/your organisation regarding your PIPEDA obligations?

6. Other than OPC, what sources do you/your organisation use, if any, to inform yourself of existing or emerging privacy issues and practices?
Please specify.

7. Do you have any final comments or observations regarding the OPC Toronto Office?

Annex 9 – Toronto Office Budgets Summarized 2010-2015

Toronto Office Budgets Summarized 2010-2011

Description	2010-2011
	Allocated Budget	Actual
Salaries & Wages	$ 261,920.00	$ 109,277.30
Suppl. Personnel Costs		$ 123,436.92
Allowances & Other Benefits
Accounts Receivable
Salary TOTAL	$ 261,920.00	$ 232,714.22
Acq. Informatic Eqpm/Parts		$ 1,232.33
Advertising
Exposition & Related Services
Business Services
Food & Beverages		$ 31.09
Informatics
Management Consulting		$ 27,900.00
Other services	$ 399,134.00	$ 104,452.70
Communication Prof. Services		$ 12,894.00
Temp. Help		$ 18,067.65
Misc. Goods & Products		$ 8,834.81
Miscellaneous Expenditures		$ 538.05
Paper & Publications
Postage, Freight & Cartage		$ 167.73
Printing and Imaging Services		$ 101.65
Protection Services		$ 1,186.56
Public Servant Travel
Relocation		$ 95,301.14
Rental - Informatics Equipment
Rental-Mach/Office Furn.		$ 2,040.00
Repair Machinery/Equipment
Special Fees & Services		$ 2,569.90
Telecommunications services		$ 23,679.93
Training & Education
Operating & Maintenance TOTAL	$ 399,134.00	$ 298,997.54
Salary + Op & Maint TOTAL	$ 661,054.00	$ 531,711.76
Differences Budgeted vs. Actual
Difference Salary TOTAL	$ 29,205.78
Difference Op & Maint. TOTAL	$ 100,136.46
Difference Overall TOTAL	$ 129,342.24

Toronto Office Budgets Summarized 2011-2012

Description	2011-2012
	Allocated Budget	Actual
Salaries & Wages	$ 425,851.00	$ 316,615.92
Suppl. Personnel Costs		$ 94,819.91
Allowances & Other Benefits
Accounts Receivable
Salary TOTAL	$ 425,851.00	$ 411,435.83
Acq. Informatic Eqpm/Parts
Advertising	$ 20,000.00	$ 3,151.68
Exposition & Related Services
Business Services
Food & Beverages
Informatics
Management Consulting	$ 80,000.00
Other services	$ 140,000.00	$ 235,335.57
Communication Prof. Services		$ 4,200.00
Temp. Help		$ 10.00
Misc. Goods & Products
Miscellaneous Expenditures	$ 15,000.00
Paper & Publications
Postage, Freight & Cartage
Printing and Imaging Services
Protection Services
Public Servant Travel
Relocation	$ 70,700.00	$ 69,863.42
Rental - Informatics Equipment
Rental-Mach/Office Furn.
Repair Machinery/Equipment
Special Fees & Services	$ 6,000.00	$ 665.10
Telecommunications services		$ 601.52
Training & Education		$ 130.00
Operating & Maintenance TOTAL	$ 331,700.00	$ 313,957.29
Salary + Op & Maint TOTAL	$ 757,551.00	$ 725,393.12
Differences Budgeted vs. Actual
Difference Salary TOTAL	$ 14,415.17
Difference Op & Maint. TOTAL	$ 17,742.71
Difference Overall TOTAL	$ 32,157.88

Toronto Office Budgets Summarized 2012-2013

Description	2012-2013
	Allocated Budget	Actual
Salaries & Wages	$ 357,567.00	$ 305,850.19
Suppl. Personnel Costs		$ 66,785.96
Allowances & Other Benefits
Accounts Receivable
Salary TOTAL	$ 357,567.00	$ 372,636.15
Acq. Informatic Eqpm/Parts
Advertising	$ 7,500.00	$ 11,726.88
Exposition & Related Services	$ 500.00
Business Services	$ 6,000.00	$ 950.84
Food & Beverages
Informatics	$ 1,200.00
Management Consulting
Other services	$ 125,597.00	$ 121,534.50
Communication Prof. Services
Temp. Help	$ 7,000.00	$ 9,402.95
Misc. Goods & Products	$ 2,000.00	$ 2,404.10
Miscellaneous Expenditures
Paper & Publications		$ 247.70
Postage, Freight & Cartage	$ 950.00	$ 577.65
Printing and Imaging Services	$ 2,320.00	$ 2,154.72
Protection Services	$ 1,105.00	$ 853.02
Public Servant Travel
Relocation	$ 17,500.00	$ 23,573.02
Rental - Informatics Equipment	$ 4,680.00
Rental-Mach/Office Furn.		$ 4,080.00
Repair Machinery/Equipment		$ 20.00
Special Fees & Services		$ 1,134.07
Telecommunications services	$ 30,900.00	$ 24,875.24
Training & Education		$ 188.00
Operating & Maintenance TOTAL	$ 207,252.00	$ 203,722.69
Salary + Op & Maint TOTAL	$ 564,819.00	$ 576,358.84
Differences Budgeted vs. Actual
Difference Salary TOTAL	-$ 15,069.15
Difference Op & Maint. TOTAL	$ 3,529.31
Difference Overall TOTAL	-$ 11,539.84

Toronto Office Budgets Summarized 2013-2014

Description	2013-2014
	Allocated Budget	Actual
Salaries & Wages	$ 514,819.00	$ 479,003.07
Suppl. Personnel Costs
Allowances & Other Benefits
Accounts Receivable
Salary TOTAL	$ 514,819.00	$ 514,819.00
Acq. Informatic Eqpm/Parts
Advertising		$ 12,947.00
Exposition & Related Services	$ 10,140.00
Business Services	$ 2,000.00	$ 834.34
Food & Beverages		$ 13.92
Informatics
Management Consulting
Other services	$ 16,000.00	$ 9,916.48
Communication Prof. Services
Temp. Help	$ 15,000.00	$ 42,489.40
Misc. Goods & Products	$ 3,201.00	$ 4,254.89
Miscellaneous Expenditures
Paper & Publications		>
Postage, Freight & Cartage	$ 600.00	$ 558.97
Printing and Imaging Services	$ 1,700.00	$ 1,126.52
Protection Services	$ 710.00	$ 911.50
Public Servant Travel
Relocation	$ 20,000.00	$ 26,219.55
Rental - Informatics Equipment
Rental-Mach/Office Furn.	$ 4,080.00	$ 2,380.00
Repair Machinery/Equipment
Special Fees & Services	$ 9,000.00	$ 333.64
Telecommunications services	$ 24,000.00	$ 22,730.41
Training & Education
Operating & Maintenance TOTAL	$ 106,431.00	$ 124,716.62
Salary + Op & Maint TOTAL	$ 621,250.00	$ 603,719.69
Differences Budgeted vs. Actual
Difference Salary TOTAL	$ 35,815.93
Difference Op & Maint. TOTAL	-$ 18,285.62
Difference Overall TOTAL	$ 17,530.31

Toronto Office Budgets Summarized 2014-2015

Description	2014-2015
	Allocated Budget	Actual
Salaries & Wages	$ 558,968.00	$ 513,341.82
Suppl. Personnel Costs
Allowances & Other Benefits	$ 10,000.00	$ 2,050.50
Accounts Receivable		$ 17,553.17
Salary TOTAL	$ 568,968.00	$ 532,945.49
Acq. Informatic Eqpm/Parts
Advertising		$ 13,760.95
Exposition & Related Services	$ 15,000.00
Business Services		$ 982.08
Food & Beverages
Informatics
Management Consulting
Other services
Communication Prof. Services
Temp. Help	$ 800.00	$ 988.13
Misc. Goods & Products		$ 3,343.92
Miscellaneous Expenditures	$ 1,085.00
Paper & Publications		>
Postage, Freight & Cartage	$ 600.00	$ 1,549.67
Printing and Imaging Services
Protection Services	$ 915.00	$ 1,171.00
Public Servant Travel	$ 10,111.00	$ 23,741.29
Relocation
Rental - Informatics Equipment
Rental-Mach/Office Furn.
Repair Machinery/Equipment
Special Fees & Services		$ 269.46
Telecommunications services	$ 12,000.00	$ 20,545.86
Training & Education
Operating & Maintenance TOTAL	$ 40,511.00	$ 66,352.36
Salary + Op & Maint TOTAL	$ 609,479.00	$ 599,297.85
Differences Budgeted vs. Actual
Difference Salary TOTAL	$ 36,022.51
Difference Op & Maint. TOTAL	-$ 25,841.36
Difference Overall TOTAL	$ 10,181.15

Annex 10 – Bibliography

Independent Summative Evaluation of the OPC’s Toronto Office

Office of the Privacy Commission of Canada

American Marketing Association Toronto (2015).

Canadians Bankers Association (2016).

Chartered Professional Accountants Canada (2016).

Consumers Council of Canada (2016).

Digital Advertising Alliance of Canada (2016).

Direct Marketing Association of Canada (2016).

Information and Privacy Commissioner of Ontario (2016). PHIPA.

Institute of Communication Agencies (2016).

International Associations Privacy Professionals (2016).

Justice Laws Website (2016). Personal Information Protection and Electronic Documents Act,

Ministry of Economic Development, Employment and Infrastructure (2016).

Office of the Information and Privacy Commissioner of Ontario (2016).

Office of the Privacy Commissioner of Canada (2010). Annual Report to Parliament 2010.

Office of the Privacy Commissioner of Canada (2010). I & I 2010-2011 Branch Business Plans.

Office of the Privacy Commissioner of Canada (2011). Annual Report to Parliament 2011: Privacy for Everyone.

Office of the Privacy Commissioner of Canada (2011). Privacy and the Public: It’s all about them! A New Way of Working at the OPC.

Office of the Privacy Commissioner of Canada (2011). Privacy on the Go: 10 Workplace Tips for Protecting Personal Information on Mobile Devices.

Office of the Privacy Commissioner of Canada (2012). Annual Report to Parliament 2012: Privacy and Your Reputation. Who Shapes Your Identity Online?

Office of the Privacy Commissioner of Canada (2012). Government of Canada Position Description – Executive Group.

Office of the Privacy Commissioner of Canada (2012). OPC Scorecard – March 2012.

Office of the Privacy Commissioner of Canada (2012). OPC Toronto Outreach Strategy.

Office of the Privacy Commissioner of Canada (2012). PIPEDA Investigations 2012-2013 Branch Business Plan.

Office of the Privacy Commissioner of Canada (2012). Ten Things HR Professionals need to know about Privacy.

Office of the Privacy Commissioner of Canada (2012). Toronto Quarterly Outreach Report – Q2 2012-13.

Office of the Privacy Commissioner of Canada (2012). Toronto Quarterly Outreach Report – Q3 2012-13.

Office of the Privacy Commissioner of Canada (2013). Annual Report to Parliament 2013: Online Privacy Transparency.

Office of the Privacy Commissioner of Canada (2013). OPC Monthly Scorecard – March 2013.

Office of the Privacy Commissioner of Canada (2013). OPC Toronto Outreach Strategy.

Office of the Privacy Commissioner of Canada (2013). PIPEDA Investigations Branch 2013-2014 Branch Business Plan.

Office of the Privacy Commissioner of Canada (2013). Ten Tips for a Better Online Privacy Policy and Improved Privacy Practice Transparency.

Office of the Privacy Commissioner of Canada (2013). Toronto Quarterly Outreach Report – Q1 2013-14.

Office of the Privacy Commissioner of Canada (2013). Toronto Quarterly Outreach Report – Q2 2013-14.

Office of the Privacy Commissioner of Canada (2013). Toronto Quarterly Outreach Report – Q3 2013-14.

Office of the Privacy Commissioner of Canada (2013). Toronto Quarterly Outreach Report – Q4 2012-13.

Office of the Privacy Commissioner of Canada (2014). Annual Report to Parliament 2014: Privacy Protection, A Global Affair.

Office of the Privacy Commissioner of Canada (2014). Departmental Performance Report 2013-14.

Office of the Privacy Commissioner of Canada (2014). OPC Monthly Scorecard – March 2014.

Office of the Privacy Commissioner of Canada (2014). OPC Toronto Outreach Strategy.

Office of the Privacy Commissioner of Canada (2014). PIPEDA Investigations Branch 2014-2015 Branch Business Plan.

Office of the Privacy Commissioner of Canada (2014). Privacy Legislation in Canada.

Office of the Privacy Commissioner of Canada (2014). Ten Tips for Communicating Privacy Practices to your App’s Users.

Office of the Privacy Commissioner of Canada (2014). Ten Tips for Reducing the Likelihood of a Privacy Breach.

Office of the Privacy Commissioner of Canada (2014). Toronto Quarterly Outreach Report – Q1 2014-15.

Office of the Privacy Commissioner of Canada (2014). Toronto Quarterly Outreach Report – Q2 2014-15.

Office of the Privacy Commissioner of Canada (2014). Toronto Quarterly Outreach Report – Q3 2014-15.

Office of the Privacy Commissioner of Canada (2014). Toronto Quarterly Outreach Report – Q4 2013-14.

Office of the Privacy Commissioner of Canada (2015). Anti-spam law’s changes to Canadian federal privacy law: A guide for businesses doing e-marketing.

Office of the Privacy Commissioner of Canada (2015). Anti-spam law’s changes to Canadian federal privacy law: Helpful tips for businesses doing e-marketing.

Office of the Privacy Commissioner of Canada (2015). Collecting from Kids? Ten Tips for Services Aimed at Children and Youth.

Office of the Privacy Commissioner of Canada (2015). Findings under the Personal Information Protection and Electronic Documents Act (PIPEDA), #2015-001 to #2015-009.

Office of the Privacy Commissioner of Canada (2015). Nomination Narrative – Case Summary Team.

Office of the Privacy Commissioner of Canada (2015). OPC Monthly Scorecard – March 2015.

Office of the Privacy Commissioner of Canada (2015). OPC Office of the Privacy Commissioner of Canada Organizational Chart

Office of the Privacy Commissioner of Canada (2015). OPC Office of the Privacy Commissioner of Canada – PIPEDA Investigations Organizational Chart.

Office of the Privacy Commissioner of Canada (2015). OPC Toronto Outreach Plan – 2015-16.

Office of the Privacy Commissioner of Canada (2015). Organisational Structure.

Office of the Privacy Commissioner of Canada (2015). PIPEDA Compliance Eco-System.

Office of the Privacy Commissioner of Canada (2015). PIPEDA Funding Pressure 2015-16 ER, Request for Additional Funding for 2015-16.

Office of the Privacy Commissioner of Canada (2015). PIPEDA Investigations Branch 2015-2016 Branch Business Plan.

Office of the Privacy Commissioner of Canada (2015). Position Description – Executive Group (Toronto Regional Office)

Office of the Privacy Commissioner of Canada (2015). Report on Plans and Priorities 2015-16.

Office of the Privacy Commissioner of Canada (2015). Ten tips for avoiding complaints to the OPC.

Office of the Privacy Commissioner of Canada (2015). The OPC Strategic Privacy Priorities 2015-2020: Mapping a course for greater protection.

Office of the Privacy Commissioner of Canada (2015). Toronto Quarterly Outreach Report – Q1 2015-16.

Office of the Privacy Commissioner of Canada (2015). Toronto Quarterly Outreach Report – Q4 2014-15.

Office of the Privacy Commissioner of Canada (2015). Toronto Quarterly Outreach Report – Q1 2015-16.

Office of the Privacy Commissioner of Canada (2015). Total Investigations Completed from 2008 to 2015.

Office of the Privacy Commissioner of Canada (2016). About Us.

Office of the Privacy Commissioner of Canada (2016). Annual reports to Parliament.

Office of the Privacy Commissioner of Canada (2016). Commissioner’s Findings.

Office of the Privacy Commissioner of Canada (2016). Complaint Volumes, GTA

Office of the Privacy Commissioner of Canada (2016). Contact Us.

Office of the Privacy Commissioner of Canada (2016). Delivered speeches and Requested Speeches GTA/Other

Office of the Privacy Commissioner of Canada (2016). Federal Privacy Laws.

Office of the Privacy Commissioner of Canada (2016). FreeBalance Financial Management System – Free Balance Report – Summary Selection Criteria 2014-15.

Office of the Privacy Commissioner of Canada (2016). FreeBalance Financial Management System – Free Balance Report – Summary Selection Criteria 2013-14.

Office of the Privacy Commissioner of Canada (2016). FreeBalance Financial Management System – Free Balance Report – Summary Selection Criteria 2012-13.

Office of the Privacy Commissioner of Canada (2016). FreeBalance Financial Management System – Free Balance Report – Summary Selection Criteria 2011-12.

Office of the Privacy Commissioner of Canada (2016). Frequently Asked Questions.

Office of the Privacy Commissioner of Canada (2016). GX OPCOIC-PROD 10.7.5 – General Budget Management Report.

Office of the Privacy Commissioner of Canada (2016). Parliamentary Activities.

Office of the Privacy Commissioner of Canada (2016). PIPEDA Inventory by Age.

Office of the Privacy Commissioner of Canada (2016). Report on the Personal Information Protection and Electronic Documents Act.

Office of the Privacy Commissioner of Canada (2016). Toronto Staffing and HR History – Contextual and Narrative Background.

OPC Toronto Office (2015). OPC Toronto Office Site Visits.

Public Works and Government Services Canada (2016).

Retail Council of Canada (2016).

The Interactive Advertising Bureau of Canada (IAB Canada) (2014).