AI-Assisted Qualitative Analysis for Program Evaluation

Planning & Decision-making · Inform

What it collects

Operational data

Anonymized data

Published literature, program reports, and non-protected qualitative materials gathered through interviews, surveys, and literature reviews used for program evaluation.

Run by: Canadian Heritage (PCH)
Where: No fixed location
Kept: Not stated by the Helpful Places.
Shared with: Accountable organization
Your copy: You cannot see the data it holds about you. What you can do

What it is for

Planning & Decision-making

Inform

This system uses ChatGPT Teams to help Canadian Heritage employees analyze large amounts of qualitative information — such as interview notes, survey responses, and literature reviews — to identify themes and patterns for program evaluation. It also provides plain language editing support for reports. The tool operates on non-personal, non-protected information only, and is currently in a pilot phase to gather best practices before wider adoption.

What it collects and what happens to it

Data taken in

Operational data

Anonymized data

Published literature, program reports, and non-protected qualitative materials gathered through interviews, surveys, and literature reviews used for program evaluation.

Processing

Language Models

ChatGPT Teams (OpenAI) — a large language model platform used for thematic analysis of qualitative text and plain language report editing.

What it does

Understanding (Semantic AI)

Human decides

Reads and extracts meaning from qualitative texts (interview notes, survey responses, reports) to surface themes and patterns for human evaluators to review and act upon.

Creating (Generative AI)

Human decides

Generates plain language rewrites and editing suggestions for evaluation reports; all generated content is reviewed and approved by GC employees before use.

Outputs

A recommendation or prediction

Anonymized data

Thematic summaries and pattern identifications presented to GC employees as advisory outputs to inform program evaluation findings; also includes edited plain language report drafts for human review.

Run by

Canadian Heritage (PCH)

Canadian Heritage (PCH) is the Government of Canada department deploying this ChatGPT Teams pilot to support internal program evaluation functions.

Government of Canada AI Register — 2526-PCH-004

Built by

OpenAI

OpenAI provides the ChatGPT Teams platform licensed by Canadian Heritage for this pilot.

Government of Canada AI Register — 2526-PCH-004

Kept for

Not stated by the Helpful Places.

Shared with

Not available to me

Members of the public do not have access to the outputs or data processed by this system, which is used exclusively by GC employees for internal evaluation purposes. AI use is not disclosed to external users.

Available to the accountable organization

Outputs (thematic analyses and edited reports) are available to Canadian Heritage GC employees involved in program evaluation.

Stored

Not stated by the Helpful Places.

How to read the colours

Can it identify you?

Anonymized data: Data about people with the link to who is broken. Stripped of identifiers, blurred, aggregated, or noised so this system can’t reasonably tie a record back to an individual.
Pseudonymous data: Each person’s data is tied to a token (hash, ID, template) that lets this system recognise the same person across events, but the token itself doesn’t reveal a name. Reidentification is possible with extra information.
Identifiable data: The data either contains a direct identifier (name, address, account name, recognisable face or voice, plate number) or carries a token this system uses to look up legal identity during processing.

Who completes the loop?

Human decides: This mode suggests; a person decides what to do next. The AI is always advisory — a human is in the loop on every decision. Example: a triage tool ranks cases for a clinician who chooses which to see first.
Human executes: This mode decides; a person carries out the result. Example: an optimizer plans the day’s trash-collection routes, and drivers run them.
Autonomous: This mode decides and acts on its own. No person reviews each decision or carries out the resulting action.

Definitions from the DTPR standard. Amber is about your data, violet about who decides. The fuller the shape and the deeper the colour, the more identifying the data or the less a person is involved.

What you can do

Ask about this system

Questions go to the Helpful Places, not the vendor.

Your rights

Right to Be Informed of AI UseThe register indicates that AI use is not currently disclosed to users (disclosed: N). As this system is used internally by GC employees and does not affect members of the public directly, no public notice mechanism is described. Employees using the tool are expected to be aware of the pilot through internal communications.
Right to Algorithmic TransparencyThis DTPR disclosure provides transparency about the system's purpose, capabilities, and data practices. The system is in a pilot phase; best practices and lessons learned are being gathered to inform longer-term standards within Canadian Heritage.

Risks and safeguards

Reputational harmLLM-generated thematic summaries or edited text may contain inaccuracies, mischaracterize qualitative findings, or introduce unintended biases that could affect evaluation conclusions and program reputations.Safeguard: All outputs are reviewed and approved by GC employees before use; the pilot is focused on internal, non-binding analysis only and does not directly determine program outcomes.
Societal & cultural harmUse of a US-based commercial LLM for government program evaluation may introduce cultural or linguistic biases that affect how Canadian cultural programs are assessed.Safeguard: The pilot is exploratory and restricted to non-protected information; lessons learned are being gathered to inform longer-term, locally appropriate solutions and standards within Canadian Heritage.