Ethical AI in Therapy: A Framework for Clinicians (and How We Apply It)

The ethics of AI in therapy are not optional and not abstract. Every clinician who uses a language model in their workflow is making concrete choices about privacy, consent, scope of practice, and clinical judgment — usually without realizing they are making them. This piece lays out the framework we use, the lines we will not cross, and the way we apply both inside TherapistAssist.

If you have not read the companion piece, AI for Therapists: A Clinical Guide, start there. This piece assumes you already know what a large language model is and where it tends to fail.

Four principles, in order of precedence

When two principles conflict, the higher one wins. We have ordered them deliberately.

1. Privacy is non-negotiable

Client information is the most sensitive data your practice holds. AI tools, by their architecture, are designed to ingest text and surface patterns — exactly the wrong properties for PHI handling unless aggressively controlled. The non-negotiables:

• No PHI in consumer-tier tools. ChatGPT, Claude, Gemini, Copilot in their default consumer plans are not appropriate vessels for any client information, paraphrased or not. The model does not have to remember your text for the privacy obligation to be breached — it is breached at the moment of transmission.
• A BAA or no PHI. If you want to use AI on identifiable client data, you need a signed Business Associate Agreement with the vendor. The BAA is the bare minimum, not the gold standard.
• De-identification at the door. Initials only. No DOB, no exact age when small, no employer, no school, no neighborhood, no rare diagnosis combined with a small population, no quoted client speech long enough to be searchable.
• No transcript ingestion. Session recordings and verbatim transcripts are the densest PHI artifacts in mental health. They do not belong in any AI tool that is not purpose-built, BAA-covered, audited, and explicitly consented to.

2. Informed consent must be ongoing, not buried

If AI plays any meaningful role in a client's care — drafting their treatment plan, generating their homework, summarizing their intake — they have a right to know in plain language. The right does not survive a single mention in a 14-page intake packet.

The minimum practice:

• A short, plain-language paragraph in the treatment agreement describing what AI is used for and what it is not used for.
• A verbal mention at intake, in the same breath as fee structure and confidentiality limits.
• An opt-out path for documentation tasks that does not penalize the client (e.g., slower turnaround on certain paperwork is fine; refusing care is not).
• Re-consent if your AI use expands materially after initial intake.

3. Scope of practice does not transfer to the model

The model cannot diagnose. The model cannot conduct risk assessment. The model cannot decide what intervention this client at this moment needs. These are scope-of-practice decisions that belong to a licensed clinician with the case in mind, and they remain there regardless of how good the model gets.

Concretely, the bright lines we hold:

• Diagnosis. Models generate differentials worth considering; clinicians diagnose.
• Risk assessment. Suicide, homicide, child welfare, and intimate partner violence assessments are clinician-led, clinician-documented, in clinician language, every time.
• Safety planning. Stanley-Brown style safety plans are co-created with the client and the clinician. Pre-generated templates from a model bypass the protective work of the conversation itself.
• Treatment recommendations to non-clinicians. A model output advising "you should consider EMDR" sent directly to a client without clinician filtering is practicing therapy without a license, even if the model wrote it.

4. The model's bias is your bias unless you correct it

Large language models are trained on internet-scale text, which over-represents dominant-culture, English-language, North-Atlantic, college-educated, neurotypical, cis-het perspectives. Without active correction, that bias propagates into every output: the worksheets it writes, the conceptualizations it offers, the language it defaults to.

Correction looks like:

• Specifying cultural and identity context in prompts when relevant. "This is a Cambodian-American client in their fifties whose primary frame for distress is somatic" produces a meaningfully different draft than "this is a client presenting with depression."
• Reviewing every client-facing output for cultural fit. Reading level, idiom, examples, metaphors, religious framing, family-system assumptions. The model defaults will fit some clients and quietly miss others.
• Watching for diagnostic skew. Models over-diagnose certain conditions in certain demographics in line with the training data's biases. Hold your own clinical judgment as the corrective.
• Not letting AI standardize away difference. The efficiency gain of templated language is real; the cost is a kind of clinical flattening if you let it run unchecked.

A decision tree for clinicians

When you are deciding whether to use AI for a specific task, run it through these three questions in order.

Does this task involve identifiable client information?

• Yes → BAA-covered, purpose-built tool only. If you do not have one, do the task without AI.
• No → continue.

Does this task require licensed clinical judgment that the client is paying you for?

• Yes → AI may draft, you must author. The clinical decision and the words on the page must be yours after review.
• No → AI may assist more freely (generic templates, psychoeducation drafts, formatting).

Would the client be surprised to learn AI was involved?

• Yes → either update your informed consent before using AI for this task, or do not use AI for this task.
• No → proceed, but audit your own use periodically.

How we apply this in TherapistAssist

We built TherapistAssist to be opinionated about exactly these principles. The product-level rules that follow from them:

No PII anywhere, ever. The app does not ask for, store, or display client names, emails, phone numbers, addresses, dates of birth, emergency contacts, employers, or any identifying combination. Clients in the workspace are identified by initials only. This is enforced at the schema level, not as a guideline.

De-identified by design in AI features. The AI Homework Builder generates worksheets from clinician-described situations using role terms ("the client", "the partner") rather than identifying detail. The prompt template does not have a field for a client name because the clinical workflow does not require one.

No training on clinician or client content. We use vendors with enterprise data-handling agreements and contractual no-train guarantees, and we re-verify quarterly. Your usage does not become anyone else's model.

Clinician-completed risk tools. Safety planning, lethal-means counseling, and intake items that touch suicide or homicide are completed by clinicians inside the workspace, not generated by AI. The scaffolding is ours; the clinical content is yours.

Transparent AI surface area. Every tool that uses AI is labeled. Clinicians always see what was AI-generated and what was clinician-authored. There is no hidden assistance happening behind progress notes or treatment plans.

Opinionated defaults, editable everything. The app suggests; the clinician decides. Every AI-generated artifact is fully editable before it leaves the workspace.

What we do not do, and will not do

A short list of things we have been asked for and declined.

• No session transcription or summarization from audio. The risk-reward calculus does not work in our judgment: the privacy exposure of session audio is too high relative to the documentation lift.
• No "AI therapist" companion features. The relational work is the work. We are not interested in products that let clients talk to a model instead of a clinician.
• No predictive diagnostic outputs. We will not ship a feature that takes intake data and spits out a diagnosis or risk score. Differentials are for clinicians to hold.
• No insurance-facing AI advocacy. Generating prior-auth letters that lean on language designed to maximize approval against medical-necessity standards is a path we are not interested in walking.

A working clinician's checklist

Print this. Tape it next to your monitor.

• [ ] No PHI in any consumer AI tool, ever.
• [ ] BAA on file for any AI tool that touches identifiable data.
• [ ] Informed consent covers AI in plain language, with opt-out.
• [ ] AI drafts, I author. Every output is reviewed before it lands anywhere that matters.
• [ ] Diagnosis, risk assessment, safety planning — clinician-led, full stop.
• [ ] Prompts are neutral. Outputs are checked for cultural fit. Bias is named when I see it.
• [ ] Workflows that use AI are documented in my policies.
• [ ] Audit my own AI-assisted work monthly for drift.

Where to go from here

If you want the more concrete version of how we apply this, the AI for Therapists guide is the companion. If you want to see the philosophy in product form, the AI Homework Builder is the cleanest example — opinionated about scope, transparent about what it does, designed so the clinical decision stays with the clinician.

The framework is not a constraint on what AI can do for our field. It is the condition that makes the help worth accepting.