← All articlesArcana Recovery blog

Responsible Use of AI in Behavioral and Physical Health

Artificial intelligence can support documentation, screening, clinical decision-making, and professional development. It can also introduce errors, bias, privacy risks, and unsafe recommendations. This guide explains how clinicians can evaluate AI while keeping professional judgment and client welfare at the center of care.

What artificial intelligence is

Artificial intelligence describes computer systems designed to perform tasks that normally require human judgment, pattern recognition, communication, or decision-making.

AI does not think or understand a client the way a clinician does. Most systems identify patterns and calculate likely outputs based on the data, instructions, and examples they were given.

Let’s break down a few common types of AI, how they work, and how they may be used in healthcare.

Rules-based systems

Rules-based systems follow instructions programmed in advance. When specific information meets a set condition, the system produces a predetermined response. For example, a screening tool may calculate a patient’s score and create a clinical alert when the score reaches a certain level. These systems can support consistent processes, but they cannot make decisions outside the rules they were given.

Machine learning

Machine learning systems analyze historical data to identify patterns and predict outcomes from new information. In healthcare, a system might review past patient records to identify factors connected to hospitalization or treatment dropout. These predictions can help clinicians identify possible risks, but they do not explain why something will happen or guarantee it will.

Deep learning

Deep learning is a more complex form of machine learning that uses layers of connected calculations called neural networks. It can process large amounts of information and identify patterns that may be difficult for a person to detect. In healthcare, deep learning is often used to analyze medical images, recognize speech, or detect patterns within complex clinical data.

Natural language processing

Natural language processing helps computers analyze and organize spoken or written language. In healthcare, it may identify diagnoses, medications, symptoms, or other important information in case notes. It can also support transcription and documentation review. However, the system may misunderstand clinical language, context, or the meaning behind a person’s words.

Generative AI

Generative AI creates new content based on patterns learned from existing information. It can produce text, images, audio, and other forms of content. In healthcare, it may help draft progress notes, treatment plans, discharge summaries, or educational materials. Review anything it creates because the information may be incomplete, inaccurate, or inappropriate for the individual receiving care.

Large language models

Large language models are a type of generative AI designed to work with language. They generate responses by predicting the next most likely word based on patterns learned during training. This allows them to create conversational responses and professional-sounding clinical documentation. It does not mean they understand the client, know whether the information is true, or can replace clinical judgment.

A large language model can produce information that sounds confident and credible but is inaccurate or completely fabricated. This is commonly called an AI hallucination. In a healthcare setting, hallucinations can become especially dangerous if inaccurate information is added to a case note, treatment plan, diagnosis, or clinical recommendation without being carefully reviewed.

Current uses in healthcare

Behavioral health

• Clinical documentation and session transcription
• Drafting SOAP, DAP, BIRP, and progress notes
• Drafting treatment plans and discharge summaries
• Depression, anxiety, suicide, and substance use screening
• Appointment reminders and patient engagement
• Measurement-based care
• Digital therapeutic programs
• Mental health and recovery chatbots
• Predicting treatment dropout, relapse, or crisis risk
• Reviewing case records for quality and compliance
• Translating patient education materials
• Matching clients with providers or levels of care

Physical health

• Radiology and medical-image interpretation
• Cancer detection and pathology
• Cardiac monitoring
• Predicting clinical deterioration
• Medication safety
• Remote patient monitoring
• Clinical decision support
• Prior authorization and claims review
• Appointment scheduling and care coordination
• Patient messaging and documentation
• Robotic and computer-assisted procedures

By 2024, the FDA had authorized hundreds of AI-enabled medical devices, with radiology as the largest category. Many support screening, diagnosis, or triage. 

What current research tells us

The evidence is promising but not settled.

A 2025 randomized trial involving 106 adults found that a purpose-built therapeutic chatbot was associated with reduced symptoms of depression, anxiety, and eating-disorder concerns compared with a waitlist. The study did not establish that the system was equivalent to clinician-delivered therapy. 

A separate 2025 study found that general-purpose language models sometimes expressed mental health stigma, reinforced delusional thinking, or responded inappropriately to high-risk situations. The researchers concluded that these systems should not replace mental health professionals. 

Early studies of ambient AI scribes suggest that they may reduce documentation burden. However, researchers have also identified transcription errors, incorrect medications, missing information, and statements that were never made. Clinical review remains necessary. 

The responsible conclusion is that AI is neither inherently safe nor unsafe. Safety depends on the specific tool, population, task, safeguards, clinical oversight, and consequences of an error.

Real world ethical issues

Ethical problems can appear at different points in care. The risk depends on what the AI is asked to do, what information it receives, and whether a qualified professional reviews the result.

Case notes

AI-generated case notes may include information that was never discussed or leave out details that affect safety and care. For example, a note might state that a client denied suicidal thoughts even though suicide risk was never assessed. Once an error enters the clinical record, it can influence later treatment decisions.

Treatment plans

AI can produce treatment plans that sound appropriate while remaining generic or disconnected from the client. A recommendation for cognitive behavioral therapy may fail to consider culture, trauma history, disability, identity, readiness, or personal preference. Treatment planning must remain collaborative and individualized.

Diagnosis

AI may encourage diagnostic anchoring when an early suggestion shapes the rest of the assessment. A clinician could accept an AI-generated diagnosis before gathering enough information or considering other explanations. AI may help organize information, but it should not replace an independent clinical assessment.

Interventions

A chatbot or generative AI tool may recommend an intervention that is unsafe, unsupported, or inappropriate for the client's circumstances. Examples include suggesting that a client stop medication without consulting a prescriber or confront an abusive person. Clinical recommendations require professional review and an understanding of the client's safety needs.

Crisis support

AI may miss indirect language, changes in tone, cultural expressions, or other warning signs that a trained clinician might recognize. A system could respond casually to an ambiguous statement about suicide or self-harm. Crisis tools need clear escalation procedures and cannot be treated as a substitute for emergency or clinical support.

Risk prediction

Risk prediction tools can create false reassurance or unfairly classify a person when the available data is incomplete or biased. A client may be labeled low risk because important information was never documented. Predictions should be considered alongside direct assessment, clinical judgment, and current circumstances.

Session recording

Recording or processing a session without meaningful consent can violate privacy and damage trust. Clients should understand that AI is being used, what information will be collected, where it will be stored, who may access it, and whether it will be used to train a model. Consent should be informed and voluntary whenever a choice is available.

Documentation review

Clinicians may develop automation bias when they trust an AI-generated note because it appears polished or complete. Signing a note without checking it can place inaccurate information into the record and transfer the consequences to the client. The clinician remains responsible for verifying what is documented.

Payer decisions

Algorithms used by insurers or healthcare organizations may prioritize cost reduction over client need. An automated system could influence whether treatment is considered medically necessary without fully accounting for clinical complexity. These decisions require transparency, a meaningful appeal process, and human review.

Bias

AI can reproduce bias found in clinical records and historical healthcare decisions.

Bias may enter through

• Underrepresentation of Black, LGBTQIA+, disabled, rural, and non-English-speaking populations
• Historical differences in diagnosis and treatment
• Clinician language contained in case notes
• Using healthcare spending as a substitute for healthcare need
• Differences in insurance coverage and access
• Labels created through policing, child welfare, or criminal justice systems
• Cultural expressions being interpreted as symptoms
• Missing data caused by unequal access to care

A system can appear neutral while relying on variables such as ZIP code, missed appointments, insurance status, or previous service utilization that reflect structural inequality.

Exploitation

Behavioral health data may be especially valuable because it includes trauma histories, substance use, relationships, sexual behavior, legal involvement, and crisis disclosures.

Exploitation may occur when

• Patient information is used to train commercial systems without meaningful consent
• A company encourages emotional dependence to increase engagement
• Sensitive information is sold or used for advertising
• AI-generated recommendations benefit the payer or provider more than the client
• Clients are offered AI because human care is unavailable or considered too expensive
• Organizations test unproven systems on underserved populations
• Frameworks for ethical analysis
• Biomedical ethics

Each use of AI can be examined through six established principles.

Autonomy
Autonomy asks whether the client understands how AI is being used and has a meaningful choice whenever possible. Clinicians should consider whether consent is informed, whether declining will affect access to care, and whether the client can ask questions or request another option.

Beneficence
Beneficence focuses on whether the AI tool is likely to help the client. A tool should have a clear clinical purpose and credible evidence supporting its use for the task, setting, and population involved.

Nonmaleficence
Nonmaleficence means avoiding preventable harm. Clinicians should consider what could happen if the system makes an error, omits important information, exposes private data, or gives an inappropriate response.

Justice
Justice asks whether the system performs fairly across different populations. An AI tool should be evaluated for unequal error rates, barriers to access, and outcomes that may disadvantage people because of race, language, disability, gender identity, location, or income.

Fidelity
Fidelity focuses on trust within the therapeutic relationship. AI should not interfere with the clinician's responsibilities, reduce the client to a data point, or weaken the honesty and reliability expected in care.

Veracity
Veracity requires clinicians and organizations to be truthful about what an AI system does, what information it uses, and where its limits are. Clients should not be led to believe that an automated tool has human understanding or clinical authority that it does not possess.

World Health Organization principles

The World Health Organization identifies six principles for AI in health.

• Protect human autonomy
• Promote well-being, safety, and the public interest
• Ensure transparency and explainability
• Foster responsibility and accountability
• Ensure inclusiveness and equity
• Promote responsive and sustainable AI

NIST AI Risk Management Framework

The NIST AI Risk Management Framework gives organizations a practical way to examine AI throughout its use. It organizes the work into four connected functions.

Govern
Govern means establishing who is responsible for AI decisions and oversight. An organization should define acceptable uses, approval requirements, staff responsibilities, documentation standards, and when a tool must not be used.

Map
Map means understanding the context in which the AI will operate. This includes identifying the intended users, client population, purpose, available data, affected workflows, and possible harms if the system fails.

Measure
Measure means testing how the AI performs before and during use. Organizations should examine accuracy, bias, privacy, reliability, security, and clinical performance instead of relying only on a vendor's claims.

Manage
Manage means acting on the risks that have been identified. This may include adding safeguards, limiting access, monitoring performance, reporting incidents, correcting problems, or stopping use when the system becomes unsafe.

The current AI RMF is voluntary and is being revised. Its four functions remain the published operational model.

Professional codes of ethics

Professional codes do not all address AI in the same level of detail. However, their existing duties still apply when technology is used in assessment, documentation, treatment planning, communication, or decision-making.

NASW

The NASW Code of Ethics centers client dignity, self-determination, social justice, competence, informed consent, confidentiality, professional boundaries, and accurate records. Its technology standards also make clear that social workers remain responsible for protecting clients when services or information are delivered through technology.

ACA

The ACA Code of Ethics requires counselors to protect client welfare, collaborate in treatment planning, maintain accurate records, obtain informed consent, protect confidentiality, and practice within their competence. When technology is involved, counselors must also consider security, accessibility, appropriateness, and whether the tool is effective for the intended purpose.

APA

The APA Ethics Code is grounded in beneficence, nonmaleficence, integrity, justice, and respect for people's rights and dignity. Psychologists must remain competent, protect confidentiality, use appropriate assessment methods, maintain accurate records, and take responsibility for the professional services they provide, including work supported by AI.

AAMFT

The AAMFT Code of Ethics requires marriage and family therapists to protect client welfare, obtain informed consent, maintain confidentiality, work within their competence, keep accurate records, and manage professional boundaries. These duties continue when technology assists with communication, documentation, or service delivery.

NBCC

The NBCC Code of Ethics emphasizes competence, nondiscrimination, informed consent, privacy, secure records, collaborative treatment planning, appropriate assessment, and protection from harm. A counselor using AI must still evaluate whether the tool is appropriate and protect the client from foreseeable risks.

AMHCA

AMHCA provides direct guidance for counselors using AI. Clinicians should disclose when AI is involved, obtain the client's consent, evaluate whether the tool is safe and appropriate, and remain accountable for assessments, treatment plans, interventions, and other clinical work produced with its assistance.

NAADAC

The NAADAC Code addresses electronic informed consent, privacy, encryption, recording consent, and disclosure of AI use. It also identifies risks involving autonomy, misdiagnosis, surveillance, algorithmic bias, and unfairness. Addiction professionals remain responsible for reviewing AI-assisted work and protecting the client's rights.

AMA

The AMA's guidance on augmented intelligence emphasizes patient welfare, competent medical care, confidentiality, transparency, clinical validation, human oversight, equity, privacy, security, and accountability. AI should support physicians and care teams without removing human responsibility for medical decisions.

ANA

The ANA Code of Ethics emphasizes human dignity, patient advocacy, privacy, safety, health equity, professional accountability, and ethical working environments. Nurses using AI remain responsible for questioning unsafe outputs and protecting the patient's interests when technology influences care.

The clearest AI-specific guidance currently comes from AMHCA and NAADAC.

AMHCA states that clinicians must disclose AI involvement, obtain consent, and remain responsible for evaluations, treatment plans, interventions, and assessments. AMHCA AI addendum

The 2025 NAADAC Code specifically identifies informed consent, autonomy, privacy, transparency, misdiagnosis, abandonment, surveillance, algorithmic bias, and unfairness as AI concerns. It also requires informed consent before AI is used to record or process live clinical interactions. 

ACA requires accurate documentation, collaborative treatment planning, technology competence, informed consent, encryption, appropriate use, and evaluation of effectiveness. Its official website currently identifies the 2014 code as the governing version while revisions are completed. 

HIPAA and AI

HIPAA does not prohibit using AI. It regulates how covered entities and business associates use, disclose, store, and protect protected health information.

Before entering PHI into an AI system, an organization should determine

• Whether the vendor is acting as a business associate
• Whether a valid business associate agreement is in place
• What information the system collects
• Where information is stored and processed
• Whether prompts or recordings are retained
• Whether information is used to train models
• Which subcontractors can access the information
• How access, deletion, security incidents, and breach notification are handled
• Whether the organization completed and documented a security risk analysis
• Whether the information shared is limited to what is reasonably necessary

HHS states that a cloud provider that creates, receives, maintains, or transmits electronic PHI is generally a business associate, even when the provider cannot view encrypted information. A BAA and appropriate risk analysis are required. 

A vendor calling its product “HIPAA compliant” does not make every use of that product compliant. The healthcare organization must still configure, govern, and use it appropriately.

Addiction treatment providers may also be subject to 42 CFR Part 2, which provides additional protections for substance use disorder records.

Central practice standard

AI may assist with clinical work, but it should not become the unverified author of the clinical record or the unaccountable decision-maker in a person’s care.

The clinician remains responsible for what is assessed, documented, recommended, signed, and communicated.

Putting ethical AI into practice

AI can support clinical work when it is used carefully, but usefulness alone does not make a tool ethical or safe. Clinicians and organizations must understand how the system works, examine the evidence behind it, protect confidentiality, look for bias, obtain meaningful consent, and verify every clinical output. Biomedical ethics, WHO principles, the NIST framework, professional codes, and privacy standards all point to the same conclusion. AI should support human care without replacing professional judgment, accountability, or the relationship at the center of treatment.

About Arcana Recovery

Arcana Recovery is a Pennsylvania-based behavioral health and recovery organization working to improve what happens before, during, and after treatment. Through education, recovery resources, continuity-focused services, and consent-aware technology, Arcana helps individuals and behavioral health organizations strengthen engagement, care coordination, discharge support, and long-term connection. Our work is person-centered, trauma-informed, and culturally responsive, grounded in the belief that recovery is a relationship, not a transaction.

Learn more at arcanarecovery.com

Build stronger continuity of care with Arcana.

Schedule Demo