It is projected that $11.1 billion in venture capital will be invested in AI healthcare companies in 2024.
For therapists, that means many shiny new tools will be coming to market, with large sales budgets directed at you.
To be honest, in allied health professions, we are not accustomed to this level of investment—and we’re not used to fielding these types of sales pitches.
That’s why I recorded this podcast course with Dennis Morrison, PhD, where we look at some of the latest research about ambient AI documentation tools (and identify the most important questions therapists need to ask about any new tools that are presented to them).
This course was recorded with occupational therapists in mind, but I hope other allied health professionals (including physical therapy, speech therapy, and behavioral health providers) find this information helpful—and that they are asking similar questions!
We need to band together to make sure AI tools that actually help us and our patients are the ones being built and adopted.
With the right questions and input from therapists, I believe these tools will transform our work for the better.
Okay, here are the AI evaluation questions I took away from my conversation with Dennis.
1. What Type of AI Technology is Being Used?
When it comes to AI for documentation, tools can be categorized into 3 main buckets:
- Voice transcription with editing capabilities: This type of tool turns spoken words into text—which the program then cleans up (similar to what Grammarly can do).
- Voice interpretation that generates the beginning of your note: This type of tool goes a step further to analyze and interpret spoken words in the context of that particular conversation (e.g., by creating a progress note based on session audio and then parsing it into a SOAP note format).
- Voice interpretation + note generating assistance + clinical decision support: This type of tool adds clinical inferences and insights to a spoken transcription that are not explicitly found anywhere in the verbiage that was spoken. This could involve the AI:
- Analyzing information from past sessions (longitudinal data)
- Analyzing and applying information from similar sessions by other therapists
- Incorporating practice guidelines and treatment support recommendations
Most tools on the market fall into categories 1 and 2, but we should expect to see more offerings at level 3 in the next year or so.
2. How Was the AI’s Large Language Model (LLM) Developed?
AI documentation tools leverage large language models (LLMs) to generate content, and you need to know how the AI’s large language model was built.
🛑 If the large language model is SOLELY a general-purpose model like ChatGPT, this should be a red flag for you. 84% of ChatGPT’s LLM was scraped from the internet, and as a result, it carries all the bias found on the internet.
✅ Ideally, the tool’s LLM is a purpose-built model specifically trained on clinical data that is relevant to your practice. This means the output it generates is more likely to be accurate and useful in a clinical setting. These companies may still leverage ChatGPT technology in the development process, but it should not be the only source of training information. Furthermore, the data you input into the tool should never be exposed to the public-facing version of ChatGPT. This is the approach taken by behavioral health AI tool Eleos Health. Learn more about why they believe healthcare AI needs to be built specifically for your discipline here.
3. What Privacy and Compliance Measures Does the Tool Have in Place?
Privacy is one of the main concerns when using these tools. In the United States, it should be a given that any healthcare AI tool is HIPAA compliant.
Beyond that, the degree to which patient information is stored can vary widely. That’s why it’s important to ask:
- How patient data is stored
- Who has access to it
- How long it is retained
- Whether you can request that it be deleted
- When it gets deleted (and whether it is ever totally deleted)
- What components of the data the customer owns (versus the components the AI tool owns)
- Who authorizes third parties to access the data
- How the tool anonymizes data
- What measures are in place to protect patient information from breaches (and, who is liable in the event of a breach)
4. Does the AI Tool Require Client Consent?
Find out if the use of the tool requires explicit consent from your clients. If the tool records or stores any form of patient interaction, it’s essential to inform the client and obtain their consent.
I would also inquire about what—if any—support the company provides to help educate clients about the tool and obtain their consent.
5. How Well Does the Tool Integrate with My Current EHR System?
🛑 If you have to copy and paste documentation into your EHR system, walk away. This means extra work for you, and a copy-paste workflow is prone to errors.
Integration with your existing Electronic Health Record (EHR) system is crucial for a seamless workflow. Ask if the AI tool can connect directly with your EHR, how this integration works, and whether it requires additional effort or coding from your IT team.
6. Can the AI Tool be Customized to Fit My Documentation Style?
Every therapist has a unique documentation style and voice. Sophisticated AI products should accurately capture not only the content of what a therapist writes, but also the writer’s voice. Inquire whether the AI tool can be tailored to fit your specific needs—for example, by including verbatim patient goals or generating summaries in particular formats like DAP or SOAP notes.
Customization ensures the AI tool enhances, rather than disrupts, your workflow.
7. What are the Potential Efficiency Gains?
One of the main promises of AI documentation tools is increased efficiency and accuracy—especially compared to cutting and pasting note content. A good AI tool should generate note suggestions that are unique to this—and only this—session.
Ask for specific data on how the tool has improved documentation time in real-world therapy settings—and whether there are any studies or pilot programs that demonstrate its effectiveness. For example, I like how this behavioral health tool is actively researching its effectiveness.
8. How Does the Tool Impact Patient Engagement?
Look for tools that allow you to remain fully present with your clients during sessions, minimizing task-switching and administrative distractions. Ask about any feedback mechanisms or studies indicating improved patient satisfaction, engagement, and outcomes.
In this research, 100% of clinicians reported being able to give the client their full attention when using the AI tool—versus 66% when using the EHR alone.
This improved client-provider interaction may even lead to better client retention. I was pretty blown away 🤯 that this study showed that when an AI documentation tool was used patients attended, on average, 67% more sessions compared to those whose therapy sessions were conducted without the tool.
9. What Training and Support are Provided?
Ensure that the AI vendor offers comprehensive training and ongoing support. This includes initial training sessions, user manuals, and accessible customer service to swiftly address any issues or questions that arise.
10. Are There Additional Features to Enhance Patient Outcomes?
Beyond documentation, some AI tools offer features that can directly enhance patient care. So, it’s worth asking: Can the tool generate plain language summaries for patients? Does it provide alerts or suggestions based on patient history? These additional features can significantly impact patient outcomes and should be taken into account.
11. Is The Tool Geared Toward Helping Therapists See as Many Patients as Possible?
🏃⛰️ Run for the hills if a company focuses solely on allowing you to see as many clients as possible.
🛑 This smells of of a company that not understand rehab and healthcare.
Increasing therapist caseloads is one way to boost organizational revenue and justify the cost of these tools. But, so is attracting the best therapists and reducing burnout and turnover. Your AI tool should be designed to improve the care experience—and help you retain the client through their full course of care. If the company behind the tool is focused on these two metrics, then it is a company that understands healthcare.
I will also note that how therapists spend “freed up” time is a management decision, and clinicians should be advocating to management about leveraging these tools to help support and restore their work-life balance (i.e., by not having to chart after hours and weekends).
Conclusion
By asking these critical questions, therapists can navigate the AI landscape more effectively, ensuring that the tools they adopt are secure, efficient, and genuinely beneficial for both clinicians and patients.
I encourage you to check out the AI tools I’ve identified for occupation therapy documentation. And, if you ever hop on a call with them, use this question list!