AI Coaching Raised Parent Fidelity. Clinicians Still Edited 31% of What It Said.

A single-case study in the Journal of Autism and Developmental Disorders, published September 11, gives the first experimental numbers for an AI system that coaches parents from video. The system is PaiCoach. Parents' implementation fidelity rose in all four families. The system was also wrong often enough that a trained reviewer corrected almost a third of what it produced.

The turn: the useful number is the edit rate

The fidelity gains are what most write-ups will lead with. For a practice owner weighing AI feedback tools, the more useful figure is how much clinician time the AI saved and how much it still needed. The study reports both.

What the study did

• Participants: four mother–child dyads in Western New York. The children were 45–56 months old and minimally verbal, with roughly 20–30 words. All procedures were remote.
• Design: concurrent multiple baseline across participants. The routine was shared book reading, and the strategies were modeling, mand-model and time delay.
• How it worked: parents uploaded short reading videos. PaiCoach used speech recognition, computer vision and a large language model to identify the strategies used, score fidelity and draft time-stamped feedback. A human reviewed every output before the parent saw it.

What it found

• Parent fidelity: Tau-U of 1.00 for all four dyads, meaning no overlap between baseline and intervention. Parent 3 was at 0% throughout baseline.
• Child responsiveness: Tau-U from 0.72 to 1.00, mean 0.84.
• Edit rate: across 53 sessions and 879 coded instances, 31.4% of AI outputs needed expert editing. The edit rate was 37.7% for fidelity scores and 34.5% for feedback text.
• Time: coding a 5-minute session by hand took an experienced coder 20–30 minutes. Reviewing the AI's output took 5–10.

The errors followed a pattern. Children's gestures were missed when the book blocked the camera. Overlapping speech confused strategy detection. Acknowledgments like "okay" versus "good job" were misclassified, which dropped correct implementations from a fidelity score of 4 to 3.

The limits, in the authors' terms

All outputs "were reviewed and validated by human researchers prior to parent delivery," so this tested a human-in-the-loop system, not an autonomous one. "Child communication behaviors appeared substantially more difficult for the system to classify reliably than parent strategy use." The authors call the results "preliminary evidence supporting the feasibility" of the approach. The study had four families, all mothers and one routine, and no comparison with conventional coaching. The first author is an associate editor at the journal and was not involved in handling the paper.

What you must know or do

• If you use or are piloting AI scoring or feedback tools for caregiver training: budget a clinician review of every output, not a sample. On this study's numbers, that's about 5–10 minutes per 5-minute video.
• Audit where the errors land. Take ten AI-scored sessions and have a BCBA re-score them. Check separately how often child responses were wrong versus parent behavior. This study found child behavior was the weak point.
• Keep the reviewer's corrections in the record. Cigna's ABA policy says reimbursement follows the service description regardless of AI use, so the documented clinician review is what supports the claim.