CMS Wants Hours Set by DSM Severity Level. Clinicians Can't Agree on the Level.

Page 37 of the CMS State Medicaid & CHIP Applied Behavior Analysis Toolkit, published in August, tells states this: “Best practice is that the number of hours detailed in the ITP be aligned with these levels; for example, 10, 20, and 30 hours for children in levels 1, 2, and 3, respectively.”

On August 21, Autism Research published the reliability figures for those levels.

What the study found

Morrier and Schwartz had four doctoral-level clinicians independently evaluate 49 toddlers aged 16 to 31 months. Each child got both a telehealth and an in-person comprehensive diagnostic assessment. Earlier work on the same cohort had shown high clinician agreement on whether the child was autistic.

Agreement on the severity specifier was a different story: “Agreement between assessment protocols was fair for SCI (κ = 0.318, 95% CI [-0.114, 0.352]) and slight for RRB (κ = 0.195, 95% CI [-0.046, 0.436]).” Both confidence intervals cross zero.

It was not one bad rater or one bad modality. There were “no significant differences in severity ratings based on individual clinician, clinical team, or assessment protocol.” The authors' reading: “assigning a diagnosis and determining the level of support needed represent distinct clinical decisions.”

The collision

A level assigned this unreliably is a poor instrument to hang a number of hours on. It is worth noting that the toolkit half-concedes the point on the same page, immediately before making the recommendation: “there is no consensus on a set number of hours recommended for treatment,” and, immediately after it, “[i]ntervention intensity and frequency are not correlated with improvement in outcomes, in that high levels do not equate to better outcomes compared to lower levels.”

The toolkit binds nobody. It is written for state Medicaid and CHIP agencies, and CMS says it is not a regulation. It is also the menu states are ordering from.

The limits

The sample is 49 toddlers between 16 and 31 months. This is evidence about severity ratings at first diagnosis, not about a level assigned to an eight-year-old, and the study does not follow whether those ratings become more stable with age. The kappas compare two assessment protocols on the same children rather than two independent clinicians working blind, and both confidence intervals include zero. The paper is behind a paywall; the figures above are from the published abstract and the statistics it reports.

The study is also not an argument against remote assessment. Protocol made no significant difference — the unreliability sits in the severity construct, not in the telehealth.

What you must know or do

• Stop importing a referral's severity level into your treatment plan as a finding. If your intake template copies “Level 2” from a diagnostic report into the justification for hours, that is a number you cannot defend on this evidence. Justify intensity from your own assessment data.
• Check whether any payer you bill already sets hours by level. Read the medical-necessity criteria for the phrase “severity level” or “requires substantial support.” Where it appears, this paper is the citation for an appeal.
• Where you assign severity yourself, write down what you based it on. The authors' conclusion is that factors beyond the two core domains are driving these ratings. Naming your basis is what separates your rating from the ones this study measured.
• Watch your state's response to the toolkit for the 10/20/30 line specifically. If it appears in a draft manual or a plan's criteria, that is the moment to comment, and this is what to comment with.