The difficulties of early diagnosis: Insights from Sally Ozonoff

If you've read any early autism studies, you've probably heard of Sally Ozonoff.  At UC Davis's MIND Institute, and working with 11 other sites across the country, she heads up prospective studies looking for ways to diagnose autism earlier. At Northwestern yesterday, she gave an overview of her major findings over the past ten years.  What she said will disappoint people hoping for earlier diagnosis for their children (or wishing they'd been diagnosed earlier themselves), but will please those who see early diagnosis as leading to interventions that do more harm than good.

According to the latest CDC report, the average age of diagnosis is the same as it was two years ago--53 months (or about 4 1/2 years).  Yet parents are first concerned about their child's development at an average of 18-19 months.  Parent reports certainly have flaws, as most are not trained in child development and are looking at the big picture rather than the specific details of behavior that might provide the best clues (e.g., frequency of pointing vs. other gestures).  In particular, they might reliably distinguish a child with atypical development, but not be able to tell if autism is the appropriate diagnosis--even trained psychologists seem to have difficulty with this.  However, Ozonoff thought, if parents can identify potentially autistic children as early as 18 months, shouldn't psychologists with all their developmental training and research tools do at least as well?

The methods used for investigating the earliest signs of autism have changed.  Originally, there were two main sources of information: interviews with parents and home videos.  Parent interviews attempted to get the most detailed information possible by asking about "landmark events," memorable things that happened when the child was a particular age.  If a family traveled to visit the grandparents when the child was 12 months old, for example, the researchers would ask them to picture it and would ask if, for example, the child was toddling around the kitchen, or pointing to items of interest at Grandma's house.  The information provided seems to be accurate, but not fine-grained enough.  Home videos are better because you can watch the behaviors unfold yourself and count how often behaviors of interest occur.  However, they take hours to analyze, and videos may not include relevant, but less presentable behavior--e.g., Grandma will not want to see a video of Johnny failing to respond to his name twelve times in a row, or having a meltdown.  So now, Ozonoff's team takes a different approach.  They identify so-called "high risk siblings," babies who have an older ASD sibling.  The matched control group is babies who have no first, second, or third degree ASD relatives (that means no grandparents, aunts, uncles, or cousins--which is increasingly hard to find these days).  These babies are first seen at 6 months old, and are followed up at least at 12, 18, 24, and 36 months.  At 36 months, all are tested for ASD or other atypical development and assigned to a diagnostic category.  This approach allows Ozonoff's team to retrospectively look at how ASD children developed over time, and at what point they first diverged from peers with typical or other atypical development.

The biggest problem for such research is separating ASD babies from other atypically developing peers, so let's talk about how "other atypical development" was defined.  First, babies who were within 3 points of the ADOS cutoff for ASD, but still below it, were "atypically developing."  Second, babies who scored at least 1.5 standard deviations below the mean on cognitive, verbal, or motor skills on the MSEL (a standardized direct observation test) were atypically developing.  The resulting group could be expected to be a heterogeneous mix of children with subthreshold autism traits and those with various developmental delays or learning disabilities.

I had expected Ozonoff's team to look only at socially relevant behaviors (things like eye contact, looking at faces, responding to name, joint attention, etc.).  However, they cast an impressively wide net.  They looked at sensory and motor skills--babies' abilities to gradually follow a moving object with their eyes (called "smooth pursuit"), and their ability to integrate auditory and visual information when listening to speech. Even head circumference was measured.

Unreliable Early Markers
Most of the measures examined didn't differ in high and low risk babies.  For example, high risk babies were just as responsive as low risk ones to the "still face" paradigm, where parents interact normally with their babies for a while then suddenly stop interacting, with a completely blank face.  Babies, who were previously smiling and making eye contact with the parent, now start looking away, losing positive affect, and sometimes become distressed.  When the parent returns to playing with them, though, babies normally return to making eye contact and smiling at the parent.  High-risk babies, as a group, show the exact same pattern.

Another behavior looked more indicative of autism, but didn't pan out.  High risk babies were more likely to look at the mouth than low risk ones, who looked at the eyes1.  However, although 10 out of 11 kids (91%) who focused on the mouth were high risk babies, most of the high risk group still looked mostly at the eyes.  Furthermore, none of the mouth lookers turned out to be autistic, with most typically developing.  Meanwhile, all four with ASD tended to look at the eyes rather than the mouth.  While looking at the mouth does not predict autism diagnosis, but it does predict expressive language skills, on both a direct test (MSEL) and a parent questionnaire (McArthur CDI).  (This is a pretty common relationship in autism language research, but Ozonoff said that as a clinical psychologist, she was unaware of it.  She talked a lot about being a clinical psychologist and how she hoped to get other perspectives from her hosts in the Northwestern Communication Sciences and Disorders department).

Ozonoff's team also looked at children's response to their name, because this is the most common concern parents have.  Again, it wasn't a reliable marker.  Again, high risk babies were more likely not to respond to their name than low risk ones, but only 14 high risk babies failed out of 101 (14%).  Of these 14, 5 turned out ASD (36%), 6 (43%) atypically developing, and 3 (21%) typically developing.  Meanwhile, this marker missed 4 ASD and 22 atypically developing babies.  The best thing one can say about failure to respond to name is that it can identify atypical development in general (80% of such children are atypically developing), but not ASD in particular.  And even for atypically developing kids, most were not identified this way.

So, at what point were researchers able to distinguish the autistic babies?  There were no group differences at all at 6 months, and differences on almost every measure by 12 months.  Even so, only one baby was diagnosed at 12 months, 15% by 15 months, and under half by 18 months.  The average age of diagnosis was 24 months--about the age at which these experienced clinicians would feel comfortable making a diagnosis anyway.

What changes from 6 to 12 months in autistic babies?
The change from 6 to 12 months was dramatic enough that Ozonoff's team explored further with their another group of babies, adding an observation in the middle at 9 months.  Here, she found the only significant results so far.

At 6 months, ASD babies looked, smiled, and vocalized at parents and experimenters just as much as their peers.  However, from 6 to 12 months, their frequency of looking dropped steeply, their smiling declined slightly, and their vocalization remained the same.  Meanwhile, typically developing babies dramatically increased their rates of looking, smiling, and vocalizing at interaction partners.  This pattern existed across the entire group, not just the roughly one third estimated to have lost language or social interaction skills.2  Furthermore, this finding was based on a direct count of looks, smiles, and vocalizations during a real-time interaction, not a standardized measure--it captured the behavior itself, not just the relative frequency compared to a normative group.

Experimenters and parents were asked to rate their child's frequency of looking, smiling, and vocalizing at the end of each session based on general impression.  Experimenters reported the same drop as the coders did, and so did 71% of parents.  Interestingly, 29% of ASD parents reported rates of looking, smiling, and vocalizing that stayed the same from 6 to 12 months, and did not differ from ratings for typically developing babies.  Ozonoff's team assumed these parents were simply bad reporters, as these babies ended up with a diagnosis of ASD, but this is a problematic assumption and we'll talk about why later.

A similar pattern held for synchronous behavior, which was defined as either looking/smiling/vocalizing at each other at the same time, or responsively (with the response following within 3 seconds).  In fact, at 6 months, ASD babies were more in sync with their parents than typically developing peers.  By 12 months, however, ASD babies were less synchronous while typically developing babies had become more so; typically developing babies were now more synchronous than ASD ones.

There's another difference that I, personally, find more interesting.  At 12 months, babies were given a variety of toys to play with, including a shiny round lid, which had two sides that looked different from each other.  Typically developing babies, given that lid, liked to put it on their face or in their mouth, drop it, or bang on the table with it.  They seemed to enjoy the noise this made, and be focused on how the object felt, moved, and sounded rather than how it looked.  They also were less focused on the toy, looking more frequently at the experimenter and around the room.  ASD babies were more focused on the toy, with fewer looks elsewhere; they were more likely to attend to it without interruption for 10 seconds at a time.  They also had a different pattern of playing with the toy.  Unlike the typically developing babies, they actually noticed that the two sides of the lid looked different.  They were more likely to rotate the objects to stare at the different sides; more likely to hold it up close and squint at it; and more likely to spin it, or drop it and watch it wobble. In addition to this set of visual behaviors, they still engaged in the more tactile and auditory play behaviors the typically developing babies preferred.  Each of these visual behaviors, and their composite, were highly predictive of group membership; crucially, they were much less common in other atypically developing babies (but given the low base rate of autism in the population, this is still unlikely to enable reliable early diagnosis).

The differences in play here were fascinating because they are among the earliest evidence of autistic people's fascination with visual stimuli, particularly things that spin.  The differences from other atypically developing babies also surprised me, because atypical sensory processing occurs across all developmental disabilities.  It would be interesting to know whether, across the lifespan, fascination with shiny, spinning things really is more common in autism than in other conditions with atypical sensory processing.

The problem of distinguishing ASD babies from atypically developing ones...again
Atypically developing kids could be identified either by developmental delays on the MSEL, or by high but subthreshold ADOS scores.  76% were identified based on high ADOS scores, 14% on high MSEL scores, and 10% on both--meaning that 86% could be considered to have the "broad autistic phenotype" and only 24% could be considered developmentally delayed.

Such a group makes it all the harder to separate autistic from other atypically developing kids early on, because the autistic kids aren't a different type of disabled child, they just have more extreme versions of the same traits.

Are atypically developing kids just autistic kids who received early intervention?
The atypically developing group's rate of early intervention wasn't significantly different from typically developing participants.  This was despite Ozonoff's team's aggressiveness in informing their parents about the risks of ASD diagnosis and the benefits of early intervention for this group. Thus, these children's failure to develop ASD has nothing to do with being helped by early intervention.  This is a point you're not likely to see discussed much--Ozonoff herself only brought it up in response to a question about diagnostic reliability in her studies--so I think it's important to emphasize.

Questions and Problems
1. How unreliable are parents, really?
Almost 30% of parents of autistic kids rated them as displaying the same rates of basic social behaviors as their typically developing peers.  Ozonoff assumed these were poor reporters because these children were later diagnosed with ASD.  But she did not say whether she examined the measured rates of social behaviors in these kids to see if there actually was any difference.  And in fact, it's possible that these kids really did have high rates of social looking, smiling, and vocalizing, while still being autistic.  There are several ways this could happen.

First, these babies could have mastered these particular social behaviors, which are early developing and do not require complex motor skills or advanced concepts.  Yet, they might have more difficulty with other skills measured by the ADOS, such as pointing, triadic joint attention (between a person and an object), or conversation.

Second, the babies could have been inconsistent, as often occurs in atypically developing populations. Whatever their real level of social communication skills, they performed well in one context (interaction with parents or experimenters) but not in another (the ADOS examination).

Third, the babies could have diverged from typically developing kids in social communication behaviors later than 12 months.  They may have had typical social behaviors at 12 months, but different enough to receive an ASD diagnosis by 36 months.

This would appear to be a case where the assumptions of a discipline (i.e., that clinical psychologists, as experts, know more than poor, ignorant parents) get in the way of a truly scientific examination of a question (i.e., testing other possible explanations for an inconsistent set of results).

2. The definition of synchrony
First, the good news.  Unlike the "reciprocity" studies I've discussed in an earlier post, this study did measure the behavior of parents as well as their children.

The bad news: synchronous behavior was defined as either displaying a social behavior at the same time as parents, or as a response within 3 seconds.  But what if ASD babies are slower to respond?

This could be for any number of reasons.  If they are focusing their attention on one thing (perhaps a toy both are playing with), they may be slow to shift their attention to something else.  Slow shifting of eye movements and attention have often been found in autistic kids and adults, as discussed in a previous post.  If they have to shift their attention first, that could delay the response.  Second, slow responses in general sometimes occur in autism, particularly when speech is required.  These 6-12 month olds aren't using speech.  But if vocalizations for them work in a similar way to speech for older people, one might expect their responses to take longer than 3 seconds.

So, are ASD babies really less synchronous at 12 months?  Maybe.  I'd be more confident in this result if the time window for response were a little wider.

3. Assumptions I don't like
Like many clinicians--including most of the audience at her talk--Sally Ozonoff assumes that any early intervention is a good thing, no matter what it consists of. That if early enough, intervention can even prevent all the characteristics of autism from emerging at all.

She is obviously not familiar with the borderline abusive therapies that some children receive, perhaps because there is little awareness for bad autism interventions.

She also loves the Autism Speaks parent kits, particularly the "first 100 days" one, and actually recommends them to parents.  Full disclosure: I have been working with parents and autistic adults to make an alternative that would be more helpful, and less depressing.

The Upshot
In this study, children received a diagnosis at an average of 24 months, which was a lot earlier than in the general population (53 months).  However, with all the behavioral and electrophysiological data they collected, they still weren't any faster than they would have been using clinical judgment alone.

She's been studying high-risk babies since 2003.  All this time and money has brought us nowhere near having a method that could be widely used to identify autistic babies and toddlers, much less distinguish them from similar atypically developing kids.  At this point, the effort seems almost pointless.

There is, however, one exception.  Babies with an autistic sibling have much higher rates of an autism diagnosis.  Across the 12-site Baby Sibs Research Consortium, which tested 664 high-risk babies, 19% were diagnosed with ASD.  The rate was particularly high in males, with about 25% of males and only 9% of females receiving an ASD diagnosis.  The rate was higher still in families with multiple ASD siblings, where almost 50% of males and about 18% of females diagnosed with ASD.

Interestingly, all sorts of much-touted risk factors didn't make a difference in this large sample.  The sex and functioning of the older ASD siblings didn't matter.  The younger sibling's age at enrollment didn't matter.  The birth order, parent age, and demographic variables--even ethnicity and SES3--didn't affect diagnosis rates.  Only the child's sex and the number of ASD older siblings mattered.

Therefore, Ozonoff argued, we should be paying more attention to screening younger siblings than we are, especially boys and those with multiple ASD older siblings.  Pediatricians should know that the ASD rate is much higher in this group and watch their development particularly closely.4

So there you have it: early diagnosis is nowhere near existing in reality, except, perhaps, for younger siblings.
...
1 Notably, Ozonoff's study measured looks to the face in this and the still face paradigm in an unusual and somewhat artificial way.  The mother was in another room and her face was projected onto a screen in front of the baby, so that a camera could more easily measure where the baby was looking.  Most babies still had typical responses to the still face in this setup, which suggests that they still thought they were interacting with their mothers.
2 This particular study did not measure regression, but Ozonoff reported that other studies estimate it at about 1/4-1/3 of the autistic population.  
3 The lack of ethnicity and SES differences in diagnostic rates surprises me.  It may be due to the fact that participants were recruited with diversity in mind, with a concerted effort to sample ethnic minority groups.  It could be due to a lack of SES diversity--the sample, though ethnically diverse, was mostly middle-class, and the restricted range may have precluded finding an SES effect.  It could also be due to the fact that participants were recruited from an area around three major California universities, where autism awareness is high and participants were likely to receive a diagnosis.  Or, it could stem from the MIND Institute staff's thorough diagnostic evaluation process, which was uniform across participants, regardless of their ethnicity or SES.  One should not expect this lack of ethnic and SES identification bias in the general population.
4 There is one other group that might be worth screening early: those with older siblings with related diagnoses, such as ADHD or specific language impairment (SLI).  However, Ozonoff did not discuss this group, compare their elevated risk of autism diagnosis to that of those with autistic siblings, or recommend more aggressive screening for them.