big little number problem

Big little number problem

The Contribution of Diagnostic Substitution to the Growing

Administrative Prevalence of Autism in US Special Education by Paul

Shattuck, is about to be published in Pediatrics.

Paul Shattuck started out working in social services and then went to

study neurodevelopmental disablities. He sounds like he's quite the

advocate for the needs of autistic children in Wisconsin. He seems to

have done some serious fundraising advovcacy for the Waisman center,

too.

New study questions suggestions autism is on the rise

BY SUSANNE RUST

Milwaukee Journal Sentinel

MILWAUKEE - Despite warnings of a national autism epidemic, there's

little data to substantiate such a claim, according to new research

compiled by a University of Wisconsin-Madison scientist.

Indeed, special education figures that are being used to suggest an

autism explosion are faulty and confounded, said Paul Shattuck, a

researcher at the university's Waisman Center and author of the

study, which appears in Monday's issue of the journal Pediatrics.

From 1993 to 2003, statistics compiled by the U.S. Department of

Education showed a 657 percent increase in autism across the

country - an explosive jump that signaled an epidemic to many.

But Shattuck discovered that, at least in most cases, the numbers

are not only misleading, they're also likely inaccurate. On one

hand, they don't support a dramatic increase in autism prevalence,

but on the other, the figures could be underestimating the absolute

number of children with the condition.

In other words, "special education trends cannot be validly used to

substantiate claims of the presence or absence of an autism

epidemic," Shattuck said.

One of the problems is what researchers call the process of

"diagnostic substitution": While the number of reported autism

cases has increased, diagnoses of mental retardation and learning

disabilities in schools have correspondingly decreased.

Before the early 1990s, the Department of Education didn't have an

autism classification for children with special needs. Therefore,

children who would now be considered autistic were often diagnosed

with other disorders, invalidating direct comparisons of data

collected from the early 1990s and now.

The study suggests researchers may need to use different sorts of

data to accurately quantify the prevalence of autism in this

country.

"In public health, we make a distinction between `population-based

estimates' and `administrative-based estimates' of the prevalence

of any given condition or disease," Shattuck said.

Population-based estimates are those in which researchers actively

go out into a community and make a "no stone left unturned" effort

to find every person who has a particular disease or condition. "As

you might imagine, these kinds of studies are difficult to execute

and cost a lot of money," Shattuck said.

So, data collectors generally opt for a faster route, which is to

quantify the number of people who are already enrolled in a system

and receiving services for a particular diagnosis.

In the case of special education counts for children with autism,

the administrative prevalence is simply the number of students with

a primary classification of autism divided by the total number of

students in that given region, whether it's a state, district or

county.

This means, among other things, that data collected in this manner

often underestimate "the true population prevalence because, for

instance, schools do not go out into the community and actively

seek out and evaluate all kids for autism," Shattuck said.

For instance, consider data collected in Wisconsin: In 1992, 18

children were counted in special education programs as being

autistic. By 2002, that number had jumped to 2,739.

"The conclusion is that the prevalence of autism has grown by

15,117 percent. This is ridiculous," Shattuck said. "No credible

clinician or scientist in the field would ever suggest there were

actually only 18 children with autism in all of Wisconsin in 1992."

Also consider this: In 2002, there were about 1.3 million children

between the ages of 6 and 21 living in Wisconsin.

"If we assume that true prevalence of autism is about six in every

1,000 children," an estimate based on smaller, more accurate

population-based evidence analyses, "then we would expect the true

number of kids between the ages of 6 and 21 in Wisconsin with an

autism spectrum disorder to be around 7,800 ... a lot higher than

the 2,739 actually identified in special education that year,"

Shattuck said.

In other words, the special education numbers not only discount the

fact that the diagnosis for autism has changed over the years -

making yearly comparisons faulty and inaccurate - but they also

underestimate the number of children who actually have it.

"This is not anything new," said Glen Sallows, a clinical

psychologist and director of the Wisconsin Early Autism Project,

which is based in Madison. "Other people have been saying the same

thing" about diagnostic substitution for quite a while.

But he suspects this is the first paper that has really proved the

case.

"Researchers have pointed out that the definition of autism has

changed over time," Sallows said. It's now one of 13 categories

schools use to classify children identified as having special

needs.

In addition, there are no uniform diagnostic practices or

guidelines among states or school districts. That means how

children are classified can vary dramatically between states and

school districts.

Shattuck did discover that California does not follow the pattern

he noted among other states: While autism numbers in the state

grew, there was no corresponding decrease in mental retardation.

This indicates that California should not be considered

representative of the rest of the nation, which it has been in many

press reports.

In a commentary that accompanies Shattuck's paper, Craig

Newschaffer, an epidemiologist at the Johns Hopkins Bloomberg

School of Public Health in Baltimore, wrote that despite the flaws

in administrative-prevalence studies, and the potential for

diagnostic substitution, it "remains difficult to ascribe all the

observed autism prevalence increase to this particular phenomenon."

But he thinks it will be hard to find data that support or refute

an increase in the disorder in the past two decades. Therefore, he

believes it's time to accept that and instead focus on the causes

of autism.

Shattuck points out that California's IDEA numbers are quite

problematic when compared to the rest of the country. California's

IDEA numbers are problematic all by themselves, too.

As you can see from this pie chart which represents the total student

population of the State of California in the 2004-2005 school year,

Hispanic students are shown by the dark pink wedge that comprises

about half the pie.

The ratio of ethnicities among the disabled student population pretty

closely matches that of the total population.

In 2001, Hispanic students comprised 43.19% of the total school

population and 45.77% of the disabled student population. African

American students were slightly over-represented among the disabled

students (around 11% among the disabled) and the Caucasian students

represented about the same percent in both total population and among

the disabled, 35%.

So, a little more than a third of the Kindergarten through 12th grade

students are Caucasian in California.

But for some reason nearly half of all the autistic students are

Caucasian, in 2004-2005, it was 47.5%

The Hispanic students were seriously underrepresented in the autism

category, comprising only 27.86% where one would expect about 45%. The

other ethnicities are all a little off, with African American, Asian

and Filipino kids being slightly overrepresented in autism.

To say it again, in California, Caucasian students are significantly

over-represented and Hispanic students are underrepresented in autism.

There are considerably fewer than the predicted 1 in 166 students in

California with an ASD diagnosis according to the IDEA stats, which

goes along with what Shattuck's paper said, that the autistic kids are

not being reached with services, at least not under the "autism"

label.

One big message that we need to keep in mind, however, is that autism

spectrum kids in California only represent 4.3% of the total disabled

student population. If one is immersed in "autism, autism, autism,"

(to quote Sally Bernard of CAN) one can start to imagine that it's the

ONLY disablity that kids and families EVER deal with.

But back to Wisconsin and the Waisman Center where Paul Shattuck

works.

The following is from the Waisman Center website from a few weeks ago:

Experts question prevalent stereotypes about autism...

As theories about autism spread like wildfire in the media and the

general public, a panel of autism experts will reflect on the

validity of four widely held - and potentially inaccurate -

assumptions about the developmental disability.

...

"With the surge in both scientists and society turning their

attention toward autism, there comes responsibility," says Morton

Gernsbacher, a Vilas Research Professor of psychology at UW-Madison

and the symposium's chair and organizer. "It behooves us as

scientists to distinguish uninformed stereotypes from scientific

reality and to move beyond myths and misconceptions."

During her talk, Gernsbacher will cast doubt on the prevalent

notion among autism researchers that autistic individuals lack a

"theory of mind." The belief that autistic children lack a sense of

both their own minds and those of others emerged about 20 years

ago, becoming a seemingly undisputed tenet in the literature since

then, says Gernsbacher.

When the psychologist began delving into the question, however, she

found that scientists usually ascertain how well individuals

perceive the mind with tasks that require a relatively

sophisticated level of linguistic ability. Since a common

diagnostic criteria for autism is the impairment of communication

skills, Gernsbacher says it's not surprising that most autistic

children don't fare well on such theory-of-mind tests.

"I think we as a society fall prey to a slippery slope when we

begin talking about members of our society as not appreciating that

they or others have a mind," says Gernsbacher. "An uncritical

acceptance of the hypothesis that autistic individuals lack a

theory of mind can seriously compromise how autistic individuals

are treated in the workplace, the community and society in

general."

The other panelists will similarly address other stereotypes about

autism. Judith Grether, an environmental epidemiologist who works

for the state of California, will contest the popular notion that

North America is reeling from an autism epidemic. Grether will make

the point that a higher number of reported autism cases - due to

looser diagnostic criteria - doesn't necessarily translate into an

actual rise in the overall number of cases.

...

[Gernsbacher:] "I would like scientists to become more skeptical of

the stereotypes that flourish about autism and members of society

to become more skeptical of the myths that are circulated."

It looks like Wisconsin is leading the way in with

anti-autism-epidemic research from the U.S..

Science, worldwide, is starting to see plainly and state plainly that

there has been nothing like an epidemic of autism.

Hopefully, soon, very soon, wherever autism is discussed,

all will look back on the bad old days when they believed

there had been a devastating tsunami of autistic trainwrecks crashing

across the United States,

threatening to leave destroyed economies in its devastating wake,

tsunamiwreck

and laugh nervously and change the subject.

Heh.

Autism Diva

hopeful

posted by Autism Diva at 11:33 PM

12 Comments:

Anonymous mike stanton said...

The study by Laidler came to similar conclusions last year.

As has been shown, the USDE data on autism are

at odds with studies of autism prevalence, largely

because the criteria used by the school districts (the

source of the USDE data) to categorize children as

autistic are neither rigorous nor consistent. They are

inconsistent over time, as are the medical criteria,

and are inconsistent from region to region. The

USDE data are not reliable for tracking the prevalence

of autism, and they in fact never were meant to

fill this need.

3:44 AM

Blogger notmercury said...

Wow, good story and I look forward to reading the JPeds paper

when it's published.

By 2002, that number had jumped to 2,739. "The conclusion is

that the prevalence of autism has grown by 15,117 percent.

Is that so hard to believe?

5:43 AM

Blogger Joseph said...

There is some diagnosis substitution going on, but this is a

minor part of what's going on in autism. The diagnoses of

mental retardation, for example, have remained close to

population growth levels in California. On the one hand,

there's a bit of diagnosis substitution, but on the other,

there could be a sligh increase in the proportion of

individuals who seek CDDS eligibility.

What actually explains the bulk of the 'autism epidemic' are

two factors:

1) Increasing recognition of autism in the population with

mental retardation.

2) Increasing recognition of autism in the population without

mental retardation.

I've come to this conclusion in my latest blog entry titled

Regional Differences And Quarterly Growth Due To Two Factors.

And I've also argued that there hasn't been an autism epidemic

in No Autism Epidemic: Summary of the Numerical Evidence which

Autism Diva already linked to in one of her recent posts.

9:10 AM

Blogger notmercury said...

Joseph,

Will you blog on this once you've read the paper? I'm looking

forward to your take on it.

10:26 AM

Blogger Jockey said...

I went to a conference on autism education in which a woman who

gives kids educational diagnoses for the Chicago Public Schools

explained and emphasized that educational diagnoses are not the

same as psychiatric diagnoses; the educational diagnosis is

supposed to asign children to the type of classroom/teaching

method that will be most beneficial to it, so children with

AS/HFA do not get educational diagnoses of autism in the CPS,

but kids with, say, Rhett's, will, as might some kids with

mental retardation who don't actually have any PDD.

10:39 AM

Blogger Heraldblog said...

Wisconsin is very advanced when it comes to caring for families

of autistic children. Under the state medicare waiver, the

state pays for ABA therapy for the first three years following

a diagnosis. After that "intensive therapy" period, each family

has access to $10,000 during the "post intensive" period, to

spend on ABA or other approved therapies, as well as

instructional materials and adaptive equipment. I've met

several families who moved here from other states just for the

services.

Ken, Milwaukee

11:18 AM

Blogger Autism Diva said...

Did you hear that??? It's a howl, a howl of pain and rage from

a wounded animal... or maybe its the mercury parents screaming

bloody denial...

---------

Autism Community Calls New Study in Journal Pediatrics

Misleading, Declares Autism Epidemic Real

4/3/2006 2:20:00 PM

To: National Desk, Health Reporter

Contact: Jamie Shor, 202-628.7772

WASHINGTON, April 3 /U.S. Newswire/ -- A study appearing in

next week's edition of Pediatrics titled "Diagnostic

Substitution and Changing Autism Prevalence" is being

questioned by the autism community. The report, authored by Dr.

Paul Shattuck, uses U.S. Department of Education data to

support the hypothesis that real autism rates have not

increased over the last two decades, and that reported

increases are a function of reclassification of students from

learning disabilities and mental retardation categories. This

theory has been rejected in a number of scientific studies. The

autism community would like to see scientific studies based on

more valid databases in order to determine accurate prevalence

trends.

The autism parent organizations including SafeMinds (

http://www.safeminds.org ), the National Autism Association (

http://www.nationalautismassociation.org ), A-CHAMP (

http://www.a-champ.org ), and Generation Rescue ( http://

www.generationrescue.org) see this latest article as part of a

phenomenon of epidemic denial that inhibits open scientific

investigation of autism's causes and blocks allocation of

needed resources into autism. The groups note that the

prevalence of autism now far exceeds other high profile

disorders such as cerebral palsy, cystic fibrosis, and juvenile

diabetes. They request that autism be recognized as a national

emergency and that unbiased epidemiological studies be

conducted that count both older and younger people with autism

to see if the increases are real.

The paper's use of Department of Education data to conclude no

epidemic exists is troubling. The study author himself said

that the data set is inconsistent and is subject to

administrative and policy changes by the states. He notes that

28 of the 48 states included in the analysis do not support his

theory of reclassification as a reason for autism increases.

"Each state has its own rules and the autism rates by state

vary greatly, so aggregating the state-level data to a US

average is not good statistical practice," explained Mark

Blaxill of SafeMinds. "Other, more reliable data sets, like the

California Department of Developmental Disabilities, do show a

real increase in autism."

Autism Groups voiced support for the commentary by Dr. Craig

Newschaffer that accompanies the Shattuck article in

Pediatrics. This commentary makes a number of valid points

regarding Dr. Shattuck's approach and conclusions. The autism

groups note that the hypothesis of reclassification, or

"diagnostic substitution", has been examined and rejected in

several scientific papers as a likely major factor in reported

autism increases. One study was authored by Dr. Newschaffer. A

study by Robert Byrd and a study by Blaxill, Baskin and Spitzer

have also ruled out diagnostic substitution.

More information about the Shattuck study and autism prevalence

may be found at http://www.safeminds.org or by calling Jamie

Shor at 202-628-7772.

http://www.usnewswire.com/

-------

The Byrd study has been seriously challenged by the California

DDS itself. It's a very poor study based on voluntarily

submitted questionnaires that tended to select for a certain

type of responders.

The MIND will likely divorce itself from the Byrd study in

Autism Diva's opinion.

These very howling parent groups are the ones who decided that

the IDEA data was a good way to represent their "epidemic"

belief system. Even though their ideas have been totally

debunked and shredded. Blaxill is a mercury dad totally loaded

down by his (leather covered Gucci) agenda and is not a

scientist by any stretch of the imagination.

The IDEA numbers as evidence for the "epidemic" have been

debunked by Gernsbacher et al and Laider and others, in peer

reviewed published papers, besides the volume of scientific and

skeptical blogging.

Newschaffer is likely to be backpeddling quite a bit from here

on out, he has leaned toward the epidemic in the past... which

is the financially beneficial thing to do... get funding based

on the supposed impending doom of an epidemic... see Dr.

Fombonne's presentation on video at the MIND for more on that.

2:03 PM

Blogger Joseph said...

A study appearing in next week's edition of Pediatrics titled

"Diagnostic Substitution and Changing Autism Prevalence" is

being questioned by the autism community.

I dislike their use of the autism community terminology.

Anyway, at this point the ball is in the court of the autism

epidemic proponents to prove that an autism epidemic has

occurred. The burden of proof has clearly shifted, because:

1) The characteristics of autistics have changed as time goes

by.

2) The 'epidemic' is geographically isolated.

3) Characteristics linked to autism are not on the rise.

It's their turn. I'm waiting to hear back from them.

2:19 PM

Anonymous Anonymous said...

I already received an email this AM stating that the study is

potentially "biased" by "conflict of interest" and "probably

falsified" data. The demand is for a full inquiry by the

University of Wisconsin.

i.e., Don't do the analysis when you can just tar the

researcher.

BTW--the paper and the commentaries by Shattuck and Newschaffer

are up on the Pediatrics website if you have access.

I'm not commenting on the Shattuck and Newschaffer paper(s)

because I haven't gotten a chance to read them yet, but as Mike

noted, Laidler already pointed out some of the problems.

Laidler also mentioned the possible ASD/MR diagnosis inversion

in a talk I attended last summer but I think that his paper was

already in prepress at that time.

Regan

2:20 PM

Blogger Alyric said...

Blaxill - Bleh

Whatever is in the the water at Uni Wisconsin should be use to

spike the DAN and CAN water coolers. Aren't those guys

weonderful?

3:55 PM

Blogger notmercury said...

"The autism community would like to see scientific studies

based on more valid databases in order to determine accurate

prevalence trends."

Translation: Those of us who believe mercury causes autism

don't appreciate studies contrary to our

litigation.....err....investigations concerning the cause

(which we all know is mercury) of the increase or decrease in

autism rates as it serves our purposes.

6:03 PM

Blogger Heraldblog said...

SafeMinds begs to differ. From their press release:

A study appearing in next week's edition of Pediatrics titled

"Diagnostic Substitution and Changing Autism Prevalence" is

being questioned

by the autism community. The report, authored by Dr. Paul

Shattuck, uses US Department of Education data to support the

hypothesis that real autism rates have not increased over the

last two decades, and that reported increases are a function of

reclassification of students from learning disabilities and

mental retardation categories. This theory has been rejected in

a number of scientific studies. The autism community would like

to see scientific studies based on more valid databases in

order to determine accurate prevalence trends.

12:40 PM

Links to this post: