While there were a number of procedural issues asked about the presentations on Tuesday night regarding the district AIM program, much of the hard questions focused particularly on the identification and testing issue.
Board Member Madhavi Sunder noted that the importance of process is not only to ensure equal access, but to make sure that the board receives information that is vetted.
However, Board Member Susan Lovenburg said that they saw nothing unusual about the process and Board President Alan Fernandes said that the agenda item followed board policies that allowed for items to be placed on the agenda by board members.
The board heard presentations from two UC Davis professors, presenting their findings regarding AIM assessment and identification data.
Madhavi Sunder pointed out that the two tests serve very different populations, with the OLSAT given to the general population and thus obtaining a smaller percentage of qualifying scores. The TONI serves two populations – the first are those who scored within five points on the OLSAT and the second is the “search and serve” group ‒ whether they have a huge discrepancy in scores or they needed more time.
Tobin White, an Associate Professor at the UC Davis School of Education, will discuss his assessment of current testing practices at DJUSD. Currently they use two exams, the OLSAT (Otis-Lennon School Ability Test) and the TONI (Test of Nonverbal Intelligence) to identify students eligible for the AIM program.
According to Professor White, the OLSAT identifies about 24% of the AIM students, the TONI 49% of AIM students, and 27% are identified through some other test.
The TONI is by design more inclusive, as it is used to retest students who come within five points of qualifying for enrollment on the OLSAT test. Therefore, it is not completely surprising that students administered the TONI were “six times more likely to qualify than those taking only the OLSAT.” They were also nine times more likely, according to Professor White, to score in the 99th percentile.
He writes, “These are radically different measures, yet they are being treated as equivalent in program placement decisions.”
He adds, citing research, “The TONI was not designed to replace broad-based intelligence tests but rather to provide an alternative method of assessment when a subject’s cognitive, language, or motor impairments rendered traditional tests of intelligence inappropriate and ineffectual.”
The TONI was designed specifically for populations of deaf and hearing impaired, those with aphasia, dyslexia and other disorders related to spoken and written language, and those not proficient with written and spoken English.
The TONI is also the more ethnically diverse testing protocol. Currently the District’s AIM program is 60% White, 19% Hispanic, 16% Asian, and 3% Black. The OLSAT generates 48% White qualifiers, 44% Asian, 6% Latino and 1% Black. The TONI is more ethnically diverse with 43% white, 31% Latino, 12% Asian, and 8% Black.
“Generally there is some reason to believe that these particular students might have a false negative on the OLSAT,” she said. “They may actually have some gifted potential that was not apparent on the OLSAT.”
Professor Tobin White noted that it’s really one-third of the population that is affected. However, the purpose of the TONI is to identify students where language is an issue and where a verbal test might not be the best measure. “To get up to a third seems like a pretty big jump,” he said.
Professor White concludes that “the current re-screening process is fundamentally flawed.” The “TONI and OLSAT have dramatically different psychometric properties, one score should not simply replace the other.”
He noted, “The TONI is designed to address only a limited range of the search and serve criteria, yet is being applied for all of them.”
He concludes, “The TONI is clearly identifying a more diverse group of students than the OLSAT, but as a function of student selection rather than test bias.”
Madhavi Sunder noted that the TONI doesn’t just look at language learners, but all categories of disadvantage, like students who didn’t go to preschool and other such disadvantages like family history and student background.
“When you come from a low socio-economic background and you’re not necessarily going to have all of the those enrichment opportunities around language, and it’s precisely those kids who we think are at a disadvantage on the (OLSAT), there’s a lot of language that’s required to do that test,” she said. “That’s really the heart, are we testing the wrong kids with the TONI?” She argues, “We should be using these non-verbal tests for a larger population.”
Ms. Sunder argued, “Fairness isn’t how many tests each person gets to take, but is every kid getting the opportunity to take the appropriate test for them.” She argued we need to be using the practices for the method of test that we are offering.
Board Member Susan Lovenburg said, “The big conclusion is there is no perfect test.” She said, “What is concerning here is that one test is being thrown out in favor of a second test for some kids. That’s the kind of problematic element of what’s happening here.”
She wants data and evidence to look at as we make our decisions. She said, “Particularly in the world of education research, you can find research to support almost any decision as long as your going to ignore the research that conflicts with it.”
Ms. Lovenburg added, “Ultimately the question that is being asked is are we appropriately identifying the students that need these special services.” She said, “I’m not in favor of dismantling the self-contained GATE program. I don’t speak for any of my colleagues, they can speak for themselves. But I do see that value in the program for students who truly need the services.”
However, she stated, “I do have some significant concerns that we’re over-identifying students into the program… We need to address that as a district.”
Board Member Barbara Archer said, “When I look at this data, I see that 50 percent of the students in this dataset qualified through the TONI for the AIM program. That caused me to wonder, well what were the reasons?”
She said she wanted to understand the reasons why students were being re-screened through the TONI. “What I was told is that we don’t keep track of the reason why 50 percent of our students qualify using the TONI, and that’s concerning to me.”
“The other point of concern is that it seems to me statistically impossible that 28 percent of students could score in the 99th percentile on a test – that doesn’t make sense to me,” she said.
Ms. Archer said, “When you see that 50 percent of students were being qualified with the TONI, and you realize it is possible – although we don’t know because we don’t have the data – that you’re lumping together margin of error kids, ELL kids, low SES, and the various other reasons, that seems to me a suspect metric because you’re lumping so many different groups into a non-verbal category.”
She added, “It seems like the TONI is advantaging everyone.”
Board Member Tom Adams said, “The overall test results would raise questions.” He noted that issues of reliability and validity depend on what the test is trying to measure. In this case the question is whether you are GATE, and in need of GATE services. “The question is, can you really swap out – as has the district done – the identification that occurs under OLSAT with TONI?”
“Do these tests really measure the same thing?” he asked. “That’s a test that I really don’t think has been addressed, because I don’t think they measure the same thing.”
He acknowledged the need for more than one test, particularly with people who lack the language skills for the OLSAT. “In this sense, it’s not that anybody’s against GATE identification, it’s not that anybody is trying to prevent anybody from trying to get into the program, the real question is… what is the best measure of those kids in identifying them in terms of them getting these services that they need.”
Susan Lovenburg would note that we lack the resources to throw a full range of tests at every student. Tom Adams asked whether we using this in a manner that reflects best practices.
Finally, Madhavi Sunder asked, what is the alternative to the current way of evaluating AIM identifiers?
The challenge that the district faces is that, while it is easy to argue that the TONI perhaps sets too low a bar, with the OLSAT alone, GATE would be 92% White and Asian with only 8% Black, Latino, and American Indian.
These are challenges not easily resolved and an issue that tends to create sharply held beliefs and divisions.
—David M. Greenwald reporting
Ms. Lovenburg added, “Ultimately the question that is being asked is are we appropriately identifying the students that need these special services.”
No, DJUSD is not identifying students that need special services. What they have created is an AP elementary school tract if half of the students qualify from the testing provided.
If you were then only about 3%-5% of the students would qualify and those students would be of about the same racial makeup as the school population, some will have gone to pre-school, some won’t be able to read, some will get all A’s, some will get all F’s……
What is wrong with an AP elementary school tract that raises the bar for learning? We should be encouraging our children to exceed the standards, not just meet them. I see not problem with an AIM program that is over inclusive. Parents should have the choice to place there students in a learning environment that best fits their child’s needs.
I never said there was a problem with having an AP tract. Just change the name and keep everything else the same.
The problem that I have is that you are not serving the gifted children in the district as the program was intended. They need services that are different than just being put into AP classes. In Davis, those children are not being identified or being helped.
When the district decides to start peering performance results for the remaining 75% of the student body – the majority of families might be more supportive.
There is a confusing message about AIM identification. There is the implication that AIM identified students have certain innate (“natural”) abilities that distinguish them from others. Therefore the AIM identification test (TONI or OLSAT) isn’t something that one can study or prepare for.
On the other hand there are analogies comparing GATE/AIM identification to selection for audition/try-out school activities like Madrigals and varsity football:
Chander & Sunder, 1/29/2013, Gifted kids need appropriate instruction:
I understand screening processes for Madrigals and varsity football to rely substantially on developed talent. A successful Madrigal singer or varsity football player doesn’t show up unprepared for their audition. Plenty (years) of practice and preparation goes into a successful tryout. Most folks don’t have an issue with Madrigal or varsity football selection because it is perceived as an “earned” achievement. It is a nature vs. nurture situation. Can one practice and prepare for a successful GATE/AIM qualification?
At one time I remember seeing a web page on the DJUSD website suggesting ways that parents could foster GATE/AIM characteristics in their kids, such as enrolling them in music. I haven’t found that page in recent years. Perhaps I misunderstood that information?
That is my point. Davis runs an AP program that you can practice and prepare to qualify for. A true GATE program identifies and assists only gifted children. Those gifted children learn differently and have different issues in school than non-gifted children. That is not something that someone can pratice and prepare for.
Look at the other end of the spectrum. Children that have been identified with low intelligence and dyslexia also have special needs that should be addressed. Should Davis just give all of the children that do not qualify for GATE an IEP? No, because then the program designed to serve children with learning disabilities is then changed and those who need the services are not identified do not receive them.
the essential problem that i see here is how can gate/ aim be at the same time inclusive and exclusive. madhavi seems to want the big tent approach – open up goat, make sure it is allows in people of color. archer and lovenburg seem to want it to be much more exclusive. so you have the olsat model which would create a traditional high achiever framework – heavy on whites and asians, light on blacks and hispanics. the toni model on the other hand seems to water things down to the point where you have 28% scoring in the 99th percentile.
the problem is that we seem to want to have our cake and eat it too – exclusive and inclusive – and that doesn’t work. why? because we don’t have an inclusive high achieving society.
so to me we have to fix that first – fix the achievement gap before we can fix gate.
If it was an actual GATE program it would include an ethnic population similar to the students.
DP: madhavi seems to want the big tent approach – open up goat, make sure it is allows in people of color.
goat? do you mean GATE?
A set of numbers we do NOT seem to have in this discussion is the population matrix of White/Asian/Black/Latino-Hispanic/Native American residents of the district boundaries. Can someone provide those numbers, to put the AIM/Gate numbers in fuller context?
Meaning what? That you think there might be a large number of AIM/GATE students from outside the district who are driving this discussion?
What da’ F&*%, WDF!?!
No, what I question is that the columns shown in the table above, are in Gate/Aim (total district), the two main tests, and “other” tests.
I just don’t believe the residents of the District including, if you will, any “imports”, have the similar demographics to the table.
Give it a rest WDF! All I asked was a simple question which you for some reason, want to “spin”… Just think Data Tells All, and it might not be such a ‘scary’/nefarious question. for you.
No need to react that way. Your point wasn’t originally clear to me, thanks for clarifying. Context and voice inflection don’t always come over very well in comments.
It is the Truman show.
All this debate activity about the actors in the play, and nobody seems willing to question the play itself.
Programs are developed because the employees of the system can market them to get more money from taxpayers. But programs end up being a form of segregation for students.
There should be only one program (one play). It should simply be “school”. And the model should be “school of one” with each student being seen as unique and needing a customized education path.
Until we do that we will forever be arguing about the various programs that segregate students into groups with some getting advantaged at the cost of others.
” And the model should be “school of one” with each student being seen as unique and needing a customized education path.”
When he’s right, he’s right.
Once again, a solution too simple for the convoluted thinkers to grasp.
;>)/
When GATE testing was done by teacher recommendation of a largely white upper class pool, no one complained of “over identification”. Now that we have universal testing and the percentage of minorities in the GATE program has risen dramatically, people want to talk about “over identification.” Seems like something our social justice advocates would be interested in backing.
There is a legal backdrop to the current testing methods:
http://www.davisenterprise.com/local-news/schools-news/davis-school-board-settles-gate-admission-policy-lawsuit/
https://davisvanguard.org/2012/08/discussion-the-issue-of-gate-figures-to-rise-again-in-this-years-sb-elections/
If they discontinue the TONI test, it seems they would be inviting litigation.