Analysis: Five Questions About the AIM Proposal

There has been some pretty good discussion on the Vanguard about some of the pros and cons of the current AIM program and whether the new proposal makes things better or worse. From my perspective, the biggest positive about the AIM proposal is it seeks to fix what seemed to be the biggest problem in the previous system, identification.

Tobin White found that under the previous system, 27 percent were identified through private testing, 49 percent through the TONI, (Test of Nonverbal Intelligence) and 24 percent through the OLSAT (Otis-Lennon School Ability Test). While the OLSAT itself is subject to criticism, with the district acknowledging “the test has been shown to reflect a higher level of success for white and Asian students” – and to be biased against identifying underrepresented groups of students including English Learners, low income, Hispanic and African-American – the district proposal does clean up the identification process.

Gone is the private testing which was a key source of claims of unfairness. Streamlined is the rest of the process.

If the district had simply cleaned up this portion of the identification process, they probably would have complete community consensus for moving forward.

However, the district goes further than setting the process for selection as the OLSAT plus risk factors, and then a prescribed test for people with a given set of risk factors. And that leads us to a series of questions.

First, why did the district select the 98th percentile? As some of our readers pointed out, it is not immediately clear what the educationally-based justification is for the 98th percentile. They note in their explanation, “Local districts are directed by the state to develop their own program design and identification criteria. As mentioned earlier, research for this report suggests that the qualification score ranges from 90-99 percentile in GATE programs throughout California.”

That would appear to suggest that there is no agreed upon standard or number. The district seems to suggest a different rationale.

They say: “The current DJUSD qualification score for AIM-identification is the 96 percentile. Raising or lowering the qualification score will have a direct effect on the projected number of students who qualify.” This at least implies that the district is focused more on a bottom-line number than an educationally-based rationale.

They then add, “Analysis from relevant research as well as conversations with GATE teachers, principals and community input has led the administration to select a qualification score that is meant to best serve the DJUSD student population.” But they never explain what exactly that means. Are they simply trying to end up with a certain number or is there some non-arbitrary rationale for the decision on the cut score?

That leads us to a second question, what is achieved by shrinking the size of the program? Over the last decade, research has shown that there are between 100 and 130 students in the AIM program. The district projects, with the qualification score raised to the 98th percentile and the elimination of private testing, there would be between 63 and 73 students in self-contained classrooms.

Throughout this discussion, we have heard some critics believe that there should only be self-contained classrooms for those who struggle in the mainstream classroom. That is different from the program that we have had, but that is not what this proposal entails. Staff writes, “All 3rd graders would take the OLSAT and students scoring 98th percentile or above will qualify for AIM.”

That suggests that the OLSAT score is what matters and we will see high achieving students as well as perhaps underachieving students in the AIM self-contained classrooms.

That leads us to the question: why do we need to shrink the size of the classrooms? Does staff believe that there are too many students in AIM? Does staff believe that the students scoring in the 96th and 97th percentiles are not benefiting from AIM?

What is the ultimate purpose of this change? How does it benefit the schools?

As we mentioned, the district acknowledges problems with the OLSAT in that “the test has been shown to reflect a higher level of success for white and Asian students,” and to be biased against identifying underrepresented groups of students including English Learners, low income, Hispanic, and African-Americans.

So that begs the question, why use it all? In 2012, the Wall Street Journal reported that the New York City gifted program had re-worked its testing formula where the Naglieri Nonverbal Ability Test will count for two-thirds of the student’s score and the OLSAT would count for one-third (down from 75 percent of the total).

One of our commenters noted that there are problems with the Naglieri, which this district plans to use to assess students with economic risk factors, while students with risk factors related to language or culture will be assessed with TONI.

Another commenter noted, “Relying on OLSAT scores so heavily, and on other standardized tests is a problem. What those scores mean is much less objective than it appears.” They suggest the need for qualitative assessments by educators.

But there are problems there, as well. It was Jann Murray-Garcia who in 2002-03 who sounded the alarm that “no African-American or Latino third-graders in the entire Davis school district were recommended by teachers to sit for the GATE test.”

The school district is hanging part of its cultural diversity hopes on the HOPE (Having Opportunities Promotes Excellence) scale, which they plan to start as a pilot that they will run concurrently with the current process.

Staff writes, “The HOPE Scale assessment was designed to identify and serve high-potential students from low-income families. Classroom teachers complete the HOPE scale for each of their students by answering eleven questions using a six point frequency response scale. The future use of the HOPE in DJUSD may mitigate for the inherent biases associated with other assessments.”

Are things better now than in 2002-03? Certainly the district is more conscious of racial bias and their discussion here confirms that belief. On the other hand, the district continues to struggle with the achievement gap, something that may be broadened with the wrong AIM approach. And while the district has utilized unconscious bias training in the past, we were told that it had been five years since the district received that training – not only have current teachers likely forgotten the training but there has been a very high turnover in the last few years, meaning many have not received any unconscious bias training at all.

In short, will the HOPE test work better than previous teacher-based assessments of gifted for minority and underrepresented populations?

That leads to the fifth question, what does the AIM program look like in terms of racial breakdown? We see in the analysis the belief by staff that raising the qualification score cut-off to the 98th percentile along with the end of private testing will likely reduce the size nearly in half, bringing it down to between 63 and 73 students.

That leads to a question: what does the racial breakdown look like? One poster yesterday suggested, “Why should we worry about the ethnic breakdown of the program when it should be about the needs of the children regardless of their race?” But then they couldn’t answer a question from another poster about the fairness of a system that identifies 95 percent of its students as white or Asian.

This isn’t about quotas or producing an exact replica of the district’s ethnic breakdown. Instead, it is about creating a system that is fair, that can assess underserved populations just as well as it assesses whites and Asians. The OLSAT has problems there and the district attempts to remedy that by establishing risk factors (which go beyond simply race and ethnicity) as well as using the HOPE scale pilot.

Tobin White’s research suggested that, at the previous 96th percentile level, the OLSAT-identified population was 48 percent white and 44 percent Asian, suggesting that an OLSAT-only program would produce a program that underrepresented both Latinos and Blacks.

On Saturday and again on Monday, I sent two questions to Superintendent Winfred Roberson. First, what is the projected racial/ethnic breakdown of the proposed revised AIM program? Second, what percentage of the current minority students in AIM would qualify under the proposed changes?

They seem like reasonable and critical questions. If the superintendent does not know the answer, that is fine, but he did not even respond. That is disappointing.

It was pointed out that this is, in fact, a slow process. The school board will not act on these recommendations this week. To me, though, that requires that we raise the tough questions and hope that the district will take them into account as they refine and finalize this policy.

For me this isn’t about defending the status quo. The more I looked into the issue of identification, the more I realized that we needed to change that. Private testing has some rationale, but late arrivals into the school district should be offered school district-based assessment. That seems like an easy fix.

The second question is whether the district’s second level assessments are the right tools. The district added some tools, but interestingly enough did not eliminate the TONI, which seemed a huge source of controversy.

I agree with the need to tweak and refine the identification process. Where I am less sure is the need to make the major changes to the program that the district did. I am not necessarily opposed to them, but want to hear a more objective explanation of how these are educationally-based changes rather than changes motivated simply by the desire to cut the size of the program for other reasons.

—David M. Greenwald reporting

Categories:

Tags: