Rethinking School Assessment: 3rd Grade Math TCAP Performance in Pueblo

In recent posts I have argued for a more nuanced approach to evaluating school performance in Pueblo. I also illustrated the strong negative State-wide correlation between % students who qualify for free or reduced lunch and Math TCAP scores.

It’s clear to me that student household income plays a very important role in determining school performance on Math TCAP. But, it’s not at all clear to me how this is taken into account by the Colorado Department of Education. It’s certainly not evident in this “mission statement” that economic disparities will play any role in assessment.

The Transitional Colorado Assessment Program (TCAP) is Colorado’s standards-based assessment designed to provide a picture of student performance to schools, districts, educators, parents and the community. The primary purpose of the assessment program is to determine the level at which Colorado students meet the Colorado Model Content Standards in the content areas assessed. The TCAP is collaboratively developed by the Colorado Department of Education, the Colorado teaching community and CTB/McGraw-Hill. The data should be used to keep abreast of individual student, school, and district progress toward attaining higher student achievement levels. The fact that TCAP is based on the Colorado Model Content Standards will ensure that all districts are held to the same challenging standards that Coloradans expect for their students regardless of whether they live in urban, suburban, or rural areas.

– See more at: http://www.cde.state.co.us/assessment/coassess-about

Do we plan to only reward school districts with the highest student household incomes? Do we plan to ignore all the tremendous work being done by teachers who are willing to take on the challenges presented by low income schools? What will we do when the State takes over some of the lower-income school districts only to find that they can do no better? At that point won’t it be too late for the school district, the teachers, the parents and the students?

Well, here’s one way to evaluate individual schools and school districts that takes into account economic circumstances.

Using the scatter plots and regression equations I developed using the Math TCAP data, I’ve produced “predicted” TCAP scores using % Free and Reduced Lunch as the independent variable. These predicted scores are represented in the “y-hat” column below. Actual scores (2014 % Prof & Adv) are then compared to predicted scores to calculate a residual value (i.e., actual score minus predicted score). Positive residuals indicate schools where test performance is exceeding expectations. Likewise, negative residuals indicate schools where test performance is below expectations. For easier interpretation I’ve then added the residuals to 100 producing a “Teaching Effectiveness Index” or TEI shown in the right-most column in the table below.

Using this approach, I’ve ranked 33 public elementary schools in Pueblo. The top 3 schools are all from District 60. So are the bottom 5.

So the narrative you’ve been hearing about District 70 good, District 60 bad is based on a lazy analysis of the TCAP results and lacks credibility. The reality is far more complicated. Both school districts are doing a great job in some places and not so great in others. Of course, that story doesn’t sell as many newspapers. The Pueblo Chieftain should apologize to readers for recklessly throwing D60 and, with it, the City of Pueblo, under the bus. Instead of using test scores to malign entire school districts who have been dealt a difficult hand why not use score results to highlight the accomplishments of individual schools where teachers and their students are beating the odds? Isn’t that the whole idea behind assessment?

Well, that is exactly the idea I have in mind. So, without further ado, here are my awards for outstanding teaching effectiveness in elementary mathematics.

• Gold Medal: South Park Elementary School (D60)
• Silver Medal: Ben Franklin Elementary School (D60)
• Bronze Medal: Sunset Park Elementary School (D60)
• Honorable Mention: Vineland Elementary School (D70), Haaff Elementary School (D60) and Carlile Elementary School (D60)

Congratulations to the teachers and staff at these top schools for their outstanding performance!

Note to School Administrators: Let’s reward the teachers at these schools with more resources and higher compensation levels. And let’s involve these teachers when it comes time to plot a strategy and implement tactics to raise performance at schools on the bottom of the list.

Read the Previous Post in this Series | Read the Next Post in this Series