Tag: TCAP

Rethinking School Assessment: 10th Grade Math TCAP Performance in Pueblo

This will probably be the last post in my series on TCAP scores and Pueblo Schools. If this is the first you’ve seen of the series you might want to go back and read the first 4 posts:

In this post I continue my assessment of TCAP test performance by looking at Pueblo’s High Schools. Let’s get straight to it.

There are only 6 public High Schools with sufficient data so the Gold, Silver, Bronze thing doesn’t make as much sense here but it’s too late to change it up. Instead of 3 honorable mentions I will only recognize 1 runner up because, hey, in the education landscape after passage of No Child Left Behind there have to be losers, right?

  • Gold Medal: Centennial HS
  • Silver Medal: Pueblo County HS
  • Bronze Medal: East HS
  • Honorable Mention: Central HS

Congratulations to the hard-working teachers at these High Schools!

PuebloSchoolsTeachingEffectiveness-Math10

As a Centennial graduate I’m proud to see my Bulldogs on top (TCAP Bell rings Red!) but it’s a bit disappointing that none of the area high schools outperformed relative to the trend line. A more nuanced look at TCAP performance and school/district evaluation would look at how performance, again relative to the trend line, improves or declines year over year. Looking only at absolute scores is not fair to the teachers in low-income schools and school districts.

My approach may not be a perfect solution to the thorny problem of public school assessment but, in my view, it beats the hell out of what I see going on now. Rewarding or punishing school districts without any consideration for socioeconomic variability makes no sense.

If you agree, post a link to these articles on the CDE Facebook page or on Twitter. In addition, you might consider (warning: you’ll have to endure a horrible website replete with pop-up ads if you follow this link) sending a letter to the editor of the Pueblo Chieftain letting them know their reporting on local TCAP results was completely misleading.

 

Read the Previous Post in this Series

More

Rethinking School Assessment: 7th Grade Math TCAP Performance in Pueblo

Did anyone notice the Chieftain softening its tone in their editorial this morning? They acknowledge that “external, socioeconomic factors” can impact student performance. Of course, they can’t admit any error in their rush to judgement so they quickly remind us that even if a child is deprived of adequate food, shelter and a supportive family it “does not preclude him or her from learning entirely” as if to say, now that we’ve acknowledged students don’t do as well if they’re hungry we feel entitled to revert back to our grumpy stance regarding the failures of D60 since their union is, generally speaking, opposed to our political agenda.

Sort of like the Grinch giving Suzy-Who a drink of water and sending her back to bed before continuing to rob her home.

Oh well. I suppose I should be happy they included the word “socioeconomic” in the conversation. In any case, if you missed my first few posts in this series go back and read these before exploring my Middle School rankings further below.

And now, here are my rankings of Pueblo’s public Middle Schools in terms of Math Teaching Effectiveness as demonstrated during 2014 TCAP testing. If you haven’t already you’ll want to be sure and read Part 3 to better understand the methodology for coming up with the TEI values.

  • Gold Medal: Goodnight Elementary School (D60)
  • Silver Medal: Pleasant View Middle School (D70)
  • Bronze Medal: The Connect (D70)
  • Honorable Mention: Corwin International (D60), Craver Middle School (D70) and Beulah Middle School (D70)

Congratulations to all the hard-working teachers and staff at these Middle Schools!

And, I’ll repeat my suggestion…

Note to School Administrators: Let’s reward the teachers at these schools with more resources and higher compensation levels. And let’s involve these teachers when it comes time to plot a strategy and implement tactics to raise performance at schools on the bottom of the list.

PuebloSchoolsTeachingEffectiveness-Math07

 

Read the Previous Post in this Series | Read the Next Post in this Series

More

Rethinking School Assessment: 3rd Grade Math TCAP Performance in Pueblo

In recent posts I have argued for a more nuanced approach to evaluating school performance in Pueblo. I also illustrated the strong negative State-wide correlation between % students who qualify for free or reduced lunch and Math TCAP scores.

It’s clear to me that student household income plays a very important role in determining school performance on Math TCAP. But, it’s not at all clear to me how this is taken into account by the Colorado Department of Education. It’s certainly not evident in this “mission statement” that economic disparities will play any role in assessment.

The Transitional Colorado Assessment Program (TCAP) is Colorado’s standards-based assessment designed to provide a picture of student performance to schools, districts, educators, parents and the community. The primary purpose of the assessment program is to determine the level at which Colorado students meet the Colorado Model Content Standards in the content areas assessed. The TCAP is collaboratively developed by the Colorado Department of Education, the Colorado teaching community and CTB/McGraw-Hill. The data should be used to keep abreast of individual student, school, and district progress toward attaining higher student achievement levels. The fact that TCAP is based on the Colorado Model Content Standards will ensure that all districts are held to the same challenging standards that Coloradans expect for their students regardless of whether they live in urban, suburban, or rural areas.

– See more at: http://www.cde.state.co.us/assessment/coassess-about

Do we plan to only reward school districts with the highest student household incomes? Do we plan to ignore all the tremendous work being done by teachers who are willing to take on the challenges presented by low income schools? What will we do when the State takes over some of the lower-income school districts only to find that they can do no better? At that point won’t it be too late for the school district, the teachers, the parents and the students?

Well, here’s one way to evaluate individual schools and school districts that takes into account economic circumstances.

Using the scatter plots and regression equations I developed using the Math TCAP data, I’ve produced “predicted” TCAP scores using % Free and Reduced Lunch as the independent variable. These predicted scores are represented in the “y-hat” column below. Actual scores (2014 % Prof & Adv) are then compared to predicted scores to calculate a residual value (i.e., actual score minus predicted score). Positive residuals indicate schools where test performance is exceeding expectations. Likewise, negative residuals indicate schools where test performance is below expectations. For easier interpretation I’ve then added the residuals to 100 producing a “Teaching Effectiveness Index” or TEI shown in the right-most column in the table below.

Using this approach, I’ve ranked 33 public elementary schools in Pueblo. The top 3 schools are all from District 60. So are the bottom 5.

So the narrative you’ve been hearing about District 70 good, District 60 bad is based on a lazy analysis of the TCAP results and lacks credibility. The reality is far more complicated. Both school districts are doing a great job in some places and not so great in others. Of course, that story doesn’t sell as many newspapers. The Pueblo Chieftain should apologize to readers for recklessly throwing D60 and, with it, the City of Pueblo, under the bus. Instead of using test scores to malign entire school districts who have been dealt a difficult hand why not use score results to highlight the accomplishments of individual schools where teachers and their students are beating the odds? Isn’t that the whole idea behind assessment?

Well, that is exactly the idea I have in mind. So, without further ado, here are my awards for outstanding teaching effectiveness in elementary mathematics.

  • Gold Medal: South Park Elementary School (D60)
  • Silver Medal: Ben Franklin Elementary School (D60)
  • Bronze Medal: Sunset Park Elementary School (D60)
  • Honorable Mention: Vineland Elementary School (D70), Haaff Elementary School (D60) and Carlile Elementary School (D60)

Congratulations to the teachers and staff at these top schools for their outstanding performance!

Note to School Administrators: Let’s reward the teachers at these schools with more resources and higher compensation levels. And let’s involve these teachers when it comes time to plot a strategy and implement tactics to raise performance at schools on the bottom of the list.

PuebloSchoolsTeachingEffectiveness-Math03

Read the Previous Post in this Series | Read the Next Post in this Series

 

More

Math TCAP Performance Highly Correlated with % Free-Reduced Lunch

In my initial post on the topic of TCAP scores I pointed out that Pueblo District 60 Schools (D60) have more students who qualify for Free Lunch than the total number of students in all of Pueblo County District 70 (D70). I also said D60 has a more difficult task not only due to the fact that they are a much larger school district overall but also because D60 students come from households with far lower incomes, on average.

Some readers may not be convinced that lower economic status translates into lower test scores. Let’s see if I can convince those folks that there is indeed a strong relationship between household income and academic performance.

Rather than focusing on Pueblo Schools I want to look at a bigger picture to better understand the relationship between test performance and economic status on a State-wide basis. So these data displayed in the charts below cover all over Colorado’s public schools. Many schools have too few test takers to be considered statistically significant so those small schools were eliminated from the data set. I also limited data selection to 3rd grade, 7th grade and 10th grade. These grade levels are the middle grade (or close to it) for typical elementary, middle and high schools, so I think it’s an appropriate subset to explore. If I look at every grade level I worry the quantity of data displayed starts to become too much to digest. In addition, I’m only looking at Math scores and, for now, omitting Reading and Writing scores for two reasons. First, again, it’s difficult to look at too much data all at once and might lead to confusion. Second, in my opinion, Math scores tend to minimize cultural bias and, as such, provide a more accurate reflection of actual academic capability.

On each of the charts below the horizontal (x) axis represents the percentage of students who qualify for Free or Reduced Lunch programs. The vertical (y) axis represents the Math TCAP test score results. Each point on the chart represents one school’s Math TCAP scores and their corresponding % of students who are eligible for Free or Reduced Lunch. I’ve also included a trend line with the associated regression equation and R-Squared value (included in the upper right corner of each chart). Don’t worry if the equations make you nervous – these details are only there for reference.

grade03-math-freelunch
grade07-math-freelunch
grade10-math-freelunch

As the charts clearly illustrate, or at least they clearly illustrate this to me, there is a very strong relationship between economic status and TCAP Math performance. As % Free-Reduced Lunch goes up, school TCAP Math performance goes down. Given the strength and consistency of this relationship, School District evaluation efforts and media coverage of local school performance results should focus on school performance within this context.

Furthermore, instead of blaming public school systems for failing our children perhaps we should consider blaming socioeconomic policy for producing too many students who are too hungry to succeed in school.

Read the Previous Post in this Series | Read the Next Post in this Series

More

Let’s Compare Apples to Apples When Evaluating Pueblo’s Public Schools

In recent editorials the Pueblo Chieftain has been singing the praises of Pueblo District 70 Schools (D70) and, in parallel both implicitly and explicitly, criticizing Pueblo District 60 (D60) for failure to achieve similar TCAP testing results. I needed a good set of data for my Statistics classroom so I decided to dig in to the TCAP data hoping to learn more about TCAP scores in general while exploring how well D70 was actually performing relative to D60.

The TCAP data are like most data sets produced by large government agencies, i.e., a big mess and far more difficult to interpret than necessary. But, I’ve seen a lot worse.

I downloaded the 2014 Math, Writing and Reading scores to see what I could learn. Boy, there’s a lot of data to sift through. Probably someone could do a doctoral dissertation analyzing the Colorado TCAP data. I don’t have time for that…at least not at the moment. So, for now, let’s look at the difference in economic circumstances between the two school districts.

The most readily available indicator of economic status by school and by district are the % Free and Reduced Lunch figures. Conveniently, the Colorado Dept of Education breaks down the testing results into 3 categories: (1) students qualifying for free lunch, (2) students qualifying for reduced lunch and (3) students who do not qualify for free or reduced lunch.

Here’s a table summarizing what I found:

freelunch

Anything stand out to you?

What jumps off the page at me is that D60 educates more children receiving Free Lunch than the total number of students in D70. Somehow, I missed that in the Chieftain’s reporting.

Let’s say you’re a teacher and the principal asks you to choose between 2 classrooms, both with 30 students. The principal reminds you that the fate of your job, the school and the entire school district depends on you getting your students to succeed on a set of culturally biased standardized tests. As a professional educator you know that academic success is closely correlated with economic status. In classroom A you have 19 kids receiving free lunch and in classroom B you have only 9 kids receiving free lunch. Which classroom would you choose?

If we’re going to hand out trophies to D70 and castigate D60 let’s be sure to provide appropriate context. D60 has, by far, the more difficult task. To read the Chieftain you might mistakenly assume both districts are in the same boat.

I plan to dig further on this. Next post will likely explore the relationship between TCAP scores and % Free-Reduced Lunch eligibility. So stay tuned and, in the meantime, don’t be so quick to throw D60 under the big yellow bus.

Read the Next Post in this Series

 

 

More