• Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that have been used in the blog.
  • Bloggers
    Bloggers Search for your favorite blogger from this site.
  • Archives
    Archives Contains a list of blog posts that were created previously.
  • Login
    Login Login form

Posted by on in Assessment

So about that actionable data...

One of the frequently-offered reasons for the Big Standardized Tests is that they are supposed to provide information that will allow classroom teachers to "inform instruction," to tweak our instruction to better prepare for the test better educate our students. Let me show you what that really means in Pennsylvania.

Our BS Tests are called the Keystones (we're the Keystone State-- get it?). They are not a state requirement yet-- the legislature has blinked a couple of times now and kicked that can down the road. Because these tests are norm-referenced aka graded on a curve, using them as a graduation requirement is guaranteed to result in the denial of diplomas for some huge number of Pennsylvania students. However, many local districts like my own, make them a local graduation requirement in anticipation of the day when the legislature has the nerve to pull the trigger (right now 2019 is the year it all happens). The big difference with a local requirement is that we can offer an alternative assessment; our students who never pass the Keystones must complete the Binder of Doom-- a huge collection of exercises and assessment activities that allow them to demonstrate mastery.  It's no fun, but it beats not getting a diploma because you passed all your classes but failed on bad standardized test.

Why do local districts attach stakes to the Keystones? Because our school rating and our individual teacher ratings depend upon those test results.

So it is with a combination of curiosity and professional concern that I try to find real, actionable data in the Keystone results, to see if there are things I can do, compromises I can make, even insights I can glean from breaking that data down.

The short answer is no. Let me walk you through the long answer. (We're just going to stick to the ELA results here).

The results come back to the schools from the state in the form of an enormous Excel document. It has as many lines as there are students who took the test, and the column designations go from A to FB. They come with a key to identify what each column includes; to create a document that you can easily read requires a lot of column hiding (the columns with the answer to "Did this student pass the test" are BP, BQ and BR.

Many of the columns are administrivia-- did this student use braille, did the student use paper or computer, that sort of thing. But buried in the columns are raw scores and administrative scores for each section of the test. There are two "modules" and each "module" includes two anchor standards segments. The Key gives a explanation of these:



Last modified on

Posted by on in Assessment

Last Sunday afternoon the set was struck and the stage swept clean. We've come to the end of this year's spring musical. As always it was one of the highlights of my year, and as always, it reminded me of how inadequate so many of our educational models are.

There are weeks of rehearsal, learning music, learning choreography, working on blocking and lines and the underlying character work that goes with all of that. We have a cast of students in 7-12 grade in very many levels of skill and experience.

That means that in the course of assembling the show, each student learns a different set of lessons that depend a great deal on what roles they receive and what skills they bring to the table, as well as their ambition and adventurousness of spirit.

So this educational experience is extremely personalized, and that means far more than I have twelve lessons to choose from and a computer picks the next one based on how the last one turned out. My lead actor may need to learn about comedic timing, while one of my chorus folks may need to learn about the importance of the chorus in a show. My leading actress may need to learn about how to flesh out a character when the writers haven't given you much to work with. But the list of lessons will be different for every different role and every different cast member.

The lessons also vary with directors. This program is a co-op that allows my school to join in with a school just across town, and I split directing duties with an old friend who heads up the other school's program. We've divided up duties many different ways over the years, and it works because we work well together. Every theater production is a collaboration of some sort, and that collaboration is always shaped by the approaches of the people involved. Some directors have a very specific vision for the actors to bring to life, while others like to leave spaces for the actors to fill in with their own choices. We tend toward the latter, but some actors are more comfortable with the former and all sorts of combinations can get good results (and the requirements of the script itself also make a difference). All of which means that if you showed up with a specific program for exactly how a director should put together a show, I would laugh at you. Here we are with a performance based task that literally comes with a script-- and yet only a fool would claim that the script is all you need to produce a great show.

Likewise, putting on a show is the very definition of a performance-based learning experience. Yet if we were to follow the PBL model currently favored, we would break the show down into a checklist. Does the actor know the lines? Check. Does the actor know the blocking? Check. Can the actor put on her costume? Check. And on and on and even if I have checked off every micro-credential on the list, that is not the same thing as actually performing the show. Nor do we build toward that performance capability by working down the list one separate performance task at a time, because they are all part of a greater whole.

And those tasks would be performed for an evaluator, an assessor of some sort, which is not the ultimate goal. Our show was performed in front of an audience, and because it was a comedy, the audience reaction was a critical part of performance (in fact, on our second night, I saw something I've never seen in school or community theater before-- the show was stopped by audience laughter). Unlike competency-based education, which presumes that competencies can be approached as separate, discrete skills that can be measured through proxies, tasks that aren't the real thing. There is no checklist that would have substituted for dress rehearsal, no assessment more valuable than audience reactions in performance.

And speaking of assessments-- at no point in the eight-week process of preparing the show would a multiple-choice standardized test have been useful.At no point in the process did anyone think, "Hey, we need to do some assessments here to make sure that everyone is on track for a good performance." It would have been a pointless, useless waste of time.

In fact, standardization of any type is useless in this process. I have no idea how many productions of The Addams Family have been put on in community and school theaters at this point, but I will bet you the farm, the rent money, and a full box of donuts that not one of those productions looks exactly like any other. It's true that nobody who saw our production would have mistaken it for Hamlet or Oh Calcutta, but every production exists at the intersection of a specific cast, director, school, community, and stage (ours has no fly gallery, so that affects set design considerably). School theater in particular has to make adjustments for things as simple as language and as substantial as character gender (I can tell you, for instance, that interesting things happen to the subtext of Disney's Beauty and the Beast when Belle's crazy father Maurice is replaced by Belle's crazy mother Marie). It is those specific variations that most often give the special flavor and quality to the local production; the deviations from the standard are a source of excellence, not treatment-demanding flaw.

I love working with students and theater (despite the giant chunks of my life that it demands) because it is an experience that, in an absolutely authentic manner, helps each student grow and learn and discover new greatness in herself. It is an absolutely real learning and growth experience, which is why I'm always struck by how completely it does not match any of the assumptions about real learning made by the forces of ed reform. This is what real learning and growth look like, and they don't resemble the whole standard-driven test-centered punishment-fueled system that has been forced on us for the past fifteen years.

Last modified on
Hits: 1967 Comments

Posted by on in Assessment

This work is Romantic because the author used lots of Romantic ideas, and the characters behave in a Romantic way that captures just how very extremely Romantic the work really is. The author has really infused Romanticism into the whole writing in a way that makes in undeniably Romantic.

Welcome to my world. While this is not a direct quote of an actual student essay, it's of a type that English teachers often see. Call it support via assertion, or argument by modifiers (the more adjectives and adverbs you throw in, the more absolutely very clearly definitively true your argument is).

It is one of the few things that the Common Core actually gets right-- if you are going to make a case for a point, you need to provide evidence.

Evidence can take many forms, but it needs to be specific. It needs to be true.

Repetition is not evidence. Here's another archetypical essay paragraph.

Good parents need to be patient, because you need patience to be a good parent. A good parent is able to be patient. If you can't be patient, then you will not be a good parent. Every day, good parents must display patience, because if you are not patient, you cannot be a good parent.

It's hard to say exactly where students pick up the technique of un-supported ideas. Certainly we can reinforce it in school without meaning to. Tests where the student just has to mention a key idea or fact without backing it up help push the notion that we just want you to say the right thing. And of course our young humans come with plenty of pre-packaged ideas from home-- it must be true because it's what I learned from my folks, what do you mean I have to back it up with something.

Nor do I think test-centered schooling has helped. We have taught students that answers are "found," not created or built or supported, but plucked from a pre-created array of possible answers. We just hunt down the "correct" answer and mark it. Even when a standardized test pretends to involve support, that support is itself just one more "find the right answer" exercise.

And of course, it is tried and true in our culture that evidence is not really necessary. Yes, I can make the easy point that our current President and his administration are huge on the whole Just Repeat It Till People Believe It approach. Biggest inauguration crowd ever. Huge margin of victory. Millions of illegal voters. Urban hell holes. Just keep saying it and insisting that anyone who contradicts you is a liar, a faker, a Bad Person, even as you offer not one shred of evidence of the truth of what you say.

Yes, I could point at Herr Trump and say, "See! Our President does it. How am I supposed to teach children to do better, to use evidence?" But that would be the low-hanging fruit, and it would treat us all to the soothing notion that Trump somehow emerged out of the ether, full-blown flush with his lies and his fact-free anti-evidence zone.

But that would be going to easy on our culture. It's no coincidence that the Trumpistan flag was first planted on television, where citizens are bombarded with a constant stream of thirty-second playlets built on spin, deception, half-truths, and plain old bullshit. We soak in lies all the time, soak in them so that we can be softened up to be happy consumers of things we don't need that offer magic that doesn't work in order to solve problems that we don't have. We watch longer dramas that tell us lies about how people think, how the world works, what makes human beings click and work and become their best.

Where in our culture would students find examples of the notion that an idea should be grounded in truth, built out of evidence, supported by substance. What do we have in our culture that works that way?

The best I can do is present the practical notion that you have to do some sort of work in order to convince people to agree with you. The idea of pursuing the truth as a value in and of itself is a far bridge indeed. Evidence? That's a hard sell. We can all do better.

Last modified on
Hits: 1891 Comments

Posted by on in Assessment

Test 1

Recently the Tampa Bay Times reported on a great new program being pursued by Pinellas County schools to raise school ratings. The program could best be described as "Just stop having school and devote your time to test prep instead."

The article focuses on differences that are emerging between biweekly test results for 3-6 grade students and K-2 students. In doing so the article completely breezes past the fact that these schools are giving biweekly tests to K-2 students.

There is so much educational malpractice jammed into this whole stupid package.

2e1ax elegantwhite entry pinellasThe biweekly testing is being done in Pinellas "transformation zone" schools, aka "schools with lousy ratings" aka "poor schools." Pinellas County (that's St. Petersburg etc) schools have seen a transformation common in Florida, with shrinking enrollment and huge piles of money being funneled into mismanaged charter scams. But the story in Pinellas County is even worse than that, because the Pinellas County school board purposefully manufactured these failing schools. Let's pause for a history lesson.

You can read the full story here, or my shorter version here. But let me lay out the short ugly version. But if you remember the story of "failure factories" in Florida from a year or so ago-- well, that's where we are.

So the district created transformation zones in which they promised to focus on these poor schools and get them what they should have had (and used to have) all along. Last spring Pinellas County was looking for "transformational leaders" to run their elementary and middle schools. So what do transformational schools get?

They get Antonio Burt, a roving ronin of school transformation with experience within Tennessee's "innovation zone." What else do they get?

They get testing every other week for their littles. Every other week. What possible justification is there for biweekly testing? Well, according to the Tampa Bay Times:

The tests, which are new this year and are only being given in those schools, are being used to help teachers identify how well they have taught the state standards and to catch students' weak areas earlier in the year. 

Oh, baloney. This is training. This is the rankest kind of test prep. This is making the students well-rehearsed little test-taking machines. It is throwing up your hands and admitting that the Big Standardized Tests are not legitimate measures of anything except test-taking prowess, and while I applaud the recognition of reality, this is terrible education malpractice.

First, a generation of students is being taught that you go to school to take a test, and that's all education is. This is the worst kind of lie, a selfish inexcusable lie told to our most vulnerable children.

Second, just what has been cut out of the curriculum to make room for all this testing? If each administration of the test only ate only one day, that would still be eighteen days of school given over to testing, which is a almost four weeks, a month. A month of actual instruction lost to these students.

Third, these are the students who are going to be least helped by an education that is all about doing well on a Big Standardized Test. The deck is already stacked against them, and being well-versed in the taking of standardized tests is not going to help them.

This kind of baloney is most damaging to the small children, but it's bad news for all the students in Pinellas County.

Other misguided "transformational" ideas are hinted at in the article.

Antonio Burt, who is leading the Pinellas transformation effort, said teachers are not waiting to expose students to advanced concepts. For example, a standard usually scheduled to be taught in February — one that could count as much as 40 percent on the Florida Standards Assessment — now is introduced to students in August, giving them more time to practice.

SMH. First of all, this is the very definition of test-centered curriculum, which is an absolutely indefensible practice. Second of all, how does this even work-- students, I know we haven't laid the groundwork for any of this, and it involves concepts you haven't been taught yet, but we're just going to skip to chapter twenty-three on the text-book. I mean, I guess this is genius-- we can just "introduce" the quadratic formula to Kindergartners because if we introduce it sooner, they'll do better on the test, right?

Transformational schools are all about the test. Here's one super-swell motivational piece--

At Sandy Lane Elementary, principal Tzeporaw Sahadeo adds some encouragement for the children. She created the 80 Percent Club to recognize students who scored at least an 80 percent on their biweekly tests. 

Those students get to cut the lunch line for the week and are given 80 "shark shillings" — enough for a bag of coveted Takis spicy chips from the school store. Incentives also are given for children who barely miss the mark and earn 70 percent.

Yes, the school ties when you get to eat to your test score. That's not just a bizarre example of an extrinsic motivator, which we've long known is not a healthy sort of motivation to saddle a kid with. It also means that every day at lunch, students are lined up publicly in the cafeteria according to test results. If you thought a data wall was bad, how do you feel about a data lunch line?

The hook for this article is the mystery of decreasing test scores. The littles do well on the tests, but older kids do not, particularly on the literacy test. What could explain it? The article considers two explanations. One is that the standards get harder and more complex. And Burt suggests that there are "pockets of teachers" who "need reinforcement on what the standards are." I would suggest some other theories. One is that the standards are bunk. Another is that standardized literacy tests don't really test literacy. Yet another would be that the older students get, the less inclined they are to jump compliantly through hoops that they see as useless and pointless and part of an educational system that is not offering to give anything to them, but instead only wants to get them to produce scores for the school's benefit.

Test-centered education is ultimately always backwards. The school is not there to serve the students by providing them with an education. Instead, the students are there to serve the school by generating the numbers the school wants to get.

It is possible to have some understanding for Pinellas school leaders, who are staring down the barrel of Florida's immensely stupid, damaging, and unhelpful test-based school grade system. Throughout Florida, many schools face that one basic choice-- do they actually work at providing students with a real education, or do they make their school test centered in an effort to avoid punishment for low scores? In a state that is determined to break down its public schools, the better to drive parents and students into the arms of the charter industry, that's not a small or easy dilemma for public schools to face.

But Pinellas County has chosen poorly (and the Tampa bay Times has, on this occasion, reported lazily by not asking for evidence that any of these practices actually work). Test-centered education isn't good for anybody except the businesses selling test materials. Pinellas County has lost its way, but it's the students who are getting abandoned in the wilderness.

Last modified on
Hits: 2638 Comments

Posted by on in Assessment


What are we grading for?

During teacher development I provide, grading is one of the topics most commonly brought up. We as teachers have an obsession (and rightly so) with how any activity, instructional method or new tactic fits into that little book (or software) where we calculate student grades.

If you think about it, grades mean so much to teachers because, a lot of times its what we are being graded on. I've never heard a parent thank a teacher for assessing a student as earning an "F" in their class. I have, however, heard a lot praise for teachers who have high numbers of passing students.

So what's in a grade?

Last modified on