Keynote speech by Michael Wimberly at the CERA annual conference, November 2018

Michael Wimberly was the keynote speaker on the morning of the first day of the California Education Research Association annual conference on November 13. Over 45 minutes, he walked us through the Air Force’s approach to preparing him to succeed as a jet engine mechanic and as an officer. The talk left many at my table puzzled. Why was he talking about his military experience? What did that have to do with the theme of the conference:  preparing students of all ages for careers and college?

Well, perhaps the conference theme – “Readiness At All Ages” – and his talk were intended to raise a question: what have school districts and schools of education done to prepare educators to measure the progress of learning? (Since he now works at the CDE doing assessment development, I wish he’d told us more about his prep for that job.)

The Air Force has a structured method to preparing its members to succeed. Mr. Wimberly explained that the bodies of knowledge candidates for each occupational specialty must master are clearly defined. The leadership and technical skills that officer candidates must learn are codified, and they are teachable. And the many steps of the Air Force’s training staircase are separated by assessments that candidates must pass if they are to advance to the next level. Advancement is also expected, and building skills leads to promotion.

I suspect that Mr. Wimberly was holding up the Air Force’s approach to career prep as a model against which everyone in the room might consider their own preparation for the jobs they hold in districts and county offices of education.

This leads me to offer three “wish list” ideas to CERA’s leaders.

  • I wish CERA would poll its members, and ask them if they’ve ever studied psychometric fundamentals, either during their formal education or after they started work. And if some of its members reply in the affirmative, wouldn’t it be interesting to ask to what degree this training prepared them to do their jobs successfully.
  • I wish CERA would build a list of the top ten measurement skills needed to interpret assessment results, and ask their members to report the degree to which they believe they are technically qualified to do those tasks. But of course, this begs three questions. What is the definition of “technically qualified?” Who decides what tasks deserve measurement attention? And against what yardstick should one measure the quality of work performed?

You may be surprised to learn that these questions (and more) have been answered definitively. I am referring to the Master Data Use Standards created in 2014-15 by a multi-state consortium of 27 educator, working under the sponsorship of the Statewide Longitudinal Data Systems grant program. One set of standards was written for administrative leaders, one for data leads and a third set for teachers. This group of 27 has done the hard work of defining the knowledge, skills and professional behaviors for people in all three roles.

This leads me to my third suggestion.

  • Perhaps CERA could survey its members to learn how many of them are aware of the existence of these Master Data Use Standards. Simply asking the question would call their members’ attention to this foundational document.

The job specific training Michael Wimberly enjoyed has no equivalent for California educators pursuing an administrative credential. Nationally, I am only aware of two institutions that offer courses of study that strive toward the military’s standard: Harvard’s Strategic Data Project and an emerging certification program – Leading with Evidence –at Columbia University’s Teachers College, led by Alex Bowers. (Note, School Wise Press is a partner of this Teachers College program.)

The lack of preparation in measurement skills – measurement of learning, human resources, money or operations – is endemic in education management. This is not my conclusion alone. It is the conclusion of scholars like Ellen Mandinach (WestEd) and Edith Gummer (Arizona State University). And it is a belief most strongly asserted by author, and educator of educators, Jim Popham, whose blunt critique of the blindspots of the education profession, Unlearned Lessons, includes a chapter titled “Abysmal Assessment Literacy.”

Whether you agree, or disagree, I want to hear from you. Our team at School Wise Press’s K12 Measures Project, is doing something to help district and school leaders and school board members augment their measurement skills. Let’s talk.