The point of visualizing data is discovering human stories
I just returned from the annual conference of the California Education Research Association, feeling both pleased and disturbed. What pleased me was the conference theme itself: the power of visualizing data. That’s an entirely suitable topic for the members of an organization in charge of communicating the meaning of quantitative measures. What disturbed me was that the presenters I heard [...]
Do you really know the work your assessment director is doing?
Over the past 20 years, I’ve met at least 60 assessment directors among the 240 districts we served. When I asked them to tell me about their work, I was struck again and again by a common theme. Most of them felt misunderstood by their peers, and under-valued by their boss. Assessment directors are navigators. Superintendents are captains of [...]
Supes are asking for help with LCAPs: candid criticism and appeals for technical support
What has happened to the high hopes policy heavies had for Local Control Accountability Plans? After reading dozens of districts’ plans, and after working side by side with district LCAP planning teams in three counties, I suspect those high hopes have for the most part remain unmet. Have those hopes all been dashed? I don’t think so. But with a [...]
Steve Gervais: a K12 mapmaker who’s making sense of student data
Steve Gervais is an explorer. He looks for patterns … in particular, geographic patterns that could help solve riddles that school and district leaders wrestle with daily. When he learned that his district, San Bernardino City USD, was wrestling with high chronic absentee rates, he wondered if those students missing school might tend to live near each other. He built [...]
What data skills should K12 leaders expect from their planning teams
After reading dozens of district LCAPs this year, I’m appalled and worried. What appalls me most is not their length or the poor writing that prevails. (If you doubt me, take your own district’s LCAP in hand and just identify every sentence that’s expressed in the passive tense.) What appalls me most is the prevalence of sloppy logic and weak quantitative [...]
What’s wrong with data-driven decision-making?
Let me know if this surprises you. I don’t believe in data-driven decision-making. In fact, when I hear someone affirm this phrase, my b.s. detectors start ringing. When does data ever drive anything? People drive things. And passion drives people (let us hope). The desire to eat well drives me to cook a tasty meal. But I am not driven [...]
How much does funding vary among neighboring districts?
If you go to lunch with a dozen of your co-workers, and ask them what they think is at the heart of the variation in district funding, I bet you’d hear a dozen answers. Some would say: “Wealthier communities fund their schools better.” Someone else would say: “Districts with more kids who are harder to educate get more money. Another [...]
Why “continuous improvement” is never a straight line
“The key question is whether teaching can shift from an immature to a mature profession, from opinions to evidence, from subjective judgments and personal contact to critique of judgments.” — John A.C. Hattie, Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement, Routledge (2009), pg. 259. I am no longer surprised when reading California school districts’ Local Control Accountability Plans [...]
Steve Ayon’s popular visualizations of CAASPP/SBAC and accountability data
Steve Ayon is a one-man wiz whose smart work visualizing CAASPP results and CDE accountability findings has proven to be quite popular. His CAASPP/SBAC visualizations have been used more than 4,000 times, and the accountability results have enjoyed more than 8,000 uses (stats courtesy of the Riverside CoE Tableau Public profile page). Thankfully, the Riverside County Office of Education has [...]
Does the Dashboard’s combining status and change help or hurt?
The Dashboard has changed a little each year in the way it presents and interprets measures of districts’ and schools’ vital signs. But one thing that has remained unchanged is the Dashboard’s reliance on combining the status of a vital sign with its rate of change. Has cobbling together two attributes into one color indicator helped or hindered the public’s [...]
Two more Dashboard mismeasurements: English learners and grad rates
Two more areas of mismeasurement are baked into California's accountability policies, and are expressed in its Dashboard. One area is English learners. The other is graduation rates. The transient category of "English learners" I'd like to meet the policy people who believed it was legitimate, fair and meaningful to look at the year-to-year change in test results for students deemed [...]
Five Measurement Flaws That Compromise the Dashboard
The CDE’s Dashboard has become the new interpreter of schools’ and districts’ health. Its dials and color matrixes have replaced numbers as the measurement language of accountability. Its rules determined in December 2018 that 374 districts deserved “differentiated assistance” and the money that comes along with it (between $100,000 to $300,000 per district, depending on enrollment). The Dashboard’s signals have [...]
The End of the Era of Statistical Significance (and Why You Should Care)
The editors of the official journal of the American Statistical Association have put a spike in the heart of a monster of their own creation. The monster is the test of statistical significance, a measure of the quality of a sample of evidence, and a magical line that accepts results when p<.05 and rejects evidence when p>.05. Let us all [...]
Nine Ways to Put Student Level Data to Work That FERPA Allows
There's a hypersensitivity in California school district offices to the use of student record level data. Internally, district leaders call it a litigation risk, deferring to the advice of their legal counsel. But externally, district leaders defend this in the name of protecting student privacy. This rubs me the wrong way for two reasons. First, the internal motivation is the [...]
Missing John Mockler: His Wit and Wisdom
It’s been four years since John Mockler died, and I miss his voice, his views, his candor and his humor. I'd like to hear him jousting with the leaders of the K12 establishment. I imagine he'd be ready, willing and able to to interrupt the chorus of conformity that’s all too common when people talk about what’s right or wrong [...]
Collateral damage from the accountability wars
[This interview with me appeared in Jenny Rankin's PSYCHOLOGY TODAY blog on January 19, 2019. Its title: "War Stories From the Accountability Battlefield."] Jenny Rankin (JR): What are some faulty conclusions you have seen educators draw from data in the California School Dashboard, the state's accountability measurement system? Steve Rees (SR): Let’s start with two examples of schools’ vital signs [...]
CERA’s gentle nudge to improve members’ measurement readiness
Keynote speech by Michael Wimberly at the CERA annual conference, November 2018 Michael Wimberly was the keynote speaker on the morning of the first day of the California Education Research Association annual conference on November 13. Over 45 minutes, he walked us through the Air Force’s approach to preparing him to succeed as a jet engine mechanic and [...]
Site and district leaders are learning online how to put numbers to work
Prof. Alex Bowers, Teachers College, Columbia University Meet Alex Bowers. He is a hard-charging, energetic professor at Columbia University’s Teachers College who is opening doors for school and district leaders. Those doors lead to the numbers way of knowing. Numbers that can help district leaders solve riddles like these: Are students missing more days of instruction in schools [...]
Good news awaits if we stop measuring ELs and start measuring “Ever-ELs”
The California accountability system’s interpretation of the progress of English learners is wrong. That’s the bad news. Its flaws are explained in a recent article, “Hidden Progress of Multilingual Students on NAEP.”* that appeared in the August/September 2018 issue of AERA’s bimonthly journal, “Educational Researcher.” The irony is that we are missing a happy conclusion. If we look at students who were [...]
Why I’m a skeptic of “best practices”
I spent September 13 at a special event in Palo Alto to discuss the potential for evidence generated when students use online instructional materials. About 30 people gathered from around the country, and they shared deep experience and concern with building better evidence, enabling applied research to improve K12 education. The star of the day was Mark Schneider, the newly appointed chief [...]
