Someone should really make a documentary about this. A “rock-star” Stanford Education professor who helped inspire math de-tracking in San Francisco and who subsequently helped re-write the California math curriculum with an emphasis on equity has been accused of dozens of cases of “citation misrepresentation.”
A Stanford University professor, whose research was credited with inspiring San Francisco’s failed experiment to ax 8th grade algebra, is facing allegations of “reckless disregard for accuracy” in her work, according to an official academic complaint filed Wednesday with Stanford’s provost and dean of research.
The anonymous complaint, backed by a California-based group of math-and-science focused professionals, alleges that Professor Jo Boaler—the most prominent influence on California’s K-12 math framework that nudges schools away from accelerated math pathways—has in 52 instances misrepresented supporting research she has cited in her own work in order to support her conclusions. These include the notions that taking timed tests causes math anxiety, mixing students of different academic levels boosts achievement, and students have been found to perform better when teachers don’t grade their work. This pattern of “citation misrepresentation,” the complaint alleges, violates Stanford’s standards of professional conduct for faculty, showing a disregard for accuracy, and may violate the university’s research integrity rules.
The complaint is here. Many of the examples come from the California Math Framework (CMF) and were first identified by Dr. Brian Conrad. I’ll talk more about him in a moment but first here’s a sample of the kind of thing we’re talking about.
Claim as written in the CMF:
“…positive outcomes for achievement and longer-term academic success from keeping students in heterogeneous groups focused on higher-level content through middle school.” “The researchers found that the students who learned in the heterogeneous classes took more advanced math, enjoyed math more, and passed the state Regents test in New York a year earlier than students in traditional tracks. Further, researchers showed that the advantages occurred across the achievement spectrum for low and high achieving students (Burris, Heubert, and Levin, 2006).” SFR Ch. 9 Line 178-181 & 285-289
Dr. Boaler used this same argument in her own work as a way to argue in favor of de-tracking or keeping students at different levels all together in one class rather than separating them by ability. But here’s what the complaint says about the citation quoted above.
“One naturally wonders: what is that higher-level content in middle schools? Looking at the paper, it is Algebra I for all 8th grade students (in a “diverse suburban school district”). The CMF omits to mention this extremely pertinent fact. In other words, the study shows benefits from accelerating all students to take Algebra I in 8th grade and so the CMF cites the paper for its great success in heterogeneous education but doesn’t tell the reader that this was done with Algebra I in 8th grade (that wouldn’t fit the CMF’s preferred narrative). This omission puts the focus on heterogeneity and away from the highly relevant context of 8th grade Algebra I…So here there are a variety of very positive outcomes, all attributed to the CMF’s recommended methods “de-tracking” and “heterogeneous classes”. But New York’s Sequential Mathematics sequence was exactly an early incarnation of the Integrated Math rearrangement of content (I know this because I grew up in New York at the time it was first introduced), so here is a more direct description: it was an acceleration program to get everyone taking Integrated 1 in 8th grade. The tip-off that this is an acceleration program is that the cited 2006 paper of Burris et al. has “acceleration” in its title. The CMF should not hide behind obfuscatory buzzwords and instead be crystal-clear that it is actually offering great praise for teaching the Grade 9 math class to everyone in 8th grade. (I am not personally advocating that everyone should take Algebra I in 8th grade. Rather, I am just noting the actual meaning of what the CMF is praising in a specific paper it chooses to cite.).” Conrad – CMF Public Comment #2, Section 4.2, pg. 14 and Section 5.3, pg. 16:
So while this citation was used to argue for de-tracking, the citation was actually about having students take Algebra I in Middle School. Compare that with results of de-tracking in San Francisco, a decision which seems to have been partly inspired by Dr. Boaler. In 2014, SF adopted de-tracking which meant that more advanced students would no longer be taking Algebra 1 in 8th grade but would instead wait to take it with everyone else in 9th grade. At the time SF adopted this plan, the superintendent of San Francisco’s school district wrote an article praising Jo Boaler as a rock-star.
Bringing a rock-star education professor to our schools this week is not a coincidence. Boaler’s research follows the principles of our new core curriculum, which aligns with the Common Core State Standards.
What this means is, we will be giving students more time to dive into math concepts so they really understand them. Instead of asking kids to rush for the right answer, we will be asking them to make sense of the concept. We will be helping them connect important mathematical concepts from problem to problem and course to course so that math doesn’t feel like a bunch of isolated rules. We will ask them to keep trying when they tackle open-ended problems.
Contrary to Dr. Boaler’s suggestion that de-tracking should improve things, in San Francisco it was a complete failure. It was adopted with the explicit goal of closing the racial achievement gap, but it did not achieve any of its equity goals. What it did do was make it much harder for advanced math students to take the classes expected by universities. Students with engineering or science aspirations either had to take a year of math in a condensed summer school format or take two math classes at once in 9th grade in order to be competitive with the students around the country who’d taken Algebra 1 in Middle School. San Francisco reversed the de-tracking plan last month.
Despite the fact that de-tracking didn’t work as advertised, the San Francisco adoption of it was an inspiration for the California Math Framework which was drafted by, you guessed it, Dr. Jo Boaler and four other authors.
And that brings us to Professor Brian Conrad, the director of undergraduate math studies at Stanford. Back in 2022, he read the entire California framework and pointed out many of the bogus citations contained in it, most of which seem to be duplicated in Dr. Boaler’s own work. I wrote about his lengthy review of the California framework here. Last October, the Atlantic published a summary version of professor Conrad’s findings. Here’s a bit of what he said.
When I decided to read every word of California’s 1,000-page proposal to transform math education in public schools, I learned that even speculative and unproved ideas can end up as official instructional policy. In 2021, the state released a draft of the California Mathematics Framework, whose authors were promising to open up new pathways into science and tech careers for students who might otherwise be left behind. At the time, news reports highlighted features of the CMF that struck me as dubious. That draft explicitly promoted the San Francisco Unified School District’s policy of banishing Algebra I from middle school—a policy grounded in the belief that teaching the subject only in high school would give all students the same opportunities for future success. The document also made a broad presumption that tweaking the content and timing of the math curriculum, rather than more effective teaching of the existing one, was the best way to fix achievement gaps among demographic groups. Unfortunately, the sheer size of the sprawling document discouraged serious public scrutiny.
I am a professional mathematician, a graduate of the public schools of a middle-class community in New York, and the son of a high-school math teacher. I have been the director of undergraduate studies in math at Stanford University for a decade. When California released a revised draft of the math framework last year, I decided someone should read the whole thing, so I dove in. Sometimes, as I pored over the CMF, I could scarcely believe what I was reading. The document cited research that hadn’t been peer-reviewed; justified sweeping generalizations by referencing small, tightly focused studies or even unrelated research; and described some papers as reaching nearly the opposite conclusions from what they actually say.
Here’s one more example:
On the question of timed tests causing “math anxiety,” Boaler has asserted that “researchers now know that students experience stress on timed tests that they do not experience even when working on the same math questions in untimed conditions.” As evidence, she cites a study by psychologist Randall Engle. However, Engle’s paper in question deals with “working memory” rather than student anxiety, and Engle himself called the assessment a “huge misrepresentation” of his work.
Anna Stokke, a mathematics professor at the University of Winnipeg who has studied this claim and found that it contradicts available evidence, said many math teachers nonetheless seem to believe it—and that their belief seems to stem from Boaler.
The problems with the CMF and Dr. Boaler’s work on it have been out there in public for a while. It’s good to see it finally being challenged directly with Stanford University. Elite schools ought to hold their professors to elite standards.