I will protect your pensions. Nothing about your pension is going to change when I am governor. - Chris Christie, "An Open Letter to the Teachers of NJ" October, 2009

Monday, June 12, 2017

Should Families Really Have to "Choose" Healthy Schools?

Looks like it's going to be a big week for school "choice." The National Charter School Convention is in full swing (I've been following the hashtag #ncsc17, and a better "reform" cliche lexicon you will not find). The folks at CREDO released yet another charter school report that once again invalidly translates small effect sizes into "days of learning" in a vain effort to show charter schools are full of chartery awesomeness.

And the doyen of "choice," Eva Moskowitz, has apparently picked up the coveted Broad Prize for her work in expanding a network of charter schools with practices so "innovative," she wants to put them on a digital platforms and share them with the world.

If there was any justice in this world, that video would have been running in a loop behind Moskowitz as she accepted Eli Broad's dough...

As those of us who follow education policy and live in the greater-NYC area know, Moskowitz's Success Academy has benefitted enormously from philanthropic giving. $8.5 million from a hedge-funder in 2015. $9.3 million in one night later that year. $25 million from another in 2016. Plus another $10 million from some pikers...

This is all in addition to the monies SA gets from the government for the students it enrolls. Moskowitz's powerful friends have even made sure that she doesn't have to play by the rules that everyone else must follow. The result is a school system swimming in money -- a system that relies on funds that no one else gets to access.

Understand, it's not just the kids on SA's "got-to-go" list who miss out on all this Wall Street largesse; any NYC student, public or charter, who isn't in Moskowitz's network misses out on the benefits of all this extra cash.

In the case of Success Academy and other big-profile charter networks, the benefits of lots of extra resources come at the cost of not having your child enrolled in a democratically and locally controlled public school, with greater transparency and greater access to due process and student/family rights.

Why I am bringing all this up today? Well...
PATERSON – Monday's heat wave prompted city school officials to send elementary students home at 1 p.m. on Monday.
High school students already were getting out early because of exams, according to district spokeswoman Terry Corallo. The district has more than a dozen schools that are more than a century old and lack air conditioning.
Staff members on Monday were required to stay at work until after 3 p.m., prompting criticism from the president of the teachers union. [emphasis mine]
And it's not just Paterson:
Thermometers are rising and more than 20,000 students in public schools in PlainfieldTrenton, and other districts throughout the state are being sent home early over the next two days. 
With the pressure of finals in the air, many students and school employees also have to contend with rising classrooms temperatures.
Few examples so elegantly show the wide disparities in school conditions in New Jersey.
In some districts, the rising temperatures won't mean much and the learning process will continue unabated.  In other districts, schools will be forced to shutter and students will lose precious hours of instruction. 
In what is often a clear divide between affluent and poorer districts, some students and school employees will learn in comfortable climate controlled classrooms, while others will struggle to learn and teach in classrooms with temperatures approaching and sometimes exceeding triple digits. [emphasis mine]
It would be an overstatement to say that every classroom in every affluent district in New Jersey has A/C; I know of several examples personally where that's not the case. But there's no doubt a student in the leafy 'burbs is much more likely to have A/C in her school than a child in an urban public school...

Unless that child is enrolled in a well-funded, well-connected charter school. From 2015:
Meanwhile, in Camden, the aunt of a student at Bonsall Elementary School posts a video (which I can't embed here because it's on Facebook, so click the link to watch) showing how students on the two lower floors are sweltering in classrooms with no air conditioning.

But up on the third floor, it's nice and cool. Why? Because that floor was taken over by the Uncommon charter chain, which somehow allowed the district to magically acquire the funds necessary for the school's renovation. Except somehow, when it came to HVAC needs, the floors housing classrooms for the public district schools didn't get refurbished in time for the start of the school year.
Bob Braun has been reporting on the disparity in A/C between charters and public district schools in Newark for years. In addition, when the new, modern Teachers Village was constructed in Newark, its three school spaces went immediately to charters; NPS schools were left to wither in the sun.

NYC Mayor Bill de Blasio, to his great credit, has made getting air conditioning units into NYC schools a priority. That's great, but I have to ask: why the wait? Why do charters like SA have access to millions of dollars in extra funding to keep their kids cool while over one-quarter of NYC classrooms swelter?

Why do Paterson public school students have to end their school day early while New Jersey gives out millions in financing to a publicly-funded charter that pays rent to a private entity? Why was there plenty of money for Teachers Village, occupied solely by charter schools, but scant few dollars to renovate Newark Public Schools' aging infrastructure -- including a plumbing system full of lead? Why did charter students in Camden get air conditioning while CPS students in the very same building did not?

No family should have to "choose" a healthy school for their child. If you really believe, as many of the fine folks partying it up at the NCSC Convention tonight apparently do, that we need more "cooperation" between district and charter schools, why would you stand for a school funding system that advantages high-profile, well-connected charters over public district schools and mom-and-pop charters that may not have hedge fund-types writing them big checks?

Again, there is no question that more affluent public school districts across the country have been unfairly enjoying a resource advantage over less affluent districts. But allowing vastly wealthy people to pick and choose which charter school networks they like, and then setting those schools up with both "no excuses" discipline and A/C, hardly seems like an equitable plan.

Rather than picking a few urban charter schools Hunger Games-style to get decent facilities, why don't we instead tax the donors to Success Academy a few percentage points more and use the money to make sure all schools are safe, clean, and healthy? 

Is anyone really against that?

h/t UFT

ADDING: Via Twitter, NPS staff report 100+ degrees today in some schools, yet no early dismissal.

Golly, I wonder what the temperature in Chris Cerf's office was...

I have always stated that the Paterson State Appointed District Superintendent Dr. Donnie Evans never fully understood the adverse impact that these inhumane conditions have on our students and our employees. I believe that this is especially true as he works from his air-conditioned 4th-floor corner office located at 90 Delaware Avenue. Be this as it may, while sorting through the OPRA request, I could not help but notice one particular document and the message it sends to our students and employees.
According to district records, on July 26, 2016, a receipt was paid in the amount of $250.77 for the following service/repairs, “AC not working, needs service.” According to these same records, the air condition repairs were made to a vehicle listing Dr. Donnie Evans as the driver. For those who do not know, Dr. Evans is provided the use of a District school vehicle. The irony here is that the repairs listed on the invoice I am referencing is for the same luxuries Evans has denied the students and staff without air conditioning for years.
A reminder: Evans serves at the pleasure of Governor Chris Christie, who wants teachers to work in the summer for no extra pay in classrooms with no A/C.

But not his state-appointed superintendents. Lovely.

Saturday, June 10, 2017

New Jersey's Totally Screwed Up Charter School System, Clifton Edition

Here we go again:
CLIFTON — City school board members remain united in opposition to the state education department's mandate that the district allocate $2 million toward a controversial charter school group.
The state Department of Education issued a directive to the Clifton school board late in the budgetary process, said Clifton school officials.
The directive calls for the funding of 225 students set to attend Passaic Arts and Science Charter schools, including a new building opening this fall on Clifton Avenue. The school formerly housed the Sacred Heart Elementary School.
The PASC is affiliated with the ILearn LLC Network, which is the subject of investigations due to allegations of fraud.
Business administrator Edward Appleton said the school system learned of the major increase the final week in February. The state’s directive created a spike in the board’s imposed charter school tuition from $2.69 million in the 2016-2017 budget to $4.68 million for next year, he said. [emphasis mine]
And so it continues with New Jersey's insane charter authorizing system: Trenton mandates a school district must give up funds to support a charter school that the district had no say in approving. Worse, the district cannot exercise any oversight authority over the charter: iLearn can spend the funding the state mandates the district provide any way they wish, so long as NJDOE approves.

It's worth pointing out that, according to data from the Education Law Center, Clifton has suffered from persistent underfunding of state education aid under the Christie administration: cumulatively $73.8 million since 2010. And yet the same administration mandates that Clifton taxpayers put up millions of dollars every year for a charter school the community may not even want.

The standard answer from reformsters to this complaint is that "families can vote with their feet": if the charter can't attract students, it will close. This argument fails on several levels, not the least of which is that schooling is a community concern -- not simply the concern of parents -- and that even taxpayers who don't have children in the school system have every right to expect that their hard earned dollars are being used efficiently and effectively.

And there's little reason to believe NJDOE is currently up to that task, no matter what the charters may say:
Dawn Fantasia, iLearn’s communications director, said the new school campus is located at 43 Clifton Ave., and denied Daley’s allegations that school has not publicized its financial reports.
“Passaic Arts and Science Charter School is a public charter school,” Fantasia said. “As per the NJDOE regulations and guidelines, all audit reports are made public and may be easily found and accessed with a simple search.”
She stated the charter's latest public audit, available on the state's education website, contradicts the “alleged lack of transparency purported by Commissioner Daley.”
Um, no. Calling this short document an "audit" is a gross misrepresentation. The plain fact, as Julia Sass Rubin and others have pointed out, is that New Jersey charter schools are far less transparent in their financial dealings than public school districts. There is no way to know, looking at this document or the charter's CAFR, what relationships and contracts exist between the charter, its board, and its contractors, vendors and lease-holders.

As NorthJersey.com pointed out in their extensive investigation of iLearn, not only is it difficult to get information about these relationships; the NJDOE has become a revolving door for regulators looking to make careers in charter school management:
Connections run deep among people involved with the schools, Gulenist groups and Turkish charter schools elsewhere in the U.S:
  • Two of the New Jersey schools, for example, have a founder who has served as a director at the New York-based Alliance for Shared Values, considered the voice of the Gulen movement in this country. 
  • The CEO of iLearn Schools Inc. – an Elmwood Park-based non-profit that manages four of the local charter schools – comes from a charter network in Texas that the Turkish government claims is linked to the Gulen movement.
The schools and their vendors have successfully courted prominent public-school educators and political figures.
  • The state’s top charter school regulator, Harold Lee, left his post last summer for a job at iLearn. 
  • Security consulting contracts at four of the schools worth more than $90,000 a year are held by ex-Bergen County Sheriff Leo McGuire, who took a 10-day trip to Turkey before he left office in 2010 with his family and local Turkish nationals tied to the schools. It was paid for in part by a Gulenist group.
More than $30 million in long-term, low-interest loans have been granted by the state to benefit the Paterson science and technology charter despite its continuing financial and academic troubles:

In 2014, a Wall Street ratings agency downgraded the bonds issued for its expansion to junk status because the school’s revenues had fallen. Last year, Wall Street lowered its overall outlook on the bonds to “negative.”

Tracking tax dollars spent by the schools can be difficult because of loopholes in state law:

  • ILearn, which is set to add a fifth charter to its chain this year, declined to answer routine requests for information about its payroll, saying that as a private contractor it is not subject to the state Open Public Records law. 
  • State officials said it is unclear if such charter-management organizations fall under the law, even though charters draw their funding directly from the tax-funded budgets of regular public schools. [emphasis mine]
Is anyone seriously suggesting the Clifton BOE ought to just accept all this? That they don't have a fiduciary responsibility to their constituents to make sure Passaic A&S and iLearn are using revenues appropriately -- especially when the town's public schools are being short-changed by the very state administration that forced them to fund this charter school?

Which brings us to the second way the reformster argument for laissez faire charter regulation falls apart: how is Clifton supposed to "compete" for students when it doesn't get the resources the state's own law says it needs, even as it serves a different population of students compared to Passaic A&S?

We see this over and over again: Charter schools enroll a much smaller percentage of special education students compared to their hosting public school districts. This is certainly the case here; in addition:

As is typical for New Jersey, the students with the most costly special education disabilities are concentrated in the public district schools, and not the charters. Which means that even as charters like Paterson A&S are syphoning funds away from an already underfunded district, they aren't taking the students who are most expensive to educate.

How does this manifest in spending?

The "Budgetary Costs Per Pupil" -- which the state itself says "...are the costs of governance, support, and instruction that are considered common to all school districts and generally are uniform among them" -- between CPS and Passaic A&S are about the same. But CPS spends much more on instruction and support services, while Passaic A&S puts its money into administration and plant costs. Support services help all students, but are particularly targeted to those with special education needs.

It's clear what's happening: Passaic A&S has proportionally far fewer special needs students, and therefore puts its money into other spending categories. Does anyone argue against the idea that the taxpayers of Clifton have every right to know how those monies are being spent? That they ought to know exactly where plant and administration funds flow? That the Clifton BOE has an affirmative obligation to protect its town's interests and properly regulate how charter monies are being spent? That the charter's inability to leverage economies of scale might make it a bad deal for the city's taxpayers?

Let's look at that classroom instruction category a little more, with the understanding that most of those funds are spent on teacher salaries:

I've pointed out many times that charter teachers tend to have far less experience than public district school teachers. But Passaic A&S has one of the least experienced staffs I have seen in the NJ data. Nearly 4 in 5 of its teachers have less than 3 years of experience; that's simply astonishing.

A charter school that only hires inexperienced teachers is arguably "free riding" on salaries: it takes advantage of the fact that teachers might be able to transfer to a better paying public district school later in their careers, and therefore will accept less pay now with the promise of more later. But Passaic A&S takes this a step further: it way underpays its teachers relative to CPS even controlling for experience, allowing more funds to be plowed into administration and plant costs.

Does NJDOE care about any of this? Certainly, Chris Christie doesn't; he loves Gulen-linked charter schools so much he visits them repeatedly, singing their praises while simultaneously pushing insane school funding plans that would decimate the state's urban school districts. He says these schools "do more with less" -- but he never asks how the students might differ.

I've done variations of this graph before. It takes a little explaining, but I find it very telling. What I've done here is use a linear regression model to explain variation in scores on the PARCC Algebra I exam based on several student population characteristics: free-lunch eligibility, special education percentage*, Limited English Proficiency percentage, and racial characteristics.** The model here isn't as robust as for other assessments, as only 22 percent of the variation in scores is explained. But it does reveal something I've seen before that implies much about how charters get their "gains":

Algebra I is a course that you take in Grade 8 if you are relatively advanced in math (Grade 7 if you're really advanced). If math isn't necessarily your thing, you're much more likely to take it in Grade 9; in other words, high school. See what happened here? The CPS middle schools scored quite well on the test; better, in fact, than Passaic A&S if we take into consideration the variables in my model.

But Clifton HS students didn't do so well. Would anyone make the case that the teachers are worse at the high school, compared to the middle school, based on this? You'd be foolish if you did: Clearly, we are justified in thinking the difference between the middle and high school scores is due at least in part to student characteristics we can't see in the data.

Well, if that's the case, why shouldn't we at least ask the question whether the same thing is going on with the charter school? The students there are self-selecting into Passaic A&S; isn't it likely they are different from students who don't choose to go to the charter? Shouldn't we at least stop to consider that charter school "gains" are due, at least in some part, to self-selection?

Charter researchers have, in some cases, tried to get around this problem by using lotteries to claim they are setting up a "random" experiment that accounts for these differences. The problem is three-fold: first, the charter "treatment" might include things like free-riding on salaries, which allows for an extended day, which could lead to higher test scores. In this case, we aren't putting the public district schools and the charters on equal footing; in other words, we're confusing "charteriness" with advantages that are paid for by the hosting district schools.

Second, we aren't accounting for peers. I know some have tried to do so in their work, but I find their methods to be weak (I know this deserves more discussion -- this summer, I promise). The fact is that concentrating special needs students in the hosting public district schools likely gives charters an advantage that is, again, not due to their governance structure.

Third -- and this is the big one -- We can only generalize "random" charter school studies to students who enter charter school lotteries. Which is what I'm asking folks to consider with the chart above. If we can see self-selection play out this way in public district schools, why shouldn't we expect similar effects from charter schools?

Let me bring this back to the local level: 

Here's Clifton, NJ, a city school system that is being screwed over by the state, which refuses to follow its own law and give the schools what they need. That same state then forces the community to fork over millions of dollars to a charter school whose operations and governance are far from transparent. The charter doesn't take its fair share of special needs students; spends money on administration and plant costs rather than instruction and support; and free-rides on teacher salaries.

The Clifton BOE rightly turns to the state and says: this is not fair to the students we must educate, and it's not fair to our taxpayers who demand accountability and transparency. 

What's the state's response?
NJDOE spokesman Michael Yaple said the agency declines to address local issues through media outlets and instead attempts to work directly with municipal officials.
"The Department takes into account a number of factors when making decisions regarding charter schools, ranging from student academic performance to the demand among parents," Yaple said in a statement. "Throughout the process, any correspondence and concerns from both school districts and charter schools is important to the Department, and we work directly with local school officials the best we can."
In other words: the good people of Clifton can take a flying leap.

At some point, this madness must come to an end. Even if you are a supporter of charter schools, nothing can possibly justify New Jersey's totally screwed up system of charter approval, regulation, and funding. We can't keep asking cities like Clifton to sacrifice their local public schools just so a few folks can have a "choice" that is negatively affecting the entire system.

h/t the great Rob Tornoe.

* A 3-year average from 2012 to 2014; I haven't yet been able to get more recent data, but this, in my opinion, is a good enough metric for our purposes here.

** Percent black and percent Asian, which I've found avoids problems with multicollinearity. 

Sunday, May 28, 2017

Facts About Newark Charter Schools

It's just so old:
Their main criticisms of charters are that they skim the best students from the district and draw money away from district schools. But Newark's universal enrollment system puts a firm hand on the scale to ensure that charters take their fair share of harder-to-educate kids. And in Camden, charter leaders run neighborhood schools that guarantee enrollment to all area students. A kid's fate is not left to a lottery. [emphasis mine]
That is the Star-Ledger Editorial Board, once again writing a love letter to charters. Once again, I am here to set them straight.

I just went over the clear differences between charter school and public, district school student populations in Camden. A few tweaks of the code, and here's pretty much the same data for Newark.

As a proportion of total population, the Newark Public Schools enroll many more students with the costliest special education disabilities. We've been over this time and again: while some Newark charters have upped their enrollments of special education students, the students they do take tend to have the less-costly disabilities: Specific Learning Disabilities (SLD) and Speech/Language Disabilities (SPL).

The charters take very few students who are emotionally disturbed, or hearing impaired, or have intellectual disabilities, or any of the other higher-cost disabilities. And when you look at the counts of students as opposed to the proportions, you can see just how stark the differences between the two sectors really are.

There's also the disparity in Limited English Proficient (LEP) students, who also cost more to educate:

It is unconscionable that Newark's charters are being "held harmless" in their funding while the district bears this enormous fiscal burden.

Just to be clear: I don't think charters should be attempting to educate these students with special needs. By all indications, they don't have the capacity to do the job correctly.

NPS has a much lower "student load" per support staff member than the charters. These support staff include counselors, occupational and physical therapists, nurses, psychologists, social workers, learning disability teacher consultants, reading specialists, sign language interpreters, speech correction specialists, and so on. It would be highly inefficient to staff every charter school and network in the city with all of these staff.

But that's precisely the problem with charter proliferation: it creates redundant school governance that taxes the entire system. And that's not the only issue:

As I've documented many times, charter school teaching staffs have far less experience than public school staffs. Not only does this create a fiscal burden on charters, who must continually retrain their churning teaching staffs; it creates a "free rider" problem for hosting public schools.

In the aggregate, teacher salaries in the first decade for district and charter teachers in Newark are fairly close. Some of the large charter networks, like TEAM/KIPP and Uncommon/North Star, pay better than the district; most of the mom-and-pop charters pay worse. But that's only until about year ten; after, NPS teachers enjoy a significant bump in pay.

But the charters don't have to worry about competing with NPS on salary at that point: because their staffs churn so much, few of their teachers have more than a decade's worth of experience. What economist Martin Carnoy has suggested is that the charters are getting a "free ride":
The “free rider” aspect of teacher costs in private schools, whether voucher or charter, means that the supply of young people entering the teaching profession is maintained by the salary structure and tenure system in public education. Without this structure, many fewer individuals would take the training needed to become certified to enter teaching. Since teaching salaries are low compared with other professions, the prospect of tenure and a decent pension provides the option of security as compensation for low pay. This pool of young, trained teachers is available to voucher and charter schools, generally at even lower pay than in the public sector and without promise of tenure or a pension, but with the possibility of training and experience. Thus, the public education employment and salary system “subsidizes” lower teacher costs in private and charter schools. In other words, for private schools to have lower costs, it is necessary to maintain a largely public system that pays teachers reasonable (but still low) salaries and provides for a teacher promotion ladder and job security. [emphasis mine]
There is, however, a cost for maintaining an inexperienced staff:

Newark charters spend much more per pupil on administration than NPS schools. Part of this is due to charters being unable to leverage economies of scale like the district schools. Part of it is undoubtedly because the schools need more administrators to support more inexperienced teachers.

Fortunately for some of the big charter networks -- TEAM/KIPP, Uncommon/North Star, and Great Oaks in particular -- they are able to access resources from their regional and national networks. This includes large amounts of philanthropic funding, capital for building new facilities, and tutors whose costs are supported from federal funds.

The SLEB claims charters get "no capital funds." I don't know where they got this idea, but it's just wrong; charters may not draw from the same state pool as district schools, but plenty of other monies are available to them for facilities.

NPS gave a sweetheart deal to charter operators, who now get to use public monies to transfer public assets to private control. Back in 2014, Owen Davis reported that all of the qualified school construction bonds New Jersey received went to charters. Three charters got to locate in Teachers Village, built with all sorts of financial goodies offered by the state and the feds. And let's not forget that a whole bunch of the Zuckerberg money went to the charter sector.

Let's review:
  • Newark charters enroll far fewer students who have costly special education disabilities, or who are Limited English Proficient, than NPS schools.
  • Compared to NPS schools, Newark charters do not have the staffs in place to properly educate these high-needs students.
  • Newark charter staffs have much less experience than NPS staffs.
  • Newark charters are, consequently, "free riding" on NPS schools, which reward teachers for their experience.
  • Newark charters have much higher administrative costs than NPS schools.
  • Newark charters have access to all sorts of outside funding and capital that NPS cannot utilize.
I've not been one to call for closing Newark's charter schools; at this point, the upheaval might -- might -- be more detrimental to the students enrolled in those schools than the damage being done to the whole system. Some might not like my point of view on this, but I don't think it's right to chance making Newark's schooling system even more chaotic than it already is.

But there is a clear fiscal drain on the Newark Public Schools due to charter proliferation. And it's very hard to justify the harm this causes by pointing to some, frankly, underwhelming gains on standardized tests. 

But the SLEB doesn't see it that way:
The key threats now facing charters in New Jersey are a moratorium on all charter growth, and a local vote being required to open any new charter school. 
Because top charters in Newark and Camden expand year by year, a moratorium would force sixth graders to transfer back to the district schools, rather than being allowed to continue on to 7th grade. That would clearly hurt kids. And since the teacher's union dominates school board elections with low turnout, giving the board veto power is akin to strangling any new charter in its crib. The better gauge is clear parental preference.
First of all: Maybe turnout is low in school board elections because Newark has been under state control for over two decades, which means the election is largely meaningless.

Second: What "would clearly hurt kids" who are enrolled in NPS is the growth of a system that holds charter schools harmless in the face of state aid cuts, concentrates special needs children into the public schools, free-rides on teacher salaries, drives more funding into administration, doesn't allow for economies of scale, and puts public assets into private hands.

Don't the kids enrolled in NPS deserve well-resourced schools? Don't their parents deserve democratic control of their school system? Shouldn't they have the same say over their schools that suburban parents have over theirs?

A moratorium on charter growth in Newark -- and across the state -- is a reasonable policy. Don't agree? Fine, but the idea that charters are unquestionably beneficial to cities like Newark is shallow and ignorant.

The Star-Ledger, after its flight from Newark (h/t Wallace Strobe).

Saturday, May 27, 2017

The Facts About Charter School Finances in Camden, NJ

Here's Part I of my U-Ark series.

Here's Part II.

This post really should be called "U-Ark Screws Up A Charter School Revenue Study, AGAIN: Part III," as it is a follow-up to my last two posts about how the University of Arkansas Department of Education Reform messed up its latest study of charter school revenues. I'm retitling it, however, because I think the facts I present below have significance beyond critiquing U-Ark's report.

As I showed, U-Ark's claim that Camden's charter sector is receiving far less funding than the city's public schools is completely bogus (and this almost certainly applies to the other cities in their study). They double count revenues going to charter schools without attributing them to those schools' students. They don't understand that since the district has the responsibility for transporting charter students, revenues allotted for that purpose shouldn't be attributed to the district.

I'd go on about the many other methodological failures in the study, but Bruce Baker covered them when he reviewed U-Ark's report from 2014. Amazingly, U-Ark cites Baker in their latest report -- but then goes back and makes the same mistakes. Which wouldn't be so bad if the study was being ignored and wasn't influencing policy makers; unfortunately, it appears stakeholders are heeding the study's conclusions.

This, by the way, is a huge problem in education policy journalism right now: reporters will often trumpet a "new study" that knowledgable critics haven't had time to properly vet. Even if it's a piece of junk, it still gets attention; at best, the critics are given a "he said-she said" positioning that leads stakeholders to believe that even if the study has problems, it's probably still making at least a somewhat valid point.

Let's use Camden's charters and the U-Ark study as an example of why this is such a problem. Here's a quote from page 14:
Inequitable funding between public charter schools and TPS could be due to differences in the number of students identified as requiring SPED [special education] services, as described in Table 2. To test this ubiquitous claim regarding the charter school funding gap, we depart from our normal approach of focusing exclusively on revenues and consider SPED expenditures by both school sectors.

The Table 3 column labeled “SPED Expenditure Gap Per Student” presents the results from subtracting the amount of dollars spent per student in the charter sector from the amount of dollars spent per student in TPS sector. All totals are positive, indicating that TPS spend more on SPED than charters in all 14 of our metropolitan areas. The largest SPED expenditure gap is in Camden, where TPS spend $3,383 more per student on SPED than charters spend. The smallest SPED expenditure gap is in Tulsa, where TPS only spend $32 more per pupil on SPED than charters do. [emphasis mine]
As I've noted previously, the documentation of data sources in this report is so bad it's impossible for me to replicate the report's analyses. There's also a serious problem when making these comparisons, as the amount spent on special education is not necessarily the amount that should be spent. In other words, if the charters or TPSs are short-changing what ought to be spent to properly educate a special needs student, that's going to affect the spending "gap." In no way, then, is this analysis adequate for making claims about charters being underfunded.

There's one additional flaw here, and it's huge: Not all special education students are the same, which affects how much a school spends. So far as I can tell -- again, the documentation of methods is so lacking I can't say for sure -- U-Ark treated special education status as a binary variable: either a child is classified or she is not. But that is a terrible way of assessing the true reasons for a charter school revenue "gap":

2014 is the last year for which we have good data on the types of learning disabilities students have. Specific Learning Disabilities (SLDs) and Speech/Language disabilities (SPL) are lower-cost classifications (that comes directly from a report commissioned by the state in 2011). Students with the other disabilities, such as autism, traumatic brain injury, visual impairment, and so on, require more resources to receive an adequate education.

Very few students with the costliest learning disabilities enroll in Camden's charter schools; this is also true across New Jersey, and across the nation

Let's look at this as a percentage of total student population:

As a percentage, Camden City Public Schools enroll far more special needs students with the costliest disabilities. It's worth noting that most of these students in the charters are classified as "Other Health Impairment," a catch-all phrase that includes students with asthma and ADHD. Which means many of these students could still have disabilities that are relatively easily treated and don't require much support.

This alone is enough evidence to dismiss U-Ark's blanket claim that special education status isn't a major factor in Camden's revenue "disparity." But let me point out a few more pertinent facts U-Ark simply doesn't consider:

The percentage of Limited English Proficient (LEP) students in CCPS is consistently greater than the percentage in the charter schools. This obviously requires CCPS to spend more per pupil, as LEP students needs special services.

Camden has many free lunch-eligible (FL) students in both the charters and in CCPS schools. The New Jersey funding formula, SFRA, directs more resources to schools with more at-risk students (as it should). But recently, CCPS moved to a universal enrollment system for subsidized student meals. I certainly applaud this, but it makes research more difficult. There is now no incentive for CCPS families to enroll in the program; in contrast, charters do rely on FL enrollment numbers to pass more money through to them from the district. Which makes the FL statistics unreliable.

But the measure was never very good to begin with: all it shows is how many students fall below 130 percent of the poverty line. There are lots of reasons to believe there's great variation within that group of students. I'm not sure U-Ark could have done anything about this, but they should at least acknowledge the issue.

Some other points:

The charter sector relies on staffs that have far less experience than CCPS teachers. We know teachers gain the most in effectiveness in their first few years, so this is a serious concern. But there's another problem related to U-Ark's study.

CCPS has many more teachers with a decade or two of experience than the charters. This affects the payrolls of each sector.

I was a bit surprised when I saw this, because teacher salaries tend to be much lower in charter schools. But starting around year 3, salary schedules are about the same... until you get to the second decade. That's when CCPS salaries start to rise at a steeper slope, leaving charter salaries behind.

Why does this matter? As Martin Carnoy has recently pointed out, it sets up a "free rider" problem for CCPS. Charter teachers accept less pay because they don't expect to stay in the job very long. Maybe they will move on to another field; maybe they'll switch over to a public school district, which will give them better pay in the future. The charters can offer less pay now because teachers can expect more pay later when they aren't working in the charter.

Between the special needs students and the free-riding on salaries, it's increasingly clear the charter sector couldn't sustain its model without the public schools spending more money. But the charters still draw significant revenues. What do they spend it on?

Charter school administrative spending is far in excess of public school spending. It's possible charters have to spend more because they churn their teachers, requiring more administrators to work with perpetually inexperienced staffs. It's also likely charters are too small to achieve economies of scale, a fact Bruce Baker pointed out recently.

Charter schools spend far less on student support services, likely because they have fewer students who need those services. Services include child study teams, therapists, school nurses, and other staff who spend much of their time serving students with special education needs. Some charters don't report any spending on these services.

None of these realities are explored in U-Ark's report. And yet every one of them is relevant to U-Ark's claim that: "Public charter schools in Camden, New Jersey, are the most underfunded, receiving an average of $14,771 less in per-pupil funding that TPS, representing a 45 percent funding inequity."

This is a bogus, sloppy, unwarranted comparison; no policy should be enacted based on U-Ark's claims. In fact, based on their analysis of Camden alone, the entire report should simply be dismissed as unreliable.

Again: this isn't anything new. U-Ark has been repeatedly criticized for its work in this area. And yet they plow ahead, with the blessing of both their funders and the university that hosts them. Why?

You tell me.

Sunday, May 21, 2017

Random Thoughts On Using VAM for Teacher Evaluation

You may have read the piece in the New York Times today by Kevin Carey on the passing of William Sanders, the father of idea of using value-added modeling (VAM) to evaluate teachers. Let me first offer my condolences to his family.

I'm going to skip a point-by-point critique of Carey's piece and, instead, offer a few random thoughts about the many problems with using VAMs in the classroom:

1) VAM models are highly complex and well beyond the understanding of almost all stakeholders, including teachers. Here's a typical VAM model:

Anyone who states with absolute certainty that VAM is a valid and reliable method of teacher evaluation, yet cannot tell you exactly what is happening in this model, is full of it.

There was a bit of a debate last year about whether it matters that student growth percentiles (SGPs) -- which are not the same as VAMs, but are close cousins -- are mathematically and conceptually complex. SGP proponents make the argument that understanding teacher evaluation models are like understanding pi: while the calculation may be complex, the underlying concept is simple. It is, therefore, fine to use SGPs/VAMs to evaluate teachers, even if they don't understand how they got their scores.

This argument strikes me as far too facile. Pi is a constant: it represents something (the circumference of a circle divided by its diameter) that is concrete and easy to understand. It isn’t expressed as a conditional distribution; it just is. It isn’t subject to variation depending on the method used to calculate it; it is always the same. An SGP or a VAM is, in contrast, an estimate, subject to error and varying degrees of bias depending on how it is calculated.

The plain fact is that most teachers, principals, school leaders, parents, and policy makers do not have the technical expertise to properly evaluate a probabilistic model like a VAM. And it is unethical, in my opinion, to impose a system of evaluation without properly training stakeholders in its construction and use.

2) VAM models are based on error-prone test scores, which introduces problems of reliability and validity. Standardized tests are subject to what the measurement community often calls "construct-irrelevant variance" -- which is just a fancy way of saying test scores vary for reasons other than knowing stuff. Plus there's the random error found in all test results, due to all kinds of things like testing conditions. 

This variance and noise causes all sorts of issues when put into a VAM. We know, for example, that the non-random sorting of students into teacher classrooms can create bias in the model. There is also a very complex issue known as attenuation bias when trying to deal with the error in test scores. There are ways to ameliorate it -- but there are tradeoffs. 

My point here is simply that these are very complicated issues and, again, well beyond the apprehension of most stakeholders. Which dictates caution in the use of VAM -- a caution that has been sorely lacking in actual policy.

3) VAM models are only as good as the data they use -- and the data's not so great. VAM models have to assign students to teachers. As an actual practitioner, I can tell you that's not as easy as it sounds. Who should be assigned a VAM score for language arts when a child is Limited English Proficient (LEP): the ELL teacher, or the classroom teacher? What about special education students who spend part of the school day "pulled out" of the homeroom? Teachers who team teach? Teachers who co-teach?

All this assumes we have data systems good enough to track kids, especially as they move from school to school and district to district. And if the models include covariates for student characteristics, we need to have good measures of students' socio-economic status, LEP status, or special education classification. Most of these measures, however, are quite crude.*

If we're going to make high-stakes decisions based on VAMs, we'd better be sure we have good data to do so. There's plenty of reason to believe the data we have isn't up to the job.

4) VAM models are relative; all students may be learning, but some teachers must be classified as "bad." Carey notes that VAMs produce "normal distributions" -- essentially, bell curves, where someone must be at the top, and someone must be at the bottom.

I've labeled this with student test scores, but you'd get the same thing with teacher VAM scores. Carey's piece might be read to imply that it was a revelation to Sanders that the scores came out this way. But VAMs yield normal distributions by design -- which means someone must be "bad."

General Electric's former CEO Jack Welch famously championed ranking his employees -- which is basically what a VAM does -- and then firing the bottom x percent. GE eventually moved away from the idea. I'm hardly a student of American business practices, but it always struck me that Welch's idea was hampered by a logical flaw: someone has to be ranked last, but that doesn't always mean he's "bad" at his job, or that his company is less efficient than it would be if he was fired.

I am certainly the last person to say our schools can't improve, nor would I ever say that we have the best possible teaching corps we could have. And I certainly believe there are teachers who should be counseled to improve; if they don't they should be made to leave the profession. There are undoubtedly teachers who should be fired immediately.

But the use of VAM may be driving good candidates away from the profession, even as it is very likely misidentifying "bad" teachers. Again, the use of VAM to evaluate systemic changes in schooling is, in my view, valid. But the argument for using VAM to make high-stakes individual decisions is quite weak. Which leads me to...

5) VAM models may be helpful for evaluating policy in the aggregate, but they are extremely problematic when used in policies that force high-stakes decisions. When the use of test-based teacher evaluation first came to New Jersey, Bruce Baker pointed out that its finer scale, compared to teacher observation scores, would lead to making SGPs/VAMs some of the evaluation but all of the decision.

But then NJDOE leadership -- because, to be frank, they had no idea what they were doing -- made teacher observation scores with phony precision. That led to high-stakes decisions compelled by the state based on arbitrary cut points and arbitrary weighting of the test-based component. The whole system is now an invalidated dumpster fire.

I am extremely reluctant to endorse any use of VAMs in teacher evaluation, because I think the corrupting pressures will be bad for students; in particular (and as a music teacher), I worry about narrowing the curriculum even further, although there are many other reasons for concern. Nonetheless, I am willing to concede there is a good-faith argument to be made for training school leaders in how to use VAMs to inform, rather than compel, their personnel decisions.

But that's not what's going on in the real world. These measures are being used to force high-stakes decisions, even though the scores are very noisy and prone to bias. I think that's ultimately very bad for the profession, which means it will be very bad for students.

Carey mentions the American Statistical Association's statement on using VAMs for educational assessment. Here, for me, is the money quote:
Research on VAMs has been fairly consistent that aspects of educational effectiveness that are measurable and within teacher control represent a small part of the total variation in student test scores or growth; most estimates in the literature attribute between 1% and 14% of the total variability to teachers. This is not saying that teachers have little effect on students, but that variation among teachers accounts for a small part of the variation in scores. The majority of the variation in test scores is attributable to factors outside of the teacher’s control such as student and family background, poverty, curriculum, and unmeasured influences. 
The VAM scores themselves have large standard errors, even when calculated using several years of data. These large standard errors make rankings unstable, even under the best scenarios for modeling. Combining VAMs across multiple years decreases the standard error of VAM scores. Multiple years of data, however, do not help problems caused when a model systematically undervalues teachers who work in specific contexts or with specific types of students, since that systematic undervaluation would be present in every year of data. 
A VAM score may provide teachers and administrators with information on their students’ performance and identify areas where improvement is needed, but it does not provide information on how to improve the teaching. The models, however, may be used to evaluate effects of policies or teacher training programs by comparing the average VAM scores of teachers from different programs. In these uses, the VAM scores partially adjust for the differing backgrounds of the students, and averaging the results over different teachers improves the stability of the estimates [emphasis mine]
Wise words. 

NJ's teacher evaluation system, aka "Operation Hindenburg."

* In districts where there is universal free-lunch enrollment,parents have no incentive to fill out paperwork designating their children as FL-eligible. So even that crude measure of student economic disadvantage is useless.

Saturday, May 20, 2017

U-Ark Screws Up A Charter School Revenue Study, AGAIN: Part II

Here's Part I of this series.

Here's Part III, which is actually a presentation of relevant facts about Camden, NJ charter school finances.

If this is true, it's really disturbing:
Colorado’s General Assembly on Wednesday passed a bill giving charter schools the same access to a local tax funding stream as district schools have, The Denver Post reported.
The bipartisan compromise measure, which supporters say is the first of its kind in the nation, would address an estimated $34 million inequity in local tax increases. It came a day after the University of Arkansas released a study that found charter schools receive $5,721 less per pupil on average than their district counterparts — a 29 percent funding gap. [emphasis mine]
It is, of course, standard operating procedure for outfits like the U-Ark Department of Education Reform to claim their work led to particular changes in policy; that's how they justify themselves to their reformy funders.  Maybe the connection between the report and the Colorado legislation (which is really awful -- more in a bit) is overblown...

But if the U-Ark report did sway the debate, that's a big problem. Because the report is just flat out wrong. 

As I explained in Part I, the claim that Camden, NJ, has a huge revenue gap between charters and public district schools seems to be based on an utterly phony comparison: all of the revenue, both charter and district, is linked to only the CCPS students -- not the charter students. Because the data source documentation in the report is so bad, I can't exactly replicate U-Ark's figures, so I invited Patrick Wolf and his colleagues to contact me and explain exactly how they got the figures they did.

So far, they remain silent.

But that isn't surprising. When U-Ark put out its first report in 2014, Bruce Baker tore it to shreds in a brief published by the National Education Policy Center. The latest U-Ark report cites Baker's brief, so they must have read it -- but they never bothered to answer Baker's main claim, which is that their comparisons are wholly invalid.

Further, what I documented in the last post is only one of the huge, glaring flaws in the report. Let me point out another, using Camden, NJ again as an example. We'll start by looking U-Ark's justification for using the methods they do:
This is a study of the revenues actually received by public charter schools and TPS. Revenues equal funding. Revenues signal the amount of resources that are being mobilized in support of students in the two different types of public schools. Some critics of these types of analyses argue that our revenue study should, instead, focus on school expenditures and excuse TPS from certain expenditure categories, such as transportation, because TPS are mandated to provide it but many charter schools choose not to spend scarce educational resources on that item. [emphasis mine]
"Choose" not to spend the revenues? Sorry to be blunt, but that statement is either deliberately deceptive or completely clueless.* In New Jersey, hosting public school districts are required to provide transportation for charter school students. The charters don't "choose" not to spend on transporting the kids; they avoid the expense because the district picks up the cost.

Baker pointed this out explicitly in his 2014 brief -- but U-Ark, once again, refuses to acknowledge the problem, even though we know for a fact they read Baker's report, because they cite it repeatedly.

And it gets worse.

For the sake of illustration, here's a simplified conceptual map of what Camden's public district school bus system might look like. We've got neighborhood schools divided into zones, and buses transporting children to their neighborhood school.** There are exceptions, of course, primarily for magnet and special needs students, but the system on the whole is fairly simple.

Now let's add some "choice":

There has been a marked decline in "active transportation" -- walking or biking -- to school over the past few decades, and school "choice" is almost certainly a major contributor. As we de-couple schools from neighborhoods (which may well have many other pernicious effects), transportation networks become more complex and more expensive.

As I said: New Jersey law requires public school districts like Camden to pay for transportation of charter school students. Which means all of these extra costs are borne solely by the district.

And how much does this cost Camden's charter sector?

So any comparison of revenues that doesn't exclude transportation -- and, again, it appears that U-Ark didn't exclude it, although their documentation is so bad we can't be sure -- is without merit. Claiming that charter schools have a revenue gap when they use services paid for by public district schools makes no sense.

Folks, this issue is so simple that it doesn't require an advanced understanding of school finance or New Jersey law to understand it. Which makes it all the more incredible the U-Ark team didn't account for it in their findings. And again: if the Colorado Assembly made their decision to raise the funding for charters -- at the expense of public district schools -- on the basis of a report that is this flawed...

Let's take a look at some better -- not perfect, but better -- financial comparisons between Camden's charters and CCPS next.

 * Granted, it might be both...

** It's worth noting that in a dense city like Camden, many of the students will be within walking distance of their neighborhood school. But when you introduce "choice," you make the school system much less walkable, because students are likely traveling greater distances. I was at a conference at Rutgers yesterday where researchers were looking into this issue -- more to come...