How can we make aca­d­e­m­ic pub­lish­ing bet­ter? I was think­ing a bit about this ear­li­er in the week, prompt­ed by the jour­nal hor­ror sto­ries re­count­ed on Dai­ly Nous and the thread about pay­ing peer re­view­ers $450 a pop on Hack­er News.

I think I have a bet­ter idea than pay­ing per re­view: of­fer sab­bat­i­cals for aca­d­e­mics to spend a chunk of their time re­view­ing and the rest of their time how­ev­er they want. Fund re­view­ers, not re­views.Loose­ly in­spired by pro­pos­als to fund peo­ple, not projects.

Prob­lems with Pay-Per-Re­view

Many of aca­d­e­m­ic pub­lish­ing’s prob­lems come down to lack of in­cen­tives for re­view­ers to (1) ac­cept re­view re­quests, (2) do a good job on their re­views, and (3) com­plete their re­views quick­ly.There are oth­er prob­lems, of course, some of which might call for more rad­i­cal changes than those I’ll be talk­ing about. But even if we should abol­ish pre-pub­li­ca­tion peer re­view, it still seems that there will be a need for post-pub­li­ca­tion peer re­view. So there’d still be rea­son to in­cen­tivize these things.

Pay­ing re­view­ers might help with each of these, as­sum­ing that one gets paid less for late or poor re­views. But over­all I don’t think it’s a good idea.

One po­ten­tial prob­lem: where will the mon­ey come from?

Of course El­se­vi­er and Springer and the like make ab­surd prof­its that they could put to good use by im­prov­ing their prod­uct and com­pen­sat­ing their la­bor­ers. But... well, let’s just say I’m not san­guine about that. I sup­pose they could just raise their prices to keep up their prof­its, but even that wouldn’t be easy to con­vince them to do. What’s in it for them?

In any case, the big­ger is­sue here is that jour­nals should be free to read and pub­lish in, but with­out a se­ri­ous fund­ing ef­fort such jour­nals will not be able to pay their re­view­ers. And if the non-free jour­nals start pay­ing their re­view­ers, the free jour­nals will have an even hard­er time find­ing re­view­ers than they al­ready do, which will make them less at­trac­tive to sub­mit to, which will even­tu­al­ly make them less pres­ti­gious, which will make them even less at­trac­tive to sub­mit to, and so on. This would be bad.

I don’t think we should re­ject the pro­pos­al on this ba­sis, though. We just have to add to it a re­quire­ment for a se­ri­ous fund­ing ef­fort on be­half of the free and open jour­nals. It would be bet­ter to sig­nif­i­cant­ly im­prove aca­d­e­m­ic pub­lish­ing with­out need­ing a se­ri­ous fund­ing ef­fort, but why think that’s pos­si­ble? Un­til some­one sug­gests a cheap­er al­ter­na­tive, pricey­ness should not knock the pay-re­view­ers-and-fund-open-jour­nals pro­pos­al out of the run­ning.

The oth­er prob­lem, though, is worse: mon­ey isn’t that great of an in­cen­tive for get­ting aca­d­e­mics to re­view.

If you ask the de­clin­ing re­view­ers why they de­cline, the slow re­view­ers why they’re slow, and the poor re­view­ers why they didn’t do a bet­ter job, I’d bet they wouldn’t say “You get what you pay for.” They’d say they didn’t have enough time.

That might be bull­shit in many cas­es, and weak enough bull­shit for $450 to cut through in some. But in many oth­er cas­es—I’d ex­pect most—it isn’t. Get­ting an ex­tra $450 isn’t go­ing let you spend less time teach­ing or com­plet­ing your ad­min­is­tra­tive du­ties, and it shouldn’t get you to spend less time on your own re­search, at least if you don’t yet have tenure. So it’s only good mo­ti­va­tion for those who val­ue what­ev­er non-work time they have at less than $450/(how­ev­er many hours they take per re­view). I think this will not that big of a group, es­pe­cial­ly when in­ter­sect­ed with those who get the most re­view re­quests.

A high enough price could over­come this. Pay some­one enough to re­tire on and they’ll do the re­view on time. But I sus­pect that any fea­si­ble price range will not im­prove things much.

I also wor­ry that in some cas­es pay­ing for a time­ly re­view might make things worse. Sup­pose you’re a well-enough-paid aca­d­e­m­ic with lots of time com­mit­ments. You ac­cept a re­view re­quest with the un­der­stand­ing that you get paid if you get it done with­in 4 weeks, but not oth­er­wise. But as the weeks pass and oblig­a­tions ac­cu­mu­late, you find your­self think­ing “$450 would be nice, but I’m just too busy for it to be worth it. I’ll just fin­ish that re­view next month and not get paid.” And then you might not feel so bad about turn­ing in a late re­view, since it feels to you like you’ve shoul­dered some of the cost your­self and be­cause the con­tract al­lows you to sub­mit late if you’re will­ing to for­go the $450. On top of that there’s now no longer any in­cen­tive to get the late re­view done quick­ly or well, since they’re al­ready not pay­ing you. And I doubt im­pos­ing fines for late re­views would fly. It’d just en­cour­age a lot more de­clin­ing and hasty/slop­py re­view­ing.

So pay-per-re­view seems un­like­ly to im­prove things much, even if you can get the mon­ey for it.

An Al­ter­na­tive: Re­view­er Sab­bat­i­cals

What’s keep­ing aca­d­e­mics from re­view­ing quick­ly enough, well enough, and, well, enough is that they don’t have the time. So why not pay them with time? Give them a re­view­er sab­bat­i­cal.

Sup­pose some time-pres­sured aca­d­e­mics are of­fered not to be paid per re­view on top of their oth­er du­ties, but are in­stead giv­en a se­mes­ter-long sab­bat­i­cal to work on what­ev­er they want with no teach­ing or ad­min­is­tra­tive du­ties, so long as they ac­cept ~2 re­view re­quests per week and do the re­views quick­ly (like with­in a week or two) and well. Many would jump at the chance.

And this would do a lot of good. If there are 10 such sab­bat­i­cal-re­view­ers in a 15-week se­mes­ter, that’s ~30 re­views each, so ~300 good, very quick re­views in a se­mes­ter, and ~600 in the year. This would at least put a dent in phi­los­o­phy’s re­view­ing needs,It’s hard to know how many to­tal re­views there are every year across all phi­los­o­phy jour­nals. For a rough es­ti­mate it’s help­ful to look at the stats that some jour­nals pro­vide, but none re­port how many to­tal re­views they used per year. I also sus­pect there should be more re­views than there are, and the ques­tion we real­ly want to an­swer is how many re­views there should be every year, which is of course even hard­er to es­ti­mate. and of course go­ing to 20 or 30 re­view­ers per se­mes­ter would be even bet­ter. Seems like a good idea to me.

“But who is gonna pay for it? How are the re­view­ers picked? Are re­searchers too spe­cial­ized for there to be 30 sub­mis­sions in a se­mes­ter that they can com­pe­tent­ly re­view? Would be­ing a sab­bat­i­cal re­view­er give one too much pow­er in shap­ing a sub­field?”

All good ques­tions. Here are some first thoughts.

Who pays?

Pay­ing with time, as I’m propos­ing, is real­ly just an­oth­er way of pay­ing with mon­ey, since some­one needs to pay the re­view­er’s salary (or at least a sig­nif­i­cant chunk of it) while they’re on their re­view­er sab­bat­i­cal. So where does that mon­ey come from?

Not the for-prof­it pub­lish­ers, I as­sume. And I’d rather the sab­bat­i­cals not come from them any­ways, since they’d prob­a­bly re­quire you to re­view only for their own jour­nals. Per­haps the pub­lish­ers can be re­quired to con­tribute to who­ev­er else is pay­ing, though, at least if they want to get re­views from the quick re­view­ing sab­bat­i­cal­ists.

Uni­ver­si­ties might be a bet­ter op­tion. I’d think they care some­what about im­prov­ing aca­d­e­m­ic re­search and the pub­lish­ing process, so if they could be con­vinced that giv­ing out re­view­er sab­bat­i­cals would do that, per­haps they could be tempt­ed. Es­pe­cial­ly if they’re also con­vinced that com­peti­tor uni­ver­si­ties will do the same.

But even if this could hap­pen it’d prob­a­bly be only the rich uni­ver­si­ties that would be able to do it, which is not ide­al for var­i­ous rea­sons. It would also be dif­fi­cult for uni­ver­si­ties to co­or­di­nate about var­i­ous as­pects of this that we’d like to be co­or­di­nat­ed.

The best op­tion, it seems to me, would be to have some cen­tral grant­i­ng agency fund the re­view­er sab­bat­i­cals, with sup­port from uni­ver­si­ties, for-prof­it-pub­lish­ers (while they still ex­ist), pri­vate foun­da­tions, and/or gov­ern­ment funds. The agency could is­sue se­mes­ter-long re­view­er sab­bat­i­cal grants to re­searchers at any uni­ver­si­ty, based on the ex­pect­ed qual­i­ty of the re­view­ing and need for re­views with­in the re­searcher’s re­search area(s).

How much would it cost? It de­pends. But with a $50,000 stipend for a se­mes­ter, that’s $1M of grants per year for the 10 re­view­er sab­bat­i­cals per se­mes­ter sce­nario. There will also be op­er­at­ing costs and prob­a­bly some ad­di­tion­al costs that I haven’t thought of, but this gives us a sense of the or­der of mag­ni­tude. And it’s loose change for the Har­vards, El­se­viers, Tem­ple­ton Foun­da­tions, and NEH/NSFs of the world.

My guess is that spend­ing the mar­gin­al $1M in this way would do more to ad­vance re­search as a whole than the same mon­ey spent fund­ing par­tic­u­lar re­searchers to pur­sue a spe­cif­ic projects.And my guess is that this would be true up to, I dun­no, maybe $5-10M a year for phi­los­o­phy? One would have to fig­ure out how many re­views are need­ed per year and how much val­ue comes from fund­ing tra­di­tion­al re­search projects, among oth­er things. It would also im­prove many re­searchers’ lives, which is a nice bonus.

Who gets the sab­bat­i­cals?

Hav­ing a grant­i­ng agency set­up also helps with this ques­tion. There would be some kind of re­view board which would get in­for­ma­tion about and sim­ple ap­pli­ca­tions from po­ten­tial re­view­er sab­bit­i­cal­ists, and dis­trib­ute fund­ing for re­view­er sab­bat­i­cals on this ba­sis.

They would look for ev­i­dence that a can­di­date is a good re­view­er. Ed­i­tors can be giv­ing out some kind of com­men­da­tions to good re­view­ers or mak­ing nom­i­na­tions to help with this.Kind of like the British Jour­nal of Phi­los­o­phy of Sci­ence’s Ref­er­ee of the Year Awards. More of this kind of thing should be hap­pen­ing any­ways. Can­di­dates can also sub­mit the re­views they’ve pro­duced and in­for­ma­tion about how quick­ly they com­plet­ed them. A nice side ef­fect is that this in­cen­tivizes good re­view­ing for those who aren’t yet on a re­view­er sab­bat­i­cal but would like to get one.

They should also make sure that the can­di­date is com­pe­tent to re­view in an area broad or pop­u­lar enough to get enough sub­mis­sions to re­view in a sin­gle se­mes­ter. If they aren’t but are oth­er­wise a good can­di­date, per­haps fund­ing a teach­ing re­duc­tion rather than a sab­bat­i­cal would make sense. This deals with the ques­tion about re­searchers be­ing too spe­cial­ized to re­view 30 pa­pers in a se­mes­ter.

If there are too many can­di­dates that clear­ly meet the cri­te­ria, a lot­tery can de­cide who among them gets fund­ed. This might be com­pli­cat­ed, since there will be var­i­ous over­lap­ping mix­es of re­view­ing com­pe­tences among can­di­dates and dif­fer­ent amounts of need for re­views in dif­fer­ent sub­fields, but I don’t see there be­ing enough com­pli­ca­tion here to sink the pro­pos­al.

Too Much Pow­er?

Hope­ful­ly the process of re­view­ing the re­view­ers would elim­i­nate the worst cas­es of bi­ased re­view­ing. Even so, one con­cern about the pro­pos­al is that it will give a small num­ber of peo­ple a lot of in­flu­ence over a sub­field. If some­one re­views 30 pa­pers in their niche in a sin­gle se­mes­ter, they might have more in­flu­ence over its di­rec­tion for the year than we would want.

I’m not too wor­ried about this, though. We can re­quire that the sab­bat­i­cal­ists not re-re­view sub­mis­sions for mul­ti­ple jour­nals. So they can keep a giv­en pa­per out of at most one jour­nal. And since most jour­nals will re­quire two re­views for ac­cep­tance, they won’t have all that much pow­er in get­ting sub­mis­sions ac­cept­ed. Sec­ond, giv­en that their in­flu­ence wouldn’t be very strong, a se­mes­ter isn’t a long time for ex­ert­ing in­flu­ence. Third, I as­sume (as­so­ciate) ed­i­tors at pop­u­lar jour­nals al­ready have more in­flu­ence than what a sab­bat­i­cal re­view­er would have, even a re­peat sab­bat­i­cal re­view­er.

I’m sure there are a lot of com­pli­ca­tions that I haven’t thought of and I wouldn’t be too sur­prised if there’s some deep flaw I’ve missed. But even if there isn’t, I have no idea how to make the pro­pos­al real­ly hap­pen. Any­body know some­one who is or can in­flu­ence some­one with plen­ty of spare mil­lions that they’d want to use to aid philo­soph­i­cal re­search? █