No more free labour by me for Synthese

UPDATE: March 2019
Well, it's been over three years. The editors never apologized, although they did give a kind of half-hearted implied acknowledgement of error. Everyone has moved on. I spent three years declining special issue invitations, referee requests, etc. I am disappointed that more people didn't join me in this kind of statement, but I guess the profession on the whole thought enough had been done. Reluctantly, and a little sadly, I'm ending my little protest now. The next time I refuse to referee a Synthese paper, it will be because I'm too busy, not because I'm unwilling to contribute my labour to this particular journal.

Original post follows:





Like most philosophers who are active on social media, I was amazed and horrified this week to learn from Feminist Philosophers about an article by Jean-Yves Beziau, nominally on logical pluralism, published in Synthese and containing irrelevant and incoherent sexist and homophobic ramblings. (See also the Daily Nous coverage/discussion here.) The article represents an extremely serious editorial failure; in my opinion, its publication is inconsistent with Synthese's status as a high-prestige philosophical journal. The editors need to apologize (really apologize) and retract the article. Unless and until that happens, I plan to exercise the little bit of personal power that I have in this matter, and refrain from contributing any more of my labour to a journal that has so dramatically failed to live up to its responsibilities.

I have a high opinion of Nous and PPR

While I can see some risk that I could be suffering a moral blindspot here, I feel like, as someone who's recently been speaking out about philosophy journal editorial practices, and also someone who worked closely for a couple of years with Nous and PPR, that I should make a point of making public my opinion that those journals seem, from all the evidence available to me, to be very well-run. I'm unaware of any editorial misconduct there, and I do not suspect that there has been any. I think the discipline would be a better place if more editors were more similar to Ernie Sosa.

I don't expect anybody to take my opinion as significant evidence on this matter, but I wanted to at least put myself on the record.

Challenges for Anonymous Review

I'm continuing to think about journals. There are a lot of serious problems, and it's really unclear how to fix them. I want to lay out another problem I haven't seen remarked on before. This one's more about referees and authors than editors. I'll start with a problem that has been remarked on before:

As has recently been observed several times, even with the best editorial practices in the world, it's not very difficult for a referee who isn't committed to anonymous review to google the title of a submission and learn the author. If, like most of us, you don't submit your paper until after it's been around in various forms for a while—you've presented it at workshops, posted drafts online for comments, etc.—the anonymity of the review process is contingent on the whim of the referee. It's possible to take some steps here as an author—you can change the title prior to submission, for example. This will make it harder for a referee to discover your identity, but likely not impossible.

Here's the twist on the problem: the reason that anonymous refereeing is important is to avoid bias—preferential treatment of some submissions (those by established figures, white people, native English speakers, men, people educated at prestigious departments, etc.) over others. So while it's possible for authors to go to extraordinary measures to protect their anonymity, in many cases, it's against their own selfish interests to do so. Suppose for the purpose of argument—and this strikes me as fairly plausible—that I am someone who is more likely to be unfairly benefitted, than unfairly harmed, by a breakdown in anonymous review. I'm at least most of the kinds of people mentioned in that parenthetical. (I've never quite figured out whether I'm a white person or not.) When I submit my paper to a journal, the journal has a rule, and the profession has a norm, that I should anonymise my paper. I remove self-citations or put them into the third person, I omit acknowledgements, etc. But now I have a choice to make: in addition to removing that straightforward information from my submission, should I take my draft down from my website, and change the title? Should I paraphrase key sections that are too similar to the abstract of a talk I gave last year that's still up on a workshop website? If anonymous review were very important to me, I could go to extra effort to preserve it. But if I'm thinking selfishly—and I do think it's reasonable for individuals participating in this system to be making these kinds of decisions selfishly—I'm not going to be very motivated to do so.

It is only those who need the protection from negative bias who are incentivised to go to great lengths to ensure anonymous review. And so now we get to the next layer of the problem: if, as seems likely, the people most incentivised to ensure anonymous review are more likely than others to take the steps to render their submissions ungooglable, then ungooglibility becomes evidence a submission by someone referees are biased against. If a referee decides to google the paper, and finds nothing, the referee can take this to be evidence against the paper's having been written by someone "important".

I can only see two solutions to this problem; both seem very difficult to implement. The first is that authors don't treat these kinds of extraordinary measures as optional—journal policies or disciplinary norms could make clear that authors must not include drafts online, or references to titles, or submit papers with language too similar to language on workshop websites, etc. This seems like a very intrusive requirement, but if the rules could be articulated clearly enough, it might do the job. The other is the obvious one: that referees don't go seeking this information out. This requires sound judgment on the part of referees, and is hard to enforce, but it would also work. The difficulty here, of course, is that it's so hard for editors to get good referees (or any referees), because there's so little incentive to do the job well, ethically, or at all.

I don't know how to make the economics work, but I feel like a lot of things would be much easier if there were some way we could pay referees for their work.

Journal Editorial Practices and an Anecdote

I was interested to read the recent Daily Nous conversation about journal editorial practices. Much of the discussion there focused on whether double-anonymous refereeing (where the authors and referees are ignorant of each others' identities) was good enough, or whether triple-anonymous methods (where the editors are also unaware of the authors' identities) are necessary for a fair system.

I was interested to see many journals' editors describing their methods, and how they try to avoid bias. One question I didn't see addressed, however, concerns oversight. Is there any system in place to confirm whether journals are actually run the way they're said to? I would consider this a somewhat paranoid question, except that I know firsthand of at least one high-profile journal which at least sometimes, at the discretion of the editor, suspends its official policy of double-anonymous refereeing.

My experience happened shortly after I finished my PhD, when I was a postdoc. I knew at the time it was a problematic situation, but I decided it probably wasn't prudent to make a fuss at that stage in my career. I'd been intending to wait until I was tenured, but I think that given the conversations happening in the discipline at the moment, now is an appropriate time for the story to come to light. This is the story of "Quantifiers and Epistemic Contextualism," one of my first papers defending contextualism, which was eventually published in Philosophical Studies.

I apologize for the length of the narrative. I'd make more of a point of being concise, but I want to make sure that I explain exactly what happened, without editorial summarizing. (I'll express some opinions after telling the story.)

So here's the story.

After defending my PhD in 2008, and beginning work as a postdoc in St Andrews the same year, I started to develop some of the ideas from my dissertation into publishable papers. I submitted one of them, a defence of contextualism, to Philosophical Studies in March of 2009. That journal uses an online program for submissions where an author can watch the progress of his submission as it goes through various stages—'editor assigned', 'reviewer assigned', 'review completed', etc. I noticed in July that four months had passed and it still just said 'editor assigned'. (I knew from a previous paper that it ought to be progressing through other stages.) So I submitted a query through the online submission, asking whether my paper might have fallen through the cracks.

I received a response to that question in the form of a personal email from Stewart Cohen, the editor of the journal. Cohen wrote that "the reason your paper is listed as editor assigned, is that I'm going to review it myself." He said he hoped to get a chance to review it soon, and invited me to email him directly if I had any further questions. This happened in July. (Potentially relevant background not everyone might know: Cohen wrote several influential papers in defence of contextualism in the 1980s and 1990s; he is one of the prominent figures in this subfield of epistemology.)

In September 2009, Cohen was hired as a part-time professorial fellow at Arché, the same institution where I was working.

In October, Cohen came to St Andrews in connection with his new job. He spoke briefly with me about my submission then, saying that he generally liked it, but had some particular concerns. He sent me a lengthy email detailing them, and we had a rather involved back-and-forth exchange on various of the substantive philosophical questions. At the conclusion of this exchange, after I'd agreed to make various changes to the paper, we'd gotten down to just one major remaining point of disagreement. In my paper, I argued that my neo-Lewisian form of contextualism avoids a challenge that Cohen himself had leveled against Lewis, concerning his ability to handle both Gettier cases and lottery cases. Cohen had argued, in a 1998 paper, that Lewis can't have it both ways; but I suggested in my submission that the issue could be avoided. Cohen was unconvinced by my treatment of this part of the paper, but invited me to work on it further. He wrote to me in an email:

Let's call it a conditional acceptance. I'm not sure you've made any advance in applying the RA view to Gettier cases and so I don't want to commit to publishing that part of the paper. You may convince me that I'm wrong, but even if you don't, you can always just cut that part. So unless you think you'll refuse to cut it, even if I'm not satisfied, you can list it as forthcoming.

In November 2009, I submitted (via direct email to Cohen) a revised version of the paper. We corresponded briefly about it in January when Cohen was visiting Arché again, but he didn't read and respond to my new draft until April 2010, when he wrote back, saying that he was still unconvinced by my treatment of lotteries and Gettier cases. We had another email exchange, which continued until his next visit to Arché in May. On May 24, we met over coffee about whether the section in question ought to be published. By the end of that meeting, he did agree to publish it, and on May 25 I received an official acceptance from the journal, and an invitation to upload my final version. The paper did end up being published, including the revised version of the section defending Lewis from Cohen.

Ok, those were 'just the facts'. From here on out, I'm editorializing. These seem to me the important problematic points to emphasize:

• The editor of Philosophical Studies seems to be treating submissions in his own area of research interest very differently from the way he treats other submissions, sometimes deciding to referee papers himself. As far as I know, Cohen was the sole referee for my submission.
• At least sometimes, refereeing at Philosophical Studies isn't even double-anonymous. (Indeed, in this case it wasn't even single-anonymous.) I spent two hours sitting with my referee over coffee, arguing about whether his concern about my paper was correct. (This is at odds with their explicit policy of a "double-blind review procedure".)
• There is lots of research showing the potential for various sorts of bias to come into play when one isn't making these decisions via anonymous review. But this is an extremely clear instance in which privilege worked to my advantage. I was a Rutgers PhD, with a job at a prestigious research institution, where my colleague literally down the hall (well, down the hall and around several corners) was the editor, who was also the referee. I was also the kind of person willing to be bold enough to argue with a senior figure who was repeatedly suggesting that my idea wasn't suitable for publication. (It helped that I was already on a first-name basis with him before the story began.) These factors, which had nothing to do with the quality of my submission, played a huge role in getting this early publication. And even with the advantages I had, I was still a fresh PhD without a tenure-track position—my desperation for a publication, combined with the strange relationship I was in with a powerful figure, was disconcerting to say the least. I very much doubt that Cohen understood how awkward my position was, or how uncomfortable it made me.
• In addition to worries about anonymous review, I think it's also potentially problematic that Cohen's biggest concerns about my paper had to do with the particular section where I engaged critically with his own work. I don't think for a minute that he was motivated by a desire not to have critiques of his work published. (And I'm not also totally sure that his worries were unfounded—I think my paper probably improved when I took his worries more seriously.) But I do think that it's easy to be biased in favour of our own views, and that this is something a journal should be very careful to avoid. It wouldn't be practical or particularly desirable, I think, to prohibit referees who are being criticized in submissions—but I do think that this is a situation that calls for an editor (who is not the referee!) to observe particularly carefully, and take the referee's advice in the appropriate context. I don't know to what degree my experience represents a pattern, but I think it would be a very poor feature of a journal if submissions criticizing the ideas of the editor were quite likely to (a) be refereed by the editor himself, and (b) be scrutinized particularly critically. There is a very nearby possible world where I simply deleted the section of my paper critical of Cohen. 

On the whole I'm very confident in asserting that in at least one instance, Philosophical Studies engaged in seriously problematic editorial procedures. I don't know how atypical my story is.

I also want to explicitly note that this story occurred five years ago. I see that Jennifer Lackey and Wayne Davis were both added as Associate Editors of Philosophical Studies in 2011, after my paper was accepted in its final form. (Cohen is still the editor-in-chief.) I don't know how much change that represents. For all I know, it may be that a system is now in place such that stories like mine couldn't occur again. But I'd like to know. I'd like to call for a statement from Philosophical Studies about its editorial practices. (Phil Studies was not one of the journals whose practices were described in the Daily Nous post linked above.) In particular, does the editor still have the discretion to decide to referee a paper himself, with full knowledge of the author? Does the journal sanction the kind of back-and-forth discussion between editor–referee and author I describe? If not, are there procedures in place that prevent it from happening?

I'm not an anonymous-review absolutist; if a journal prefers not to have an anonymous refereeing system, I'm OK with that. But transparency is important; if Philosophical Studies is not a journal run with a commitment to anonymous review, then the philosophical community should probably be thinking of that journal in a very different way than it does.

Brief Follow-up

Some people have asked me why I am speaking out publicly about Brian Leiter’s threat of litigation. If the only thing I cared about was getting Brian Leiter to leave me and my wife alone, silence would probably be the prudent response to the letter we received. But I think the philosophical community is entitled to know when freedom of speech within it is under threat. That's why it's important to me that Leiter's threatening actions are brought to light.

The things we have said about Brian Leiter constitute protected speech. They were not misleading, and we stand by them. If Leiter carries out his threat to sue, we will vigorously defend. We will also have the right to counterclaim against Leiter for false and defamatory statements he has made about us during the last year, including those contained in the letter from his lawyer he published and his commentary on it.

A Recent Email from Brian Leiter's Canadian Lawyer

On December 15, Professor Brian Leiter notified me and my wife Carrie Jenkins, through a Toronto lawyer, that he "is prepared to seek redress in the courts of Canada" against us over various Internet postings which he alleges defame him.

Professor Leiter claims to be defamed by:

1. Carrie’s pledge on her tumblr blog to behave with civility towards other philosophers and colleagues;
2. Carrie’s post to Facebook of the complete text of Professor Leiter’s email of July 2, 2014 regarding that pledge;
3. the so-called “September Statement”; and
4. the post on the Feminist Philosophers blog entitled “Sometimes An Apology Doesn’t Help.”

His Toronto lawyer has demanded that Carrie and I publish on the Internet, for a continuous period of at least six months, a lengthy apology and retraction (which his lawyer drafted).  If we do not, we are warned, Professor Leiter “will pursue his legal remedies against [me and my wife]” and “perhaps others among the original signatories to the ‘September Statement’.”  We are also warned that Professor Leiter’s Canadian lawsuit against me and my wife will involve “a full airing of the issues and the cause or causes of [Carrie’s] medical condition;”  a reference, it would seem, to posted information about the impact of Professor Leiter’s actions on Carrie’s health, her capacity to work, and her ability to contribute to the public discourse as a member of the profession.

Carrie and I have instructed our lawyer to inform Brian Leiter that anything we have posted about him on the Internet is lawful free speech under Canadian law and under the First Amendment to the US Constitution.

High School Popularity: A Modest Proposal

One of the difficult things about high school is figuring out which people to be friends with. After all, it can make a big difference in your life! Being friends with popular kids is a good way to become more more popular yourself. Plus, your teachers might treat you better, you'll get more cool stuff (from being friends with the popular kids, who are also often the richer ones), etc. In the status quo, however, freshmen often enter high school without a very clear idea of who the popular kids are, so they're aiming their friendship aspirations pretty haphazardly.

But here's an idea for an enterprising popular kid to provide an invaluable service to everybody. He gets a group of his friends together, and they rate everybody in the school (or at least everyone they think is at least minimally popular) for popularity. Then he can make the results known to the whole school, free of charge! Now everyone worth thinking about trying to be friends with comes along with a numerical popularity rating. Sure, everybody's going to try to be friends with that one girl who was already a 4.8, and she doesn't have the time or energy to be friends with everyone, but since she's the most popular, it makes sense for her to be able to be the most selective about choosing her friends. And aren't the best candidates for friends the ones who deserve to have access to the most popular kid's friendship?

Now I'm the first to admit this system won't be perfect. The popular kids are likely to get ever more popular, since everyone will know that they're the people to try to be friends with. And of course everyone will have some incentive to be friends with that one kid who started the rating system, and with the raters that make up his circle of friends. (The latter can be mitigated somewhat if that first kid occasionally makes changes to the roster of kids who do the ratings.) So yeah, maybe there are better possible systems. But in the status quo, people are just trying to guess who's most popular by asking a couple of people or -- even more unfairly -- by judging by superficial cues like race and attractiveness and athletic ability. How is that fair.

Philosophy Book Review Tips?

I'm reviewing a book for the first time; do any philosophers have tips on how to plan/organize/read/etc.? This is all new to me, and I'd welcome any advice from veterans on how to proceed. Do you like to take notes along the way? Should I plan on reading cover-to-cover more than once? How do you decide what to focus on?