A report in Thursday's edition of Times Higher Education concerns a matter which was drawn to their attention by the two of us. The report says:
An academic has called for authors to have the right to see everything written about them in referees' reports after discovering that positive comments made about a rejected manuscript had been removed.
Mark Harvey, director of the Centre for Research in Economic Sociology and Innovation at the University of Essex, submitted a paper to the Routledge journal Economy and Society last year. It was rejected after being reviewed by five referees, some of whom had been suggested by Professor Harvey.
But by comparing the version of the reviewers' comments he received with those submitted by two of his recommended referees, he found that several positive comments about his paper had been removed.
Professor Harvey told Times Higher Education he feared that the comments may have been removed to justify the rejection. He added that the form for referees' comments used by Economy and Society did not suggest that reports could be edited before being shown to authors.
You can read the rest here. The report concludes by quoting Irene Hames, a member of the Committee on Publication Ethics, as follows: 'journals shouldn't use selective editing of reviewer reports to help them justify or be a better fit for a decision... Decision-making should be transparent.' That is also the view that we urged upon the Managing Editor of Economy and Society, Fran Tonkiss, but so far, it seems, without success. Here we offer a brief account of this episode - which we regard as an instance of academic malpractice, albeit of a low level kind - and follow up by appending the communication we sent to Times Higher Education.
Last year Mark Harvey submitted a paper to Economy and Society for a special issue of the journal that was in prospect and he was invited to suggest some referees for the paper himself, which he accordingly did. One of his suggested referees was Norman Geras, who in due course sent in his report; it recommended publication. In the event, MH's paper was rejected by the journal - a decision that both author and referee have not challenged - but when MH sent NG, by way of information, the copies of the referees' reports he had received from Economy and Society, NG noticed that there were discrepancies between the version of his own report that had been sent by the journal to MH and the integral version of it originally submitted by NG to E&S.
Troubled by these discrepancies, the two of us then wrote to the Managing Editor, Fran Tonkiss, on 18 December of last year. We found her reply to our email patently unsatisfactory, since its main point was to claim that the journal excluded explicit recommendations regarding publication from its digests of referees' reports relayed to authors; but this attempt to account for what had happened didn't meet the case. Not only, as we record in our letter to the THE reproduced below, was a negative recommendation (that is, against publication) left in one of the other referee's reports sent by the journal to MH, but some of the sentences omitted from the edited version of NG's report were simply part of his assessment of the strengths of MH's paper as he saw these, rather than explicit recommendations of publication.
In any event, the further correspondence between us and Fran Tonkiss having reached no satisfactory outcome, generating only a proposal from E&S that the journal now amend its referee report form to reflect its wayward current practice, we decided to publicize the issue by writing to the THE. This is our letter to them, sent on 9 April:
Undermining the integrity of peer review
At a time when much importance is being attached to publication in highly regarded academic journals for the purposes of the Research Excellence Framework, we wish to draw attention to a case of abuse of editorial responsibility, undermining the integrity of peer review, in the journal Economy and Society. This would be a serious matter at any time, but is particularly so now given the significant consequences of the REF.
As, respectively, the author of and one of the referees for a paper recently submitted to that journal, we should state at the outset that we have at no stage questioned either the editorial decision to reject the paper, or the right of the editors to make their own decision even if the balance of referees' reports points in a different direction. The abuse of editorial power lies elsewhere. It emerged by accident, in consequence of the particular circumstances of a special issue of the journal for which the paper was submitted and its author invited to nominate referees.
The editorial malpractice came to light when the referee's report communicated to the author was compared to the original version sent to the journal by the referee, and was found to have been carefully doctored in such a way as to support the editorial decision not to publish. Four positive statements had been excised from the report, significantly watering down its positive tenor. Subsequently, another report - also by a nominated referee - was found to have been edited to similar effect. In this case, the sentence following the excision had been amended so as to disguise the fact of an omission from the integral text.
As author and referee, we raised objections with the Managing Editor to what had happened - though in terms only of its undermining the integrity of the peer review process; we accepted both the decision itself and the journal's sovereignty over such decisions. As we pointed out, Economy and Society's Referee Report form clearly suggests to referees that the content of their reports will be conveyed to authors in full, and not in an expurgated version as has happened here. We specifically objected to editing that led to distorting the referee's evaluation in such a way as to support the editorial decision. The Managing Editor's initial defence of this was to say that only statements embodying direct recommendations about publication had been excised. This was false. A negative report contained the sentence (unexcised from the version sent to the author), 'This is not for E & S'. In other words, negative recommendations, supporting the journal's rejection of the article were left in, whereas positive recommendations had been cut.
In subsequent correspondence, the Managing Editor offered two further defences of Economy and Society's handling of the matter: first, that it is normal practice to convey a 'digest' of referees' reports to authors; and, second, that the journal will henceforth alter the wording on its Referee Report form to make clear to referees that that is in fact its practice. We have responded by saying that the alteration would merely give the editors licence to go on behaving as they have in the present case - persisting with the professional malpractice of doctoring referees' reports, while being more open about the fact that this is what they are permitting themselves to do. We pointed out that for peer review integrity to work there must be trust for both referees and authors that any editorial digest is a fair and accurate representation of the original referees' report. The editorial board of Economy and Society undermines that trust, both by the practice we have given an account of here and by their pusillanimous defence of it. We do not know whether this is an isolated case or whether it has become routine at the journal. That is the problem once trust has been undermined.
The two of us have discussed this episode with colleagues at several universities, some of whom have served on editorial boards of equally prestigious journals. Their response has been uniformly one of surprise and shock, and we have been urged to bring the issue to public attention, as we are doing now. We do it with the sole objective of ensuring that Economy and Society and its publisher, Taylor and Francis, will take the necessary and transparent steps to restore the reputation of its editorial processes and peer review integrity. If called on to do so, we can of course make public all the documentary evidence necessary to support our account.
Mark Harvey (author), Professor and Director, Centre for Research in Economic Sociology and Innovation, Department of Sociology,University of Essex.
Norman Geras (referee), Professor Emeritus in Politics, University of Manchester.
To some this episode may seem like no more than a spat between oversensitive academics. But others - and that includes the two of us - see it as an important issue of principle. Academics in the UK are currently under considerable pressure to publish in journals, and in some cases their careers could suffer if the papers they submit are improperly dealt with. Fortunately for both of us as senior academics (one of us now in retirement), we are not in this vulnerable position. And we can afford to put the issue in the public domain. In doing so, we hope to make it more difficult for journals to exercise arbitrary editorial power, and hope to contribute to a greater collective oversight of the editorial conduct of academic journals. In the end, it matters for the whole university community what is made public and what is left in the dark. From informal conversation with friends and colleagues we have both gained the impression that untoward experiences in this area are not as rare they should be. (Norman Geras and Mark Harvey)