## Henning Bergenholtz* & Rufus Gouws *

## Proposals for the Writing of Peer Reviews in Lexicography

## Abstract

In lexicography a good review is important for the dictionary maker(s), the publishing house and the whole lexicographical community. It is also important for the reviewers because it can expand their research record. Up to a few years ago reviews were still acknowledged in research databases. Currently they can be included in a database, but they do not count as scientifi  c outputs. The situation for peer reviews is similar. Peer reviews are an important quality assurance tool in the scientifi  c publication process. Good peer reviews have some mutual characteristics with reviews, especially regarding ethical aspects. But there are essential differences. These issues are discussed in this paper and some methodological and ethical proposals for peer reviews are made. One of the proposals could create a debate because it argues for an open peer review process and not for the so-called double blind peer review. Another proposal focuses on the role of the editor and his ability to decide if a peer review should be rejected and not be forwarded to the author.

## 1. Reviews and peer reviews as form of quality assurance

Everything produced by man can be reviewed: fi  lms, buildings, paintings, exhibitions, promotions, etc. However, in this paper the focus will be on scientifi  c reviews in general and more specifi  cally dictionary reviews. Reviews always deal with new works. In the evaluation of older publications one does not have a review but rather a general discussion or historical presentation. The object of peer reviews can also come from a wide variety of thematic fi  elds. In business, insurance companies or medical care centres peer reviews are used to ensure quality control in their ongoing activities. It is also employed in the assessment of research proposals. In this paper the focus will only be on peer reviews of scientifi  c publications in the fi  eld of lexicography - unpublished contributions submitted for publication in journals or as chapters in books.

Both reviews and peer reviews have a long tradition. The history of peer reviews goes back to the 17th century when Henry Oldenburg, a theologian and founding editor of Philosophical transactions of the Royal Society, felt he was not in the position to evaluate contributions from the fi  eld of natural science for publication in his journal. He therefore asked colleagues from the natural sciences to assist him (Wikipedia 2014). Reviews have an even longer history, as can be seen from the well-known review of several hundred pages of an Italian dictionary, cf. Hausmann (1989).

The contents of peer reviews has to be of signifi  cant value to the author of the reviewed contribution. A negative assessment gives the editor of a journal arguments for the rejection and the non-publication of a contribution. Therefore both reviews and peer reviews are important for quality assurance in science.

In spite of this important role for individual scientists and science as a whole one notices in the internal evaluation of research at universities an increasing lack of consideration of reviews and peer reviews. A single example is given that is typical of universities in Europe, Asia, Africa and North and South America. Table 1 shows how credits were formerly obtained at the Aarhus School of Business - also for reviews:

* Henning Bergenholtz Centre for Lexicography Department of Business Communication Aarhus BSS, Aarhus University Jens Chr. Skous Vej 4 8000 Aarhus C Denmark hb@bcom.au.dk

*  Rufus Gouws Department of Afrikaans and Dutch University of Stellenbosch Stellenbosch Republic of South Africa rhg@sun.ac.za

107

Table 1. Points allocated to research outputs

| Peer reviewed paper in a journal or a book     |   5 |
|------------------------------------------------|-----|
| Non-peer reviewed paper in a journal or a book |   2 |
| Book                                           |  10 |
| Presenting a paper at a conference             |   1 |
| Paper in the proceedings of a conference.      |   2 |
| Dictionary                                     |  10 |
| Textbook                                       |  10 |
| Contribution to a dictionary or a textbook     |   3 |
| Review                                         |   1 |
| Peer review for a journal or a book            |   2 |

Today no credits are given for either reviews or peer reviews. Academics are, instead. requested to refrain from these activities in order to write more papers in peer-reviewed journals. Even when reviews and peer reviews are included in research outputs they do not count anything in the evaluation of scientifi  c activities. This tendency at a Danish university is symptomatic of a world wide tendency that is extremely disadvantageous for science. Fortunately, in spite of this situation, there still are researchers who are willing to write reviews and peer reviews and who try to do this as well as they can.

## 2. Role players in and features of a good peer review

Both reviews and peer reviews are vital components in, amongst others, the quality-assuring process in lexicography. The focus in this discussion will be on peer reviews of contributions in the fi  eld  of  lexicography  submitted  for  publication  in  scientifi  c  journals  and  books. Although  the system of peer reviews has not received ample attention in the fi  eld of lexicography it has been a topic of discussion in a broader scientifi  c debate, compare for example online sources like the following:

http://codeandculture.wordpress.com/2013/11/18/youbrokepeerreview/#comment-3066

http://orgtheory.wordpress.com/2011/05/31/the-editors-speak-what-makes-a-good-review/

http://violentmetaphors.com/2013/12/13/how-to-become-good-at-peer-review-a-guide-foryoung-scientists/

http://www.theguardian.com/higher-education-network/blog/2013/sep/27/peer-review10-tips-research-paper

A variety of the features mentioned in these sources but also some of the aspects regarding reviews discussed in the preceding sections of this paper also apply to peer reviews of lexicographic work, e.g. fairness, ethical considerations, no self-promotion, etc. These aspects will not be discussed at length in this section but where applicable reference may be made to some of these features. Naturally there are also differences between reviews and peer reviews. Where any dictionary user is able to write a review of a dictionary this approach does not have a parallel in the writing of peer reviews - only experts in the specifi  c fi  eld are capable of writing a peer review.

The system of peer reviews of academic contributions is maintained for primarily two reasons, namely to assist the editor in making the correct decisions regarding the publication of an article or book chapter and to help the author to improve his/her contribution. However, a comprehensive look at peer-reviewing goes beyond these two mentioned reasons. It needs to include all the role players in the overall peer review process. The success of a system of peer-reviewing relies on the participation of all the relevant role players. This constitutes a signifi  cant feature of a good peer review. In this paper attention will be given to aspects of the responsibility and obligations of all the role players in the process of peer-reviewing lexicographic papers. The active participation and contributions of these role players will ensure the realisation of the features needed for a successful peer review.

## 2.1.  Role players in the peer review process

There are four role players involved in the process of peer-reviewing - three are generally acknowledged but the fourth is seldom accounted for. The well-known three participants are the peer reviewer, the editor of the journal or book and the author of the paper or chapter for a book. A fourth and equally signifi  cant participant is the scientifi  c community within which the peerreviewing is done. Like each one of the other three role players this fourth participant also has a very specifi  c responsibility in this process. Aspects of the role and responsibilities of each one of these four role players will briefl  y be discussed in the following paragraphs. A stronger emphasis will be placed on the role of the scientifi  c community because this role player has received less than enough focus in prevailing discussions.

## 2.1.1.  The scientifi  c community

If peer reviews are required in order to ensure quality control in scientifi  c lexicography these demands come from the scientifi  c tradition and the overall publication environment within which the specifi  c journal or book will be published. In this regard the editor or editorial board and the peer reviewer should only be seen as instruments functioning within this process. They have to be accountable to a higher body, namely the scientifi  c community, but they also have to respond to the demands and criteria set by this community. A bidirectional relation should be maintained between the editor and the scientifi  c community. Just as the scientifi  c community can expect the editor to require peer reviews in order to ensure the quality of papers and chapters in books, the editor could expect assistance from the scientifi  c community - and not only demands. A vital part of this assistance would be to enhance the status of the peer review system as an academic and scientifi  c endeavour.

Within the fi  eld of metalexicography, dictionary criticism is regarded as one of the components of a general theory of lexicography, compare Wiegand (1984). Criticism in a scientifi  c environment should be based on specifi  c criteria. Within lexicography the importance of expertise in criticism should never be underestimated. This also implies that the role of criticism cannot be restricted to dictionaries but also has to include metalexicographic publications. The scope of scientifi  c publications in the fi  eld of metalexicography goes beyond books, articles and chapters in books. It also includes peer reviews of any lexicographic contribution 1 . This demands that a peer review, giving critical comments on contributions submitted for publication in scientifi  c journals and books, should be regarded as a fully-fl  edged scientifi  c contribution and this implies that it needs to be written by an expert and should represent an application of a scientifi  cally-based critical look at the paper or book chapter submitted. Just as dictionary criticism is part of lexicographic theory, peer reviews form part of the fi  eld of lexicography and dictionary research. Within the fi  eld of academic and scientifi  c writing peer reviews should be regarded as a genre in its own right and contributions written in this fi  eld of scientifi  c writing could then also be entered in the publication list of the author and in the research data base of universities and other institutions.

In the bidirectional relation between the scientifi  c community and the editor of a journal and his peer reviewers, criteria need to be set for peer reviews but attention should also be given to the responsibility of the scientifi  c community. Criteria for peer reviews and peer reviewers and the obligations of the editors and peer reviewers will be dealt with in subsequent sections. The focus in this section will be on a few aspects of the responsibility of the scientifi  c community.

1 We fi  nd lexicographic contributions not only in journals for lexicography, but also in other journals, e.g. linguistic journals. But we don't see lexicography as a part of linguistics, e.g. lexicology, see Bergenholtz/Gouws (2012).

## Responsibility of the scientifi  c community

As a prominent role player in the process of peer reviewing the scientifi  c community does have a range of responsibilities. A few of these will be identifi  ed and discussed in this section. In this regard there should be close cooperation between the scientifi  c community and the editors of relevant journals and books in order to determine the respective tasks, obligations and responsibilities.

Many responsibilities of the scientifi  c community reside in the overall obligation to ensure the academic and scientifi  c status of peer reviews. Currently peer reviews have little if any status as scientifi  c contributions. This is detrimental in various ways. Editors often have trouble in fi  nding appropriate people who are willing to write peer reviews. Too often when someone does agree to write a peer review it is done in a hurried and insuffi  cient way - merely to satisfy the editor's responsibility that every contribution has to undergo a process of peer-reviewing. When and where reviewers go to a lot of trouble in the writing of a peer review their endeavours receive no form of acknowledgement besides a word of thanks from the editor or a credit by being mentioned in the journal as one of a panel of peer reviewers. The prevailing approach does not foster the idea of peer reviews being truly scientifi  c contributions and an integral part of the scientifi  c publication process.

One of the issues obstructing the acknowledgement of peer reviews as a scientifi  c genre is the so-called confi  dentiality approach that is prevails in the notion of blind and double-blind peer reviews. It has become an accustomed practice in peer-reviewing to submit the contribution anonymously to the peer reviewer and to receive anonymous feedback from the reviewer. There are a number of good reasons for this practice. The reviewer does not know who the author is and cannot intimidate or be intimidated by the author. When writing an anonymous peer review the reviewer may feel free to give critical comments that would have been withheld if the author would have known who had made the comments.

In a relatively small community like the lexicographic circle even blind peer reviews are not always that blind. Ever so often it is possible for the reviewer who is familiar with research in this fi  eld to identify the author of a contribution. Likewise, the author is often in a position to identify the peer reviewer on account of some of the remarks or references included in the peer review.

Science is open and does not function anonymously. Scientifi  c research is not published anonymously. Scientists are not afraid to expose themselves and their research results to the scrutiny of the scientifi  c community. Reviews of scientifi  c publications, with severe but also constructive criticism, are not done anonymously. The scientifi  c community acknowledges the fact that publications are open to criticism and this criticism often helps to enhance the standard of future publications. This could also be a way to approach a system of peer-reviewing.

Scientifi  c honesty and integrity should dominate the peer review process - also when it is not done anonymously. A junior lexicographer should not feel intimidated when peer-reviewing a contribution submitted by a senior or established researcher. Likewise the established researcher should not eschew the value of critical remarks made by junior colleagues. A peer review may never be used as a platform to promote your own standing, to attack your personal or academic rivals, to humiliate either your superiors or your junior colleagues or to give praise where praise is not due. Scientifi  c integrity demands an objective and unbiased approach to peer reviews - this applies to both the writer of the peer review and the author of the peer reviewed contribution. Moving away from an anonymous process of peer-reviewing towards an open system may help to establish a stronger scientifi  cally-based approach to this genre of scientifi  c writing.

A big advantage of open, non-anonymous peer reviews would be the possibility to have these peer reviews scientifi  cally assessed by the broader academic community - also after the publica- tion or rejection of a given contribution. In addition the peer reviewers can include these peer reviews in their list of publications, constituting another subdivision of their academic and scientifi  c publications. More emphasis on the value and scientifi  c merits of peer reviews may help to ensure their academic and scientifi  c status and may eventually contribute to them being recognised as research outputs.

## 2.1.2.  The author

A primary role player in the process of peer-reviewing is the author who has to submit a contribution to the editor of a journal or book. Although the author currently does not have an active role once the paper has been submitted, besides making the corrections suggested by the peer reviewer, this situation could be changed by introducing a new approach to the process of peer-reviewing. This can be achieved when the system of anonymous peer-reviewing is abolished.

Currently the authors of scientifi  c papers know that their contributions will be submitted for peer-reviewing. They know they can expect feedback from the peer reviewers and that they need to respond to this feedback. Editors have the responsibility to evaluate the feedback before sending it to the authors, and authors should know that the feedback they receive needs to be considered as valid and as an attempt to enhance the quality of the paper - even when it is suggested that the contribution should be rejected in its present form. Authors often realise that the peer review was written in a hurry with few comments or suggestions that refl  ect an expert opinion or a thorough reading and interpretation of the paper. This situation diminishes the response of the author to the peer review and decreases the status of this important process.

Where authors do not share the opinion of the peer reviewer they have no opportunity to engage in a direct and scientifi  c discussion with the peer reviewer. The only option the authors have, is to put their questions and responses to the editor who has to settle the issues. Scientifi  c discussions are at the heart of scientifi  c progress. Authors should have the opportunity to engage in a discussion with the peer reviewer. Elements from such a discussion could even be included in the paper if scientifi  cally relevant for the topic. This will most probably delay the publication of the paper but it could lead to an improved version and could help to strengthen the academic debate. A peer review system which is no longer dominated by anonymous bias may elevate the author to a more active role player in the process, especially in the post-peer review phase, and should lead to fruitful scientifi  c discussions.

In the selection of peer reviewers the editor should not opt for someone with either positive or negative feelings towards the author. It is not always possible for editors to know the nature of relations between authors and potential reviewers. Although authors should never be involved in the selection of their peer reviewers they may sometimes be aware of a specifi  c person or persons not suitable to peer review their work. In such a case the authors should have the liberty to indicate to the editor that a specifi  c person should rather not be involved in peer-reviewing their contribution.

## 2.1.3.  The editor

The editor has to be regarded as the central fi  gure in the peer-reviewing process. He or she has to be accountable to the scientifi  c community, has to select peer reviewers and assess their comments and has to give feedback to the authors. The editor also has to negotiate a range of practical issues. These include sending the peer reviewer a list of guidelines to be followed. Here special mention should be made to guidelines regarding the nature of the review and the types of comments to be made. Peer reviewers should be made aware of their responsibility as role players in a scientifi  c process. Therefore their feedback to the editor should not be restricted to the ticking off of a number of points on a standard list. The reviewers should know that their review has to assist the author and the editor should inform them that the peer review is not an opportunity for them to write their own hitherto unpublished paper. Another practical issue regards the review time. The editor has to request peer reviews to be submitted punctually so that the necessary corrections can be made in time for publication in the relevant volume of the journal. If a journal allows the authors to engage in discussion with the peer reviewers it could impede the publication process and the editor needs to negotiate this in the planning of the journal.

The increasing focus on ethical values and clearance also comes to the fore in the process of peer-reviewing. A set of editorial guidelines should therefore be complemented by ethical guidelines, as, for example, can be seen in the document of COPE, i.e the Committee on Publication Ethics: http://publicationethics.org/fi  les/Peer%20review%20guidelines.pdf.

As mediator between the author and the peer reviewer the editor has the important task to assess the comments made by the peer reviewers and to decide on what to relay to the author as feedback. If editors can deviate from a system of anonymous peer reviews and if peer reviews are elevated to fully-fl  edged scientifi  c publications peer reviewers would perhaps be less subjective in their remarks and the editor will receive more balanced comments on the papers submitted for peer review. Currently peer reviews often refl  ect a clear bias or a negative feeling towards the author or the theoretical approach of the author. This is not what should be assessed by the peer reviewer and where comments of this nature form the basis of the peer review the editor should ignore such a peer review and rather invite someone else to assist. The following comments are part of the feedback from a peer reviewer that an editor returned to an author:

'This is a manuscript written in the XXX tradition, whereby lexicography is redefi  ned, its terms and concepts also being redefi  ned, after which some pseudo-theoretical novelties are dished out. No doubt great fun for the in-crowd, but extremely boring for anyone who has long concluded that this type of metalexicography is a dead end. For XXX journal this thus poses a problem: embracing this means inviting more of this gibberish. Most other lexicographic journals have long refused to even consider more from XXX.'

Such a peer review clearly lacks objectivity. The editor should not have relayed these comments to the author. The command 'You shall not kill' also applies to peer reviews.

Editors have more than enough to do. Consequently it is important that a journal should have a review editor that can exclusively focus on peer reviewers and peer reviews. Having a clear set of criteria for peer-reviewing will lighten the burden of the review editor and will enhance the quality of the peer-reviewing process.

## 2.1.4.  The peer reviewer

In the prevailing peer-reviewing process the peer reviewer has a dual responsibility, i.e. towards the editor and towards the author. According to Raft (2013) they have to:

alert the journal editor to any problems they identify, and make recommendations as to whether a paper should be accepted, returned to the authors for revisions, or rejected

This can be regarded as their main assignment. The editor still has to read every contribution but should be able to rely on the assessment made by the peer reviewers. Therefore the selection of peer reviewers is of extreme importance. If a potential peer reviewer feels uncomfortable with the request to peer review a given contribution they should rather decline the invitation. Peer-reviewing demands expert knowledge and the peer reviewer must feel well equipped to undertake the assignment. In the past peer reviews ever so often have been completed in a hurry and without the necessary scientifi  cally-based assessment. An increased scientifi  c status allocated to peer reviews as a scientifi  c genre will compel the peer reviewer to write a better peer review. This will assist the editor much more than many of the attempts editors receive today.

Their second obligation is to supply the author with comments, both positive and negative, that can help them to improve the contribution. Many criteria that apply to reviews also apply to peer reviews. Unfortunately peer reviewers often deviate from these criteria. One of the ways in which a peer reviewer transgresses is when they attempt to change the contribution to such an extent and in such a way that it should resemble something that the peer reviewer would have preferred to see. Peer reviewers may not impose their personal point of view and demand in the peer review that the contribution should be changed according to their theoretical or ideological preferences. The peer review must be unbiased and it should evaluate the contribution; not the author.

If the academic status of peer reviews can be enhanced and if peer reviewers do not write anonymous reviews the peer reviewer gets a third important responsibility, i.e. to make a signifi  cant contribution to a scientifi  c genre and to stimulate and strengthen the lexicographic discussion. By having access to published peer reviews young or less experienced peer reviewers can get assistance and they can produce a better product.

## 3. Proposals for a policy of peer-reviewing

In this section a number of proposals are made that can help to establish a policy of peer-reviewing. It is important to note that these are presented as proposals and not as criteria. The proposals often are similar to those for reviews but although the same proposal applies the details may differ .

## Proposal 1: Peer reviews have to be fair

As previously stated a peer review has to be unbiased and objective.

- The peer review may not be clouded by the personal relations, either positive or negative, between the peer reviewer and the author of the contribution.
- Neither theoretical nor ideological issues should infl  uence the peer review.
- It is important to focus on both positive and negative features of the contribution but it is always good to start with a discussion of positive features.

## Proposal 2: Peer reviews should adhere to ethical guidelines

Ethical considerations apply to peer reviews and should be negotiated by both peer reviewers and editors.

- Comments should not be directed at the author but at the contribution being peer reviewed.
- Especially if a peer review moves away from an anonymous approach the peer reviewer should not try to praise or criticise the contribution in order to satisfy a personal grudge or to get personal gain.
- The peer review is not a platform for self-promotion or self-praise.
- There is no room for sarcasm in peer reviews.

## Proposal 3: Do not repeat the contents of a contribution

Too often the emphasis of a peer reviewer is on giving an account of the contents of the contribution. The peer review has to assess; not repeat. From a peer review the editor must be able to make a fi  nal judgement about the publication of the contribution and the author needs to fi  nd guidance with regard to possible improvements but also with regard to the strong points of the contribution. Neither editor nor author will benefi  t from a mere repetition of the contents.

## Proposal 4: Identify the innovative features

In the development of lexicographic literature it is important that new focal areas and new ideas have to emerge. One of the important contributions of a peer review is to identify the innovative features of a contribution. How good or bad these new ideas are, is not that important for the peer reviewer but an editor will appreciate a clear indication of an original contribution made by the author.

## Proposal 5: Peer reviews are scientifi  c publications

Although the writing of peer reviews has to be regarded as a service to the scientifi  c community in order to ensure and enhance quality assessment and control, the function of peer reviews goes beyond a mere community service and the scientifi  c status of peer reviews needs to be acknowledged by the all the role players in this important endeavour. As a scientifi  c genre in its own right peer reviews are entitled to be included in the publication list of a researcher.

## 4. Conclusion

The writing of reviews and peer reviews form an integral part of the academic and scientifi  c publication process. Much more emphasis on the nature, extent and role of reviews and peer reviews is needed. Although this paper focuses on peer reviews with regard to lexicographic publications the discussion and proposals are largely also relevant to other subject fi  elds. The arguments in favour of an open system of peer reviews can enhance the status of peer reviews and establish them as a fully-fl  edged genre. As such they can be cited and discussed in lexicographic publications. Authors could also have the opportunity to respond to the comments made in peer reviews. This response is not restricted to the journal to which the contribution was submitted but it can also be used in future contributions in other journals.

## 5. References

Bergenholtz, Henning/Rufus Gouws 2012: What is Lexicography? In Lexikos 22 , 31-42.

- Hausmann, Franz Josef 1989: Kleine Weltgeschichte der Metalexikographie. In Wiegand, Herbert Ernst (ed.), Wörterbücher in der Diskussion. Vorträge aus dem Heidelberger Lexikographischen Kolloquium . Tübingen: Niemeyer, 75-109.
- Raft,  Jennifer  2013  [online].  http://violentmetaphors.com/2013/12/13/how-to-become-good-at-peer-review-a-guidefor-young-scientists/) http://de.wikipedia.org/wiki/Peer-Review (accessed 8 November 2014).
- Wiegand, Herbert Ernst 1984: On the structure and contents of a general theory of lexicography. In Hartmann, R.R.K. (ed.), LEXeter '83. Tübingen: Max Niemeyer: 13-30.
- Wikipedia 2014 : Peer-Review [online]. http://de.wikipedia.org/wiki/Peer-Review (accessed 8 November 2014).