A new paper sheds light on how current publication practices may distort science. The article, by researchers in the US, uncovers some of the flaws in the current system and wonders whether the digital age will cause the system of publication to evolve yet further and exorcise 'the winner's curse' {1}.

By taking the process of scientific publication back to basics, the authors are able to predict many negative effects from the current model of journal-based scientific publishing. Herding, the economic term used to describe 'follow-the-leader' behaviour, is proposed to narrow and focus the field of research on certain areas that are determined by the papers with the highest profiles. 'The winner's curse', the theory that the winner in an auction will, on average, have paid more that the item is worth, is likened to the proposed exaggeration of results of papers in the most desirable journals. The authors, possibly controversially, suggest that trials with negative results could be published with the same frequency as those with positive results since they should be judged on the quality of the experiment rather than simply positive findings.

The digital age has already extended the reaches of personal networks and allowed personal opinion to be heard across continents. Journal clubs and isolated researchers can highlight the best articles of their reading to others using services like F1000 and, given the apparent limitations of conventional publishing models, surely this can only be a good thing.

Reference: {1} Young et al., PLoS Med 2008, 5:e201 [PMID:18844432]

References:

1. http://www.mathunion.org/fileadmin/IMU/Report/CitationStatistics.pdf 

References:

1. Hirsch JE. An index to quantify and individual's scientific research output. Proceedings of the National Academy of Sciences 2005 102: 16569-16572.

2. Barendse W. The strike rate index: a new index for journal quality based on journal size and the h-index of citations. Biomedical Digital Libraries 2007, 4:3

However, some are calling into question the use of citation rates as determining factors in the assessment of performance. This is especially pertinent since the Higher Education Funding Council for England recently announced that the assessment and funding of science-based disciplines will now be based on citation rates per paper, aggregated for each subject group at each institution {1}.

In a recent letter published in Nature {2}, the author suggests that the use of citation rates in determining funding hinders research in developing countries. It is argued that in such countries, researchers are often forced to publish their findings in national journals as a result of local legislation. Such journals rarely feature in the most-cited rankings and lack impact and readership on an international level.

A second letter, also published in the same volume of Nature {3}, goes on to say that poor practices by authors reduce the worth of citation rates. This piece highlights that citation rates can be altered by excessive citation of an author's own work, inappropriate citation of irrelevant papers, and geographical and language biases, as well as groups of authors forming 'citation coalitions'.

This is why the F1000 factor is such a useful indicator of a paper's worth - it is a weighted average with positive bias based on the number of evaluations an article receives, as well as the ratings of those evaluations.