Register | Sign In


Understanding through Discussion


EvC Forum active members: 65 (9162 total)
4 online now:
Newest Member: popoi
Post Volume: Total: 915,806 Year: 3,063/9,624 Month: 908/1,588 Week: 91/223 Day: 2/17 Hour: 0/0


Thread  Details

Email This Thread
Newer Topic | Older Topic
  
Author Topic:   The System of Scientific publishing
jar
Member (Idle past 393 days)
Posts: 34026
From: Texas!!
Joined: 04-20-2004


Message 16 of 23 (609369)
03-18-2011 4:01 PM
Reply to: Message 15 by slevesque
03-18-2011 3:54 PM


The system isn't perfect, but it is an adversarial by design system.
There will likely be some changes but over all, I'd say that it ain't broke.

Anyone so limited that they can only spell a word one way is severely handicapped!

This message is a reply to:
 Message 15 by slevesque, posted 03-18-2011 3:54 PM slevesque has not replied

  
Wounded King
Member
Posts: 4149
From: Cincinnati, Ohio, USA
Joined: 04-09-2003


(1)
Message 17 of 23 (609378)
03-18-2011 5:28 PM
Reply to: Message 15 by slevesque
03-18-2011 3:54 PM


Given how long this article is and that the relevant bit comes towards the end it might help to have a brief precis of the suggestions Trebino makes.
1. All data and parameters associated with any open publication should be available to anyone interested in it.
2. Anyone knowingly publishing a paper that clearly contradicts the work of another group should be required, also as a condition for publication, to attempt to discuss the matter with that group well before publication.
3. Journal editors should be more aware of referee conflicts of interest. Reviewers should be required to stipulate any conflict of interest in reviewing a paper, even if it’s simply that they don’t like the authors.
4. No journal editor should be allowed to edit a Comment on a paper that he allowed to be published.
5. Comments should not be required to be so short as to prevent them from making sense.
6. Crazy rules that allow logically offensive situations, like the one that called for rejecting a Comment because the Reply is unpublishable, should be deleted immediately. And Comments and Replies need not, and should not, be published together.
7. Reviewers who competently review a Comment should also review the Reply. They’re the best qualified, as they’re already familiar with the work.
8. Reviewers (of any paper) should themselves be reviewable. Currently, reviewers can say whatever they like, and there is no check on them.
9. While removing unethical reviewers would help, improving reviews of ethical ones is also important. Currently there is no compensation of any sort for reviewers and hence no encouragement to do a good job. I believe that reviewers should be paid for their services.
10. Finally, let’s face it: some journal editors are simply too arrogant or burned out and have lost sight of the goal, which is to publish only truth. Perhaps they could be required to sign a semi-annual statement that they ascribe to this key value as a condition of taking and keeping the job.
11. setting up a competent scientific misconduct commission (I believe that one already exists for medical research), to which one could take misconduct cases in all areas of science.
12. Require scientific ethics courses in grad school.
My view ...
1. As I said already I agree with this one.
2. I think this is a fairly unsupportable position, you could contradict the work of dozens of labs if you overturn a popular line of research. All Trebino seems to want to do here is to shift the onus from the position he was in to that of the other party, which might serve to retard criticism just as effectively.
3. Not liking someone isn't a conflict of interest. The better way around this , if it is required, would be to double blind the review so the reviewers not know who the authors are.
4 and 7 I'll take together. This seems like a pretty contradictory pair, if the reviewers of the comment are best qualified to review the reply then why is the editor of the original article disqualified. This should at least be consistent and require a new set of reviewers for each article.
5. This obviously makes sense, but you reach a point where a simple comment is clearly not the appropriate type of response.
6. This seems reasonable to some extent. I would go for a statement in the journal to that effect. I don't agree with the timing issue. Things are already bad enough with people failing to pick up comments and retractions, putting comments and replies together like this minimises this problem.
8. This is just thin skinned, authors can say what they like in their response to a reviewer and what they say reflects on them in the same way what the reviewer writes reflects on them. If the editor can't appropriately judge the reviewer's comments then the problem is the editor.
9. I don't agree with this, nor do I see how it would improve things.
10. Now he's just getting personal. Also I'm not sure how he magically expects editors to be able to determine what constitutes the truth.
11. This is mostly dealt with at an institutional level although there are already bodies such as the Office of Research Integrity in the US.
12. I agree with this one, but I don't think it would necessarily solve the problem, I doubt that in most cases the people indulging in unethical behaviour don't know it.
TTFN,
WK

This message is a reply to:
 Message 15 by slevesque, posted 03-18-2011 3:54 PM slevesque has not replied

  
Taq
Member
Posts: 9970
Joined: 03-06-2009
Member Rating: 5.6


Message 18 of 23 (609380)
03-18-2011 5:43 PM
Reply to: Message 15 by slevesque
03-18-2011 3:54 PM


The reality is that in both of those cases, the accident was caused by a mulfunction in the system. What I'm asking is, is the publishing system mulfunctional, or broken ? And should the recommendations made by Trebino be applied ?
It works in the vast majority of cases. One of the frustrations is the "black box" nature of the whole process. You would really like to know what is going on inside the heads of the reviewers, if only to improve the paper for the next submission. For example, a peer of mine had a really tough time getting a paper published. Each time he submitted the paper he would be told that if certain revisions were made and if certain experiments were done it would be publishable. He made all of the revisions, and the additional experiments he ran all supported his conclusions. The paper was rejected again with a list of all new revisions that needed to be made and a list of different experiments that needed to be done. It was a lot like Lucy pulling the football at the last minute. It gets really frustrating.
If anything, I would like to see an accredited third party review board that can review contentious papers and settle these disputes between editor and author. At the same time, journals who employ jackholes will get fewer and fewer submissions and the overall quality of their journal will decline. They do have to walk a fine line between quality control and volume, but it can tip too far in one direction at times.
Edited by Taq, : No reason given.

This message is a reply to:
 Message 15 by slevesque, posted 03-18-2011 3:54 PM slevesque has not replied

  
Dr Adequate
Member (Idle past 284 days)
Posts: 16113
Joined: 07-20-2006


Message 19 of 23 (609384)
03-18-2011 8:23 PM
Reply to: Message 15 by slevesque
03-18-2011 3:54 PM


The reality is that in both of those cases, the accident was caused by a mulfunction in the system. What I'm asking is, is the publishing system mulfunctional, or broken ? And should the recommendations made by Trebino be applied ?
Bear in mind that this is a system that scientists are imposing on themselves --- if it was severely flawed, they'd have already fixed it!
The change I'd most like to see is an online repository consisting of all raw data, including stuff that never gets published (this would counter the infamous "file drawer effect"). This is something that scientists can't do for themselves, it would probably require a massive multinational governmental effort.

This message is a reply to:
 Message 15 by slevesque, posted 03-18-2011 3:54 PM slevesque has not replied

  
Buzsaw
Inactive Member


Message 20 of 23 (609385)
03-18-2011 9:18 PM
Reply to: Message 11 by Taq
03-17-2011 12:09 PM


Taq writes:
But, yeah, scientists are assholes. Be ready for that when you try to establish a career in research.
I'll second that.
Who would ever have thunk it?

BUZSAW B 4 U 2 C Y BUZ SAW.
The Immeasurable Present Eternally Extends the Infinite Past And Infinitely Consumes The Eternal Future.

This message is a reply to:
 Message 11 by Taq, posted 03-17-2011 12:09 PM Taq has not replied

  
Blue Jay
Member (Idle past 2697 days)
Posts: 2843
From: You couldn't pronounce it with your mouthparts
Joined: 02-04-2008


Message 21 of 23 (609388)
03-18-2011 10:02 PM
Reply to: Message 15 by slevesque
03-18-2011 3:54 PM


Hi, Slevesque.
slevesque writes:
What I'm asking is, is the publishing system mulfunctional, or broken ? And should the recommendations made by Trebino be applied ?
I wouldn't go so far as to say that the system is broken, but there are clearly bugs and kinks in it.
Peer review is a difficult thing to manage. Editors often have to struggle to find people who are willing to do reviews (which can raise concerns about reviewer qualifications), and they have to try to enforce deadlines (which are very frequently not met).
Reviewers and editors can be jerks or softies; they can be wordy or terse; they can be organized or sloppy; they can be more qualified or less qualified; they can be fair or unfair; they may or may not take their responsibilities very seriously, etc.
The proper way to deal with this is to accept that there will always be a human element, and that justice won't always get served the way you want it to. And, even if you're treated unfairly, you can always just go to another journal. You may not get the impact factor or the readership you want, but anybody so absurdly idealistic that they think not getting exactly what they want is a legitimate grounds to bemoan the system is frankly too self-absorbed to get my sympathy.
-----
Here's another example (from ecology). It's possible to detect the DNA of a prey item inside the gut of a predator for a certain window of time after the predator has eaten the prey. So, now ecologists are jumping on the "molecular ecology" bandwagon to rake in all the big grant moneys and get in the news for "cutting-edge" science.
The trouble is that ecologists generally don't spend a lot of time learning the particulars of DNA work, and aren't really trained in the way biochemistry works. So, once somebody decided to try to determine what the window of detection for different types of predators is using laboratory feeding trials, everybody else just picked it up and started doing it.
Now, everybody has to do detection-window trials, or their papers get rejected; even though the detection windows you can get from laboratory studies are useless for interpretation. So, this is an example of peer review enforcing the retention of a costly and problematic hoop for ecologists to jump through.
Eventually, with all the resistance to it, and the improvement of knowledge over time, ecologists are going to realize this and change it; but, in the meantime, we're all going to waste thousands of dollars on Qiagen kits and thousands of hours force-feeding beetles and spiders in the laboratory.
Nobody ever said science was particularly efficient. But, broken? No, it's not broken: it's just annoying.

-Bluejay (a.k.a. Mantis, Thylacosmilus)
Darwin loves you.

This message is a reply to:
 Message 15 by slevesque, posted 03-18-2011 3:54 PM slevesque has not replied

  
subbie
Member (Idle past 1254 days)
Posts: 3509
Joined: 02-26-2006


Message 22 of 23 (609403)
03-19-2011 11:09 AM


Pharyngula on 21st century publishing
Pharyngula has a piece on how teh interwebs might add a new dimension to science writing. Being an armchair scientist, I'm not sure I fully understand all the implications, and I don't get the impression that this means by-passing the peer review process, but I'd be curious to hear thoughts about how much this might change things.

Ridicule is the only weapon which can be used against unintelligible propositions. -- Thomas Jefferson
We see monsters where science shows us windmills. -- Phat
It has always struck me as odd that fundies devote so much time and effort into trying to find a naturalistic explanation for their mythical flood, while looking for magical explanations for things that actually happened. -- Dr. Adequate
...creationists have a great way to detect fraud and it doesn't take 8 or 40 years or even a scientific degree to spot the fraud--'if it disagrees with the bible then it is wrong'.... -- archaeologist

  
Dr Adequate
Member (Idle past 284 days)
Posts: 16113
Joined: 07-20-2006


Message 23 of 23 (609659)
03-22-2011 12:12 AM


Nice quote from Daniel Dennett:
Through a microscope, the cutting edge of a beautifully sharpened ax looks like the Rocky Mountains, all jagged and irregular, but it is the dull heft of the steel behind the edge that gives the ax its power. Similarly, the cutting edge of science seen up close looks ragged and chaotic, a bunch of big egos engaging in shouting matches, their judgment distorted by jealousy, ambition and greed, but behind them, agreed upon by all the disputants, is the massive routine weight of accumulated results, the facts that give science its power. Not surprisingly, those who want to puncture the reputation of science and drain off its immense prestige and influence tend to ignore the wide-angle perspective and concentrate on the clashes of schools and their not-so-hidden agendas. But ironically, when they set out to make their case for the prosecution (using all the finely polished tools of logic and statistics), all their good evidence of the failings and biases of science comes from science's own highly vigorous exercises in self-policing and self-correction. The critics have no choice: There is no better source of truth on any topic than well-conducted science, and they know it.

  
Newer Topic | Older Topic
Jump to:


Copyright 2001-2023 by EvC Forum, All Rights Reserved

™ Version 4.2
Innovative software from Qwixotic © 2024