Register | Sign In


Understanding through Discussion


EvC Forum active members: 65 (9162 total)
1 online now:
Newest Member: popoi
Post Volume: Total: 915,817 Year: 3,074/9,624 Month: 919/1,588 Week: 102/223 Day: 0/13 Hour: 0/0


Thread  Details

Email This Thread
Newer Topic | Older Topic
  
Author Topic:   The System of Scientific publishing
slevesque
Member (Idle past 4640 days)
Posts: 1456
Joined: 05-14-2009


Message 1 of 23 (609098)
03-16-2011 3:28 PM


So I just read this
How To Publish A Scientific Comment in 1 2 3 Easy Steps | PDF | Scientific Method | Science
And this opened up a whole lot of questions for me since I am studying to go into research, and I wanted to know the opinion of those of us here who are/were scientists and if there really some degree of problem scientific misconduct in the current system of publishing

Replies to this message:
 Message 2 by Taq, posted 03-16-2011 5:21 PM slevesque has not replied
 Message 4 by Dr Adequate, posted 03-16-2011 8:36 PM slevesque has not replied
 Message 6 by nwr, posted 03-17-2011 7:25 AM slevesque has not replied
 Message 8 by Dr Jack, posted 03-17-2011 9:43 AM slevesque has not replied
 Message 9 by Blue Jay, posted 03-17-2011 11:49 AM slevesque has not replied

  
Taq
Member
Posts: 9973
Joined: 03-06-2009
Member Rating: 5.7


Message 2 of 23 (609111)
03-16-2011 5:21 PM
Reply to: Message 1 by slevesque
03-16-2011 3:28 PM


And this opened up a whole lot of questions for me since I am studying to go into research, and I wanted to know the opinion of those of us here who are/were scientists and if there really some degree of problem scientific misconduct in the current system of publishing
Is it perfect? Hell no. Is it better than no peer review at all? Hell yes.
Science is not immune to the human element. We all hold the method and data as ideals, but at the same time we are all people with emotions and personal pride. Yes, there is a politics of science. It can even devolve into child-like playground insults. I've seen it happen. Anecdotally, I watched a grad student melt into a puddle of tears right on stage. She was giving a oral presentation at a big conference after which there is always an open floor Q&A. A prominent scientist stepped up and just started bullying her because her findings contradicted his own. This type of stuff happens, for better or worse.
With that in mind, conferences are the way to go to expose what you consider bad science. If someone publishes a paper that is wrong and concludes that your life's work is bollocks then present their work at a conference and show that it is wrong using your own findings. I have seen this done several times (including the sob story above). The reason that a conference is a good place for this type of exposure is that a lot of the big hitters in your field will probably be there if you choose the right conference. These are the same people that will probably be reviewing your future papers, and maybe even your grants. These are also the same people that you are speaking to in your papers. No one else in the world probably gives two cents about your research, but those scientists do.
Yes, there is chest thumping. Yes, there is posturing and preening. However, at the end of the day the data does win the day.

This message is a reply to:
 Message 1 by slevesque, posted 03-16-2011 3:28 PM slevesque has not replied

  
Wounded King
Member
Posts: 4149
From: Cincinnati, Ohio, USA
Joined: 04-09-2003


Message 3 of 23 (609119)
03-16-2011 5:45 PM


I would say that there is bound to be some degree, because the system is run by fallible humans.
Certainly the claim that it is easier to put a claim out there than to counter it is true. Indeed even when a piece of research has been genuinely shown to be severely flawed, even to the point of being retracted, it can still continue to be referenced for decades. Sadly people often aren't as scrupulous as they should be, and I'd hold my hand up to this, and don't read all of the papers they cite in full. I have more than once referenced something on the basis of a recommendation, or a citation in another paper, followed by a brief scan of an abstract, introduction and conclusion without going through all of the data.
I'm not sure that all of the authors suggestions are feasible, or in some cases desirable, but the one about the data being available for review is a very important one that I would agree with wholeheartedly. Unfortunately without some sort of centralised facility for hosting such data I'm not sure how feasible it is. If a lab host the data on their own site it is liable to disappear if they change institute. In many cases journals do make it a requirement that data is submitted to open data repositories, but that doesn't always work out and the journals may simply accept assurances or fail to check that the deposition has gone through.
Have to say, I'm not sure why he goes through all the rigmarole of anonymising everyone involved only to link to his comment paper so we know all the key players anyway.
TTFN,
WK

  
Dr Adequate
Member (Idle past 284 days)
Posts: 16113
Joined: 07-20-2006


Message 4 of 23 (609135)
03-16-2011 8:36 PM
Reply to: Message 1 by slevesque
03-16-2011 3:28 PM


For some reason, my computer doesn't let me read anything published on scribd. The page always loads, and then the pages are all completely blank. Could you hit me up with some highlights?
As to whether there's a problem, yes. The problem is that scientists aren't omniscient, which leads to two problems. When you see people (creationists, for example) chastising scientists, it is for one of two faults --- being excessively credulous, or being excessively skeptical. The people who edit journals are trying to walk a very fine line. No-one wants to be like the guy who published Blondlot's papers about N-rays. But also no-one wants to be like the guy who dismissed Morley's paper on plate tectonics with the comment: "This is the sort of thing that you would talk about at a cocktail party". (No-one knows who that guy was, but his comment ranks way up there with: "Guitar groups are on their way out, Mr. Epstein".)
Now, it is impossible for us to know the extent of this problem, because the only reason we know about any specific problem is that scientists have found it and corrected it. We can't point at any specific thing and say: "Look, science failed ..." without also being able to add: "... but then it worked, eventually!" We can't help but have a biased sample.
One thing we can say is that it would be difficult for scientists to adopt as an important truth something which is Just Plain Wrong. Because what it means for an idea to be important in science is that it becomes the basis for further research; and if you build your house upon the sand, you notice when it collapses.
Their errors in the negative direction cannot even be estimated. However, we should bear in mind that science is not monolithic, and that plenty of ideas which turned out to be completely crazy have managed to get a fair hearing.
In the end, though, there is a problem, which is that there is not (and, I think, cannot be) any formal method for accurately deciding when an idea is too silly to be worth considering.
Edited by Dr Adequate, : No reason given.
Edited by Dr Adequate, : No reason given.

This message is a reply to:
 Message 1 by slevesque, posted 03-16-2011 3:28 PM slevesque has not replied

Replies to this message:
 Message 5 by Taz, posted 03-17-2011 12:27 AM Dr Adequate has not replied

  
Taz
Member (Idle past 3292 days)
Posts: 5069
From: Zerus
Joined: 07-18-2006


Message 5 of 23 (609144)
03-17-2011 12:27 AM
Reply to: Message 4 by Dr Adequate
03-16-2011 8:36 PM


DrA writes:
For some reason, my computer doesn't let me read anything published on scribd. The page always loads, and then the pages are all completely blank. Could you hit me up with some highlights?
Compatibility issue? What browser you using?

This message is a reply to:
 Message 4 by Dr Adequate, posted 03-16-2011 8:36 PM Dr Adequate has not replied

  
nwr
Member
Posts: 6408
From: Geneva, Illinois
Joined: 08-08-2005
Member Rating: 5.1


Message 6 of 23 (609169)
03-17-2011 7:25 AM
Reply to: Message 1 by slevesque
03-16-2011 3:28 PM


Unless I am misreading, the cited report is about publishing comments, rather than articles. Comments are like letters to the editor. They are not peer reviewed, though they are editorially scrutinized. Some scientific journals allow them, and others don't.
Sure, the peer review system is imperfect, because it involves humans. But I wouldn't use comments as an measure of peer review.

Jesus was a liberal hippie

This message is a reply to:
 Message 1 by slevesque, posted 03-16-2011 3:28 PM slevesque has not replied

Replies to this message:
 Message 7 by Wounded King, posted 03-17-2011 9:06 AM nwr has seen this message but not replied

  
Wounded King
Member
Posts: 4149
From: Cincinnati, Ohio, USA
Joined: 04-09-2003


Message 7 of 23 (609173)
03-17-2011 9:06 AM
Reply to: Message 6 by nwr
03-17-2011 7:25 AM


In this case the comment certainly was peer reviewed. Different journals approach these things differently, but it certainly struck me that with the amount of time invested in organising and getting the comment published he would have been as well try to present it as a new paper in and of itself if it could be framed that way.
The main problem as I see it is not with the journal, although the tedious rounds of redrafting and revision can be arduous in any case, but with the authors of the first article. Their reluctance to provide the necessary data for checking really undermines their credibility. It is perfectly reasonable to jealously guard your data up to a point, but that point should be when the data is published, after that it should be available for scrutiny.
The furthest alternative is probably a system such as that used by PLOS One where anyone can comment on a paper, or even a specific section of a paper, as one might a youtube video.
TTFN,
WK

This message is a reply to:
 Message 6 by nwr, posted 03-17-2011 7:25 AM nwr has seen this message but not replied

  
Dr Jack
Member
Posts: 3514
From: Immigrant in the land of Deutsch
Joined: 07-14-2003
Member Rating: 8.7


Message 8 of 23 (609176)
03-17-2011 9:43 AM
Reply to: Message 1 by slevesque
03-16-2011 3:28 PM


Most published research is wrong, and peer review is a very low bar. A lot of very poor quality papers containing shoddy research make it through. Many of the worst research is, paradoxically, published in the most prestigious journals. Most of the scientists I know have a remarkably low opinion of Science and Nature.

This message is a reply to:
 Message 1 by slevesque, posted 03-16-2011 3:28 PM slevesque has not replied

  
Blue Jay
Member (Idle past 2698 days)
Posts: 2843
From: You couldn't pronounce it with your mouthparts
Joined: 02-04-2008


Message 9 of 23 (609191)
03-17-2011 11:49 AM
Reply to: Message 1 by slevesque
03-16-2011 3:28 PM


Hi, Slevesque.
I had a frustrating experience the first time I submitted a paper for peer review. My experience was not nearly as bad as Dr Trebino's, but it was still frustrating. My paper got generally positive reviews from both anonymous reviewers, but the editor kept harping on a single interpretive point without sending it out for any more reviews.
He wasn't really interested in engaging my explanations for why I had interpreted my data the way I had, nor in the substantial body of literature on which I had based my interpretation: he just wanted my interpretation out of the paper.
It actually got to the point where he gave me a list of words that I wasn't allowed to use in my paper, because of the interpretative connotations of the words in question (by the way, his English wasn't very good, either).
It wasn't really all that bad, but it was my first experience with the peer review system, and I remember being quite dismayed and disillusioned about peer review for a while because of the power one man with no particular expertise in the subject matter could hold over the process, in defiance of what the reviewers and the literature were telling him. But, it was just a small paper, and I've gotten over it now.
But, yeah, scientists are assholes. Be ready for that when you try to establish a career in research.

-Bluejay (a.k.a. Mantis, Thylacosmilus)
Darwin loves you.

This message is a reply to:
 Message 1 by slevesque, posted 03-16-2011 3:28 PM slevesque has not replied

Replies to this message:
 Message 11 by Taq, posted 03-17-2011 12:09 PM Blue Jay has seen this message but not replied

  
jar
Member (Idle past 394 days)
Posts: 34026
From: Texas!!
Joined: 04-20-2004


Message 10 of 23 (609194)
03-17-2011 11:55 AM


the case in point
The actual case in point though is one where the big issue is unrelated to the peer review process.
It looks like Trevio is more than likely referring to a paper that presents some methodology to replace FROG.
Okay.
That will get tested not in the world of reviews but in the labs. If the new process does turn out to be more efficient, cheaper, superior to FROG then it will likely prevail regardless of what is found in reviews and papers.

Anyone so limited that they can only spell a word one way is severely handicapped!

Replies to this message:
 Message 12 by Wounded King, posted 03-17-2011 12:50 PM jar has replied

  
Taq
Member
Posts: 9973
Joined: 03-06-2009
Member Rating: 5.7


Message 11 of 23 (609196)
03-17-2011 12:09 PM
Reply to: Message 9 by Blue Jay
03-17-2011 11:49 AM


But, yeah, scientists are assholes. Be ready for that when you try to establish a career in research.
I'll second that.

This message is a reply to:
 Message 9 by Blue Jay, posted 03-17-2011 11:49 AM Blue Jay has seen this message but not replied

Replies to this message:
 Message 20 by Buzsaw, posted 03-18-2011 9:18 PM Taq has not replied

  
Wounded King
Member
Posts: 4149
From: Cincinnati, Ohio, USA
Joined: 04-09-2003


Message 12 of 23 (609200)
03-17-2011 12:50 PM
Reply to: Message 10 by jar
03-17-2011 11:55 AM


Re: the case in point
As Trebino presents it it isn't a case of them presenting a novel alternative methodology but rather that they claim that FROG in fact does not work properly. Indeed the alternative methodology in question, auto-correlation, seems considerably older.
If people accept this at face value then they would stop using FROG, which understandably Trebino is against.
TTFN,
WK

This message is a reply to:
 Message 10 by jar, posted 03-17-2011 11:55 AM jar has replied

Replies to this message:
 Message 13 by jar, posted 03-17-2011 12:55 PM Wounded King has not replied
 Message 14 by Taq, posted 03-17-2011 1:12 PM Wounded King has not replied

  
jar
Member (Idle past 394 days)
Posts: 34026
From: Texas!!
Joined: 04-20-2004


Message 13 of 23 (609201)
03-17-2011 12:55 PM
Reply to: Message 12 by Wounded King
03-17-2011 12:50 PM


Re: the case in point
Exactly. Based on the article presented, this is something that will get decided in the lab, not the journals.
Does that FROG jump?

Anyone so limited that they can only spell a word one way is severely handicapped!

This message is a reply to:
 Message 12 by Wounded King, posted 03-17-2011 12:50 PM Wounded King has not replied

  
Taq
Member
Posts: 9973
Joined: 03-06-2009
Member Rating: 5.7


Message 14 of 23 (609205)
03-17-2011 1:12 PM
Reply to: Message 12 by Wounded King
03-17-2011 12:50 PM


Re: the case in point
If people accept this at face value then they would stop using FROG, which understandably Trebino is against.
It would then seem incumbent on Trebino to publish new research that addresses the criticisms and demonstrates that FROG does indeed work.
I didn't read all of Trebino's rather long whinge, but I did come across this abstract which seems to be the letter he was trying to get published before.

This message is a reply to:
 Message 12 by Wounded King, posted 03-17-2011 12:50 PM Wounded King has not replied

  
slevesque
Member (Idle past 4640 days)
Posts: 1456
Joined: 05-14-2009


Message 15 of 23 (609368)
03-18-2011 3:54 PM


Ok, so a lot of you just said that the system isn't immune to human elements, and so this is what explains situations such as the one linked to.
I feel this isn't really fact worth noting. It is like saying that a plan crashed because of gravity, or that the 2008 crash was caused by human greed.
The reality is that in both of those cases, the accident was caused by a mulfunction in the system. What I'm asking is, is the publishing system mulfunctional, or broken ? And should the recommendations made by Trebino be applied ?
Edited by slevesque, : No reason given.

Replies to this message:
 Message 16 by jar, posted 03-18-2011 4:01 PM slevesque has not replied
 Message 17 by Wounded King, posted 03-18-2011 5:28 PM slevesque has not replied
 Message 18 by Taq, posted 03-18-2011 5:43 PM slevesque has not replied
 Message 19 by Dr Adequate, posted 03-18-2011 8:23 PM slevesque has not replied
 Message 21 by Blue Jay, posted 03-18-2011 10:02 PM slevesque has not replied

  
Newer Topic | Older Topic
Jump to:


Copyright 2001-2023 by EvC Forum, All Rights Reserved

™ Version 4.2
Innovative software from Qwixotic © 2024