Quote from: StormBringer;225618no u

Your shortened form of "I know you are but what am I?" is kinda catchy.  You should put that on a t-shirt or bumper sticker or something.

Quote from: jhkim;225503It is standard within scientific circles to present your results rather than your raw data.  As a physicist, none of my experiments released our raw data to the public -- and as far as I know it is standard throughout the sciences.  (We shared some but not all of our code.)

Wikipedia calls Reproducibility "one of the main principles of the scientific method, and refers to the ability of a test or experiment to be accurately reproduced, or replicated, by someone else working independently."  It also defines Intersubjective Verifiability as "a core principle of empirical, scientific investigation."  Under Pseudoscience, discussing how the natural scientific method protects against pseudoscience, further says, "All gathered data, including experimental/environmental conditions, are expected to be documented for scrutiny and made available for peer review, thereby allowing further experiments or studies to be conducted to confirm or falsify results, as well as to determine other important factors such as statistical significance, confidence intervals, and margins of error."

So if you don't have access to the original data and how it was processed, how are you supposed to reproduce the results yourself?  And how can someone rigorously peer review a paper without that information?  Is peer-review simply a rubber stamp or popularity contest in practice?  

Quote from: jhkim;225503As for motivation, it sounds like you're delving into conspiracy theories here.  Yes, it is possible that any given scientific result is the result of a conspiracy among the top researchers to come up with a result.  On the other hand, it seems to me that the simpler explanation is that their results are what they actually found.  I have not seen anything to at all impugn the integrity of the research.

The problem is that their results were not simply "found".  They are reconstructions of past climates using a selection of multiple proxies (e.g., tree rings, ice core samples, etc.) and then they weigh and adjust the proxies.  That the process includes selection, weighing, and adjustments provides ample room to produce particular types of results.  For example, here Steve McIntyre shows how he can produce a highly elevated Medieval Warm Period and only modestly elevated modern temperature by using a series of proxies that have that pattern in them, not to prove the Medieval Warm Period but to show how the selection of data can produce very different results from the "hockey stick", which invariably just happen to rely on data showing that 20th century upward spike that produces the hockey stick shape.

Quote from: jhkim;225503The Ad Hoc Committee paper had a bunch of criticisms, but for the most part they were critical of current scientific process in general.  For example, they claimed some errors in statistics handling in the paper and recommended that the scientists should have worked with the Statistics Department of their university in data handling.  While that is not an unreasonable suggestion, many scientific papers have statistics errors and almost no one goes to another department to tell them how to handle their data.  Note that I can easily believe that there are errors in the 1998 paper, errors are not the same as bias.

That many scientific papers contain errors does not excuse errors.  And when we start talking about international treaties and business regulations, those errors can have a very real economic and social impact.  Regardless of whether it's a mistake or a bias, transparency would upon the work up for independent analysis that could uncover either.

Quote from: jhkim;225503It also criticized the process of peer review, noting that the community of reviewers were almost all people who knew each other and who often had worked together in the past.  That's how peer review works.  There are problems with it, but there are also problems with having amateurs or people from other fields judge your work.

I think that transparency is always preferable to opacity, particularly because opacity creates an illusion of confidence that's often not warranted.

Quote from: jhkim;225503It's true that Mann's 1998 paper presents a different picture than in, say, the 1990 IPCC Assessment report.  You could hypothesize that this is because the 1990 report is the real truth and Mann conspired to try to hide the truth.  However, I don't accept that any of the 1990, 1980, or 1970 pictures are the real truth.  Our scientific understanding is always revising our picture, and generally getting better.  I expect that the 1990 report is also different from reports from 1980.  That's how things work.

Actually, the 1990 chart is similar to a 1975 NAS graph which cites Lamb 1966 as a source.  So for at least 24 years, the interpretation was pretty constant.  Yes there can be major revisions of our understanding of things but isn't skepticism a normal response to such major shifts?

Quote from: jhkim;225503Opening data is a tricky thing.  In general, I think research should be done by specialists in the field.  I love wikipedia and the blogosphere, but I don't think that a horde of politically-motivated amateurs combing over the data is going to improve our understanding of any science.  Relative to physics, I particularly get annoyed at a lot of amateur claims regarding nuclear physics -- which itself has many political implications.  I would rather that we fund research to get more specialized experts working on the problem.

The hoard of politically-motivated amateurs are going to speculate, anyway, and the absence of data simply feeds conspiracy theories and suspicion.  When people hid things, it gives people the impression that they have something to hide.

Quote from: jhkim;225503That said, I do think that ideally scientists should be more free with their data.  However, here's the problem.  The truth is, scientists constantly make mistakes.  I've seen a few pretty egregious ones in my research time, and tons of little ones in how research gets done.  However, my observation is that scientists are more respected if they keep their data and their mistakes closed.  When someone opens their data, they get tons of criticism and little praise.  We need to change that attitude, which has a lot to do with the whole structure of academia.

Well, isn't one solution to demand transparency in peer reviewed publications and public policy-related research?  Steve McIntyre's point about due diligence is that in commercial research, government regulations require transparency through reporting and record keeping so that it's not optional.  Yes, given the choice, many scientists would prefer to keep their data hidden and to hide their mistakes, but wouldn't it encourage them to be more rigorous if they feared having their errors exposed?  Doesn't this suggest that maybe they shouldn't be given the option of concealing their research?  Is the point to help researchers be respected whether they deserve that respect or not or to do good science?

Quote from: John Morrow;225655So if you don't have access to the original data and how it was processed, how are you supposed to reproduce the results yourself?  And how can someone rigorously peer review a paper without that information?  Is peer-review simply a rubber stamp or popularity contest in practice?

Peer review doesn't consist of reproducing the full data analysis, and it is not the whole of reproducibility.  Peer review generally consists of a reviewer reading through the proposed paper and commenting on what is presented there.  The reviewer may ask for additional information of the researchers, which usually indicate such information should be included in the paper.  It is a guard against mistaken analysis, but it relies on the reviewed scientists being truthful about both what their original data was, and in describing what steps they did in analyzing it.  

You reproduce the results not by re-analyzing the researchers' own data -- which would be flawed anyway if they were willing to lie in favor of their biases.  Rather, you do a whole experiment from scratch that finds the same thing.  This is tricky if you are working in a field with limited data samples or room for experimentation, but in general you look for independent confirmation.  

And there are a number of people who work within the field of paleoclimatology, who publish other papers looking at the same or similar trends.  The graph from the 1998 Mann et al. paper is often cited by news, and likely often misused in ways, but from all that I can tell it is supported by other researchers and consistent with other findings.  Note that much of the basics have been fairly consistent with earlier reconstructions.  The 1990 IPCC assessment and the 1998 Mann et al. both have elevated temperatures in the period 1100 - 1300, followed by decreasing temperatures after that, followed by a sharp increase leading up to the present.  The 1998 was a significant change in the relative size of these, but the general trends had been clear for a while.  

Quote from: jhkimOpening data is a tricky thing. In general, I think research should be done by specialists in the field. I love wikipedia and the blogosphere, but I don't think that a horde of politically-motivated amateurs combing over the data is going to improve our understanding of any science. Relative to physics, I particularly get annoyed at a lot of amateur claims regarding nuclear physics -- which itself has many political implications. I would rather that we fund research to get more specialized experts working on the problem.

Quote from: John MorrowThe hoard of politically-motivated amateurs are going to speculate, anyway, and the absence of data simply feeds conspiracy theories and suspicion.  When people hid things, it gives people the impression that they have something to hide.

Well, these two statements are not in contradiction to each other.  I do not think that a horde of politically-motivated amateurs will do anything to advance our understanding of climate.  On the other hand, it is possible that releasing more raw data to them might change people's impressions as a public relations move.  

Quote from: John MorrowWell, isn't one solution to demand transparency in peer reviewed publications and public policy-related research?  Steve McIntyre's point about due diligence is that in commercial research, government regulations require transparency through reporting and record keeping so that it's not optional.  Yes, given the choice, many scientists would prefer to keep their data hidden and to hide their mistakes, but wouldn't it encourage them to be more rigorous if they feared having their errors exposed?  Doesn't this suggest that maybe they shouldn't be given the option of concealing their research?  Is the point to help researchers be respected whether they deserve that respect or not or to do good science?

Heh.  The conservative argues that more government regulation is needed for things to be handled effectively?  :-)  

In short, no, I don't think so.  My observation on this point is mostly anecdotal, though.  I've seen medicine become increasingly regulated, and on the whole I don't think this has improved things.  Making laws to force scientists to publicize their data in an amateur-readable form would be a major burden on how research is conducted -- and it seems like the gains are minimal.  Yes, it would motivate people to take more care in later analysis steps, but that isn't the whole of research.  Amateurs combing through one's data might correct a few mistakes, but I suspect that far more often they would be making noise that just muddles things.

ADDED: I think the regulations on companies are reasonable to prevent very clear bias.  i.e. A company wants to sell a drug, then they should be carefully regulated in how they test it.  They're also likely dealing with limited cases -- i.e. no one else can test a proprietary drug.

Quote from: jhkim;225677Heh.  The conservative argues that more government regulation is needed for things to be handled effectively?  :-)

You are arguing with an anti-intellectual who thinks that several courses in Medieval History makes one a peer with Ph.Ds.  Someone who whines about appeals to authourity while continually pointing to the same website again and again for rebuttal.  Goes on and on about 'agendas', while doing everything possible to impugn environmentalism in even the slightest manner to make it look like the entire body of science is corrupt.

In short, you will get arguments that will take whatever tack neccessary to accomplish the task of discrediting 'environmentalism'.

Quote from: StormBringer;225755In short, you will get arguments that will take whatever tack neccessary to accomplish the task of discrediting 'environmentalism'.

The trouble with those on the right is that they simply don't see that as the same sort of thing because they are convinced that their fears, biases, and hatreds are justified and correct while those of others are foolish and incorrect.

Quote from: One Horse Town;225824The trouble with those on the right is that they simply don't see that as the same sort of thing because they are convinced that their fears, biases, and hatreds are justified and correct while those of others are foolish and incorrect.

OHT,

Have I told you lately that I love you?

Jim

Quote from: James J Skach;225831OHT,

Have I told you lately that I love you?

Jim

Only in numerous PMs! Cut it out, already! :teehee:

Quote from: One Horse Town;225841Only in numerous PMs! Cut it out, already! :teehee:

That was, like, five hours ago. I don't care what the courts say, that's not stalker territory...

Quote from: One Horse Town;225824The trouble with those on the right is that they simply don't see that as the same sort of thing because they are convinced that their fears, biases, and hatreds are justified and correct while those of others are foolish and incorrect.

So, you really believe that people believing that their own fears, biases, and hatreds are justified and correct while believing that the fears, biases, and hatreds of others are incorrect and quite possibly foolish is a problem unique to those on the right?

Quote from: One Horse Town;225824The trouble with those on the right is that they simply don't see that as the same sort of thing because they are convinced that their fears, biases, and hatreds are justified and correct while those of others are foolish and incorrect.

I see what you did there.

Quote from: John Morrow;225880So, you really believe that people believing that their own fears, biases, and hatreds are justified and correct while believing that the fears, biases, and hatreds of others are incorrect and quite possibly foolish is a problem unique to those on the right?

Nope. They're your words, John. Except "left" replaced with "right" and "you" substituted with "they". Speaking of PC and the UK, post 146.

Ka-zing! :eek:

!i!

John Morrow, you are getting close to having a single post take up an entire page!

For the record (it was buried somewhere back there), yes, the current Canadian government is totally in the pocket of big oil.  Soon we'll be the only country who doesn't ratify Kyoto.

Quote from: StormBringer;225755You are arguing with an anti-intellectual who thinks that several courses in Medieval History makes one a peer with Ph.Ds.

No.  I'm simply someone who knows that a PhD isn't a writ of infallibility and that science isn't magic that's impossible for a layman to understand.  I also find it curious that someone who I suspect would cheer Carl Sagan's call for skepticism and rigorous science when applied to Creationists or Astrologers complains when I suggest applying the same criteria to Global Warming.  Does that mean that science can't withstand it's own rigors?

Quote from: StormBringer;225755Someone who whines about appeals to authourity while continually pointing to the same website again and again for rebuttal.

I'm not telling you that you should trust Steve McIntyre because of his academic degrees.  I'm telling you that you should read his arguments and assess their validity on the basis of the case he makes and the evidence he presents (and by all means read the RealClimate.org side, too).  You know, look at the details.  I guess it's easier to ignore the actual arguments and issues and attack the people making them, instead.  

Quote from: StormBringer;225755Goes on and on about 'agendas', while doing everything possible to impugn environmentalism in even the slightest manner to make it look like the entire body of science is corrupt.

I'm not trying to impugn all environmentalism, just the kooks and rash actions taken without careful consideration of the consequences.  I don't object to OSHA, shutting down major hazardous materials polluters, catalytic converters, emissions controls to reduce acid rain, the phasing out of CFCs, and so on.  But go ahead and keep shouting the end of the world is nigh if we don't act now and don't be surprised if people stat to ignore you the same way they ignore the kooks wearing the sandwich boards that say, "The end of the world is nigh!"

The story of the little boy who cried, "Wolf!" is either dead or people no longer understand the point of it.

Quote from: StormBringer;225755In short, you will get arguments that will take whatever tack neccessary to accomplish the task of discrediting 'environmentalism'.

So environmentalism is above any criticism, then?

Quote from: One Horse Town;225889Nope. They're your words, John. Except "left" replaced with "right" and "you" substituted with "they". Speaking of PC and the UK, post 146.

Your editing is more substantial than a two word change, but I'm sure you know that.  You removed the first sentence which acknowledges that the same biases exist on both sides.  Second, you didn't simply switch two words but rewrote the entire beginning, changing the scope from essentially being some people on the left (and you in particular -- thanks for again proving my point) to all people on the right.  My reply here expressed the same sentiment that the original quote did, much as your alteration of my quote expressed the same sentiment that I was replying to with the original quote.  Surprise!  You got me to say the same thing twice!

(Yes, I know there is a real point you could have made if you weren't trying to be so clever.)