What correlation means

From Maria Konnikova’s New Yorker piece on Randall Munroe and what makes science interesting:

In a meta-analysis of sixty-six studies tracking interests over time (the average study followed subjects for seven years), psychologists from the University of Illinois at Urbana–Champaign found that our interests in adolescence had only a point-five correlation with our interests later in life. This means that if a subject filled out a questionnaire about her interests at the age of, say, thirteen, and again at the age of twenty-one, only half of her answers remained consistent on both.

I think it’s totally OK to not say precisely what correlation means.  It’s sort of subtle!  It would be fine to say the correlation was “moderate,” or something like that.

But I don’t think it’s OK to say “This means that…” and then say something which isn’t what it means.  If the questionnaire was a series of yes-or-no questions, and if exactly half the answers stayed the same between age 13 and 21, the correlation would be zero.  As it should be — 50% agreement is what you’d expect if the two questionnaires had nothing to do with each other.  If the questionnaire was of a different kind, say, “rate your interest in the following subjects on a scale of 1 to 5,” then agreement on 50% of the answers would be more suggestive of a positive relationship; but it wouldn’t in any sense be the same thing as 0.5 correlation.  What does the number 0.5 add to the meaning of the piece?  What does the explanation add?  I think nothing, and I think both should have been taken out.

Credit, though — the piece does include a link to the original study, a practice that is sadly not universal!  But demerit — the piece is behind a paywall, leaving most readers just as unable as before to figure out what the study actually measured.  If you’re a journal, is the cost of depaywalling one article really so great that it’s worth forgoing thousands of New Yorker readers actually looking at your science?

 

 

 

Tagged ,

3 thoughts on “What correlation means

  1. Ruth Ann says:

    I thought the New Yorker was famous for their rigorous fact-checking. Does a statement about math not matter by their standards? I’m always bothered by things that demonstrate our society’s attitude that it’s ok not to understand math. We don’t think it’s ok to be illiterate, but innumeracy is expected! That attitude has caused (and is causing) lots of problems for us.

  2. JSE says:

    They are indeed famous for rigorous fact-checking. But I’m not sure what they do with mathematical or otherwise technical assertions that no one on the staff has the expertise to assess directly.

  3. David Bryant says:

    “… I’m not sure what they do with mathematical or otherwise technical assertions …”

    Five’ll get you twenty: they just ignore them. Why waste time checking the accuracy of a statement that nobody understands?

    I have a hard time understanding exactly what”our interests in adolescence had only a point-five correlation with our interests later in life” even means. I guess we’re talking about two variables: “interests” at say age 14 and “interests” years later, say at age 21. But that’s about as far as I can get. I don’t even see how one would go about quantifying “interests” in any sort of meaningful or reliable way. I suppose psychologists perform magical stunts like that all the time. Maybe that’s why I’ve got such a low opinion of psychology.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: