Why “I don’t know” doesn’t read as macho

Great post from Cathy about the need to be able to assert uncertainty in a, well, assertive way.  Why is this so hard?  Why do we find people trustworthy when they say, with complete confidence, “Here’s your answer?”

One reason must be that in some contexts, confidence really does correlate with knowledge; people who truly know nothing about a subject are (hopefully) more willing to express uncertainty about it.  So when we hear someone answer a question with “I’m not sure,” we have to carry out some inferential computation:  do we think they’re saying that because they’ve never thought about the question, or because they’ve thought enough about the question to understand that it’s actually difficult?

I don’t have an answer to this conundrum, but I do have an extremely scientific infographic that I hope will illustrate the issue.

 

Guest post: Stephanie Tai on deference to experts

My colleague Steph Tai at the law school wrote a long, amazing Facebook message to me about the question Cathy and I have been pawing at:  when and in what spirit should we be listening to experts?  It was too good to be limited to Facebook, so, with her permission, I’m reprinting it below.

Steph deals with these issues because her academic specialty is the legal status of scientific knowledge and scientific evidence.  So yes:  in a discussion on whether we should listen to experts I am asking you to listen to the opinions of an expert on expertise.

Also, Steph very modestly doesn’t link to her own paper on this stuff until the very bottom of this post.  I know you guys don’t always read to the bottom, so I’ve got your link to “Comparing Approaches Toward Governing Scientific Advisory Bodies on Food Safety in the United States and the European Union” right here!

And now, Steph:

*****

Some quick thoughts on this very interesting exchange. What might be helpful, to take everyone out of our own political contexts, perhaps, is to contrast this discussion you’re both having regarding experts and financial models with discussions about experts and climate models, where, it seems, the political dynamics are fairly opposite. There, you have people on the far right making similar claims to Cathy: that climate scientists are to be distrusted because they’re just coming up with scare models because these allegedly biased models are useful to those climate scientists–i.e., to bring money to left-wing causes, to generate grants for more research, etc.

 

So when you apply the claim that Cathy makes at the end of her post–”If you see someone using a model to make predictions that directly benefit them or lose them money – like a day trader, or a chess player, or someone who literally places a bet on an outcome (unless they place another hidden bet on the opposite outcome) – then you can be sure they are optimizing their model for accuracy as best they can. . . . But if you are witnessing someone creating a model which predicts outcomes that are irrelevant to their immediate bottom-line, then you might want to look into the model yourself.”–I’m not sure you can totally put climate scientists in that former category (of those that directly benefit from the accuracy of their predictions). This is due to the nature of most climate work: most researchers in the area only contribute to one tiny part of the models, rather than produce the entire model themselves (thus, the incentives to avoid inaccuracies are diffuse rather than direct); the “test time” for the models are often relatively far into the future (again, making the incentives more indirect); and the sorts of diffuse reputational gains that an individual climate scientist gets from being part of a team that might partly contribute to an accurate climate model is far less direct than the examples given of day traders and chess players or “someone who literally places a bet on an outcome.”

 

What that in turn seems to mean is that under Cathy’s approach, climate scientists would be viewed as in the latter category—those creating models that “predict outcomes that are irrelevant to their immediate bottom-line,” and thus deserve people looking “into the model [themselves].” But at least from what I’ve seen, there is *so* much out there in terms of inaccurate and misleading information about climate models (by folks with stakes in the *perception* of those models) that chances are, a lay person’s inquiry into climate models has high chance to being shaped by similar forces with which Cathy is (in my view appropriately) concerned. Which in turn makes me concerned about applying this approach.
Disclaimer: I used to fall under this larger umbrella of climate scientists, though I didn’t work on the climate models themselves, just one small input to them—the global warming potentials of chlorofluorocarbon substitutes. So this contrast is not entirely unemotional for me. That said, now that I’m an academic who studies the *use* of science in legal decisionmaking (and no longer really an academic who studies the impact of chlorofluorocarbon substitutes on climate), I don’t want to be driven by these past personal ties, but they’re still there, so I feel like I should lay them out.

 

So what’s to be done? I absolutely agree with Cathy’s statement that “when independent people like myself step up to denounce a given statement or theory, it’s not clear to the public who is the expert and who isn’t.” It would seem, from what she says at the end of her essay, that her answer to this “expertise ambiguity” is to get people to look into the model when expertise is unclear.[*] But that in turn raises a whole bunch of questions:

 

(1) What does it take to “look into the model yourself”? That is, how much understanding does it take? Some sociologists of science suggest that translational “experts”–that is, “experts” who aren’t necessarily producing new information and research, but instead are “expert” enough to communicate stuff to those not trained in the area–can help bridge this divide without requiring everyone to become “experts” themselves. But that can also raise the question of whether these translational experts have hidden agendas in some way. Moreover, one can also raise questions of whether a partial understanding of the model might in some instances be more misleading than not looking into the model at all–examples of that could be the various challenges to evolution based on fairly minor examples that when fully contextualized seem minor but may pop out to someone who is doing a less systematic inquiry.

 

(2) How does a layperson avoid, in attempting to understand the underlying model, the same manipulations by those with financial stakes in the matter–the same stakes that Cathy recognizes might shape the model itself? Because that happens as well, so that even if one were to try to look into a model themselves, the educational materials it would take to look into that model can be also argued to be developed by those with stakes in the matter. (I think Cathy sort of raises this in a subsequent post about how entire subfields can be regarded as “captured” by particular interests.)

 

(3) (and to me this is one of the most important questions) Given the high degree of training it takes to understand any of these individual areas of expertise, and given that we encounter so many areas in which this sort of deeper understanding is needed to resolve policy questions, how can any individual actually apply that initial exhortation–to look into the model yourself–in every instance where expertise ambiguity is raised? To me that’s one of the most compelling arguments in favor of deferring to experts to some extent–that lay people (as citizens, as judges, as whatever) simply don’t have time to do the kind of thing that Cathy suggests in every situation where she argues it’s called for. Expert reliance isn’t perfect, sure–but it’s a potentially pragmatic response to an imperfect world with limited time and resources.

 

Do my thoughts on (3) mean that I think we should blindly defer to experts? Absolutely not. I’m just pointing it out as something that weighs in favor of listening to experts a little more. But that also doesn’t mean that the concerns Cathy raises are unwarranted. My friend Wendy Wagner writes about this in her papers on the production of FDA reports and toxic materials testing, and I find her inquiries quite compelling. P.s. I should also plug a work of hers that seems especially relevant to this conversation. It suggests that the part of Nate Silver’s book that might raise the most concerns (I dunno, because I haven’t read it) is its potential contribution to the vision of models as “truth machines,” rather than understanding that models are just one tool to aid in making decisions, and a tool which must be contextualized (for bias, for meaningfulness, for uncertainty) at that.

 

So how to address this balance between skepticism and lack of time to do full inquiries into everything? I totally don’t have the answers, though the kind of stuff I explore are procedural ways to address these issues, at least when legal decisions are raised–for example,
* public participation processes (with questions as to both the timing and scope of those processes, the ability and likelihood that these processes are even used, the accessibility of these processes, the susceptibility of “abuse,” the weight of those processes in ultimate decisionmaking)
* scientific ombudsman mechanisms (which questions of how ombudsman are to be selected, the resources they can use to work with citizen groups, the training of such ombudsmen)
* the formation of independent advisory committees (with questions of the selection of committee members, conflict of interest provisions, the authority accorded to such committees)
* legal case law requiring certain decisionmaking heuristics in the face of scientific uncertainty to avoid too much susceptibility to data manipulation (with questions of the incentives those heuristics create for actual potential funders of scientific research, the ability of judges to apply such heuristics in a consistent manner)
–as well as legal requirements that exacerbate these problems. Anyway, thanks for an interesting back and forth!

 

[*] I’m not getting into the question of “what makes someone an expert?” here, and instead focus on “how do we make decisions given the ambiguousness of who should be considered experts?” because that’s more relevant to what I study, although I should also point out that philosophers and sociologists of science have been studying this in what’s starting to be called the “third wave” of science, technology, and society studies. There’s a lot of debate about this, and I have a teensy summary of it here (since Jordan says it’s okay for me to plug myself :) (Note: the EFSA advisory committee structure, if anyone cares, has changed since I published this article so that the article characterizations are no longer accurate.)

 

 

 

Distrusters of experts all around

The Wall Street Journal op-ed page is always good for a full-throated demand that we distrust the experts:

The general public is not privy to the IPCC debate. But I have been speaking to somebody who understands the issues: Nic Lewis. A semiretired successful financier from Bath, England, with a strong mathematics and physics background, Mr. Lewis has made significant contributions to the subject of climate change.

…Will the lead authors of the relevant chapter of the forthcoming IPCC scientific report acknowledge that the best observational evidence no longer supports the IPCC’s existing 2°-4.5°C “likely” range for climate sensitivity? Unfortunately, this seems unlikely—given the organization’s record of replacing evidence-based policy-making with policy-based evidence-making, as well as the reluctance of academic scientists to accept that what they have been maintaining for many years is wrong.

Domain knowledge, phooey — this dude is successful!

“Distrust the experts,” as a principle, does as much harm as good.  A better principle would be “Distrust people who are bad and trust people who are not bad.”  Of course, it can be hard to tell the difference — but that distinction is one we have to make anyway, in all kinds of contexts, so why not this one?

 

In defense of Nate Silver and experts

Cathy goes off on Nate Silver today, calling naive his account of well-meaning people saying false things because they’ve made math mistakes.  In Cathy’s view, people say false things because they’re not well-meaning and are trying to screw you — or, sometimes, because they’re well-meaning but their incentives are pointed at something other than accuracy.  Read the whole thing, it’s more complicated than this paraphrase suggests.

Cathy, a fan of and participant in mass movements, takes special exception to Silver saying:

This is neither the time nor the place for mass movements — this is the time for expert opinion. Once the experts (and I’m not one of them) have reached some kind of a consensus about what the best course of action is (and they haven’t yet), then figure out who is impeding that action for political or other disingenuous reasons and tackle them — do whatever you can to remove them from the playing field. But we’re not at that stage yet.

Cathy’s take:

…I have less faith in the experts than Nate Silver: I don’t want to trust the very people who got us into this mess, while benefitting from it, to also be in charge of cleaning it up. And, being part of the Occupy movement, I obviously think that this is the time for mass movements.

From my experience working first in finance at the hedge fund D.E. Shaw during the credit crisis and afterwards at the risk firm Riskmetrics, and my subsequent experience working in the internet advertising space (a wild west of unregulated personal information warehousing and sales) my conclusion is simple: Distrust the experts.

I think Cathy’s distrust is warranted, but I think Silver shares it.  The central concern of his chapter on weather prediction is the vast difference in accuracy between federal hurricane forecasters, whose only job is to get the hurricane track right, and TV meteorologists, whose very different incentive structure leads them to get the weather wrong on purpose.  He’s just as hard on political pundits and their terrible, terrible predictions, which are designed to be interesting, not correct.

Cathy wishes Silver would put more weight on this stuff, and she may be right, but it’s not fair to paint him as a naif who doesn’t know there’s more to life than math.  (For my full take on Silver’s book, see my review in the Globe.)

As for experts:  I think in many or even most cases deferring to people with extensive domain knowledge is a pretty good default.  Maybe this comes from seeing so many preprints by mathematicians, physicists, and economists flushed with confidence that they can do biology, sociology, and literary study (!) better than the biologists, sociologists, or scholars of literature.  Domain knowledge matters.  Marilyn vos Savant’s opinion about Wiles’s proof of Fermat doesn’t matter.

But what do you do with cases like finance, where the only people with deep domain knowledge are the ones whose incentive structure is socially suboptimal?  (Cathy would use saltier language here.)  I guess you have to count on mavericks like Cathy, who’ve developed the domain knowledge by working in the financial industry, but who are now separated from the incentives that bind the insiders.

But why do I trust what Cathy says about finance?

Because she’s an expert.

Is Cathy OK with this?

Startup culture, VC culture, and Mazurblogging

Those of us outside Silicon Valley tend to think of it as a single entity — but venture capitalists and developers are not the same people and don’t have the same goals.  I learned about this from David Carlton’s blog post.  Cathy O’Neil reposted it this morning.  It’s kind of cool that the three of us, who started grad school together and worked with Barry Mazur, are all actively blogging!  We just need to get Matt Emerton in on it and then we’ll have the complete set.  Maybe we could even launch a new blogging platform and call it mazr.  You want startup culture, I’ll give you startup culture!

 

Cathy O’Neil is killing it

Some great recent posts from Mathbabe, the funniest and pissed-offiest “math, the universe, and everything” blog on the tubes:

• Is Big Data Evil?
• Math in Business — great breakdown of the pros and cons of different career options for young mathematicians
• I never sit on the subway.

Math is like Earthball, not like ARML

Cathy’s post touched off a lot of discussion of math contests, and whether they do or do not, in her formulation, suck.  My thoughts on this are pretty simple.

Big good thing about math contests:  They reveal that math is more than what’s taught in school, and that there’s a whole community of kids around the world who are passionate about math.

Big bad thing about math contests:  They help promote the idea that the most important thing about math is whether you’re the best at it.

Of course you can design your contest to provide more of the big good and less of the big bad.  The question isn’t so much whether the good minus the bad is positive; it’s whether there are other ways of getting at the good that avoid the bad.  I think programs like Hampshire and MathCamp and PROMYS and Ross are like this.  But they clearly don’t scale to the size of an AMC.  Mary O’Keeffe‘s comments on Cathy’s blog were particularly interesting, since they give a good sense of what math contests are like in 2011 for those of us whose direct experience is substantially less recent.

Some people, I think, don’t think my big bad is so bad.  I disagree.  Somebody on my Facebook feed  recently linked to this letter from algebraist Donald Weidman to Science, headed “Emotional Perils of Mathematics.”  Weidman numbers among these perils the following frustration:

“The history of mathematics makes plain that all the general outlines and most of the major results have been obtained by a few geniuses who are not the ordinary run of mathematicians.  These few big men make the long strides forward, then the lesser lights come scurrying in to fill in the chinks, make generalizations, and find some new applications; meanwhile the giants are making further strides.”

This is so profoundly wrong it makes my teeth hurt.  Mathematics is like Earthball.  The weight of our ignorance is tremendous and all of us push together to move it a bit to one side over the course of our lifetimes.  We may vary in strength but what we have in common is that there’s very little we can do alone.

Cathy O’Neil blogs at Mathbabe

Cathy O’Neil is blogging!  Cathy was my classmate in graduate school — we’d meet once a week and she’d drag me through Milne’s survey on abelian varieties, which I was optimistically trying to read without knowing what a scheme was.  A theorem of hers appeared uncredited (sorry, Cathy) in this post, where it guarantees the maximal isotropy of the global cohomology group with respect to the quadratic form studied by Poonen and Rains.  Lately, Cathy’s been in the private sector, working in both the financial industry and the Internet economy.  If you know her, you’d probably guess that her blog is big on strongly held opinions and light on pulled punches.  Your guess would be right!  Here’s Cathy on working at a hedge fund:

Most of the quants at D.E. Shaw were immigrant men.  In fact I was the only woman quant when I joined, and there were quite a few quants, maybe 50, and I was also one of the only Americans.  What nearly all these men had in common was a kind of constant, nervous hunger, almost like a daily fear that they wouldn’t have enough to eat.  At first I thought of them as having a serious chip on their shoulder, like they were the kind of guy that didn’t make the football team in high school and were still trying to get over that.  And I still think there’s an element of something as simple as that, but it goes deeper.  One of my colleagues from Eastern Europe said to me once, “Cathy, my grandparents were coal miners.  I don’t want my kids to be coal miners.  I don’t want my grandchildren to be coal miners.  I don’t want anybody in my family to ever be a coal miner again.”   So, what, you’re going to amass enough money so that no descendent of yours ever needs to get a job?  Something like that.

But here’s the thing, that fear was real to him.  It was that earnest, heartfelt anxiety that convinced me that I was really different from these guys.  The difference was that, firstly, they were acting as if a famine was imminent, and they’d need to scrounge up food or starve to death, and secondly, that only their nuclear family was worth saving.  This is where I really lost them.  I mean, I get the idea of acts of desperation to survive, but I don’t get how you choose who to save and who to let die.  However, it was this kind of us-against-them mentality that prevailed and informed the approach to making money.

Once you understand the mentality, it’s easier to understand the “dumb money” phrase.  It simply means, we are smarter than those idiots, let’s use our intelligence to anticipate dumb peoples’ trades and take their money.  It is our right as intelligent, imminently starving people to do this.

I also like yesterday’s post, where Cathy speculates about people with happy childhoods and people with unhappy childhoods, and asks whether marriages should contain one of each.

Oh yeah, and for the people who don’t like when I post about women in math, try reading Cathy’s blog — you’ll hate it!  Here’s her inspirational speech for women in math:

Hi, I’m your unemployed role model.  I thought of not coming here today since, after all, I’m unemployed, and what kind of role model does that make me?  Actually it makes me a good one, and here’s why.  That job I left wasn’t good enough for me.  I didn’t get fired, I quit (although plenty of great people I know have been laid off so that’s no proof of anything).  The truth is, I deserve a job that I really like, where I’m challenged to grow and to learn and to do my best and I’m rewarded for doing so.  After all, I have a super power, which is mathematics.  So the reason I’m saying this is that you do too.  All of you have a superpower, which is mathematics.  You all deserve to work at good jobs that you actually enjoy- and if the jobs you have turn out to be bad, or if the become bad for some reason, then quit!  Get another one!  Get a better one!  I actually got a job offer on the plane over here yesterday (true!).  I know I’m going to get a good job, even in this economy, because I can do something other people actually regard as magical.  Mathematical training and thinking is something that everybody needs and not everybody can achieve, so remember that.  Never feel stuck.  This is not to say that the specific training you have right now is sellable on the open market, but since you’re a mathematician the one thing you can count on being good at is learning new stuff.  So if you decide to change fields, get ready to roll up your sleeves and work your butt off to learn the necessary stuff, but be sure that you can do it and that it will be really important to the people you work for.  And if it isn’t, or if you don’t think your work is being appreciated, go get a better job.  Thanks!