Category Archives: academia

Ranking mathematicians by hinge loss

As I mentioned, I’m reading Ph.D. admission files.  Each file is read by two committee members and thus each file has two numerical scores.

How to put all this information together into a preliminary ranking?

The traditional way is to assign to each applicant their mean score.  But there’s a problem: different raters have different scales.  My 7 might be your 5.

You could just normalize the scores by subtracting that rater’s overall mean.  But that’s problematic too.  What if one rater actually happens to have looked at stronger files?  Or even if not:  what if the relation between rater A’s scale and rater B’s scale isn’t linear?  Maybe, for instance, rater A gives everyone she doesn’t think should get in a 0, while rater A uses a range of low scores to express the same opinion, depending on just how unsuitable the candidate seems.

Here’s what I did last year.  If (r,a,a’) is a triple with r is a rater and a and a’ are two applicants, such that r rated a higher than a’, you can think of that as a judgment that a is more admittable than a’.  And you can put all those judgments from all the raters in a big bag, and then see if you can find a ranking of the applicants (or, if you like, a real-valued function f on the applicants) such that, for every judgment a > a’, we have f(a) > f(a’).

Of course, this might not be possible — two raters might disagree!  Or there might be more complicated incompatibilities generated by multiple raters.  Still, you can ask:  what if I tried to minimize the number of “mistakes”, i.e. the number of judgments in your bag that your choice of ranking contradicts?

Well, you can ask that, but you may not get an answer, because that’s a highly non-convex minimization problem, and is as far as we know completely intractable.

But here’s a way out, or at least a way part of the way out — we can use a convex relaxation.  Set it up this way.  Let V be the space of real-valued functions on applicants.  For each judgment j, let mistake_j(f) be the step function

mistake_j(f) = 1 if f(a) < f(a’) + 1

mistake_j(f) = 0 if f(a) >= f(a’) + 1

Then “minimize total number of mistakes” is the problem of minimizing

M = sum_j mistake_j(f)

over V.  And M is terribly nonconvex.  If you try to gradient-descend (e.g. start with a random ranking and then switch two adjacent applicants whenever doing so reduces the total number of mistakes) you are likely to get caught in a local minimum that’s far from optimal.  (Or at least that can happen; whether this typically actually happens in practice, I haven’t checked!)

So here’s the move:  replace mistake_j(f) with a function that’s “close enough,” but is convex.  It acts as a sort of tractable proxy for the optimization you’re actually after.  The customary choice here is the hinge loss:

hinge_j(f) = min(0, f(a)-f(a’) -1).

Then H := sum_j hinge_j(f) is a convex function on f, which you can easily minimize in Matlab or python.  If you can actually find an f with H(f) = 0, you’ve found a ranking which agrees with every judgment in your bag.  Usually you can’t, but that’s OK!  You’ve very quickly found a function H which does a decent job aggregating the committee scores. and which you can use as your starting point.

Now here’s a paper by Nihal Shah and Martin Wainwright commenter Dustin Mixon linked in my last ranking post.  It suggests doing something much simpler:  using a linear function as a proxy for mistake_j.  What this amounts to is:  score each applicant by the number of times they were placed above another applicant.  Should I be doing this instead?  My first instinct is no.  It looks like Shah and Wainwright assume that each pair of applicants is equally likely to be compared; I think I don’t want to assume that, and I think (but correct me if I’m wrong!) the optimality they get may not be robust to that?

Anyway, all thoughts on this question — or suggestions as to something totally different I could be doing — welcome, of course.

 

 

 

Tagged , , ,

Ranking mathematicians

I’m on the hiring committee, I chair the graduate admissions committee, and I’m doing an NSF panel, so basically I’ll be spending much of this month judging and ranking people’s mathematics.  There’s a lot I like about these jobs:  it’s a very efficient way to get a panorama of what’s going on in math and what people think about it.  The actual ranking part I don’t like that much — especially because the nature of hiring, admissions, and grant-making means you’re inevitably putting tons of very worthwhile stuff below the line.  I feel like a researcher when I read the proposals, like a bureaucrat when I put scores on them.

But of course the bureaucratic work needs to be done.  I’d go so far as to say — if mathematicians aren’t willing to rank each other, others will rank us, and that would be worse.

Tagged , , ,

Doctoral programs can have a strong influence on the weak-minded

Daniel Drezner:

First, I cannot stress enough the cult-like powers of a PhD program. Doctoral programs can have a strong influence on the weak-minded. Even if you’re pretty sure what you want going into a program, that can change as you’re surrounded by peers who want something different. You might think you’re strong-willed, but day after day of hearing how a top-tier research university position is the be-all, end-all of life can have strange effects on your psyche.

I really do feel this is something we handle well at Wisconsin.  Our Ph.D. graduates go on to a wide variety of positions, some in primarily teaching colleges, some in research institutions, some in industry, some in government.  We do not consider the North American research university the be-all and end-all of life.  We are not just trying to produce clones of ourselves.  We really do strive to help each of our students get the best job among the jobs they want to get.  

Tagged ,

Math Bracket 2015

March Math Madness is here!  Presenting the 2015 math bracket, as usual prepared by our crack team of handicappers here at the UW math department.  As always, remember that the math bracket is for entertainment purposes only and you should not take offense if the group rated your department lower than the plainly inferior department that knocked you out.  Under no circumstances should you use the math bracket to decide where to go to grad school.

Math Bracket 2015-page-0Lots of tough choices this year!

 

Tagged , ,

Are UW-Madison professors underpaid?

It’s well known that UW-Madison salaries are notably lower than those at peer institutions, at every level of seniority.  But wait, says Chris Rickert in the Wisconsin State Journal, that doesn’t necessarily mean our faculty is underpaid!

At UW-Madison, assistants are paid, on average, about $82,000 a year, associates about $93,000 and full professors about $123,000 — ranking them 10th, seventh and 12th, respectively, in salary compared to 11 other state-identified peer institutions, according to data from the university’s Academic Planning and Institutional Research office.

Obviously, more full professors means more people in line for full-professor salaries and greater pressure on the budget for professorial salaries overall. At UW-Madison, that’s no small detail, as about 59 percent of UW-Madison professors have attained full status, according to the university’s Data Digest.

By contrast, figures from the American Association of University Professors show that, on average, only about 31.5 percent profs at all universities and about 30.8 percent at public universities are full professors.

This is a really good point!  You could imagine that maybe our pay isn’t underscale at all — maybe we just promote people faster, so that our full professors are less senior and thus make less.  That’s Rickert’s take:

I’m left to wonder whether the university has adopted that old human resources trick of placating employees by inflating their titles more than their pay.

In an era of declining state support, this would help keep a lid on the cost of higher education while simultaneously allowing university officials to complain about how poorly paid are its best and brightest.

But this can actually be checked!  You can use the Chronicle of Higher Education Faculty Salary Survey to get the mean salary for any university at any seniority level, and the number of faculty members at each seniority level, and compute the overall faculty mean that way.  I did this for a few of our peer institutions and got:

UIUC UW Iowa OSU
full $145 816 $123 755 $135 494 $139 943
assoc $96 556 $93 252 $90 407 $94 763
asst $90 405 $82 363 $77 329 $85 502
$117,133 $106,618 $104,595 $111,172

So the mean UW tenure-track gets paid slightly more than people at Iowa, but notably less than counterparts at Ohio State and Illinois.

Tagged

Full professors make more money than bus drivers

Former Republican Congressional candidate and current UW-Madison history professor John Sharpless stands up for us against the Governor:

He said he arrives no later than 9 a.m. and leaves no earlier than 5 p.m. During that time, he said he’s either teaching, preparing lectures, doing research, attending required committee meetings, advising students and managing teaching assistants. Sharpless added that he often spends his evenings reading and grading papers.

“None of this seems like work to a guy like Walker because he lives a different life,” he said. “And I’m not going to make fun of what he does. I’m sure being a governor is a lot of work. He has to spend a lot of time in Iowa and South Carolina and North Carolina and courting other Republican big-wigs. That taxes the man horribly.”

But just to make it clear he’s still on board with GOP, he drops this in:

“I will retire with a salary that’s less than a Madison bus driver,” he said.

UW-Madison salaries are public records, so I can tell you that Sharpless’s is just under $80,000.  In 2012, only 9 employees of Metro made more than $70K.  And the ones who made that much, I’m pretty sure, are the ones who worked tons of overtime.

In other words, what Sharpless said is likely true in the strict sense of

“There exists a Madison bus driver whose salary this year exceeds mine”

but gives the wrong impression about typical full professors in the history department and typical bus drivers.

Tagged , ,

What a coincidence, I specialize in contemporary research!

In this morning’s inbox:

American International Journal of Contemporary Research (AIJCR) is an open access, peer-reviewed and refereed multidisciplinary journal published by Center for Promoting Ideas (CPI), USA. The main objective of AIJCR is to provide an intellectual platform for the research community. AIJCR aims to promote contemporary research in business, humanities, social science, science and technology and become the leading journal in the world.

I wonder what areas of research are appropriate?

The journal publishes research papers in three broad specific fields as follows:

Business and Economics
Management, marketing, finance, economics, banking, accounting, human resources management, international business, hotel and tourism, entrepreneurship development, business ethics, development studies and so on.

Humanities and Social science
Anthropology, communication studies, corporate governance, criminology, cross-cultural studies, demography, education, ethics, geography, history, industrial relations, information science, international relations, law, linguistics, library science, media studies, methodology, philosophy, political science, population Studies, psychology, public administration, sociology, social welfare, linguistics, literature, paralegal, performing arts (music, theatre & dance), religious studies, visual arts, women studies.

Science and Technology
Astronomy and astrophysics, Chemistry, Earth and atmospheric sciences, Physics, Biology in general, Agriculture, Biophysics and biochemistry, Botany, Environmental Science, Forestry, Genetics, Horticulture, Husbandry, Neuroscience, Zoology, Computer science, Engineering, Robotics and Automation, Materials science, Mathematics, Mechanics, Statistics, Health Care & Public Health, Nutrition and Food Science, Pharmaceutical Sciences, and so on.

I don’t know which I like best, “broad specific fields” or the “and so on.”  Or maybe “women studies.”  Or maybe that they list “linguistics” twice.  I can’t choose, it’s all so spamalicious!

Tagged

Why aren’t math professors sociopaths?

Great open from Chris Hayes:

Imagine you’re a scientist in some sci-fi alternate universe, and you’ve been charged with creating a boot camp that will reliably turn normal but ambitious people into broken sociopaths more or less willing to do anything.

There are two main traits you’d want to cultivate in your recruits. The first would be terror: You’d want to ensure that the experimental subjects were kept off-­balance and insecure, always fearful that bad things would happen, that they would be humiliated or lose their position and be cast out. But at the same time, it would be crucial that you assiduously inculcate a towering sense of superiority, the belief that the project they happen to be engaged in is more important than anything and that, because of their remarkable skills and efforts, they are among the select few chosen to be a part of it. You’d want to simultaneously make them neurotically insecure and self-doubting and also filled with the conviction that they and their colleagues are smarter and better and more deserving than anyone else.

He’s writing about young investment bankers, whose lives, such as they are, are described in Kevin Roose’s new book “Young Money.”  But doesn’t this boot camp actually describe the Ph.D. experience pretty well?  And if so, why aren’t math professors sociopaths?

I can think of one reason:  in finance, the thing you are trying to do is screw over somebody else.  If you win, someone has lost.  Mathematics is different.  We’re all pushing together.  Not that there’s no competition; but it’s embedded in a fundamental consensus that we’re all on the same team.  Apparently this is enough to hold back the sociopathy, at least for most of us.

Tagged , , , ,

Rescinding an offer when the candidate tries to negotiate

From the Philosophy Smoker, via Liz Harman:

CANDIDATE HOLDING OFFER:

As you know, I am very enthusiastic about the possibility of coming to Nazareth. Granting some of the following provisions would make my decision easier.
1) An increase of my starting salary to $65,000, which is more in line with what assistant professors in philosophy have been getting in the last few years.
2) An official semester of maternity leave.
3) A pre-tenure sabbatical at some point during the bottom half of my tenure clock.
4) No more than three new class preps per year for the first three years.
5) A start date of academic year 2015 so I can complete my postdoc.
I know that some of these might be easier to grant than others. Let me know what you think.

DEPARTMENT CHAIR:

Thank you for your email. The search committee discussed your provisions. They were also reviewed by the Dean and the VPAA. It was determined that on the whole these provisions indicate an interest in teaching at a research university and not at a college, like ours, that is both teaching and student centered. Thus, the institution has decided to withdraw its offer of employment to you.
Thank you very much for your interest in Nazareth College. We wish you the best in finding a suitable position.

Talk amongst yourselves.

 

Scientists aren’t experts on what makes jokes funny

Ben Lillie:

This week I finally realized what bugged me about the talk I was hearing about the science of science communication: Nothing. The issue is what I wasn’t hearing.

This was catalyzed by a short news item Lauren Rugani linked on twitter. A scientist had run a study where they discovered that sometimes a punchline is funnier if words from the punchline had been mentioned several minutes earlier. From the abstract:

“These findings also show that pre-exposing a punchline, which in common knowledge should spoil a joke, can actually increase funniness under certain conditions.”

This is shocking. Not the conclusion, which is clearly correct. The problem is that the conclusion has been known to comedians for at least the last several thousand years. When I trained in improv comedy the third class was on callbacks, the jargon term for that technique. The entire structure of an improv comedy set is based around variations on the idea that things are funnier if they’re repeated. And yet to the authors it was “common knowledge” that this will spoil a joke. There is a long tradition of people who know, from experience, how this works, and yet the idea of asking them is not evident anywhere in the paper. This is the problem — the sense that the only valid answers come from inside science and the research world.

Yes!  Everybody knows by now that when mathematicians try to do mathematical biology alone, without people with domain knowledge of biology in the room, they do crappy mathematical biology.  “Digital humanities” or “neuroaesthetics” or “culturomics” &c are just the same.  New techniques drawn from science and mathematics are fantastic research tools now, and they’re only getting better, but it seems like a terrible idea to study cultural objects from scratch, without domain experts in the room.

Tagged ,
Follow

Get every new post delivered to your Inbox.

Join 713 other followers

%d bloggers like this: