## Random squarefree polynomials and random permutations and slightly non-random permutations

Influenced by Granville’s “Anatomy of integers and permutations” (already a play, soon to be a graphic novel) I had always thought as follows:  a polynomial of degree n over a finite field F_q gives rise to a permutation in S_n, at least up to conjugacy; namely, the one induced by Frobenius acting on the roots.  So the distribution of the degrees of irreducible factors of a random polynomial should mimic the distribution of cycle lengths of a random permutation, on some kind of equidistribution grounds.

But it’s not quite right.  For instance, the probability that a permutation is an n-cycle is 1/n, on the nose.

But the probability that a random squarefree polynomial is irreducible is about (1/n)(1-1/q)^{-1}.

The probability that a random polynomial, with no assumption of squarefreeness, is irreducible, is again about 1/n, the “right answer.”  But a random polynomial which may have repeated factors doesn’t really have an action of Frobenius on the roots — or at least it’s the space of squarefree monics, not the space of all monics, that literally has an etale S_n-cover.

Similarly:  a random polynomial has an average of 1 linear factor, just as a random permutation has an average of 1 fixed point, but a random squarefree polynomial has slightly fewer linear factors on average, namely (1+1/q)^{-1}.

Curious!

## Y. Zhao and the Roberts conjecture over function fields

Before the developments of the last few years the only thing that was known about the Cohen-Lenstra conjecture was what had already been known before the Cohen-Lenstra conjecture; namely, that the number of cubic fields of discriminant between -X and X could be expressed as

$\frac{1}{3\zeta(3)} X + o(X)$.

It isn’t hard to go back and forth between the count of cubic fields and the average size of the 3-torsion part of the class group of quadratic fields, which gives the connection with Cohen-Lenstra in its usual form.

Anyway, Datskovsky and Wright showed that the asymptotic above holds (for suitable values of 12) over any global field of characteristic at least 5.  That is:  for such a field K, you let N_K(X) be the number of cubic extensions of K whose discriminant has norm at most X; then

$N_K(X) = c_K \zeta_K(3)^{-1} X + o(X)$

for some explicit rational constant $c_K$.

One interesting feature of this theorem is that, if it weren’t a theorem, you might doubt it was true!  Because the agreement with data is pretty poor.  That’s because the convergence to the Davenport-Heilbronn limit is extremely slow; even if you let your discriminant range up to ten million or so, you still see substantially fewer cubic fields than you’re supposed to.

In 2000, David Roberts massively clarified the situation, formulating a conjectural refinement of the Davenport-Heilbronn theorem motivated by the Shintani zeta functions:

$N_{\mathbf{Q}}(X) = (1/3)\zeta(3)^{-1} X + c X^{5/6} + o(X^{5/6})$

with c an explicit (negative) constant.  The secondary term with an exponent very close to 1 explains the slow convergence to the Davenport-Heilbronn estimate.

The Datskovsky-Wright argument works over an arbitrary global field but, like most arguments that work over both number fields and function fields, it is not very geometric.  I asked my Ph.D. student Yongqiang Zhao, who’s finishing this year, to revisit the question of counting cubic extensions of a function field F_q(t) from a more geometric point of view to see if he could get results towards the Roberts conjecture.  And he did!  Which is what I want to tell you about.

But while Zhao was writing his thesis, there was a big development — the Roberts conjecture was proved.  Not only that — it was proved twice!  Once by Bhargava, Shankar, and Tsimerman, and once by Thorne and Taniguchi, independently, simultaneously, and using very different methods.  It is certainly plausible that these methods can give the Roberts conjecture over function fields, but at the moment, they don’t.

Neither does Zhao, yet — but he’s almost there, getting

$N_K(T) = \zeta_K(3)^{-1} X + O(X^{5/6 + \epsilon})$

for all rational function fields K = F_q(t) of characteristic at least 5.  And his approach illuminates the geometry of the situation in a very beautiful way, which I think sheds light on how things work in the number field case.

Geometrically speaking, to count cubic extensions of F_q(t) is to count trigonal curves over F_q.  And the moduli space of trigonal curves has a classical unirational parametrization, which I learned from Mike Roth many years ago:  given a trigonal curve Y, you push forward the structure sheaf along the degree-3 map to P^1, yielding a rank-3 vector bundle on P^1; you mod out by the natural copy of the structure sheaf; and you end up with a rank-2 vector bundle W on P^1, whose projectivization is a rational surface in which Y embeds.  This rational surface is a Hirzebruch surface F_k, where k is an integer determined by the isomorphism class of the vector bundle W.  (This story is the geometric version of the Delone-Fadeev parametrization of cubic rings by binary cubic forms.)

This point of view replaces a problem of counting isomorphism classes of curves (hard!) with a problem of counting divisors in surfaces (not easy, but easier.)  It’s not hard to figure out what linear system on F_k contains Y.  Counting divisors in a linear system is nothing but a dimension count, but you have to be careful — in this problem, you only want to count smooth members.  That’s a substantially more delicate problem.  Counting all the divisors is more or less the problem of counting all cubic rings; that problem, as the number theorists have long known, is much easier than the problem of counting just the maximal orders in cubic fields.

Already, the geometric meaning of the negative secondary term becomes quite clear; it turns out that when k is big enough (i.e. if the Hirzebruch surface is twisty enough) then the corresponding linear system has no smooth, or even irreducible, members!  So what “ought” to be a sum over all k is rudely truncated; and it turns out that the sum over larger k that “should have been there” is on order X^{5/6}.

So how do you count the smooth members of a linear system?  When the linear system is highly ample, this is precisely the subject of Poonen’s well-known “Bertini theorem over finite fields.”  But the trigonal linear systems aren’t like that; they’re only “semi-ample,” because their intersection with the fiber of projection F_k -> P^1 is fixed at 3.  Zhao shows that, just as in Poonen’s case, the probability that a member of such a system is smooth converges to a limit as the linear system gets more complicated; only this limit is computed, not as a product over points P of the probability D is smooth at P, but rather a product over fibers F of the probability that D is smooth along F.  (This same insight, arrived at independently, is central to the paper of Erman and Wood I mentioned last week.)

This alone is enough for Zhao to get a version of Davenport-Heilbronn over F_q(t) with error term O(X^{7/8}), better than anything that was known for number fields prior to last year.  How he gets even closer to Roberts is too involved to go into on the blog, but it’s the best part, and it’s where the algebraic geometry really starts; the main idea is a very careful analysis of what happens when you take a singular curve on a Hirzebruch surface and start carrying out elementary transforms at the singular points, making your curve more smooth but also changing which Hirzebruch surface it’s on!

To what extent is Zhao’s method analogous to the existing proofs of the Roberts conjecture over Q?  I’m not sure; though Zhao, together with the five authors of the two papers I mentioned, spent a week huddling at AIM thinking about this, and they can comment if they want.

I’ll just keep saying what I always say:  if a problem in arithmetic statistics over Q is interesting, there is almost certainly interesting algebraic geometry in the analogous problem over F_q(t), and the algebraic geometry is liable in turn to offer some insights into the original question.

## Homological stability for Hurwitz spaces and the Cohen-Lenstra conjecture over function fields

Now I’ll say a little bit about the actual problem treated by the new paper with Venkatesh and Westerland.  It’s very satisfying to have an actual theorem of this kind:  for years now we’ve been going around saying “it seems like asymptotic conjectures in analytic number theory should have a geometric reflection as theorems about stable cohomology of moduli spaces,” but for quite a while it was unclear we’d ever be able to prove something on the geometric side.

The new paper starts with the question: what do ideal class groups of number fields tend to look like?

That’s a bit vague, so let’s pin it down:  if you write down the ideal class group of the quadratic imaginary number fields $\mathbf{Q}(\sqrt{-d})$, as d ranges over squarefree integers in [0..X],  you get a list of about $\zeta(2)^{-1} X$ finite abelian groups.

The ideal class group is the one of the most basic objects of algebraic number theory; but we don’t know much about this list of groups!  Their orders are more or less under control, thanks to the analytic class number formula.  But their structure is really mysterious.

## Random pro-p groups, braid groups, and random tame Galois groups

I’ve posted a new paper with Nigel Boston, “Random pro-p groups, braid groups, and random tame Galois groups.”

The paper proposes a kind of “non-abelian Cohen-Lenstra heuristic.”   A typical prediction:  if S is a randomly chosen pair of primes, each of which is congruent to 5 mod 8, and G_S(p) is the Galois group of the maximal pro-2 extension of Q unramified away from S, then G_S(p) is infinite 1/16 of the time.

The usual Cohen-Lenstra conjectures — well, there are a lot of them, but the simplest one asks:  given an odd prime p and a finite abelian p-group A, what is the probability P(A) that a randomly chosen quadratic imaginary field K has a class group whose p-primary part is isomorphic to A?  (Note that the existence of P(A) — which we take to be a limit in X of the corresponding probability as K ranges over quadratic imaginary fields of discriminant at most X — is not at all obvious, and in fact is not known for any p!)

Cohen and Lenstra offered a beautiful conjectural answer to that question:  they suggested that the p-parts of class groups were uniformly distributed among finite abelian p-groups.  And remember — that means that P(A) should be proportional to 1/|Aut(A)|.  (See the end of this post for more on uniform distribution in this categorical setting.)

Later, Friedman and Washington observed that the Cohen-Lenstra conjectures could be arrived at by another means:  if you take K to be the function field of a random hyperelliptic curve X over a finite field instead of a random quadratic imaginary field, then the finite abelian p-group you’re after is just the cokernel of F-1, where F is the matrix corresponding to the action of Frobenius on T_p Jac(X).  If you take the view that F should be a “random” matrix, then you are led to the following question:

Let F be a random element of GL_N(Z_p) in Haar measure:  what is the probability that coker(F-1) is isomorphic to A?

And this probability, it turns out, is precisely the P(A) conjectured by Cohen-Lenstra.

(But now you cry out:  but Frobenius isn’t just any old matrix!  It’s in the generalized symplectic group!  Yes — and Jeff Achter has shown that, at least as far as the probability distribution on A/pA goes, the “right” random matrix model gives you the same answer as the Friedman-Washington quick and dirty model.  Phew.)

Now, in place of a random quadratic imaginary field, pick a prime p and a random set S of g primes, each of which is 1 mod p.  As above, let G_S(p) be the Galois group of the maximal pro-p extension of Q unramified away from S; this is a pro-p group of rank g. What can we say about the probability distribution on G_S(p)?  That is, if G is some pro-p group, can we compute the probability that G_S(p) is isomorphic to G?

Again, there are two approaches.  We could ask that G_S(p) be a “random pro-p group of rank g.”  But this isn’t quite right; G_S(p) has extra structure, imparted to it by the images in G_S(p) of tame inertia at the primes of S.  We define a notion of “pro-p group with inertia data,” and for each pro-p GWID G we guess that the probability that G_S(p) = G is proportional to 1/Aut(G); where Aut(G) refers to the automorphisms of G as GWID, of course.

On the other hand, you could ask what would happen in the function field case if the action of Frobenius on — well, not the Tate module of the Jacobian anymore, but the full pro-p geometric fundamental group of the curve — is “as random as possible.”  (In this case, the group from which Frobenius is drawn isn’t a p-adic symplectic group but Ihara’s “pro-p braid group.”)

And the happy conclusion is that, just as in the Cohen-Lenstra setting, these two heuristic arguments yield the same prediction.  For the relatively few pro-p groups G such that we can compute Pr(G_S(p) = G), our heuristic gives the right answer.  For several more, it gives an answer that seems consistent with numerical experiments.

Maybe it’s correct!

## Southern California Number Theory Day, the airport Chili’s, Evan Longoria counterfactuals

I came back this morning from a very brief trip to California to speak at Southern California Number Theory Day, hosted this year at UC Irvine. The other speakers were terrific, well worth undergoing the pain of a red-eye flight back Midwest. (Non-math material follows below the SCNTD sum-up, for those readers who don’t cotton to the number theory.)

• Brian Conrad talked about his work (some of it with Gabber and G. Prasad) on finite class numbers for algebraic groups, and an alternative to the notion of “reductive group” over global function fields of characteristic p, where the usual notion doesn’t behave quite as well as you expect. Very clear, and very much in Brian’s style in its admirable refusal to concede any “simplifying assumptions.” Well, except the occasional avoidance of characteristic 2.
• Jeff Achter talked about a circle of results, many joint with Pries, about the geography of the moduli space of curves in characteristic p. Here you have lots of interesting subvarieties that don’t have any characteristic 0 analogue, such as the “p-rank r stratum” of curves whose Jacobians have exactly p^r physical p-torsion points. Typical interesting theorem: the monodromy representation of the non-ordinary locus (a divisor in M_g) surjects onto Sp_2g, just as the monodromy representation of M_g itself does. I asked Jeff whether we know what the fundamental group of the non-ordinary locus is — he didn’t know, which means probably nobody does.
• Christian Popescu closed it out with a beautiful talk arguing that we should replace the slogan “Iwasawa theory over function fields is about the action of Frobenius on the Tate module of a Jacobian” with “Iwasawa theory over function fields is about the action of Frobenius on the l-adic realization of a 1-motive related to the Jacobian.” This point of view — joint work of Popescu and Greither — cleans up a lot of things that are customarily messy, and shows that different-looking popular conjectures at the bottom of the Iwasawa tower are in fact all consequences of a suitably formulated Main Conjecture at the top.

On the way over I’d eaten a dispiriting lunch at the St. Louis airport Chili’s, where I waited twenty minutes for a hamburger I can only describe as withered. Last night, I got to LAX with an hour and a half to spare, and the Rays and Phillies in the 7th inning of a close game 3. And the only place to watch it was Chili’s. This time I was smart enough just to order a Diet Coke and grab a seat with a view of the plasma screen.

The airport Chili’s, late on a Word Series night, turns out to be a pretty pleasant place. People talk to you, and they talk about baseball. On one side of me was a pair of fifty-something women on their way to Australia to hang out with tigers in a nature preserve. One was a lapsed Orioles fan from Prince George’s County, the other had no team. On the other was a guy from Chicago in a tweed jacket who writes for the Daily Racing Form. He liked the Mets. We all cheered for Philadelphia, and pounded the table and cussed when Jayson Werth got picked off second in the 8th in what seemed at the time the Phils’ best chance to score. (Werth, you might remember, used to be the Orioles’ “catcher of the future”; in the end, he never played a major-league game for the Orioles, or behind the plate.)

The game went into the bottom of the 9th tied 4-4, about a half hour before I was supposed to board. I figured I’d miss the end. But a hit batsman, a wild pitch, and an off-line throw to second put Eric Bruntlett on third with nobody out. Tampa Bay intentionally walked the next two hitters to get to Carlos Ruiz.

Question 1: Was this wise? I understand you set up the force, and I understand you want to put the worst Phillies hitters in the critical spot. But even a pretty bad hitter suddenly turns pretty good if you can’t walk him. And the extra two baserunners mean that Tampa Bay is still in big trouble even if Bruntlett is out at the plate after a tag-up. Mitchel Lichtman of The Hardball Times says Joe Maddon blew this decision.

And then: well, you probably saw this on TV, but Ruiz hits a slow, goofy chopper up the third-base line. Evan Longoria charges it, but by the time he gets there Bruntlett is almost home; Longoria heaves a desperate moonball in the general direction of home plate, only much, much higher; Phillies win.

Question 2a: Would Longoria have had a play if he’d stopped, set, and thrown, instead of trying to fling the ball to the catcher mid-dive?

Question 2b: Should Longoria have tried to make the play at all? Suppose he’d just stood at third, recognizing he had no play. Maybe Bruntlett scores and the Phillies win; but maybe the ball rolls foul, sending everyone back to their base with the game still tied. My Racing Form neighbor was convinced the ball was headed foul, and that Longoria had blown the game by picking it up. Subquestion: Would any human being alive have the self-control not to charge the ball in this situation?

Question 2c: A commenter on Baseball Think Factory proposed a counterfactual ending for this game even more outlandish than what actually occurred. Say Longoria runs towards the ball, sees he has no play, decides not to pick it up and hope it rolls foul. The ball rolls past Longoria, headed towards third base, as Bruntlett crosses the plate. If the ball stays fair, Phillies win; if not, Ruiz bats again. So the ball’s rolling along the line, and meanwhile, Shane Victorino, who started on second, is rounding third — and as he passes the ball he kicks it fair. Now Victorino is clearly out for interfering with the ball in play. But in this scenario, has Philadelphia won the game?