Tuesday, April 21 — tomorrow! — brings the third lecture in the MALBEC series: Michael Coen, of computer sciences and biostat, talks on “Toward Formalizing “Abstract Nonsense”,” in Computer Sciences 1221 at 4pm. Here’s the abstract:
The idea of a category — a set of objects sharing common properties
— is a fundamental concept in many fields, including mathematics,
artificial intelligence, and cognitive and neuroscience. Numerous
frameworks, for example, in machine learning and linguistics, rest
upon the simple presumption that categories are well-defined. This is
slightly worrisome, as the many attempts formalizing categories have
met with equally many attempts shooting them down.
Instead of approaching this issue head on, I derive a robust theory of
“similarity,” from a biologically-inspired approach to perception in
animals. The very idea of creating categories assumes some implicit
notion of similarity, but it is rarely examined in isolation.
However, doing so is worthwhile, and I demonstrate the theory’s
applicability to a variety of natural and artificial learning
problems. Even when faced with Watanabe’s “Ugly Duckling” theorem or
Wolpert’s stingy cafeteria (serving the famous “No Free Lunch”
theorems), one can make significant progress toward formalizing a
theory of categories by examining their often unstated properties.
I demonstrate practical applications of this work in several domains,
including unsupervised machine learning, ensemble clustering, image
segmentation, human acquisition of language, and cognitive
(Joint work with M.H.Ansari)
Delicious food will follow the talk, as if this delicious abstract isn’t enough!
On Friday, Partha Niyogi gave a beautiful talk on “Geometry, Perception, and Learning.” His work fits into a really exciting movement in data analysis that one might call “use the submanifold.” Namely: data is often given to you as a set of points in some massively high-dimensional space. For instance, a set of images from a digital camera can be thought of as a sequence of points in R^N, where N is the number of pixels in your camera, a number in the millions, and the value of the Nth coordinate is the brightness of the Nth pixel. A guy like Niyogi might want to train a computer to distinguish between pictures of horses and pictures of fish. Now one way to do this is to try to draw a hyperplane across R^N with all the horses are on one side and all the fish on the other. But the dimension of the space is so high that this is essentially impossible to do well.
But there’s another way — you can take the view that the N-dimensionality of the space of all images is an illusion, and that the images you might be interested in — for instance, some class of images including all horses and all fish — might lie on some submanifold of drastically smaller dimension.
If you believe that manifold is linear, you’re in business: statisticians have tons of tools, essentially souped-up versions of linear regression, for fitting a linear subspace to a bunch of data. But linearity is probably too much to ask for. If you superimpose a picture of a trout on a picture of a walleye, you don’t get a picture of a fish; which is to say, the space of fish isn’t linear.
So it becomes crucial to figure out things about the mystery “fish manifold” from which all pictures of fish are sampled; what are its connected components, or more generally its homology? What can we say about its curvature? How well can we interpolate on it to generate novel fish-pictures from the ones in the input? The work of Carlsson, Diaconis, Ghrist, etc. that I mentioned here is part of the same project.
And in some sense the work of Candes, Tao, and a million others on compressed sensing (well-explained on Terry’s blog) has a similar flavor. For Niyogi, you have a bunch of given points in R^N and a mystery manifold which is supposed to contain, or at least be close to, those points. In compressed sensing, the manifold is known — it’s just a union of low-dimensional linear subspaces parametrizing vectors which are sparse in a suitable basis — but the points are not!