MICA, Payscale, and more about statistical sampling than you want to know
MICA’s media spokesman has been working hard to counter a Payscale.com survey, mentioned last week in The Atlantic, that found MICA to be one of the least cost-effective colleges in the country.[UPDATE, below] City Paper had some fun with it as well, comparing the Payscale-derived annual projected income of MICA grads to both high school grads in general and MICA adjuncts in particular—as adjuncts are gearing up for a union election.
Cedric Mobley lashed out at us with an email over the weekend, claiming Payscale’s numbers were incredible and its methodology “scientifically unsound.” He wrote much the same in this Baltimore Business Journal piece.
And he may have a better point than he even claims. A look at the Payscale methodology page finds a confusing claim, that, when estimating a 30-year median pay, the “90 percent confidence interval for small liberal arts schools is plus or minus 10 percent.”
Other stats sites, including this handy sample size calculator, define the “confidence interval” in a lower-is-better way, such that, if you were polling a group of 2,400 graduates and wanted a “90 percent confidence interval” at a 95 percent confidence level, you’d need to speak to only one student.
Now, if you wanted a confidence interval of 10 percent, you’d have to poll 92 of those 2400 students.
In other words, either Payscale doesn’t know what “confidence interval” means, we’re misunderstanding Payscale’s use of “confidence interval,” or the sample size is really, really, really tiny.
We’ve emailed and called Payscale to see which of these is the case and will update if or when we hear back.
That said, Mobley is in something of a bind. City Paper compared its adjunct instructors’ pay with that of high school graduates (we used National Center for Education Studies figures for that). We found that MICA adjuncts make, on an hourly basis, maybe a dollar or so more than the typical high school grad.
As MICA adjuncts pretty much all have at least a Master’s, and many have Ph.Ds, the comparison makes the adjunct pay look a bit worse that it would otherwise—and much worse even than the Payscale survey found for the “average” MICA grad.
UPDATE: Payscale spokesman Steven Gottlieb emailed us last night: turns out the company uses “confidence interval” upside-down to the way our little online calculator does. So when they say 90 percent, that means they’re certain to within a 10 percent margin of error. Which means they definitely heard from more than one MICA grad. Full email below:
The 90% confidence interval of 5-10% means we are 90% confident the true population median is +/- 5-10% of the reported median. This confidence interval, which represents the margin of error on the earnings figures, factors in both the sample size and spread in pay. The reason we do this is because you can have a school with a relatively small sample size (e.g., Harvey Mudd), but also a relatively small spread in pay since most graduates will earn a similar amount due to the fields of study offered there (STEM). You could also have a school that has a large sample, but a very wide spread in pay and thus your certainty about the median representing the true median is lower.