Later On

A blog written for those whose interests more or less match mine.

Lather-making, grade prediction, and other non-precision procedures

leave a comment »

I blogged earlier today about the strange obsession for perfect soap/bowl fit that some seem to exhibit. The phrase, “Lather-making is not a precision operation” came to mind: sure, the loose fit of soap to bowl may make it a little sloppier, but let’s face it: slop is what it’s about (within limits, as always: avoid the extremes is what I’m saying).

My mind is set to automatic analogy-seeking = T, and I immediately thought of one from my earlier career, during which time I worked for a large college-admissions-testing company in the Midwest. I worked there twice, in fact, and in the interregnum I served as a consultant when the company wanted to get an admissions perspective (I was a director of admissions at the time) on the upcoming new features (“advantages, and benefits” my brain automatically adds: the FAB statement from some ancient sales/marketing training).

In particular they thought they had improved their predictions of freshman year first-semester GPA, including by subject area. And there was also collecting and reporting of out-of-class accomplishment data with some quite intriguing supporting research: it turns out that out-of-class accomplishment (in those days, kids just doing things on their own) had close to zero correlation to grades (best was something like 0.30 Pearson linear-r correlation between grades in English and writing—very little—and the others were below 0.10 and above -0.10 as I recall. NOTE: I am writing this from memory (with my eyes closed…  and now:  backwards!) Thus the wise reader will realize the specifics may well be much less accurate than they are precise.

However, to continue the story, the out-of-class accomplishments turned out to correlate well with later life experience—not a great surprise; indeed, any other outcome would be surprising—because people who show that they will go and do things on their own learn through that the practical skill of doing things on their own, and so continue, and the small headstart widens considerably over the years, earlier experiences informing later.

And grades, it turns out, correlate well only with other grades: junior high grades correlate well with high school grades, those with undergraduate grades (and the tighter the connection the better the prediction: one can more accurately predict the first-semester freshman GPA of a student than his four-year overall GPA).

BUT—and here’s the rub—grades don’t correlate at all with (a) out-of-class accomplishments (as noted above) or (b) subsequent real-life accomplishments. No matter how you define “success”—e.g., number of patents, size of income, peer recognition, number of publications, and so on, with more specific measures by area of endeavor—just as the specific out-of-class accomplishments were different for each subject area. And cleverly done, too: I won’t go into details, but the options were very cleverly selected based upon research findings and reporting goals.

One amusing sidelight: One of the authors of the scale who had done much of the research in establishing it remarked that an amazing number of people objected to the scale on the basis that, once kids knew which out-of-class accomplishments would be asked about, would simply go out on their own and do those things. “So?” was, I think, the best answer. The educator making the objection had completely missed the point (and by the nature of the position was one responsible for the education of our country’s youth: not a good sign). So what if kids went out and did those things? They would still be out-of-class accomplishments, and the kid would then also acquire the experience and practical knowledge such things deliver, thus actuating the process the measure describes. (I’m reminded in Moneyball of how the old guys couldn’t grasp that the thing was to get on base, and no one cares how you got there: walk, hit, or be hit. The point is to get on base. And here the point was to see which students had, of their own initiative, gone out and done those accomplishments. (I fear that the measure may now be undermined somewhat by the accomplishments now less influenced by individual student initiative—the student may have coaches, for example—so that the initiative-measuring fact is down. But OTOH, the experience of the accomplishment is still the same, and some habits are doubtlessly established that support initiative: i.e., habit. If the student has the habit each summer of doing something new, even if the first few are coached, the habit settles and the student eventually seeks out things to do: “If I must do something this summer, I may as well figure out something I’d like to do that will be supported.” That’s the seed of initiative.)

So yes, the student can then go out and do those accomplishments. And we hope that they will. We are trying to educate them, aren’t we? To draw from them their natural talents and help them embed those in a supportable matrix? Isn’t that what it’s all about? Hello? (The condition from which the question-raiser suffers was known at an earlier age as “Lights are on, but nobody’s home.”)

Only one student—from AT&T Bell Labs, as I recall—ever showed any significant correlation between grades and success in later life. (As I recall—note caveat above—it was managerial rank at AT&T and college GPA: which could be an artifact of artificial barriers based on grades. While one does want a certain level of competence—one would not select a surgeon who had received straight failing grades (thankfully, you cannot select such a surgeon)—it turns out that once a surgeon gets at least, say, a C, the addition grade points have no correlation with later success in life (defined, as noted, any way you like). This one, for example, included income, papers, operations, and (critically) number of other surgeons who would choose the given surgeon for that type of surgery: all independent of grade if the surgeon got a least a C.

Back to the AT&T study: so, possible. But that was the only study found that showed such a correlation. And—weird thing—AT&T would not release the data.

From an admissions point of view, this new measure gave a new way to structure the incoming class: not just on academic accomplishment, but also to include a good number of those with various out-of-class accomplishments (science, writing, whatever: I don’t remember all the scales—not all, however, were academic).

So—as you can doubtless tell—I was pretty stoked by this idea. The grade prediction bit, left me cold, however. “Look,” I said, “your own research shows you that grades are unrelated both to current and later-life accomplishment outside of class, where, after all, the students will spend most of their lives. That should be the focus. Trying to improve grade prediction is,” I said, with what I thought was a splendid analogy, an attitude consistent with (and responsible for) much of my success-rate in organizations, “like trying to sharpen a turd. First, the material itself simply doesn’t admit of that level of precision. Second, even suppose somehow that you’ve succeeded, what do you now have?”

So you can see how that leapt to mind as I thought about becoming over-precise in soap shaping vis-à-vis lather production (which is, after all, the goal, no?).

Written by LeisureGuy

9 March 2012 at 10:27 am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.