Ranking Students

How does the Goblet of Fire determine who should be Champion?

A quick recap: The Triwizard Tournament pits the “top” student from each of three magical schools (Hogwarts, Beauxbatons, Durmstrang) in a series of strenuous practical tasks. Besides magical talent, a competitor has to display courage, ingenuity, and the ability to think on one’s feet. Fame and prize money would certainly attract the ambitious. And in the spirit of sports rivalry, each school would want to put up its strongest contestant as representative: its Champion.

Essentially this is a ranking task. How do you decide on the best student? As a professor, I encounter these situations regularly when selecting students for awards, fellowships, and scholarships. There is always a list of criteria to “judge” the students: GPA, research performance and productivity, service, highest score on the final exam, best poster (using a pre-determined rubric perhaps), most consistent, best improvement, etc. Sometimes there’s a clear front-runner, but often several students seem equally worthy and a tough “choice” needs to be made. Politics, favoritism, first impressions, last impressions, social skills, and much more, can factor into the final decision – usually made by committee, since alone I might be highly biased.

The Goblet of Fire is supposedly an “impartial judge” that selects the Champion amongst all worthy entrants from each school. The procedure: Entrants must write their name and school on a piece of parchment and deposit it into the fiery cup. They have twenty-four hours to do this, at which point the goblet spits out the worthy Champion’s name. Is the Goblet of Fire like the Sorting Hat? How does it figure out which characteristics are most important and rank the candidates appropriately?

Let’s start with the easier case that doesn’t require close ranking: the job of the Sorting Hat. I’ve briefly discussed this in a recent post; the Hogwarts founders put some of their “brains” into the hat so that it would pick out characteristics prized most by each of its founders. This is summarized in part of the Sorting Hat’s song in Goblet of Fire.

By Gryffindor, the bravest were

Prized far beyond the rest;

For Ravenclaw, the cleverest

Would always be the best;

For Hufflepuff, hard workers were

Most worthy of admission;

And power-hungry Slytherin

Loved those of great ambition.

Courage, Cleverness, Diligence, Ambition. These are all important qualities for a Hogwarts champion facing dangerous magical tasks. But the Sorting Hat has an easier task: it might “see” all these qualities, although it just has to figure out which is dominant. Our Muggle world has multiple equivalents on our magical Internet. Answer ten quiz questions that supposedly suss out your personal traits and you will then know which Hogwarts house suits you best. Won’t take more than a minute! And we so much of our personal data out in the cloud exacerbated by social media sharing, an appropriate bot could mop up the data, and spit out your assignment without needing the quiz.

One trend in higher education is the integration of student data, supposedly for better targeted intervention if a student seems to be veering off course. On the back end, one integrates data streams coming in from the Learning Management System, the Student Affairs database, test scores, Admissions files, student e-Portfolio, and combine this with bots that sweep up data from the wider world web. Now awash in data, an appropriate artificial intelligence (A.I.) program could rank the students, once again based on comparison to criteria of seemingly “successful” people who form the machine-learning training set. Bots are already reading resumes as part of the hiring process. That’s sort of what the Goblet of Fire is doing – if we imagine a “fair” process from a supposedly impartial judge.

But where does the Goblet get its data? Presumably each school has records on its students. Certainly the Ministry of Magic knows the identity of each Hogwarts students, where they are, and whether they might have performed magic in front of a Muggle. If individual memories in all their 3D-high-definition can be stored and viewed later, there should be no problem storing less memory-intensive records magically or not. Argus Filch, who cannot perform magic, stores student records old-school, or I should say non-magic-school. Or it might mop up data from the magical airwaves. Once the Goblet has the data, it could cross-reference if a student was under-aged, or did poorly in classes, or had a disciplinary record, and potentially access instructor records noting exemplary performance by students.

The Goblet can also be hoodwinked. Tricked. Confunded. Bamboozled. So can machine-learning algorithms. There are clever, creative ways to add “noise” to an image file to completely fool an A.I. – there’s a whole industry of research in this area. Harry Potter’s name came out of the goblet even though it shouldn’t have. The Goblet also spits out names very shortly, or almost immediately, after self-nominations close. It must work very quickly to come up with the names of the Champions. Perhaps it can do this through handwriting analysis, and not bother about pre-loaded data. After all, that’s all that the students submit – handwritten parchment. Can’t rule out the possibility, I suppose, as far-fetched as it sounds.

Somehow the Sorting Hat and the Goblet use some sort of proxy to make their selections. How good is the proxy at sorting or ranking students? I don’t know. Certainly those in Gryffindor seem to display bravery, but so do others from other houses. And the company one keeps influences how one behaves or chooses. Whenever I assess my students, I always use a proxy of some sort since I can’t magically look into a student’s head to see what they’ve learned. I should be similarly wary when a tech-company offers an A.I. claiming to do the same.

For that matter, why are we ranking students anyway? 

P.S. My previous take on Goblet of Fire? Comparing Bagman and Crouch.