The Matter of Ratings

What makes a Bacardi 1873 superior to, or a dog when compared against, a Santa Teresa 1796? On what basis do I compare my loving, cheerfully nostalgic review of the Clarke’s Court overproof “bush” variation with the disdainfulful yark-my-guts-out review of the Bundie? What exactly is the rating of the Bacardi 151, which, when you read it, is remarkably short on facts and long on humour. If I were to make a top five list of any kind – top-enders, dark rums, spicers, premiums etc – is there a consistent methodology backing up my assertions, or is it all just my opinion on which ones deserve to be in such a list?

After almost two years of drinking rum after rum and getting serious about writing about them in the last ten months since the Liquorature website went up, questions like these have pretty much forced me into revisiting the whole business of assigning a numerical rating to my faourite drink. To some extent, my methodology at the inception derived from a decades-long perusal of movie reviews; Roger Ebert’s four star system and his even more simplified (and troublesome) thumbs are almost standards in themselves, but in practice they are useless except at opposite ends of the spectrum.

After all, what distinguishes a subjectively assessed 2.5 star review from a 3 star? You know you’re getting quality with four stars and crap with one…it’s in the middle the problems crop up. Now contrast that against the New York Times or New Republic stance of not assigning a rating at all – in their opinion the writing and experience of the writer is what counts, the feeling and thinking of the critic as he watched the movie – and under such conditions, rankings and ratings aren’t required.

Then there was my issue with the way The Hippie rated his whiskies: sharp eyed constant readers (all three of them, ha ha) will note that of his first fifty whisky reviews he skewed heavily to assigning 90% or greater rankings to almost all of them. He defended this by noting that he knew what he was getting and so picked stallar examples of the craft to review, but when you think about it, if all reviews are uniformly positive, you dilute your relative ratings system and reduce their utility to a reader. You need to leaven all the positives with some negatives or neutrals to sustain a perceived objectivity and permit a baseline of sorts (to his credit, Curt has started reviewing some dogs of late).

Added to all this is the iconoclastic stance I have towards everyone’s jumping on the badwagon and attempting to recreate Jim Murray’s numerical system in that invaluable resource, his Whisky Bible. Now, not everyone can imitate, let alone approach the brilliance of his pithy one liners (I’m a decent writer, but I tend towards the verbose, and I’m in awe of of his short, sharp and scintillating literary gems), but surely alternative approaches can be taken? Surely it’s not always and only about numbers at the end of the day?

And there is the crux of it. Numbers on individual drinks mean nothing. An individual review stands or falls on its own merits: on the skill of the writer in recreating what he smelled and tasted and thought, and his ability to not only bring you along, but explain his point of view on whether he cared for the product or not; and irrespective of whether you agree, you at least take the point.

It’s in their aggregate that ratings come into their own: in their ability to stratify whiskies, rums, vodkas et al into groupings that a prospective buyer can choose from according to certain bands of rough similarity. It was this last point that has led me to abandon my struct adherence to my NYT policy of simply describing my experience (yes I try to write well, engagingly and to add some information as well, but you’re reading my rum review to understand whether it’s worth your cash or confirms your own experience, not to indulge me). Because at the end, what a reader is asking for is an understanding of how one rum stacks up against another, and if faced with a choice of two equally priced and presented rums, which one should he pick.

The mood, tone and style of my writing disguises the underlying essentials of a rum (the facts), while elevating what some see as nonessentials: my sense of humour, my delight in writing, and occasional irrelevancies (I can hear Maltmonster mutter “Occasional?” into his Bowmore). More than one reader has mentioned that my reviews are simply too long, and while my usual response is a resigned shrug – I do this for love, for free and for me, and if someone feels that way, they are welcome to advise me which words to cut, or click through to more convival (and shorter) review sites – I must concede it is a weakness when trying assemble a more objective measure of indivdial elements of quality in a rum.

Having said all the above, I’ve reluctantly decided to change my approach. Not to shorten anything, but to add two things: one, a short summary somewhere in the review, probably at the top or bottom, that encapsulates evertything I want to say in a sentence or two. And an attempt to quantify my observations. Try as I might, I simply cannot escape the long shadow of the standard nomenclature: Appearance/ presentation, Nose, Taste/Palate, Finish and Intangibles. These will be placed at the bottom of each review, in red, with the points I gave, and what the weight of each is, adding up to an even hundred for ease of calculation.

A few things have to be said right out front: “Intangibles” is going to be zero for almost everything, because it’s that extra something that simply can’t be easily quantified, but which elevates the entire experience of the drink (JM refers to this as balance, which is the way the elements relate to each other harmoniously, or not – as good a term as any, I suppose). So Clemente Tres Vieux gets nothing there, while Clarke’s Court does, as do the Zaya 12 and the EH-5. On the other hand, Clemente, EH10 and DDL 21 will get brownie points in Appearance/ Presentation, while Appleton 30 gets marked down for the cheap tin, and Clarke’s gets a big fat duck egg. Arguably the biggest points of contention  will be Nose, taste and Finish, and here’s where the largest element of subjectivity comes into play – I won’t pretend that what I experience will mirror your own memories of the drink, but it will faithfully reflect what I had. And as always, I’ll describe, in my review, the steps and thinking that went into the final ranking I gave: you may not agree with me, which is perfectly fine…but you’ll understand why I gave it the marks I did.

Perhaps I needed to crack more than fifty rums before I came to this pass: before, it was a hobby, a sort of tickling interest of mine. I always enjoyed writing, and these short little essays (well…relatively short) became a labour of love for me, an exercise in creative style and clarity of expression. Now, it’s obviously gone beyond that. Liquorature does not get a massive amount of hits per week – after less than four months in action, the sister site AllThingsWhisky.com is already overtaking this site – but the numbers are trending upwards. I believe that as more and more reviews of rums and that obscure Scottish drink get posted over the years, we will become, with perhaps four or five others, one of the resources people come to for a well written, cogently argued review of a given drink.

I dread going back to some fifty-plus rums to assign the summary and numbers (and to endure the smirks of my friends with the inevitable I-told-you-sos), but there are sound reasons for doing so. I guess I had to do enough of these reviews to understand that, and feel committed enough to finally make that plunge.