Living in Victoria in Australia, I have access to an abundance of world leading wineries to visit. One afternoon I visited two large wineries: De Bortoli and Yering Station. At Yering Station, I opted to be guided through a tasting of Yering Station’s premium wines. One wine was the Yering Station 2013 Shiraz Viognier. On tasting it, it was balanced and palatable but it did not strike me as being much better than any other decent red wine I have tasted. I was then surprised to learn, moments after tasting it, that leading wine critic James Halliday had scored the wine a hugely impressive 98 points. This made me wonder about whether scores which “wine experts” and “whisky experts” attribute to wine or whisky are subjective, or whether there are any good reasons to believe these scores are reliable indicators of the quality of a wine or a whisky.

 

In this post, I take the next step in my whisky learning as the epiphany strikes that taste in whisky is “wildly subjective” and that it is both unreliable and unfounded to give a whisky an award or a score out of 100. This is why I now have a five star rating system which is subjective and does not aim to assess a whisky’s “quality” – I simply give 5 stars for a whisky I think is excellent and love to smell and taste, four stars for a whisky I enjoy drinking, three stars for a whisky which I find palatable but do not necessarily enjoy, two stars for a whisky I find mildly unpalatable but manage to drink, and one star for a whisky I do not enjoy at all and cannot drink.

I accept that the only real value in my reviews are my tasting notes, which try to describe what a whisky tastes like as I try to illicit what chemical compounds exist in a whisky from smell and taste (just as I get great value from an expert’s wine tasting notes but not their scores). For example, an abundant ester in the “hearts” of a distillation run is  ethyl hexanoate which smells similar to apple while isomers which come from the oak barrels in which a whisky is aged – such as cis and trans-3-methyl-4-octanolide – resemble coconut flavours. So rather than say I can smell ethyl hexanoate and taste cis and trans-3-methyl-4-octanolide, I say I can smell apple and taste coconut! My olfactory (smell) and gustatory (taste) systems may detect the compounds as something slightly different – such as pear or macaron – but the descriptors may provide some clues as to what a whisky smells and tastes like.  

That being said, this leads me to ask whether taste in whisky – not just how it tastes, but whether it tastes “good” – is likely to be subjective and whether whisky awards are reliable indicators of a whisky’s intrinsic quality. I argue that, firstly, taste in whisky is likely to be subjective, and, secondly, whisky awards are useless.

Taste in “good” wine (and whisky) is “wildly subjective”

Two economists, Steven Levitt and Stephen Dubner, wrote a book titled Freakonomics. The authors of the Freakonomics website refer to  a study by winemaker Robert Hodgson, conducted in collaboration with the California State Fair wine competition,  as evidence that wine tasting is “wildly subjective”.

The study by Robert Hodgson, which was published in the Journal of Wine Economics, found that – as the Freakonomics website says – a person’s taste in wine is likely to be subjective. In the experiment Hodgson presented each panel of four wine judges with their usual “flight” of wine samples to smell and taste. The catch in the experiment, however, was that some wines were presented to the judges three times, and that wine was poured from the same bottle each time it was presented to the judges.  Hodgson found that the wine judges were inconsistent when scoring the same wine. As reported by The Guardian, the same judge scored the same wine differently in blind tastings even when the wine was tasted only minutes apart. Hodgson’s research has reconfirmed not once but twice that it is likely ‘professional palates are terrible at judging wine’. His study found, in particular, that only about 10% of the wine judges were consistent. Speaking to The Guardian, Hodgson says that “[c]hance has a great deal to do with the awards that wines win.”

There does not seem to be any reason why these study results would not also apply to whisky judging, and to whisky awards. If you are not convinced, perhaps the below arguments will persuade you. 

1. The criteria as to what makes a “good” whisky seems to be entirely subjective and random

The criteria as to what makes a “good” whisky is entirely random. This is evident, as will now be discussed, in the inconsistent scores different “whisky experts” give a whisky.

For example, differences in the scoring of Chivas Regal 18 year old by whisky guru’s Jim Murray on the one hand and John Hansell on the other is huge: Jim Murray gives Chivas Regal 18 year old 73.5 points (‘usually drinkable but don’t expect the earth to move’) while John Hansell gives it 95 points (‘A classic! All components are balanced appropriately, with the complexity and character expected in a classic’).

Granted batches in world leading blends may vary (one thinks not so significantly however), but this inconsistency in the scoring of whisky also appears with single malts and single cask offerings (single cask whisky is from one barrel, which means that it should taste the same from bottle to bottle). For example, in relation to Glenfarclas Family Cask 1961 Cask 1325 (47% abv) respected whisky writer Dave Broom scores it a 79 (‘Average. No unique qualities. Flaws possible’) whereas in Whisky Bible 2014 leading whisky guru Jim Murray scores it 87.5 (‘very good to excellent … definitely worth buying’).

Jim Murray does not seem to think much of Glenmorangie Lasanta scoring it a measly 68.5 points (‘very unimpressive indeed’) whereas noted whisky reviewer Ralphy Mitchell has scored it 87 points.  Dave Broom, in contrast to Murray, seems to like the Lasanta.

By way of another example John Hansell scored Glengoyne 40 year old (45.9% abv) 94 points (‘Outstanding! One of the best for its style. Distinctive’) whereas Jim Murray gave it 83 points (‘good whisky worth trying’).

All this seems to demonstrate that scoring whisky seems to be, just as Hodgson demonstrated with wine, wildly subjective. There seems to be no science or consistent criteria which can be used to score whisky, and it seems to depend on the judge’s personal tastes.

2. Whisky seems to be an acquired taste, and the process of acquiring taste for whisky may further refine tastes so what an “expert” enjoys may not be what an ordinary drinker will enjoy

When I first introduce people to whisky, their first sip of a neat blend or malt is usually followed by a cringe, and they say something along the lines of: “ew, its really strong how do you drink this stuff”. Obviously, to a palate unaccustomed to whisky, a whisky will seem very strong and alcoholic; often these newcomers to whisky will say that all they can taste when tasting whisky is alcohol and… well, whisky! (one person even said it tastes like what she imagines “poison” would taste like).  But just as taste in stinky cheese or heavy red wine develops as you revisit them, so too does taste in whisky develop and evolve as you revisit it.  My first taste of a cask strength whisky left me with my proverbial tail in between my legs – it was far too strong and potent to enjoy, I thought. Now, I cannot get enough of them!

So, this seems to lead me to hypothesise that what a seasoned whisky enthusiast enjoys will not likely be what a beginner in whisky will enjoy. In journal Food Quality and Preference four academics conducted a study which found that wine experts and novices seem to perceive wine differently based on differences in cognitive development concerning wine. They write:

Results showed that experts and wine consumers overall organised the wine sensory properties in similar ways, but that wine consumers were less homogeneous in their classifications, demonstrating that they lacked the shared cognitive constructs relating to Sauvignon blanc wine knowledge exhibited by the experts. Further, wine experts performed similarly across conditions, demonstrating an ability to conceptualise Sauvignon blanc from memory in much the same way that they conceptualised the wines when experiencing (i.e., tasting) exemplars. On the other hand, wine consumers’ hierarchical classifications differed across conditions. The results are discussed in terms of the cognitive processing associated with developed wine knowledge.

There are no good reasons to think that this developed familiarity with tastes associated with wine would not also apply to developed familiarity with spirits, such as whisky. It can be suggested that a seasoned spirits enthusiast would, from memory, know what kind of aromas and flavours in whisky they enjoy and score a whisky according to their cultivated tastes and palate. This will likely have no benefit to ordinary whisky drinkers, who are unlikely to have the same level of sophisticated (or trained) palate. 

What this seems to show is that “experts” have refined and developed palates, and will likely enjoy or appreciate wine or spirits that the ordinary drinker will simply perceive differently. This is because the ordinary wine or spirits drinker will not have a large mental library of wine or spirit flavours to tap into when judging wine or spirits.   

3. Important flavour compounds in whisky can be detected by some, but not by others, making whisky taste different to different people

The differences in the ability of different people to taste the same compound in whisky may mean that the same whisky will taste differently to different people. For example, the chemical rotundone has been shown in research to be an important contributor to the flavour of oak aged spirits such as whisky, but one fifth of people cannot perceive it. Therefore, two people tasting the same whisky may perceive it differently. According to five wine researchers who wrote a paper in the Technical Review:

Rotundone is very potent with as little as 16 parts per trillion able to be detected by the human nose in red wine (Wood et al. 2008). Interestingly, approximately one in five people cannot perceive it, even at levels several orders of magnitude above the average odour detection threshold.

The inability to perceive certain odors is known as “anosmia”, and when tasting whiskies it causes a blind spot; an inability to perceive certain compounds and important parts of a whisky’s character. The sense of taste works with our sense of smell, so if we are unable to smell properly (such as when we have a cold or a blocked nose) we are unable to taste  properly. Therefore, people who have the ability to detect rotundone may perceive whisky in one way, but, people who do not have the ability to detect rotundone may smell or taste that exact same whisky in a different way. 

4. Important flavour compounds in whisky may smell differently to different people, or smell the same but be described differently by different people 

Even people who are able to detect the same compound may perceive it differently. In a whisky specific study titled ‘Perception of Whisky Flavour Reference Compounds by Scottish Distillers‘ published in the Journal of the Institute of Brewing a ‘set of 16 compounds was selected from the literature as potential flavour standards for whisky profiling: acetic acid (sour), diacetyl (buttery), dimethyl tri sulphide (sulphury), ethyl hexanoate (fruity-appley), ethyl laurate (soapy), furfural (grainy), geraniol (floral), guaiacol (smoky), hexanal (grassy), iso-amyl acetate (fruity-banana), iso-valeric acid (sweaty), maltol (sweet), phenyl ethanol (floral), vanillin (vanilla), 4-vinyl guaiacol (spicy) and whisky lactone (coconut). Each compound, at 90% recognition threshold concentration, that at which 90% assessors recognise the flavour character, was added to year old grain whisky diluted to 23% v/v. The solutions were assessed by 72 distilling professionals (blenders, quality control and technical functions)’. The researchers in this study found that there was a lot of inconsistency in the way that the assessors perceived the same compound; and some described certain flavours differently:

It is an accepted problem in descriptive sensory analysis that assessors may perceive flavour in similar manner but describe it differently. However, in this experiment, assessors not only described certain flavours differently, but also appeared to perceive their character differently (e.g. grassy vs almond and sweaty vs fruity).

This demonstrates that even trained palates can describe certain flavours differently, so that a compound that smells like apple may be described as different things by different people.  This seems to add to the argument that how a whisky is described may not be how it smells or tastes; it may be how the assessors brain perceives how the whisky smells or tastes. 

Often the sheer complexity of whisky is too much for human senses, and when smelling and tasting whisky the brain must do the complex job of unpacking the aromas and flavours in a whisky. As these studies show, different people may often perceive the same compounds differently or different people will not be able to detect the same compound – therefore, the whisky may taste different to different people. This seriously puts into question the usefulness of whisky awards. 

Moral of the story?

With all this in mind, looking back on my visit to Yering Station and De Bortoli wineries, I found that the tasting notes which accompanied the wine I tasted were fairly accurate. The wine, for the most part, exhibited notes which were described in the tasting notes. The tasting notes guided me as to what notes to look for in the complex bouquet and taste of the wine, and it encouraged me to explore beyond those notes to see if I could smell and taste anything else. To me, they were useful and it was great to compare notes with the winemakers in relation to what they could smell and taste, and, what I could smell and taste. That’s where the usefulness of “expert opinions” seems to end.  

Moral of the story? While referring to tasting notes can be a fun and helpful way to explore flavour in a wine/whisky, science (and common sense) tells us that wine/whisky awards and scores are likely to be nonsense; subjective taste dressed as a numerical score and, as the above studies show, probably thought up at random. Your taste in whisky is therefore likely to be subjective. So, if you have forked out $10,000 for an award winning whisky which should cost $150 without the award, my condolences.