Getting Blind With Crafty: How We Judge – 2021 Update

February 7, 2021, by Crafty Pint

Getting Blind With Crafty: How We Judge – 2021 Update

We've been conducting blind tastings at The Crafty Pint since late 2012, both in-house and as public events across the country. They're always a lot of fun, frequently enlightening, and provide a great workout for the senses.

As we get them underway for 2021, we've tweaked the way we source and judge the beers. Not significantly, but enough to make it worth updating our guide to how we approach each tasting for those keen to know.

Sourcing The Beers


In the past, we always tried to source as much beer as possible from retailers using a three-stage process. Given much of our audience is the beer-drinking public, we figured buying as much as possible from bottleshops helped make the beers in our lineup representative of those readers would be able to buy.

We'd contact highly-rated indie bottleshops and source as many from them as possible, occasionally fill gaps with beers from major retailers, then contact breweries directly if we weren't able to locate any in retail in Melbourne.

However, the landscape has changed significantly in recent years. Far more breweries are packaging beer and fewer, as a percentage of the overall industry, sell their beer in Victoria, not least due to the ever greater focus on local. So, in order that we're not excluding breweries without stock in the state where The Crafty Pint is based, we decided to start sourcing beer directly from the breweries themselves.

We appreciate this means in some cases we'll be sampling fresher stock than readers are likely to find in their local bottlo, but figure, on balance, that keeping the tasting open to as many brewing companies as possible is preferable.

We still encourage everyone to make sure they check the date stamps – whether that's packaged on or best before – whenever buying beer. In the case of limited release beers and when buying from smaller stores with fast turnover, it's less of an issue but next time you're in a larger retailer take a look at a few cans of the same beer and the chances are you'll find multiple batches brewed weeks, if not months, apart.

When publishing the results, we'll indicate where each beer was sourced as well as the packaged on and best before dates, where applicable.

How We Judge


The other main change in the way we conduct our tastings as of 2021 is the introduction of a new column on our score sheets. We've called it "Structure", a catch-all term for the way the beer presents beyond appearance, aroma and flavour. Here, we ask panelists to consider elements such as a beer's balance, finish, mouthfeel, and so on.

Often, a good beer becomes a great beer when the brewer has crafted one in which all the disparate bits come together as a perfectly structured whole. You might pick up a beer that smells and tastes amazing, but if it's flat or flabby, slick or sticky when it shouldn't be, the enjoyment factor falls away.

Here's the judging process in full:

  • Once we've selected a style, we draw up a long list of Australian versions. If that ends up with more beers on it than is feasible for a single tasting, we'll whittle it down to a manageable number, taking in factors such as recent award-winners, achieving a spread of entrants from across the country, and occasionally putting some of the spots up for a public vote.
  • Once sourced as above, they're chilled to the appropriate temperature for the style.
  • A panel of beer experts from different areas of the industry – for example, brewers with competition judging experience, brewery reps, beer writers and educators, retail staff, those with Cicerone certification – is invited to take part.
  • Stewards present the beers in flights of three. Panelists have no knowledge of the lineup or the order in which beers will be served.
  • Beers are assessed and marked in five areas: appearance (scored out of 3); aroma (out of 6); taste (out of 6); structure (out of 5); and overall (a mark out of 10).
  • The overall score is the area in which our process differs most from official judging competitions. Because we're not awarding medals but trying to offer something of meaning to consumers as well as the brewers involved, we don't want to overly punish a good beer just because it's not to style. Ultimately, your choice when buying a beer most often comes down to whether you like it.
    So we encourage judges to use the overall score to mark up a beer they've enjoyed that's not quite to style, or to mark down a beer that meets style guidelines but isn't enjoyable.
  • In terms of other advice to judges, because we're not awarding medals there's no need to all stick to the same level of generosity or harshness in scoring. All we ask is that each participant is equally harsh or generous across all beers.
  • Each flight is judged in silence (at least there's no discussion of the beers) until all judges have completed their scoring for that flight. In this way, no one person can try to influence the others. We will discuss each flight after all judges have completed their scores in case there are major disagreements or anyone wants to sample any beer again.
  • We ensure we know which bottle or can has been served to each judge in case there is discrepancy in scoring over a particular beer. We have found two beers from the same brewery from the same store to present very differently in the past.
  • Once the judging is finished, we total the scores and get into the best part of the day: the reveal from last to first (usually with a glass of a beer that's as far from the style we've spent the previous couple of hours assessing in our hands!).

The Results


On occasion, the tasting will be accompanied by a piece on the featured style, offering background and insight as to what a particular beer style is, its history and so on. And, once completed, we publish a results article.

In the early years, we listed the beers in order right down to last place; from 2015 onwards we took the decision to list only the top half or so in order, with the remaining beers listed alphabetically. We felt it was better to focus on those that did well while offering to provide feedback to brewers on those that did less well in the hope it proves useful for future brews.

These are fun exercises we take very seriously. Indeed, if you've never attended or carried out your own blind tasting, we highly recommend anyone with an interest in beer gives it a go. It's fascinating to see how beers perform once you take away the branding and preconceptions; as one panelist said after her debut with us: "Blind tasting is such a trip!"

Find past style-based articles here and all past blind tasting-related features and results here. If you've a style you'd like us to feature, drop us a line.

If you enjoy The Crafty Pint, you can become a supporter of our independent journalism.

You can make a donation or sign up for our beer club, The Crafty Cabal, and gain access to exclusive events, giveaways and special deals.

AIBA 2024 dinner B2 BONUS
Bintani- Updated Behind The Brew- E
Cryer E