Vintage bias ... is it real?

[Roy Hersh]

Having been at a Port tasting lately where the wines were served to us -- not blind -- I found that some participants comments, were right in line with the "prevailing perception" of that vintage, but in my opinion, did not necessarily accurately portray what was in the glass.

Some have called this "vintage bias." Do you believe it is REAL and do you think that it happens to you at a tasting if the bottles are NOT poured blind?

[Glenn E.]

Label bias... vintage bias... price bias... it's all the same thing.

To answer your questions, yes and yes. I'm pretty firmly convinced that even professional tasters are biased by prior knowledge of what they are drinking, though probably less so than Joe Consumer.

I don't think it matters for most friendly tastings, but I strongly believe that professionals should only be rating wines (of all kinds, not just Port) blind.

[Frederick Blais]

I agree with Glenn. Too often I've heard people saying this wine is ok, a bit dillute but not showing much. Oops the bottle was Lafite 96 a 100pts wine. So right after, they say oh the wine is shut and and in a bad phase. Though all the components are there to live well.

Come on!! You should find more answer when you taste blind. So many people and pro taste blind just for the fun of the game and then adapt their comments once the bottle is revealed. I think one main problem of tasting blind is this, we should try to focus more and take more than a couple minutes to really understand when you blind taste. Not 30 secs and move on.

[Andy Velebil]

Yes there is and will always be some bias.

Blind tasting can be very humbling but its by no means acurate all the time. I think a taster should at least know a couple basic things like the vintage, region, and type of wine (or at the bare minimum the last two). That is the only real way to acturately give an assesment about a wine. I'll use the 2000 Noval VP's as an example. I've had them several times and most recently they were firmly shut down and not giving up much at all. Had I not known they were 2000's I'd have probably given them a poor score ( i didn't score them this time). Thats not fair to the wine, as I know from prior experience is a great wine but just in a closed phase now for awhile. So while blind tastng is fun to try and guess, it really isn't fair to the rater as they have no prior baseline to compare it with.

[Roy Hersh]

Andy,

So would you say that the people who are supplying you with the scores that help to provide you with an opinion on Port or any wine ... that you've never had or may not have before your opportunity arrives to buy it ... would your answer be, yes or no, to: -- do you want that "critic" to be tasting blind or not? --

Just a yes or no answer would be really appreciated. I realize it will be tough to ignore the urge to repeat this part of your discussion:

I think a taster should at least know a couple basic things like the vintage, region, and type of wine (or at the bare minimum the last two).

but for the moment, please just give a "yes" or "no" to the question.

I am betting your response will be, "yes."

But if the answer is truly no, please do elaborate. :help:

[Moses Botbol]

Label bias... vintage bias... price bias... it's all the same thing.

To answer your questions, yes and yes. I'm pretty firmly convinced that even professional tasters are biased by prior knowledge of what they are drinking, though probably less so than Joe Consumer.

I'd have to agree with that.

I don't think it matters for most friendly tastings, but I strongly believe that professionals should only be rating wines (of all kinds, not just Port) blind.

Not sure I agree, as context is important to the overall judgement. We had a 1908 and 1970 Graham side-by-side. The 1908 was not far off in color or youth from the 1970. Knowing that this port was a 1908 rather than something from the 1960's makes what's being evaluated much different.

Expectations are part of the overall evaluation. A 92 point rating for a $20 of wine and $200 bottle of wine should be in the context of their respective type and vintage.

[Glenn E.]

Expectations are part of the overall evaluation. A 92 point rating for a $20 of wine and $200 bottle of wine should be in the context of their respective type and vintage.

I guess I look at the ratings differently. To me, a 92-point wine is a 92-point wine. Whether it costs $20 or $200 is irrelevant - the two wines should give equal amounts of enjoyment. If the ratings change based on expectations then they're useless. What happens to the rating when that overpriced $200 wine gets discounted to $100 or $50? Should its rating go up because your expectations are now lower?

To me that's silly - it means that the ratings are meaningless.

[Andy Velebil]

If there was an absolute then i would no have to say NO.
I'll give an explaination in a bit as I dont have time to write as long of a reply as i want to.

[Tom Archer]

it means that the ratings are meaningless

They can only be viewed as a rough indication, and many commentators, especially the best known, are unreliable witnesses when it comes to making sighted assessments of expensive wines.

The sentiment that 'It was expensive, so it must be good' is widespread, and many wines, most notably Champagnes, wear the Emperors New Clothes in abundance.

To demonstrate this, get a small crowd of people, and split them into two groups. Procure bottles of Smirnoff Red label, Blue label and Black label. Get one group to rate them sighted, and the other group blind. Tell your sighted subjects how much the bottles cost, in relative terms, and don't let on that you're trying to pull a stunt.

Almost everyone will agree that the Blue label is better than the Red label, but tasted blind, most people think the slightly more expensive Black label is the worst of the three. Expect most of those making a sighted assessment to place it top or middle.

- Try it!

Tom

[Roy Hersh]

Tom,

Remember your audience. A Port analogy might go over a lot better. :?



Glenn wrote:
I guess I look at the ratings differently. To me, a 92-point wine is a 92-point wine. Whether it costs $20 or $200 is irrelevant - the two wines should give equal amounts of enjoyment. If the ratings change based on expectations then they're useless.
I agree with your sediments, 100%.

[Andy Velebil]

Since you wanted an absolute, i said no.

There will always be bias in wine reviews, its human nature to have some bias in everything we do. But without some basic information about what is being tasted a wine review can be very misleading. Allow me to explain. When I had those two 1815 Ports last year they weren't the best in the world by anymeans. On their own, totally blind, I would have given them horrible scores as a Vintage Port. If I knew nothing about what I was tasting, only that it was a VP. however, knowing their age, I was able to asses them with that in mind. And while still drinkable, and when taking into consideration their age, my notes reflected certain information that reflect that. That is very important. We can argue the merits of that to death, but that is my opinion.

Like I mentioned earlier, I much prefer a blind tastings where a little is known. Such as vintage, or range of years, and type of wine/Port. That allows the reviewer to have some baseline as to how to asses certain aspects of the wine in relation to its peers. How do you compare a 1815 Port to a 2003 Port..you can't! Yes they are both Ports, but not even in the same league at almost 200 years apart in terms of structure, ageability, etc.

But even blind tastings can have bias. That blind bottle consumed sitting on the veranda of a Quinta, at night, as the sun sets will never taste that good again. Human nature associates the environmental aspects too, something no one can completely shut off.

I read in a Parker interview when someone asked him about his score scale and how he does it. Parker replied something to the effect of... there is no scientific scale, a 89 point wine just gives me a little more pleasure to drink than a 88 point wine. I thought that was the best answer i had ever heard.

[Tom Archer]

Remember your audience. A Port analogy might go over a lot better

OK, try a little '83 horizontal with people who are wine enthusiasts, but not port experts:

Nacional, Fonseca & Offley

There's a fair chance that the average scores from a blind tasting will show the opposite outcome to a sighted one..

Tom

[Andy Velebil]

Remember your audience. A Port analogy might go over a lot better

OK, try a little '83 horizontal with people who are wine enthusiasts, but not port experts:

Nacional, Fonseca & Offley

There's a fair chance that the average scores from a blind tasting will show the opposite outcome to a sighted one..

Tom

Reminds me of the time I snuck in an LBV into a VP tasting, blind. That LBV held up quite well much to the surprise of the drinkers later. :devil:

[Glenn E.]

When I had those two 1815 Ports last year they weren't the best in the world by anymeans. On their own, totally blind, I would have given them horrible scores as a Vintage Port.

Stop right there. You're done.

It doesn't matter that they were almost 200 years old - they weren't that good. Period. End of rating. Served blind, you would have arrived at a correct rating. But because you were influenced by their age, you fudged your rating and ended up with one that is ... well, fudged. It's not accurate.

If you fudge your rating, you're deceiving anyone who reads it and relies on it. Someone might buy those Ports thinking that it's going to be pretty good and be disappointed because it is, in your own words, they weren't the best in the world by any means.

Let the reader do the fudging, if the reader so desires. The reviewer should always be as honest, accurate, and precise as possible.

[Eric Menchen]

Ahhh, not quite a port story, but I am reminded of a great story from a family friend. He is an avid wine collector, and really knows his stuff. He was dining in a restaurant in France and was troubled by the wine list. He asked the owner why some of the best wines were so cheap, and yet the lesser wines were priced more expensively. The owner explained that the prices were for the tourists that bought wines based on price, and that he was rewarding the locals that presumably knew better with the better stuff.

[Moses Botbol]

When I had those two 1815 Ports last year they weren't the best in the world by anymeans. On their own, totally blind, I would have given them horrible scores as a Vintage Port.

Stop right there. You're done.

It doesn't matter that they were almost 200 years old - they weren't that good. Period. End of rating. Served blind, you would have arrived at a correct rating.

It’s not as easy as that. What about decant time? Allowing for the correct decanting time is an indication of vintage and is now no longer “blind”. Many 100+ year old ports look the part as does 5 year old port. Just that alone does not make port “blind”. One has to recognize what they are drinking, and how it is being presented to be fair.

[Andy Velebil]

Glenn,

I disagree. If someone would have given those two to me and said this are 1985 VPs. Yes, that would have been a horrible score for an '85, as no 1985 VP should ever be like that. But they wern't 23 years old they were almost 200 years old. That does have a lot to do with giving an acurate review of those specific wines. At 200 years they were still drinkable, which is no small feat in itself.

Here in lies the problem with any score based system...people tend to only look at the score. They don't read the actual text that the reviewer wrote. My TN reflects that information so the reader will have an understanding of what I drank, what it was like, and then they can form their own opinion as to wether they think they would like it to.

How many times have you had one bad bottle out of a case. You give that one bottle a bad score. Yet from then on people only look at that one review/score and think to themselves, "That port must suck, he only gave it xx points." The same applies here. If they read the text they will then understand.

[Moses Botbol]

Here in lies the problem with any score based system...people tend to only look at the score. They don't read the actual text that the reviewer wrote. My TN reflects that information so the reader will have an understanding of what I drank, what it was like, and then they can form their own opinion as to wether they think they would like it to.

Yup. I agree with Andy.

[Glenn E.]

I disagree. If someone would have given those two to me and said this are 1985 VPs. Yes, that would have been a horrible score for an '85, as no 1985 VP should ever be like that. But they wern't 23 years old they were almost 200 years old. That does have a lot to do with giving an acurate review of those specific wines. At 200 years they were still drinkable, which is no small feat in itself.

No small feat, true. But totally irrelevant to the rating. There's a difference between a rating and a review.

200-yr old bad Port is still bad Port. 2-yr old good Port is still good Port. Rate the Port based only on its quality. Then, once you've arrived at a proper rating, add additional notes based on its age, appellation, or producer if you think it is appropriate. But don't change your rating based on those attributes. It may be more impressive that a 200-yr old Port can still achieve a rating of 85 points, but that doesn't mean that it deserves a rating of 90. It's still an 85-point Port... it's just a 200-yr old 85-point Port.

The argument I presented before based on price/expectation works just as well based on age. Let's say you taste some Port when it is 20 years old and think it deserves a rating of 90, then you bury it in your cellar and leave it there for 50 more years. (And we develop a longevity pill so that we can be around for the 2nd part of the test. :) ) Let's say it tastes exactly the same 50 years later.

What you're telling me is that you would rate the Port higher at 70-yrs old than you would at 20-yrs old even though it tastes exactly the same, simply because it is 50 years older at that point. To me, that's crazy.

Here in lies the problem with any score based system...people tend to only look at the score. They don't read the actual text that the reviewer wrote. My TN reflects that information so the reader will have an understanding of what I drank, what it was like, and then they can form their own opinion as to wether they think they would like it to.

How many times have you had one bad bottle out of a case. You give that one bottle a bad score. Yet from then on people only look at that one review/score and think to themselves, "That port must suck, he only gave it xx points." The same applies here. If they read the text they will then understand.

I understand all that, and agree with what you're trying to accomplish. I just disagree with the score you're presenting. Everything you say is just as relevant if you present the real rating.

I think you should present your real rating, then explain why that is impressive (or not) in your notes. I think it is misleading to present a fudged score and then explain why you fudged it in your notes - for exactly the reason you mentioned. If someone only reads the score and makes their purchasing decision based off of that, then the score alone needs to be accurate. If you've adjusted the score (either way, it doesn't matter which) then some people will be buying the wine based on an inaccurate rating. Reward the people who read you notes, don't deceive the people who don't.

As far as the bad bottle goes, if you think you have a bad bottle you shouldn't rate the Port. Obviously you can't always tell when that's the case, but that's why people shouldn't buy Port (or wine) based on a single rating (and also why they should always read the notes).

[Glenn E.]

It’s not as easy as that. What about decant time? Allowing for the correct decanting time is an indication of vintage and is now no longer “blind”. Many 100+ year old ports look the part as does 5 year old port. Just that alone does not make port “blind”. One has to recognize what they are drinking, and how it is being presented to be fair.

Well... decant time can be hidden. We're talking about professional tasters here... it should be pretty trivial for them to set up a truely blind tasting on a regular basis that allows for someone else to do the decanting so it can be hidden.

As far as the looks go... well that's part of the rating isn't it? Is that an ancient Port, or just an example of a lesser producer's effort from a bad year? At any rate, the visual characteristics of the Port are part of its rating, are they not?

Note that I'm really only saying that the rating and initial notes should be done blind. I don't have a problem with there being a reveal after the rating is done and the initial notes have been taken, and then additional notes added (but no changes made!) to reflect the taster's opinion after the reveal has taken place. That information is also useful to me as a consumer provided the initial rating and notes are intact. If the reviewer recognizes after-the-fact that the Port is probably just in a closed phase, that can then be indicated in the post-reveal notes and the reviewer can provide an estimation of how much higher it might climb after a few more years. But I don't want that adjustment included in the rating, because if I'm buying the Port right now I need to know what it's like right now. Besides, that fudge factor for future development is just a guess - albeit an educated guess - and may not ultimately be accurate.

There's one other element of a rating/review that is very important but which we haven't discussed - the date. Since I'm asking that Port (and wine) be rated blind, I also need to know when the tasting took place. As others have mentioned, a Port may get a poor rating because it was in a closed phase... to me that's fine, because at the time it was rated it wasn't that great. But I need to know when I see that review that it was done 8 years ago, because I can then make my own evaluation of how up-to-date that review might be. I may need to look for more recent reviews before I make my purchase, or I may need to read the reviewer's notes to see if he or she provided an estimated adjustment for future development.