... some personal considerations and opinions regarding "High-End Whisky" (with respect to price and rates):
I believe that the rating at "whiskybase.com" is quite an adequate guide for a first orientation before buying a test bottle. My strategy, if a sample is unavailable (in most cases, unfortunately), is to select a distillery whose profile I like anyway, then taking an average from ratings from whiskybase as well as from other reviewers whose taste fit well to mine in the past. Further very important is to read thoroughly the detailed notes of all reviewers to learn about the specific profile of the whisky.
At "whiskybase.com", the rating seems to fit quite well in maybe 80% of the cases if a certain minimum of votes is already available (should be at least 20-30 votes, the more the better). This does not sound self-evident because every whiskybase member can define his/her scheme to award a whisky quite individually. So for instance everyone sets another personal "standard" for a certain value for a perceived quality (whatever quality means to the member, it can be just the personal liking or the trial to rate some "neutral" properties), everyone spreads his personal scale in a different way, i.e. one member may use a wide scale between 50 and 100, another may use only a very narrow scale between 75 and 90. This leads to different statistical distributions with varying mean values and standard deviations (variances) being superimposed. From this it seems to be difficult to maintain a kind of normative result. However, practical experience shows that within a certain region (typically at least between 80 and 90 points) it works quite well. This is at least my personal feeling. The reason for this might be some statistical compensation of the outliers if a certain minimum number of votes contributes to the end result or some "swarm intelligence" where all members subliminally adapt a certain rating behaviour with more or less the same statistical mean and variance.
So, to a certain degree it seems to work. But I recognized (again my personal observation, only, of course) that this is only true for the range between 80 and 90 points. Especially when it comes to rates above 90 points, it seems that maybe expectation of something very special leads to exaggerated and not realistic verdicts. Maybe again a "swarm intelligence" effect (following trends), however this time counterproductive. For instance there might be a famous name or charismatic distillery, there might be a "lost distillery" like, e.g. Port Ellen or Brora, there might be a very old age (e.g. 40 y.o.), there might be the feeling "everything used to be better in the past" for whiskies bottled in the last century, there might be top (but still personal!) verdicts from renowned whisky gurus, there might be already a lot of good ratings to follow blindly, or there might be simply - for whatever reason - a high market price ("an expensive thing cannot be bad.") etc., etc. ...
This seems to induce some mysterious expectation leading to the rates above 90 points. I have tasted whiskies at 92, 93, 94 points, and yes, they were different than whiskies at 88, 89, 90 points. But were they better? My clear answer is 'No'. Not in general, maybe not even in average. I feel a clear quality slope between whiskies between 85 and 90, much, much more between 80 and 90, and an extremely strong slope between 75 and 90. But in the top range between 89 and 95 for me it seems more to be based on expectation and imagination.
The whiskies with >89 points are excellent in most cases, and having this excellence up one's sleeve the charismatic variety and individualism is quite high within this quality group. So we find a lot of different individual characters, but on par with each other if it comes to intrinsic quality. So, my impression is that within this high quality domain it is no longer sensible to compare these individuals by a pure number "xxx/100". Individual tasting notes, however, are still quite helpful to learn about the specific profile.
But even with the detailed descriptions of nosing and tasting one should stay careful. If you read and compare notes from different people, especially if they did not publish them at the same review site (where people first read and then write), the details can differ quite a lot. The associations arising from smell and taste can be quite different when it comes to a detailed description based on reference flavours, such as from different fruit, chocolate, coffee, herbs, and so on. The same phenomenon is known from, e.g., wine tastings, here blind tests uncovered that repeated recognition of the taste descriptions often fails, even by wine testing professionals and sommeliers. Even artificially coloured white wine was accepted to be quality red wine etc.. For example see the research results by Robert Hodgson ( http://www.spiegel.de/spiegel/print/d-63947536.html ).
But let's quote another example of expectation-driven imagination and verdicts. This comes from a completely different field, the HiFi audio community where the enthusiasts are seeking for the best sound quality. Here, the prices for technical HiFi equipment range from small amounts below 100 Euros to prices for so called high-end equipment for more than 100,000 Euros (e.g. for amplifiers). Enthusiasts even pay thousands of Euros for a simple electrical interconnection cable with special "voodoo" design (e.g. silver coated or with other special features). From pure technical perspective, all these cables have - with respect to the relevant audio frequency range - exactly the same transfer properties, so it does not matter if it is a simple wire for one Euro, only, or if it is such a special luxury wire for >1000 Euros, the sonic quality will always be equal. The same is valid for other kind of HiFi equipment, e.g. some HiFi enthusiasts believe in the extraordinary excellent sound quality of expensive mechanically damped Compact Disk drives for 10,000 Euros, however each CD player has a digital buffer chip that collects all bits and repairs all faulty bits by a software algorithm. So physically there cannot be any difference in sound quality between a simple CD drive for 100 Euros and a high-end drive for 10,000 Euros (here I do not speak about D/A converters which may differ in sonic quality). Even the professional HiFi magazines rate all different HiFi components by a sonic quality verdict with points, often between 0 and 100 points (in practice using only a sub-range). Within numerous blind tests it could be proven that all these rates, differences and points for certain groups of HiFi equipment are complete nonsense. For example the team from http://www.hifiaktiv.at.xserv08.internex.at/ has shown that most of the ratings for electrical cables, transistor amplifiers and CD players are complete nonsense.
Nevertheless, still many HiFi enthusiasts believe in the sonic differences based on the ratings from professional HiFi magazines. They seem to hear the sonic quality differences between 0...100 points (respectively within the relevant sub-range). And this comes from pure psychological imagination, in reality all these sonic differences do not exist!
0...100 points, doesn't this sound quite familiar to us whisky enthusiasts?
So what about real quality differences and pure human imagination with regard to tasting whisky? Yes, they all taste different, no doubt, but what about the intrinsic quality level, so-called complexity level and so on???
Just think about it ...
... it may put some of all these tasting verdicts, notes, ratings and quality comparisons into perspective ... ;-)