Levels of Quality
For me, there are only four levels of quality for projects:

	Top Level	High Technical Quality, High Appeal
	Second Level	Low Technical Quality, High Appeal
	Third Level	High Technical Quality, Low Appeal
	Fourth Level	Low Technical Quality, Low Appeal

At the top level, we have remarkable projects that excel for virtually every possible measure of evaluation. Examples: CASABLANCA, MONONOKE HIME, SANCTUARY (manga). Usually, it is pretty easy for reviewers to be consistent in their reviews at this level, and it is fairly simple for viewers to recognize the merits of these outstanding projects. Usually any discrepancies are either backlash against hype (see the reviews of the new STAR WARS movie), or the result of personal biases. For example, I strongly dislike Disney's BEAUTY AND THE BEAST because I am particularly sensitive to Disney's "Someday my prince will come" themes and class separation issues. Thus the movie never ceases to annoy me...
At the second level, we find projects that are not really all that great at all levels, but there are particular elements that allow an audience to embrace the project and enjoy it. Examples: BIO-BOOSTER ARMOR GUYVER, INDEPENDENCE DAY (ID4), CRAYON SHIN-CHAN, or THE A-TEAM. Titles in this list will vary from person to person according to their personal preferences, and personal biases rule whether titles fall into this category or the fourth category. For example, while I really get bored by the GUYVER, I have met so many teenage guys who were enthralled by it that I can easily justify its inclusion in this category despite my own temptation to drop it into the fourth category.
In the third level are projects that are technically sound, but yet never seem to click, or are seriously flawed in composition such that their overall appeal is missing. Examples: AMBASSADOR MAGMA (animation) JOHNNY MNEMONIC, and WATERWORLD. Somewhat similar to the second category, the titles in this list will vary from person to person according to their personal preferences, since one can place weight on certain technical aspects over others. For example, one could easily justify including INDEPENDENCE DAY (ID4) in this category if they consider the technical merits of the special effects and simply didn't like the film. Personally, I thought it was fun, but since I am story oriented, I cannot ignore the technical merits of the script, which was more plot holes than plot, and thus I put it in the second category. The weight of the technical aspects is very much like a teeter totter, with the personal biases of the reviewer determining what technical aspects will be considered and how much weight is given to each.
Another example would be GIANT ROBO. I would include the Japanese version in the top level, but I would put the English dubbed version in the third level because I feel the mocking tone of the voice actors completely distracts and undermines much of the dramatic effects from Imagawa's marvelous direction. Thus my personal opinion weights this one aspect heavier than all of the other technical aspects and lowers my opinion of its appeal.
The fourth level contains the crap. Examples : VIOLENCE JACK, anything by Ed Wood, etc. While there are those who develop an affinity for bad movies, the people who take pleasure from rotten projects usually know that they like trash and know that they move these projects from level four to level two on personal biases alone.
The different levels can be interpreted in many ways, and many levels basically boil down to two generalizations - 1) either you liked a project or you didn't, and 2) your opinion was based on personal biases or technical merit. If reviewers can break down their review, others will be able to tell if their opinion of the technical merit is similar, or if their personal biases are similar, and thus can evaluate how their opinion of the project will compare with the review.
Indeed, a good portion of my inspiration for this column was generated from a pair of magazine reviews for two recent anime titles: FUSHIGI YUGI and PANZER DRAGOON. The reviewer, quite justifiably, gave FUSHIGI YUGI poor marks for its animation quality, but did not give it any credit for its striking character designs and its excellent use of SD characterizations for comical effect. Meanwhile this same reviewer gave PANZER DRAGOON stellar marks for its animation quality without pointing out the fact that the computer generated backgrounds clash with the flat unremarkable 2D characters, the animation was choppy and of no better quality than FUSHIGI YUGI, and the direction was unenthusiastic and without any sense of tension which would have been necessary to instill a sense of drama in the show.
I could not disagree more with the reviewer's technical evaluation of the production, which kills the credibility of that reviewer in my mind. I cannot tell if the reviewer simply loved the computer game, was enthralled with the concept or story, or what. I can understand and accept any or all of these conditions, however, the reviewer did not identify those aspects of the program that were more in line with the actual product, and as a result I feel I cannot trust that person's reviews for a long time.
In my ideal world, reviewers would make this process clear and concise, but in reality, this is a rare exception. Biases, preferences, and expertise determine what a beholder will consider beauty, but if we know ourselves and present ourselves honestly and fairly, we can give others the ability to see through our eyes and evaluate beauty for themselves.