Phil Askey publishes reviews of digital cameras that are the most comprehensive on the Web. They typically run 10, 12, even more pages. Lots of sample photos and test data. They've helped guide my buying decisions.
Michael Reichmann also posts some excellent tutorals. They are not as exhaustive or systematic as Phil Askey's reviews, but they have the benefit of a widely-recognized pro photographer testing them and offering his professional judgment.
Those two models -- the reviews on DPReview and The Luminous Landscape -- are the models for the comprehensive software reviews on this site.
It's my hope that the software reviews on this site are the most comprehensive on the Web. If anyone knows of a site with more comprehensive reviews, I'd like the link so I can learn from it. This is a small community site that's growing, and just maybe the comprehensive reviews will be one of the features that sets this site apart from the others.
It would be nice to publish more software reviews. The problem is the time required. So far, every review has been a 20 plus page .PDF composed in Adobe InDesign with lots of screenshots. It is no exaggeration to say that it takes days to write such a review.
Let me give you an example. The most recent comprehensive review is for Alien Skin Bokeh. I downloaded a trial copy to get started while I waited for the review copy. I loaded the software, read the user manual, and experimented. Then I watched the videos on the Alien Skin site. It took me a couple of hours to decide on the test photos and how I was going to go about testing the software. Screenshoots took several more hours. Writing the review started in earnest early yesterday morning and ended around midnight. I think it would be fair to say that I had somewhere around 20 hours of work on the review. Half a work week. I hope the final result shows how much effort went into the review.
Most of the reviews I've read on the Web take a couple of hours to write. They draw heavily on press releases and advertising claims. Even the language in some cases is borrowed from those sources. There are a couple of strong motivations for superficial software reviews. The reviewer wants early access to free software. The publisher wants advertising from the manufacturer. Those reviews tend to be with flattering puffery. That stuff writes fast. I could have knocked out reviews of all of the noise reduction software in an afternoon or two.
Comprehensive reviews mean candid evaluations. I will tell you what gave me problems, what I especially liked, and what I'd like to see changed. You can also expect some photographer-to-photographer advice. Manufacturers read all of the reviews of their software. I use my comprehensive reviews to offer constructive feedback to them. In the Alien Skin Bokeh review, I was open with my suggestion to add a feature like the Depth Maps for the Photoshop Lens Blur filter. I also suggested a feature to apply noise or (better) a film grain effect.
My opinions are candid. I like to think they're informed judgments. You might find that your experience is different. Please, take advantage of the comment system and share your experiences.
I read this coment today on DPReview:
"I've been thinking about this plug in for some time now, but never got around to test it. I will now, when you basically made the manual for it."
I've seen something of this sort on a couple of forums, comparing my reviews to the user manual. That's the best comment possible!