Reviewers and Objectivity

Discuss products and their reviews in Genii.
Marty Demarest
Posts: 89
Joined: August 12th, 2011, 11:23 am
Location: Montana

Reviewers and Objectivity

Postby Marty Demarest » January 2nd, 2015, 10:29 am

Let me state up front that I am an admirer of David Britland's work, both as a reviewer for Genii and as a writer of his own projects. This post is prompted by his recent review of Ian Keable's book Charles Dickens Magician in Genii (January, 2015). But it addresses problems that I see elsewhere, and to a much greater extent.

It bothers me that David elected to review a book to which he had contributed in some capacity. This is something that I see happening in the magic press, and it undermines the usefulness of the journal in which it appears, and the credibility of the reviewer.

I understand that the magic world is small, and that a good reviewer is someone who is well-posted and widely acquainted with the world of magic. It stands to reason that reviewers would often have some relationship with the person whose work they are reviewing. I don't think that always needs to be pointed out. But in cases where the relationship is notable, I think it should made explicit. A good example of handling this situation appropriately would be Will Houstoun's review of Justin Higham's book The75% Production and The Trick with No Method in Genii (August, 2014). There, Will clearly states his relationship with Justin--which is extensive both personally and professionally. He also clearly states his relationship with the work under review--which was nothing. He also asserted his objectivity in that case, which was appreciated. In a single sentence he gave me all the tools necessary to interpret his review.

However, when a reviewer has contributed to the work under review--even at an early stage, even if his contributions were not in the final product--then I believe the reviewer should recuse himself. At least David is to be commended for not simply giving the Dickens book a rave or a pan--he displays discernment, and he explains his involvement with the book. On the other end of the spectrum would be, for example, the reviews for Revelation by Jamy Ian Swiss in Genii (July, 2008) and Jason England in MAGIC (July, 2008). Both Jamy and Jason were thanked personally for their contributions in the book's introduction. Yet, in their respective reviews, neither critic mentioned that they had been involved with the book. They simply gave it raves.

That sort of thing is misleading at best, and dishonest at worst. It certainly makes me question the integrity of the magazines I'm buying, and the intent of the reviewer.

I don't believe it is the duty of the magazine editor to police this, either. A reviewer should have the ethical sense to recuse himself if he knows that there is--or might be perceived to be--a conflict of interest. This is standard journalism stuff. Indeed, many organizations go further. When reporting on a company for the radio show Marketplace, for example, I was forbidden to have traded in stock for that company in a window of time that extended before and after the report. Again, I understand magic is a smaller world with smaller stakes, and such measures would be draconian. But I do expect some degree of integrity from the reviewers and the magazines that purport to offer me an objective assessment.

User avatar
Richard Kaufman
Posts: 23859
Joined: July 18th, 2001, 12:00 pm
Favorite Magician: Theodore DeLand
Location: Washington DC
Contact:

Re: Reviewers and Objectivity

Postby Richard Kaufman » January 2nd, 2015, 11:40 am

Marty, our world is much too small to adequately deal with this issue. Once you reach a certain level in our field, people tend to know the other creative people to some degree. It comes down to trusting the integrity of your review staff. I know every one of them personally and for many years. All are of the highest integrity. No one has any reason to boost a friend's product--that is not in their best interest.

In the specific case of David Britland, I know that he would simply not have reviewed the book if it was crap. This is true of all of my reviewers, because I've had this very conversation with them: if the item they're reviewing is something created by a friend, or something in which they were involved to some degree, they will not review it if it's crap (that will fall to another person on the staff), or will give it an honest review if it's good.

In my own case, for example, I know every person who reviews books for every other publication. In many cases they are friends.
Subscribe today to Genii Magazine

User avatar
Dustin Stinett
Posts: 6762
Joined: July 22nd, 2001, 12:00 pm
Favorite Magician: Sometimes
Location: Southern California
Contact:

Re: Reviewers and Objectivity

Postby Dustin Stinett » January 3rd, 2015, 1:48 am

Marty,

You know I love ya, but you of all people should know that reviews, by their very nature, are not "objective."

They certainly can and do have objective aspects: For example, when I point out sound-flawed segments of DVDs or when a credit is left out or incorrectly attributed. But for the most part, all reviews are subjective because we, as reviewers, are expressing our opinions.

Opinion has no place in an objective report or study. That's a hallmark of true journalism (and what got me disqualified from working on a school newspaper because I refused to add my "journalistic point of view"--read opinion--to hard news stories).

But a review is not a hard news report. It's an opinion piece. Hopefully people reading it know that.

I do agree with--and I always try to practice--full disclosure as needed. But I am a human being and my inherent biases are going to shape my reviews. I cannot help but wonder if trying to prevent that would not throw the pendulum too far to the other side. Hopefully people reading our work will have come to understand our likes, dislikes, and indeed the biases that all reviewers have. That's where the trust comes in.

Dustin

Marty Demarest
Posts: 89
Joined: August 12th, 2011, 11:23 am
Location: Montana

Re: Reviewers and Objectivity

Postby Marty Demarest » January 3rd, 2015, 12:10 pm

Richard, thanks for the explanation. I do understand the situation, especially with regards to the size of the magic world. And I must add that by maintaining this forum and allowing for the open expression of opinions, you go a long way toward mitigating any flaws in the system.

I have no problem with someone reviewing a friend's book, provided they make that relationship clear to me as a reader. (There is a big difference between you trusting your reviewers, and me trusting your reviewers.)

Dustin, I agree with you--every report is subjective, and every review especially so. That's not what I'm talking about. I'm referring to when the object under review (say a book) is the work of the subject (the reviewer). In that case, the subject is essentially reviewing the subject. Gazing into the mirror of self-regard. In such a case, I don't trust the review. They can pat their own back on their own dime.

I don't think anyone should review any product to which they themselves have contributed. I suspect that Genii has a book reviewer on staff who didn't contribute to Ian's book. I believe they should have taken that duty, and David should have declined it.

Doomo
Posts: 347
Joined: March 13th, 2008, 3:23 pm
Contact:

Re: Reviewers and Objectivity

Postby Doomo » January 3rd, 2015, 12:31 pm

There is a point to consider in this discussion. For example, wallets... I make them... Design, and make... I have done so for years. I know a good bit about the subject. In theory I would be a good person to review them. But do I have too much personal bias? On the other hand I see reviews by unbiased people that are totally full of crap talking out their asses, if you will, about leather quality and workmanship and originality. When the product in question has none of those qualities.

But if I were to review a wallet and found it lacking I would be criticized for being biased.

On the other hand there are so few leather-workers (even fewer than coin workers) that I happen to know all the ones who do it. So a review by me could be construed as doubly biased.

Reviewing unless it is a fawning praise of an item is usually all that is desired. People say that they want honest reviews. But watch the uproar that a bad one will produce. Basically it is a very tricky issue. Most reviewers have no knowledge of craftsmanship. They mainly judge on packaging and high end video instructions.

This will probably not be a happily viewed comment. But oh well... I will be in my basement making indexes. So go annoy someone who tells ya how nice the latest download or piece of Thailand/Chinese tat is...
RFA Productions yeah... It is cool stuff.

www.rfaproductions.com

Jim Riser
Posts: 1078
Joined: January 17th, 2008, 12:00 pm
Location: Tucson, AZ

Re: Reviewers and Objectivity

Postby Jim Riser » January 3rd, 2015, 12:46 pm

I agree with Tony on this issue. Most reviewers do not know enough to really judge the merits of the materials used nor workmanship involved. If you have not gotten your hands dirty making the object, you are not actually aware of the variables involved in the manufacturing processes. I term such reviews as the "Antique Roadshow Syndrome" - reviews by those who can barely sharpen a pencil. Hence, I never submit any of my items for reviews. My guess is that folks like Tony, Todd, and me are more critical of our items than anyone else anyway.
Jim

Ian Keable
Posts: 31
Joined: March 16th, 2008, 5:47 pm

Re: Reviewers and Objectivity

Postby Ian Keable » January 6th, 2015, 4:26 pm

I don't want to comment on Marty's viewpoint with regard to the review of my book as my thinking has been that once you've written a book you should leave it to the readers to have their say; you've had your say in the book itself.

However when it comes to reviews generally, I have noticed that some reviewers are actually much tougher on magicians they don't know personally - even better if they have never heard of them before and (better still) don't live in the same country. In a previous book of mine, the only poor review I got was from an American who had never heard of me. He was actually confronted (completely independent of me) by some people who knew me (he happened to be in the UK for a convention) and asked to explain why he had given it a poor review. He did admit that he had probably been 'overly hard' on it and hadn't read it properly (he came in with a pre-determined standpoint). What I'm certain of, though, is that if the exact same book had been written by someone he knew who lived in the US, he would have taken more care in his reading. The truth is that it is much easier to criticise someone who you don't think you are ever likely to come across in person.

Something similar happened to a friend of mine who wrote a fairly esoteric but brilliantly researched treatise on an antiquarian book. The reviewer (from North American, he might have been Canadian) gave another book a rave review, after confessing that he knew the author well. And proceeded to give an indifferent, and misinformed review (in the sense that he had completely missed the point of the book) of my friend's treatise (who lived in the UK).

All that proves, I guess, is that a review can never be that objective - whether the reviewer knows the individual well; or doesn't know the individual at all - the review will still be subject to some sort of bias.

User avatar
Andrew Pinard
Posts: 362
Joined: March 13th, 2008, 11:13 am
Contact:

Re: Reviewers and Objectivity

Postby Andrew Pinard » January 6th, 2015, 7:05 pm

While I understand the difficulty of finding qualified people to write reviews, I have to agree with Marty on this one. There were several other people who are writing reviews (for Genii no less) that were eminently qualified to review this book (and not involved in the production of the book). I agree with both Tony and Jim that sometimes it is important for a reviewer to have actual experience in manufacturing an item to provide a qualified opinion on the physical qualities of an item, but here the item being reviewed is a book. For most readers, they don't care too much whether a book is saddle stitched, perfect bound or wrapped in cloth, bonded leather or actual leather (yes, there is a difference in the latter two). The reader is concerned with the content, whether it is written in an accessible and engaging style (and learn-able in a "teach a trick" work), whether or not the scholarship is valid and that the author not only knows the material, but brings something original to the work or the field of research.

That said, the review was well written, informative and helpful. It seems a shame that some might not purchase a product based on the fact that the review might be considered unjustified hype as the reviewer had a hand in the work. There are plenty of qualified people who would be interested in reviewing products and I would hope that the reviews editor would be very hesitant to accept (or solicit) a review from a contributor. It does not help the credibility of anyone involved (producer, collaborator, reviewer or publication).

It is difficult to be critical in a field as small as ours, but even though we be few, we should not relax our standards, instead we should be even more vigilant so as to maintain well-justified good reputations in an industry focused on deception.

Personally, I look forward to reading the work. I have interest and experience in performing, researching and writing about magic history which makes me the ideal target demographic... Hoping the work does well and looking forward to seeing how it compares with the Trevor Dawson book released a couple of years ago (please, let's not divert the thread to discuss that work; anyone interested in doing that please start a new thread and I will happily participate)...

User avatar
Richard Kaufman
Posts: 23859
Joined: July 18th, 2001, 12:00 pm
Favorite Magician: Theodore DeLand
Location: Washington DC
Contact:

Re: Reviewers and Objectivity

Postby Richard Kaufman » January 6th, 2015, 7:11 pm

It all comes down to integrity: do you trust our reviewers. If not, then nothing else matters.
Subscribe today to Genii Magazine

User avatar
Andrew Pinard
Posts: 362
Joined: March 13th, 2008, 11:13 am
Contact:

Re: Reviewers and Objectivity

Postby Andrew Pinard » January 6th, 2015, 7:38 pm

It is more than trust, it is also about value. A qualified, fresh set of eyes go further to provide a consumer's perspective on an item and provide the information they need to discern whether it will have value to them. Third party review with no prior personal interactions with the producers might not be fully possible in our tiny industry, but while many of your readers might know of Britland's high personal integrity, if even one person feels that a reviewer is too close to the subject, it damages the credibility of everyone involved. It is unfortunate that this perspective was not considered before the review went to press, but it might be helpful to look at this as a chance to revisit publication policy to avoid this type of apparent conflict of interest for future reviews...

User avatar
Richard Kaufman
Posts: 23859
Joined: July 18th, 2001, 12:00 pm
Favorite Magician: Theodore DeLand
Location: Washington DC
Contact:

Re: Reviewers and Objectivity

Postby Richard Kaufman » January 6th, 2015, 8:04 pm

"If even one person thinks ...?"

I don't operate that way. Anyone can "think" anything, for any reason, justified or not.

I trust my reviewers to be fair. They don't "roll over" for anyone.
Subscribe today to Genii Magazine

User avatar
Andrew Pinard
Posts: 362
Joined: March 13th, 2008, 11:13 am
Contact:

Re: Reviewers and Objectivity

Postby Andrew Pinard » January 6th, 2015, 8:30 pm

No one (especially me) is suggesting that Britland is "rolling over" for anyone. This is about editorial policy. If you are comfortable having contributors reviewing products that they were involved with, go to town. Just don't be surprised when people stop looking to your periodical for unbiased reviews. It's your sandbox. I know that I was simply trying to provide a perspective that might have a positive impact on your subscriber base. If you prefer to entrench and ignore the apparent conflict of interest it's your prerogative, but, as a result of this thread and your response, it will be harder for me to read the reviews in Genii without questioning whether or not there is something I am not aware of. It's not like you have to worry about me dropping my subscription (being a lifetime subscriber and all), but a cavalier attitude about a question of editorial policy betrays a distinct lack of concern for journalistic integrity.

Ah, it's just a magic magazine, what does it matter?!??

It used to matter and it should still...

Should I send you my review for those copies of The Yankee Magic Collector that are sitting in the Genii mailbox? It would certainly expedite the publishing process. Nope, that would be a conflict of interest.

Or would it?

Roger M.
Posts: 1244
Joined: January 17th, 2008, 12:00 pm

Re: Reviewers and Objectivity

Postby Roger M. » January 6th, 2015, 9:46 pm

They're magazine reviews, not legal briefs in a death penalty case.

Standard operating procedure with magazine reviews on any topic is to read them with a grain of salt regardless of who wrote them.

I'm always suspicious of an agenda when I see a post that's this personal ... but then I'm suspicious of everything I read, and rarely make purchasing decisions based on anything I haven't researched extensively on my own.

Brad Henderson
Posts: 3573
Joined: January 17th, 2008, 12:00 pm
Location: austin, tx

Re: Reviewers and Objectivity

Postby Brad Henderson » January 6th, 2015, 10:29 pm

The problem with reviewing products in which you have had a hand is you run the risk of reviewing what you think is there, not what actually is.

Jon Racherbaumer
Posts: 843
Joined: January 22nd, 2008, 12:00 pm
Location: New Orleans

Re: Reviewers and Objectivity

Postby Jon Racherbaumer » January 7th, 2015, 12:47 pm

This is a stimulating discussion. I have little to add even though I've worked both sides of "imaginary divides" between producers and consumers and those who review both players. What I tried to do in writing my later reviews was to comment on why I thought a book was GOOD and what I thought might be MISSING from it.

Back in 1969 I wrote the following ramble, knowing its limitations and naiveté but it was nevertheless an attempt to launch a more serious discussion on the topic.

13 WAYS OF LOOKING AT CRITICISM

I. Begin with a phenomenon: the Performance. Everyone in the audience is a critic. Criticism itself is a Performance, with its own audience. The best spectators can be the best critics.

II. Northrop Frye: "A public that tries to do without criticism, and asserts that it knows what it wants or likes, brutalizes the arts and loses its cultural memory... The only way to forestall the work of criticism is through censorship, which has the same relation to criticism that lynching has to justice.”

III. Performers ignore critics or refute them, as though they were natural enemies. Both are part of the Total Performance, extending beyond the fringe, the falling curtain, picture fade-outs and dissolves, and the printed Finis. The audience is between, their vicarious brothers.

IV. R. Buckminster Fuller has written: "The sum of the relationship of all our experiences is always tetrahedronal." Elsewhere: "What we mean by understanding is: apprehending and comprehending all the interrelationships of experience. Understanding is symmetrically tetrahedronal." Extraordinary statements! Critics, officially appointed and credentialed to be responsible, should be interested in Understanding. They should be aware of relationships existing in the subject being examined. The coordinate system of the Performance is omni-dimensional. The interpretation-presentation should be equal to the event.

V. The Greek krisis has adjective, kritikos, meaning "able to discern - judge - discuss, " hence a critic, hence the Latin criticus, hence the English words—critic, critical, criticism, and criticize. Denotatively we are stuck with—1. A person who judges, evaluates, or criticizes.

VI. Most of the time criticism is wholly emotional and characterized by one-dimensionality, harshness, triviality, and captiousness. The eye-for-an eye school, saying, "Who are you to talk, you sonufabitch?" Criticism in our "field" (a dubious word) is usually instigated, sustained, and subdued by emotion. This energy is modified by degrees of maturity, knowledge, and intelligence. "Card tricks suck!”

VII. "To say that one does not like something is not to say that one hates it; it is only to say that one is indifferent to it, or wishes it were not there, for it may occupy the space that could be filled with something one positively likes.” This is Allen Tate speaking.

VIII. "Criticism is an indefinite set of devices for 'presenting' not 'proving' the merits of works of art. It has none of the stability of logical truth, scientific method, legal and moral law. It varies with time, place, and audience, while not being subject to these limitations.” This is Margaret Macdonald speaking.

IX. George Johnstone, half-hip and breezy columnist for The New Tops is very fond of quoting Channing Pollack's: "Critics are legless men who teach running.” Johnstone is an entertainer. He is the critic's enemy: the other side—or should we say?—another side. To his quote add Cyril Connolly's: "The entertainer suffers from no criticism whatever…The fate of the entertainer is simply to go on till he wakes up one morning to find himself obscure.” Critics, too, wake up and find themselves obscure. This is the history of Total Performance.

X. Many times, particularly if we feel like we are under attack, criticism is associated with denunciation. A put-down is not subversion. A put-down is polarization. Critic with Performer, not Critic versus Performer. Two performances, two presentations. Values, evaluation. Thoughts, afterthoughts. A marriage and metaphor that can be extended as far as you can imagine.

XI. As critics, we should describe actions rather than report or impute feelings. Taste, however, is not always rational. Is enjoyment analytic? Is applause an equation? These are performer's questions, even though they do not seek their answers. Critics are the unwanted lawyers and philosophical relatives of performers.

XII. Criticism is inescapable and indispensable.

XIII. Criticism is an aspect of loving. It has all the consequences; hence, one sees
negativity and aggression, submission and possession. Love is a creative act, transcending aggression, yet transcendence cannot be conceived apart from the expression of it.

Et cetera.

User avatar
Brad Jeffers
Posts: 830
Joined: April 11th, 2008, 5:52 pm
Location: Savannah, GA

Re: Reviewers and Objectivity

Postby Brad Jeffers » January 7th, 2015, 3:22 pm


Ian Keable
Posts: 31
Joined: March 16th, 2008, 5:47 pm

Re: Reviewers and Objectivity

Postby Ian Keable » January 7th, 2015, 3:48 pm

Just to pick up on a couple of points from Andrew Pinard's very erudite postings.

There were several other people who are writing reviews (for Genii no less) that were eminently qualified to review this book (and not involved in the production of the book).


Not sure if this is entirely true - given that Eric Mead reviewed Trevor Dawson's book on the same subject for Genii and actually used my own internet review (now removed, as my book over-rides it) as the basis of his review. In other words he quite understandably did not have enough knowledge about the subject (you had to be familiar with Dickens's life, all of his writings - not just his novels - and previous writings by magicians on Dickens) to properly assess the book himself. Without wanting to appear elitist, I don't think there was anybody sufficiently knowledgeable in the magic world of evaluating Trevor Dawson's book properly (with perhaps the exception of Eddie Dawes) - as was proved, in my opinion, by all the other reviews that appeared in the magic press. Even eminent magic historian, Dr Michael Claxton, an associate professor of English, admitted that his review of Trevor's book fell short.

Hoping the work does well and looking forward to seeing how it compares with the Trevor Dawson book released a couple of years ago.


Thank you for that kind endorsement; and I would of course be delighted if people would read my book and compare it with Trevor's. As they will discover my take is completely different from his. I am not shy in stating why I think Trevor, and indeed other magicians, including Ricky Jay, who have written on Dickens, have got it wrong.

Ted M
Posts: 900
Joined: January 24th, 2008, 12:00 pm
Favorite Magician: Dani DaOrtiz
Location: Madison, WI

Re: Reviewers and Objectivity

Postby Ted M » January 7th, 2015, 4:23 pm

Brad Henderson wrote:The problem with reviewing products in which you have had a hand is you run the risk of reviewing what you think is there, not what actually is.


I think Brad has his finger on it. I trust the Genii reviewers to review what they see, but it's difficult to really see one's own work.

That's why the best magicians employ directors, after all.

Bill Mullins
Posts: 4494
Joined: January 17th, 2008, 12:00 pm
Location: Huntsville, AL

Re: Reviewers and Objectivity

Postby Bill Mullins » January 7th, 2015, 5:21 pm

Ian Keable wrote: As they will discover my take is completely different from his. I am not shy in stating why I think Trevor, and indeed other magicians, including Ricky Jay, who have written on Dickens, have got it wrong.


I've ordered your new book (haven't received it yet) and I assume you have expanded on this there.

However, I've always thought that Ricky Jay's scholarship was pretty reliable. As far as I can tell, he's only written in any depth on Dickens in his Celebrations of Curious Characters. I don't see anything there that is at odds with your most recent article on Dickens in The Linking Ring.

How do you think Jay "got it wrong"?

Ian Keable
Posts: 31
Joined: March 16th, 2008, 5:47 pm

Re: Reviewers and Objectivity

Postby Ian Keable » January 8th, 2015, 3:16 pm

Bill Mullins wrote:

I've ordered your new book (haven't received it yet) and I assume you have expanded on this there.


Thanks for that - the book has indeed been sent; I'll email you separately about that.

How do you think Jay "got it wrong"?


The book explains exactly where Ricky Jay, in my opinion, 'got it wrong'; so I'm not going to repeat my findings here. However I would say that apart from Curious Characters, Jay also wrote quite extensively about Dickens in the first issue of Jay's Journal of Anomalies, later published in book format.

Bill Mullins
Posts: 4494
Joined: January 17th, 2008, 12:00 pm
Location: Huntsville, AL

Re: Reviewers and Objectivity

Postby Bill Mullins » January 8th, 2015, 4:29 pm

Thanks. I've got Journal, and didn't examine that. I do so soon. I look forward to getting your book.


Return to “Light From the Lamp”