R/E/P Community

Please login or register.

Login with username, password and session length
Advanced search  

Pages: [1]   Go Down

Author Topic: Why Most 'Mic Shootouts' Are Fundamentally Flawed  (Read 6194 times)

klaus

  • Moderator
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2203
Why Most 'Mic Shootouts' Are Fundamentally Flawed
« on: June 28, 2020, 01:10:37 PM »

I hate them. Never listen to sound samples, and never in my life would I evaluate microphones that way. Not even for clients who beg me.

1. Sound samples recorded with MP3 or other lossy formats obliterate the subtle characteristics we cherish in a good microphone- characteristics we reward by paying disproportionate financial premiums for that last bit of quality.

2. The intended audience is never present when these 'shootouts' are made. For all we know, totally different mics, preamps, cables, distances were used that may favor one mic over the other, regardless of a mic's true merit as recording tool in a real-world working environment.

3. Speaking of distances: here is the most overlooked and biggest flaw of shootouts - a flaw so big, it will render any comparison between two or more mics pointless and inconclusive: to position several mics at the same exact distance from the sound source.

We see pictures of totally different microphones with totally different sensitivities and timbres being meticulously lined up so that their capsules were aligned within millimeters - all in service to the illusion of scientific objectivity. As if we didn't know that every mic has its own sweet spot- a factor recording professionals pay close attention to. Who would place an SM58 and a U47 at the same distance from the singer?


There is science and then there is stupidity.

Logged
Klaus Heyne
German Masterworks®
www.GermanMasterworks.com

BluegrassDan

  • Newbie
  • *
  • Online Online
  • Posts: 29
  • Real Full Name: Dan Boner
Re: Mic Shootouts Online Are Fundamentally Flawed
« Reply #1 on: June 28, 2020, 05:35:56 PM »

The SoundPure videos are the worst. They start out by saying how crazy a person must be to even want the vintage piece of equipment in their test. Little hints throughout like “thankfully, the original LA2A didn’t malfunction during our test.”
Logged
Dan Boner
Professor/Program Director
Bluegrass, Old-Time, and Roots Music Studies
East Tennessee State University

David Satz

  • Full Member
  • ***
  • Offline Offline
  • Posts: 179
Re: Mic Shootouts Online Are Fundamentally Flawed
« Reply #2 on: June 29, 2020, 01:41:26 AM »

Klaus, I agree strongly with your critical opinion of "shootouts" (a terrible name, too), but my reasons are different from yours.

First, though, they can be good for a few things. If you have two samples of the same microphone from the same manufacturer, and one is known to be in good condition, you can check out the other one for problems. You can also get a "data point" regarding consistency of manufacture if both mikes are new. If the mikes aren't the same exact type but are variations on the same design, it may be possible to get at least a tentative idea of their objective similarity or dissimilarity.

When two microphones with distinctly different characteristics are compared, though, anyone's choice of which one sounds better in a given situation and setup--source(s), room, placement--really says very little about what that same person would prefer in any different situation or setup. And that's true whether both microphones are placed at a common location or some engineer chooses the location that supposedly sounds best for each one. In my opinion your complaint about using a common location for mike placement is fair, but the alternative would be worse, with many more variables not "controlled".

There are also basic technical problems that spoil any comparison when not dealt with--and in most listening comparisons, these problems really have no solution--which in my opinion dooms the whole genre. Here are two big ones.

- Small differences in sound amplitude (say, less than 1 dB) are usually experienced by listeners as differences in sound quality rather than amplitude. When the program material is similar in timbre between two examples, the louder sound is almost invariably preferred no matter which order the two selections are played in. If two sounds don't have similar timbre, though, all bets are off. There's simply no effective way to match levels and avoid this problem of bias. This alone makes most "shootouts" nonsensical and misleading--false evidence that appears real.

- Differences in placement of just a few inches affect the sound that a microphone picks up, and so does the presence of any other nearby solid object. The designers of stereo microphones have to be very careful so that two matched capsules in the same microphone can sound the same as each other; the number of microphones that can get an equal "bite" of a sound field is always very small. "Shoot-outs" with large numbers of microphones are celebrations of ignorance.

Basically, when you listen to any recording but you weren't there when it was made, if you're not very familiar with the performer and the room, then you know what result was obtained but you don't know how it got that way. You don't know in what way the characteristics of the mike are balancing with or against some aspect of the source, the room, or the mike placement--a balancing act that's basic to being a recording engineer. Any two engineers typically get different results even when they use the same equipment in the same general way, since so much depends on details that the engineer decides on by ear at that time and place.

In short, we never learn anything from these comparisons about the results we would get under any other recording conditions. And among real-world recording assignments, hardly any two sets of conditions are ever quite the same.

--best regards
Logged

Kai

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 436
Re: Mic Shootouts Online Are Fundamentally Flawed
« Reply #3 on: June 29, 2020, 03:34:43 PM »

Studio work is mic shootout all the time.

You set a mic, don't get the sound you're after, try another one.
Only in real world's practical use can a mic show its qualities, over time.

There are too many variables involved to arrange a test scenario with universal validity:

Today the saxophone sounds best with a Schoeps, next day the next sax player needs a Sennheiser MD421 - those mics are worlds apart in a "shootout", so are the saxes ;D

A well sorted mic locker helps.
Logged

Timtape

  • Full Member
  • ***
  • Offline Offline
  • Posts: 108
  • Real Full Name: Tim Gillett
Re: Mic Shootouts Online Are Fundamentally Flawed
« Reply #4 on: September 10, 2020, 12:37:20 AM »

It's easy to "impress" a client vocalist who knows nothing about microphones and production, that testing ten different microphones on his or her voice  - all this wonderful personal attention - which extra time the performer may well be paying for - is justified in the cause of producing a  superior product.

That the "expert" is searching for the mic that allegedly best "matches" this one voice seems superficially impressive especially if the vocalist doesnt know any better.

What is the evidence that it works or is the best method? And best method for achieving what exactly?

Tim

Logged

Timtape

  • Full Member
  • ***
  • Offline Offline
  • Posts: 108
  • Real Full Name: Tim Gillett
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #5 on: September 10, 2020, 12:55:00 AM »


3. Speaking of distances: here is the most overlooked and biggest flaw of shootouts - a flaw so big, it will render any comparison between two or more mics pointless and inconclusive: to position several mics at the same exact distance from the sound source.

We see pictures of several totally different microphones with totally different sensitivities and timbres being meticulously lined up so that their capsules were aligned within millimeters - all in service to the illusion of scientific objectivity. As if we didn't know that every mic has its own sweet spot- a factor recording professionals pay close attention to: who would place an SM58 and a U47 at the same distance from the singer?

I agree, Klaus. The SM58 is equalised for close vocals, the U47 isn't. At the same distance, they will sound very different. Used as per their respective user manuals, not nearly so different.
Logged

klaus

  • Moderator
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2203
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #6 on: October 24, 2020, 02:44:50 AM »

This is what I am talking about. Lining up the capsules of five mics in one plane, without any regard for the individual sweet spots which rarely if ever coincide.

Case im point on this picture: a dual-backplate mic design with considerable low-frequency processing (U67) will prduce an entirely different proximity effect than a single backplate mic with zero electronic frequency shaping of the low end (U47)
Logged
Klaus Heyne
German Masterworks®
www.GermanMasterworks.com

klaus

  • Moderator
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2203
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #7 on: November 13, 2022, 10:59:08 PM »

Two years after starting this thread people continue to distribute "shootouts"on the internet without documenting methodologies or identifying vital sound-shaping microphone components. Results are then consumed and commented on, again, with no inquiries into methodology or (sub)component details.
To sum up and expand on the fallacies of this approach:

1. Ignorance and incuriosity persists about the unscientific, untenable methodology applied to these comparison tests. Methods are never disclosed or detailed by the producers and never questioned by the consumers - an unscientific, wobbly foundation to forming a fact-based opinion or to making an informed purchase decision.

2. As prices for vintage originals (which are then often "shot out" against current-production copies of the same model) have skyrocketed in recent years, "shootouts" also have skyrocketed, which can give a potential buyer with limited financial means the illusion of being able to achieve sonic excellence on the cheap.

3. Test conditions are so far beyond any realistic setup and working experience encountered in professional recording scenarios that a microphone costing less than 1/10 of another one it is compared to, sounds somehow indistinguishable from it.

4. Lively discussions ensue about a 60+ year-old M49 having a "slightly preferable" midrange, compared to a brand-new M49V reissue, without verifying capsule version and condition, tube health, power supply voltages, and other vital checks. This is akin to comparing performance of two otherwise similar cars - one veteran, one brand-new, one worth millions, the other a mere fraction - without as much as opening the hood.


As demonstrated in my previous post with photo, disregarding the relevance and individual variation the proximity effect has on pressure gradient microphones does gross disservice when evaluating a mic's optimal distance from a sound source.

Here is another example of the "sin of equal distance" (and another example demonstrating the ultimate sin of compqring mics with different capsule topology at the same, unrealistic distance from the sound source): https://www.youtube.com/watch?v=BTUcnWDMPJs

What is also striking to me: true professionals working in successful commercial studios never participate in these amateur contests. It's usually people publishing these tests who have some sort of financial interest in the results, or buyers who have succumbed to confirmation bias: I own the cheaper copy, and, see? It's just as good!

So why then even bother talking about it?
In the absence of rigorous discussion outside of self-interest, and by continuing to ignore pleas to use a more scientific methodology when examining sonic impressions of high-end microphones, the current trend of sloppy mic comparisons will eventually take hold and become the new working standard.

It will cause headaches, heartaches and remorse down the road for those who took these tests seriously and made purchasing decisions based on them.
Logged
Klaus Heyne
German Masterworks®
www.GermanMasterworks.com

gtoledo3

  • Full Member
  • ***
  • Offline Offline
  • Posts: 143
  • Real Full Name: George Toledo
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #8 on: November 17, 2022, 11:20:20 AM »

One point about positioning the mics in the same place.

It doesn’t really reflect how someone would tend to position a mic in typical context. It does reveal differences with proximity effect, and possibly some other differences.

At GS, someone made a thread titled something like U47 Test 2. It may be the first time I’ve really engaged in one of these comparisons, and the main reason I did is to deflate the idea that all of the mics actually sounded like U47. It was quite easy to discern the brand of each mic, and a fair amount of that was because of the copy mics not having proper proximity response/sub bass extension. There were other discrepancies of course.

If a user just worked the mic as typical, the discrepancy in proximity effect would have been less clear because there would be another factor involved.

When it comes to recording, the proof is in the pudding. It is always fair to judge a mic by a recording created with the mic, it’s just that the judgement should be limited.
Logged

klaus

  • Moderator
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2203
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #9 on: November 17, 2022, 11:47:58 AM »

It is important to stress:

* Differences in proximity effect between microphones are magnified the closer you get to the capsule.

* Differences between capsules (and their processors which amplify the capsule's output) need to be recognized in comparison tests. Therefore, each mic should be positioned to its sweet spot, where the mic's proximity and other characteristics are subjectively perceived to be optimal.

But few testers seem to have the courage to involve what I find is paramount when evaluating microphones: making a decision at what distance a microphone sounds best. THEN you have something worth comparing.
Logged
Klaus Heyne
German Masterworks®
www.GermanMasterworks.com

Timtape

  • Full Member
  • ***
  • Offline Offline
  • Posts: 108
  • Real Full Name: Tim Gillett
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #10 on: November 25, 2022, 07:40:23 AM »

Some mic companies have printed graphs showing the lower frequency responses at various source distances, due to proximity effect. Also sometimes stated in the user instructions is a recommended singing or speaking distance such as 2" from the pop grille for an SM58.

Some users are able to determine this distance subjectively but it should normally be consistent with the manufacturers' data.
Logged

klaus

  • Moderator
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2203
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #11 on: November 25, 2022, 04:08:09 PM »

Can you share which mic companies include graphs based on realistic operating conditions?
Logged
Klaus Heyne
German Masterworks®
www.GermanMasterworks.com

Timtape

  • Full Member
  • ***
  • Offline Offline
  • Posts: 108
  • Real Full Name: Tim Gillett
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #12 on: November 26, 2022, 08:38:16 PM »

 I'm not sure if it was consistent even within one company but here's an example of the type of graph I was thinking of:

https://mynewmicrophone.com/proximity-effect/
Logged

Paul Johnson

  • Newbie
  • *
  • Offline Offline
  • Posts: 9
  • Real Full Name: Paul Johnson
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #13 on: April 07, 2023, 11:38:26 AM »

I rather agree with the idea here, despite having produced quite a few mic comparison videos.

Most of my videos don't try to create winners and losers, mainly because my opinion really doesn't matter. I even did a few videos where I bought the cheapest Chinese copies of mics - some were called U87s, and there was no point sticking one next to my real one - there would be no point at all, we know what the result would be, so I concentrated on showing practical things people could listen to.

I got a few comments saying there was no point in just talking into three or four mics, but distance, placement and aiming all impact - but Youtube is now so fixated on style - so everyone speaks into an SM7B, or other easily identified mic - so I used an SM7B, not remotely my favourite, but I used it as the starting point on every video, and then when swapped for the mics in each video, people may or may not hear differences in tone, or sometimes noise.

Lots of the videos were comfort videos really for people who don't have an SM7B, 87, 414 etc and wanted to know how using an SM58 would actually make them sound worse. I get lots of emails and private messages from people unable to post their comments that they were grateful.

Very often, people comment that, for their ears, they preferred the cheaper alternative, but daren't say it in public. I make no attempt to treat or cure - I'm doing them in my video studio, not the sound studio and the edit suite monitors are small and lacking bass. Often I go home to the audio studio and discover I put a video out with boomy bass, or even a hum. Every video has noise - the desk mounted Blackmagic recorder has a fan - so my reviews are raw and practical.

Even weirder, when I bought the rubbish U87s - some, including postage from China were less than the price of a happy meal. I put them on ebay with accurate descriptions - terrible microphone ex-youtube review, and included the link. People bought them. I found one Chinese AKG 414 copy - sold now by a few suppliers under different brand names, and cardioid only. I thought it sounded pretty bright and piercing, but the guy who bought it loves it.

The concept of comparisons is fine as long as the viewer understands the context: Got a cheap guitar, not in a good space, then hearing mics doing this job gives you enough to NOT buy something - preventing an expensive mistake maybe?

Clearly, my review of a range of cheap mics/expensive mics does not guarantee X brand is best when you do the same thing, but I do think there is a place for these. I decided the SM7B, with its forgiving nature for not changing tone too much with distance, was a good 'constant'.

On the graph front, very few give plots at different frequencies on a polar pattern graph. I did a comparision of shotgun mics, and used simple plots - and the result seemed to explain the popularity of certain mics and seeing the plot at different degrees of a circle made more sense - here's a link to that one. https://youtu.be/PPvS39mS9xU
Logged
Pages: [1]   Go Up
 



Site Hosted By Ashdown Technologies, Inc.

Page created in 0.035 seconds with 20 queries.