R/E/P Community

Please login or register.

Login with username, password and session length
Advanced search  

Pages: [1]   Go Down

Author Topic: Why Most 'Mic Shootouts' Are Fundamentally Flawed  (Read 3292 times)

klaus

  • Moderator
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2108
Why Most 'Mic Shootouts' Are Fundamentally Flawed
« on: June 28, 2020, 01:10:37 PM »

I hate them. Never listen to sound samples, and never in my life would I evaluate microphones that way. Not even for clients who beg me.

1. Sound samples recorded with MP3 or other lossy formats obliterate the subtle characteristics we cherish in a good microphone- characteristics we reward by paying disproportionate financial premiums for that last bit of quality.

2. The intended audience is never present when these 'shootouts' are made. For all we know, totally different mics, preamps, cables, distances were used that may favor one mic over the other, regardless of a mic's true merit as recording tool in a real-world working environment.

3. Speaking of distances: here is the most overlooked and biggest flaw of shootouts - a flaw so big, it will render any comparison between two or more mics pointless and inconclusive: to position several mics at the same exact distance from the sound source.

We see pictures of totally different microphones with totally different sensitivities and timbres being meticulously lined up so that their capsules were aligned within millimeters - all in service to the illusion of scientific objectivity. As if we didn't know that every mic has its own sweet spot- a factor recording professionals pay close attention to. Who would place an SM58 and a U47 at the same distance from the singer?


There is science and then there is stupidity.

Logged
Klaus Heyne
German Masterworks®
www.GermanMasterworks.com

BluegrassDan

  • Newbie
  • *
  • Offline Offline
  • Posts: 26
  • Real Full Name: Dan Boner
Re: Mic Shootouts Online Are Fundamentally Flawed
« Reply #1 on: June 28, 2020, 05:35:56 PM »

The SoundPure videos are the worst. They start out by saying how crazy a person must be to even want the vintage piece of equipment in their test. Little hints throughout like “thankfully, the original LA2A didn’t malfunction during our test.”
Logged
Dan Boner
Associate Professor/Director
Bluegrass, Old-Time, and Country Music Studies
East Tennessee State University

David Satz

  • Full Member
  • ***
  • Offline Offline
  • Posts: 160
Re: Mic Shootouts Online Are Fundamentally Flawed
« Reply #2 on: June 29, 2020, 01:41:26 AM »

Klaus, I agree strongly with what you've said here. If two microphones are the same make and model and ought to sound the same, a direct comparison is a nice, quick way to check that. If they're variations on the same basic thing (e.g. a KM 84 vs. a KM 140 or KM 184), sure. If you know something about one of the microphones, then after such a comparison you will know something about the other one, too.

But when two microphones have inherently different characteristics, your judgment that microphone "X" sounds better than microphone "Y" in setup "A" tells you nothing about which one will sound better in setup "B", let alone "C", "D" or "E". Your preference may be a total fluke--particularly if you weren't there when the recording was made. Different characteristics may come into play, e.g. differences in directional pattern in different frequency ranges.

--best regards
Logged

Kai

  • Sr. Member
  • ****
  • Online Online
  • Posts: 398
Re: Mic Shootouts Online Are Fundamentally Flawed
« Reply #3 on: June 29, 2020, 03:34:43 PM »

Studio work is mic shootout all the time.

You set a mic, don't get the sound you're after, try another one.
Only in real world's practical use can a mic show its qualities, over time.

There are too many variables involved to arrange a test scenario with universal validity:

Today the saxophone sounds best with a Schoeps, next day the next sax player needs a Sennheiser MD421 - those mics are worlds apart in a "shootout", so are the saxes ;D

A well sorted mic locker helps.
Logged

Timtape

  • Full Member
  • ***
  • Offline Offline
  • Posts: 108
  • Real Full Name: Tim Gillett
Re: Mic Shootouts Online Are Fundamentally Flawed
« Reply #4 on: September 10, 2020, 12:37:20 AM »

Studio work is mic shootout all the time.

Perhaps so but the mere fact that it occurs doesn't validate it.

The motive is important, may be honourable or not, and the practice may or may not be based on solid evidence.
 
It's easy to "impress" a client vocalist who knows nothing about microphones and production, that testing ten different microphones on his or her voice  - all this wonderful personal attention - which extra time the performer may well be paying for - is justified in the cause of producing a  superior product. That the "expert" is searching for the mic that allegedly best "matches" this one voice seems superficially impressive especially if the vocalist doesnt know any better.

What is the evidence that it works or is the best method? And best method for achieving  what exactly?

Tim

Logged

Timtape

  • Full Member
  • ***
  • Offline Offline
  • Posts: 108
  • Real Full Name: Tim Gillett
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #5 on: September 10, 2020, 12:55:00 AM »


3. Speaking of distances: here is the most overlooked and biggest flaw of shootouts - a flaw so big, it will render any comparison between two or more mics pointless and inconclusive: to position several mics at the same exact distance from the sound source.

We see pictures of several totally different microphones with totally different sensitivities and timbres being meticulously lined up so that their capsules were aligned within millimeters - all in service to the illusion of scientific objectivity. As if we didn't know that every mic has its own sweet spot- a factor recording professionals pay close attention to: who would place an SM58 and a U47 at the same distance from the singer?

I agree, Klaus. The SM58 is equalised for close vocals, the U47 isn't. At the same distance, they will sound very different. Used as per their respective user manuals, not nearly so different.
Logged

klaus

  • Moderator
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2108
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #6 on: October 24, 2020, 02:44:50 AM »

This is what I am talking about. Lining up the capsules of five mics in one plane.
Logged
Klaus Heyne
German Masterworks®
www.GermanMasterworks.com

klaus

  • Moderator
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2108
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #7 on: October 31, 2022, 07:00:54 PM »

Two years after starting this thread things are not looking up, and people continue to distribute "shootouts"on the internet whose results are then commented on. This trend is not healthy, in my opinion:

1. Ignorance persists about the unscientific, statistically questionable methodology applied to these comparison tests. Methods are never detailed by the producers and never questioned by the consumers - an untenable foundation to forming a fact-based opinion or to making an informed purchase decision.

2.As prices for vintage originals (which are then often "shot out" against copies of the same model) have skyrocketed, the phenomenon persists and has gotten more widespread in recent years. This then gives a potential buyer with limited financial means the illusion of being able to achieve sonic excellence on the cheap.

3. Test conditions are so far beyond any realistic setup and working experience encountered in professional recording scenarios that a microphone costing less than 1/10 of another one it is compared to sounds somehow indistinguishable from it.



As demonstrated in my previous post, disregarding the relevance and individual variation the proximity effect has on pressure gradient microphones does gross disservice when evaluating a mic's optimal distance from a sound source.

Here is another example of the "sin of equal distance" (let alone, the ultimate sin of using mics with different capsule topology at the same, unrealistic distance from the sound source): https://www.youtube.com/watch?v=BTUcnWDMPJs

What is most striking to me: true professionals running successful studios and engineers making a good living due to their talent never participate in these amateur contests. It's usually either people with some sort of financial interest who publish these tests or people who feed on consumers' confirmation bias: I own the cheaper copy, and, see? It's just as good!

So why then even bother talking about it?
In the absence of rigorous discussion outside of self-interest, and by continuing to ignore pleas to use a more scientific methodology when examining sonic impressions of high-end microphones, the current trend of sloppy mic comparisons will eventually take hold and become the new working standard. But it will cause heartaches, heartaches and remorse down the road by those who took these tests seriously and made purchasing decisions based on them.
Logged
Klaus Heyne
German Masterworks®
www.GermanMasterworks.com

klaus

  • Moderator
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2108
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #8 on: November 13, 2022, 10:59:08 PM »

Two years after starting this thread people continue to distribute "shootouts"on the internet without documenting methodologies or identifying vital sound-shaping microphone components. Results are then consumed and commented on, again, with no inquiries into methodology or (sub)component details.
To sum up and expand on the fallacies of this approach:

1. Ignorance and incuriosity persists about the unscientific, untenable methodology applied to these comparison tests. Methods are never disclosed or detailed by the producers and never questioned by the consumers - an unscientific, wobbly foundation to forming a fact-based opinion or to making an informed purchase decision.

2. As prices for vintage originals (which are then often "shot out" against current-production copies of the same model) have skyrocketed in recent years, "shootouts" also have skyrocketed, which can give a potential buyer with limited financial means the illusion of being able to achieve sonic excellence on the cheap.

3. Test conditions are so far beyond any realistic setup and working experience encountered in professional recording scenarios that a microphone costing less than 1/10 of another one it is compared to, sounds somehow indistinguishable from it.

4. Lively discussions ensue about a 60+ year-old M49 having a "slightly preferable" midrange, compared to a brand-new M49V reissue, without verifying capsule version and condition, tube health, power supply voltages, and other vital checks. This is akin to comparing performance of two otherwise similar cars - one veteran, one brand-new, one worth millions, the other a mere fraction - without as much as opening the hood.


As I pointed out in my previous post, disregarding the relevance and individual variation which proximity effect has on pressure gradient microphones does gross disservice when evaluating a mic's optimal distance from a sound source.

Here is another example of the "sin of equal distance" (let alone, the ultimate sin of using mics with different capsule topology at the same, unrealistic distance from the sound source): https://www.youtube.com/watch?v=BTUcnWDMPJs

What is also striking to me: true professionals owning or woking in successful commercial studios never participate in these amateur contests. It's usually people publishing these tests who have some sort of financial interest in the test results, or buyers who have succumbed to confirmation bias: I own the cheaper copy, and, see? It's just as good!

So why then even bother talking about it?
In the absence of rigorous discussion outside of self-interest, and by continuing to ignore pleas to use a more scientific methodology when examining sonic impressions of high-end microphones, the current trend of sloppy mic comparisons will eventually take hold and become the new working standard. It will cause headaches, heartaches and remorse down the road for those who took these tests seriously and made purchasing decisions based on them.
Logged
Klaus Heyne
German Masterworks®
www.GermanMasterworks.com

gtoledo3

  • Full Member
  • ***
  • Online Online
  • Posts: 119
  • Real Full Name: George Toledo
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #9 on: November 17, 2022, 11:20:20 AM »

One point about positioning the mics in the same place.

It doesn’t really reflect how someone would tend to position a mic in typical context. It does reveal differences with proximity effect, and possibly some other differences.

At GS, someone made a thread titled something like U47 Test 2. It may be the first time I’ve really engaged in one of these comparisons, and the main reason I did is to deflate the idea that all of the mics actually sounded like U47. It was quite easy to discern the brand of each mic, and a fair amount of that was because of the copy mics not having proper proximity response/sub bass extension. There were other discrepancies of course.

If a user just worked the mic as typical, the discrepancy in proximity effect would have been less clear because there would be another factor involved.

When it comes to recording, the proof is in the pudding. It is always fair to judge a mic by a recording created with the mic, it’s just that the judgement should be limited.
Logged

klaus

  • Moderator
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2108
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #10 on: November 17, 2022, 11:47:58 AM »

What I wanted to stress:
Differences in proximity effect between microphones are magnified the closer you get to the capsule. Differences between capsules (and their processors which amplify the capsule's output) need to be recognized in comparison tests. Therefore, each mic should be positioned to its sweet spot, where the mic's proximity and other characteristics are subjectively perceived to be optimal.

But few testers seem to have the courage to involve what I find is paramount when evaluating microphones: making a decision at what distance a microphone sounds best. THEN you have something worth comparing.
Logged
Klaus Heyne
German Masterworks®
www.GermanMasterworks.com

Timtape

  • Full Member
  • ***
  • Offline Offline
  • Posts: 108
  • Real Full Name: Tim Gillett
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #11 on: November 25, 2022, 07:40:23 AM »

Some mic companies have printed graphs showing the lower frequency responses at various source distances, due to proximity effect. Also sometimes stated in the user instructions is a recommended singing or speaking distance such as 2" from the pop grille for an SM58.

Some users are able to determine this distance subjectively but it should normally be consistent with the manufacturers' data.
Logged

klaus

  • Moderator
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2108
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #12 on: November 25, 2022, 04:08:09 PM »

Can you share which mic companies include graphs based on realistic operating conditions?
Logged
Klaus Heyne
German Masterworks®
www.GermanMasterworks.com

Timtape

  • Full Member
  • ***
  • Offline Offline
  • Posts: 108
  • Real Full Name: Tim Gillett
Re: Why Most 'Mic Shootouts' Are Fundamentally Flawed
« Reply #13 on: November 26, 2022, 08:38:16 PM »

 I'm not sure if it was consistent even within one company but here's an example of the type of graph I was thinking of:

https://mynewmicrophone.com/proximity-effect/
Logged
Pages: [1]   Go Up
 



Site Hosted By Ashdown Technologies, Inc.

Page created in 0.044 seconds with 21 queries.