R/E/P Community

Please login or register.

Login with username, password and session length
Advanced search  

Pages: [1] 2 3 ... 46   Go Down

Author Topic: The Chicago test results...  (Read 65193 times)

Dingo

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 51
The Chicago test results...
« on: November 19, 2005, 01:45:00 pm »

...is there a verdict?

Did you all reach a consensus - or did some hear something & others hear nothing?

Basically - did the group acertain that there was a hearable loss of bottom end (no matter how large or small) in PT - or not?
We're all dying to know - please share!

Dingo  Razz
Logged
"This has to be the only industry in the world that has spent the last decade trying to effectively simulate all of the tools that it was told it needed to throw out." - bunnerabb

http://www.backtoback.cd/TIB3.jpg

rnicklaus

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 3859
Re: The Chicago test results...
« Reply #1 on: November 19, 2005, 01:50:51 pm »

Fletcher posted a bit on the subject on the other thread that is now locked.

Logged
R.N.

rnicklaus

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 3859
Re: The Chicago test results...
« Reply #2 on: November 19, 2005, 02:10:21 pm »

So far, the only info from the test is that PT didn't suck as bad as Fletcher thought it was going to suck.

Without knowing how much he thought it was going to suck, it's a bit difficult to get a feel for what happened.
Logged
R.N.

Rail Jon Rogut

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 570
Re: The Chicago test results...
« Reply #3 on: November 19, 2005, 02:12:19 pm »

Since most Pro Tools users never experience any low end loss...  I'm sure they experienced normal converter sonic differences (as expected).

Rail
Logged
Recording Engineer

www.platinumsamples.com

Engineered Drums for BFD & Superior Drummer 2.0

Dingo

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 51
Re: The Chicago test results...
« Reply #4 on: November 19, 2005, 02:23:03 pm »

Granted, there was a little info:

1. Pro Tools sucked but not as bad a Fletcher thought it would...and sounded half decent with different converters.

2. There were dramatic differences heard during playback, which Fletcher believes won't be as predominant in the recorded files to be posted somewhere at a later date.

So what exactly were the "dramatic differences" heard during playback?

Was there a loss of low end from PT or not - I mean, that's what the test was all about, no?

Fletcher, Ron, Steve, GK, anyone willing to share?

Dingo
Logged
"This has to be the only industry in the world that has spent the last decade trying to effectively simulate all of the tools that it was told it needed to throw out." - bunnerabb

http://www.backtoback.cd/TIB3.jpg

orbb

  • Newbie
  • *
  • Offline Offline
  • Posts: 27
Re: The Chicago test results...
« Reply #5 on: November 19, 2005, 02:39:29 pm »

And who won the poker game?  How hot were the strippers?  Any pictures?  
Logged

rnicklaus

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 3859
Re: The Chicago test results...
« Reply #6 on: November 19, 2005, 02:48:30 pm »

Dingo wrote on Sat, 19 November 2005 11:23

Granted, there was a little info:

1. Pro Tools sucked but not as bad a Fletcher thought it would...and sounded half decent with different converters.

2. There were dramatic differences heard during playback, which Fletcher believes won't be as predominant in the recorded files to be posted somewhere at a later date.

So what exactly were the "dramatic differences" heard during playback?

Was there a loss of low end from PT or not - I mean, that's what the test was all about, no?

Fletcher, Ron, Steve, GK, anyone willing to share?

Dingo


You seem to have read much more from Fletcher's post than I did.

Where did it say they used a different converter?

You are taking the word dramatic to a different level than just reading what is in the post IMHO.
Logged
R.N.

rnicklaus

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 3859
Re: The Chicago test results...
« Reply #7 on: November 19, 2005, 02:51:27 pm »

Below is Fletcher's post from the other thread
 ------------------------------------------------------------ ----

Well the whole thing was pretty eye opening in some regards, at least to me. PT didn't suck nearly as much as I thought it was going to suck... and there were even some aspects of it [different clocking hardware] that were down right decent sounding... but I'm not going to get into any kind of a blow by blow... I'm sure someone else will do that.

There are files that have been recorded, though I don't believe the recorded files are quite as dramatic as the differences heard in live playback [which is an entirely different can of worms I seriously don't feel like opening at the moment!!].

There should be some files available sometime in the near future... when we have a place to put them we'll let you know.

This brings an end to this thread's broadcast day... hopefully one or more of the other participant's will post their observations... hopefully we'll be able to get the files in some kind of place where y'all can play at home and come to your own conclusions.

Thanks for spending the time to assist with the formulations of the various questions... but for now, I'm going to do what should have been done about a day ago [lock this thread]

Fletcher
Mercenary Audio

Logged
R.N.

Gannon Kashiwa

  • Newbie
  • *
  • Offline Offline
  • Posts: 17
Re: The Chicago test results...
« Reply #8 on: November 19, 2005, 03:18:52 pm »

Hi all,

Here's how I thought it went.

Background:  We did tape to Pro Tools clocked to the 192 internal, SYNC I/O and Big Ben transfers at 48k and 96K and tape to Radar at 48k and 96k.  For the blind tests, we only listened to the PT transfers with the 192 internal clock (though we will post examples of all clocks for you to listen to and decide which ones you like best - blind, of course).  

There were 18 listeners in the room.  Prior to the blind tests, Steve asked the room if anyone detected a loss of low end as that's what originally set out to check.  Not a single hand went up.  It didn't seem like anybody heard a discernable loss of low frequency information at all.

Then we did the blind tests in "A/B/X" fashion.  We took 5 examples - tape, PT 48k, PT 96k, Radar 48k and Radar 96k - and paired them up as "A" and "B" in random pairs.  Sometimes it was tape and PT96k and other times it was Radar 48k and PT 96k - you get the idea.  18 passes were performed altogther where the operator would play a 15 second section of "A", a 15 second section of "B" then randomly select one or the other and play that back as "X".  The listeners then were to identify what they thought "X" was.  

Of the 18 passes, the room had a 50% accuracy in correctly identifying the "X".  The highest score in the room was 13 correct, the lowest was something like 4.  There are differences in the sounds of these exaples to be sure, but they are subtle enough that they can only be picked out half of the time which means it's pretty clear that all 5 of these formats are very comparable in sound.  Subtly different, yes, egregiously inaccurate, no.

That's my take on it.  I'm sure others will chime in with their impressions.  We still haven't sorted out how to deliver the files to y'all, but I'm suggesting posting 20 second excerpts (the same ones we listened to plus the clock alternatives) of the raw 24/96k files and create a poll of which ones you like best and why.  You woulndn't be able to do blind comparison tests, but you can certainly tell which ones you like best.  The files should be labeled "A", "B", "C", etc and put up for a week or so then revealed.

We'll figure that out and get them up asap.  There was also a movie of the whole thing.  I'm kicking myself for not taking any pictures!

I'd like to thank Ron Steele and the kind folks at CRC for generously hosting this thing.  Azoulas, Bruce, Chris, Chrisand the rest of the crew - you guys are awesome!  Also, Steve and Fletcher for providing the tape, headstack, Radar and making the test as scientific as possible - and for keeping an open mind!  I know I was there with and open mind and was very relieved to know that "we don't suck as much as Fletcher thought we did".  Smile

-GK

Logged

bblackwood

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 7036
Re: The Chicago test results...
« Reply #9 on: November 19, 2005, 03:26:59 pm »

Gannon Kashiwa wrote on Sat, 19 November 2005 14:18

Of the 18 passes, the room had a 50% accuracy in correctly identifying the "X".  The highest score in the room was 13 correct, the lowest was something like 4.

I don't remember the number needed to 'prove' one was statistically correct in hearing differences, but I think it's close to 80% - which means maybe one person heard the diff in these tests.

Unless the data as presented is inaccurate or incomplete.
Logged
Brad Blackwood
euphonic masters

Ron Steele

  • Full Member
  • ***
  • Offline Offline
  • Posts: 230
Re: The Chicago test results...
« Reply #10 on: November 19, 2005, 03:55:50 pm »

Here is MY findings of fact:


When Steve A. presented the room with the question on the bottom end, he asked the participants to raise their hands if they heard the suggested claim.

No hands went up.



We then proceeded to transfer the 2", PT using int. and ext. clock sources {digi sync i/o and big ben} at various sample rates, and Radar {int. clk} at 48 and 96. Participants were in and out of the control room during these transfers. There was certainly a lot of  opinions being discussed about the transfers, but they were more or less about overall sonic preferences of all the transfers as opposed the low end disappearing because of the DIGI 192.


After that we moved on to the blind test. We listened to 18 passes of A/B/X using the various transfered files all mixed up by Azoulas . We all kept a score sheet of our individual answers. After we tabulated the results {we found out AE's can't count to well} we found that the participants in the room were correct with their picks about half the time.

What this proved was, that on any given playback of any given transfer, the room could only pick X out around, give or take, 50% of the time, which tells me nothing sucked.


I find this to be very telling. I also feel that if we did a blind A/B/X test with just the 2", PT int. clk. at 48 and Radar at 48, we would get similar results. At least that would be my first guess.

Honestly, it was a great experience, and clear that we all hear and perceive sound differently, and everybody is certianly entitled to their opinion.

But in the end, I believe WE our responsible for the end product WE are creating, not the equipment manufactures. They provide us with choices, so if anybody thinks they sound like shit because of X piece of gear, it's time to find something that works better for you, or it's time to take a look in the mirror.

Logged
 "I have had PLENTY of my posts torched on other boards. It kind of goes with the territory of pushing the envelope. Their house, their rules. Why can't everyone GET this?"

Rail Jon Rogut

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 570
Re: The Chicago test results...
« Reply #11 on: November 19, 2005, 04:00:26 pm »

Hi Ron

Out of interest.. how was the clock of the Big Ben wired to the system.. split to each 192 or only to the Sync I/O?  Apogee suggest that it should feed each 192 separately.

Rail
Logged
Recording Engineer

www.platinumsamples.com

Engineered Drums for BFD & Superior Drummer 2.0

The Resonater

  • Full Member
  • ***
  • Offline Offline
  • Posts: 113
Re: The Chicago test results...
« Reply #12 on: November 19, 2005, 04:10:47 pm »

Gannon,

Thanks for the general rundown on how the testing went.  From what you've already said, the day was really interesting, and I especially liked the A/B/x concept.  Hadn't seen that in the previous methodology threads.  Kool.

Obviously, if the *best* tester nailed 13 of 18, and the *worst* tester nailed 4 of 18, it indicates that there was most likely no clear consensus on picking the various formats.    Which may not be surprising, given that *perhaps* all of our ears may work a little differently OR that emotional bias creeps into the choosing of which format we're hearing.  Just like with wine, when you're tasting blind, it gets much harder to choose the $50.00 bottle from the $10.00 bottle.  So, obviously, the blind aspect of the testing was very important and may prove illuminating.

Hopefully, Steve, Fletcher, Ron et al will post more of their interpretations of the day and of the formats themselves.  Fletcher has posted a taste of his interpretation, but, given that he's given a lot of bandwidth to his pre-test feelings about the various formats, surely he'll give us a more thorough rundown on how the various formats sounded, etc.  He's probably working on it right now.  

So thanks again Gannon for your post and Ron, Steve and Fletcher, bring it on!  Inquiring minds want to know!

Logged
The Resonater

Ron Steele

  • Full Member
  • ***
  • Offline Offline
  • Posts: 230
Re: The Chicago test results...
« Reply #13 on: November 19, 2005, 04:15:37 pm »

Quote:

Hi Ron

Out of interest.. how was the clock of the Big Ben wired to the system.. split to each 192 or only to the Sync I/O? Apogee suggest that it should feed each 192 separately.

Rail



Not quite sure my-self Rail. Gannon would know better then me as I was just the food and beverage manager.

I will tell you that I prefered the digi sync i/o over the big ben. It felt less hyped overall, if that makes any sense.

Ron
Logged
 "I have had PLENTY of my posts torched on other boards. It kind of goes with the territory of pushing the envelope. Their house, their rules. Why can't everyone GET this?"

rnicklaus

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 3859
Re: The Chicago test results...
« Reply #14 on: November 19, 2005, 04:19:06 pm »

Obviously, Steve Albini hasn't posted his thoughts yet - but so far with Fletcher saying PT didn't suck nearly as much as he thought it would, combined with Gannon and Ron's posts (both went in on the side of no low end loss) it appears that the bottom octave being gone or the 6 db down at 50 or 60 hz description didn't happen.

Logged
R.N.
Pages: [1] 2 3 ... 46   Go Up