R/E/P Community

Please login or register.

Login with username, password and session length
Advanced search  

Pages: 1 [2] 3  All   Go Down

Author Topic: DDP fail  (Read 13536 times)

Allen Corneau

  • Full Member
  • ***
  • Offline Offline
  • Posts: 125
  • Real Full Name: Allen Corneau
Re: DDP fail
« Reply #15 on: March 29, 2011, 08:57:29 am »

(I think Brian from DM always says the following): When you submit a DDP, add the checksum file inside the folder, compress the entire folder (.zip or .rar) and send that to the plant.

This is what I've been doing lately and no problems so far.
Logged
Allen
=====
Allen Corneau Mastering
http://allencorneau.com/

phonon

  • Newbie
  • *
  • Offline Offline
  • Posts: 43
  • Real Full Name: Andrew Hamilton
Re: DDP fail
« Reply #16 on: March 30, 2011, 09:16:05 am »

Andrew, no not added... 30 seconds into the first track, the music disappears for a whole 60 seconds, while the CD display keeps ticking over. The only way that could happen, I should think, is if I created an edl with 60 seconds of silence in the first track, but the MD5 I included with the uploaded DDP set passes the master edl, which of course is intact. It doesn't pass the image file on the ftp server.

Thanks for clarifying, Tony...  However, I should have thought that the MD5 you created was created to be a checksum of the premaster DDP image you sent (the one that is, presumably, still on your hdd) - _not_ a bit-string of the edit decision list in the DAW, which would have a different hexcode expression from even a perfectly Dumped premaster thereof, yes?



That doesn't make the bad image file the fault of the plant - where their fault lies is in not running the checker, which would have spotted a bad file, which I could then have re-uploaded.  :'(

Indeed...   they should not have proceeded until either you gave them the green light on the redownloaded DDP or they, at least, checked the checksum, themselves, against what the image copy they had.   Otherwise, it is they who leaped without looking first - not you...

They allegedly said, "Why did the engineer send an MD5...?"    If this is the case, then they should be trained on the vagaries of FTP.


Of course, perhaps MD5 is also somewhat unreliable.   Although it may be that the only issue is the ability for two different files to generate the same hexcode and not for the same file to generate two different hexcodes...

from Wiki:
"US-CERT of the U. S. Department of Homeland Security said MD5 "should be considered cryptographically broken and unsuitable for further use,"[9] and most U.S. government applications now require the SHA-2 family of hash functions.[10]"



Andrew


Logged
Serif Sound Mastering / Dingbat Lacquer Sound Disk
www.serifsound.com
www.dingbatlsd.com
1 (513) 542-3555

tbridge

  • Newbie
  • *
  • Offline Offline
  • Posts: 13
  • Real Full Name: Tony Bridge
Re: DDP fail
« Reply #17 on: March 30, 2011, 10:06:58 am »

<snip>..I should have thought that the MD5 you created was created to be a checksum of the premaster DDP image you sent (the one that is, presumably, still on your hdd)... yes?

yes indeed, but that also includes checking the PQ, which has not been affected by a whacking great silence in Track One.

And I agree with everything else you say, except that MD5 still seems to be the industry norm for us audio engineers, maybe the CIA needs something more robust.
Logged

bkuijt

  • Newbie
  • *
  • Offline Offline
  • Posts: 17
Re: DDP fail
« Reply #18 on: March 30, 2011, 01:08:43 pm »

Of course, perhaps MD5 is also somewhat unreliable.   Although it may be that the only issue is the ability for two different files to generate the same hexcode and not for the same file to generate two different hexcodes...

from Wiki:
"US-CERT of the U. S. Department of Homeland Security said MD5 "should be considered cryptographically broken and unsuitable for further use,"[9] and most U.S. government applications now require the SHA-2 family of hash functions.[10]"

This Wiki quote is reffering to the fact that it is not impossible to crack passwords generated with the MD5 hash algorithm with brute force computing power.

However, as a means of verifying file integrity it is pretty safe.
At least more safe then .zip which relies on CRC checksums.
I have had the occassion that a corrupted .zip archive still unpacked, with a glitch in the audio as result...
It might be that this error appeared -after- unpacking, while writing the unpacked files to disk, but still WinZip did not see the error...
WinRar has performed better but is less popular, especially when dealing with Mac based clients.

MD5 checksums have saved me on a number of occasions, I use them as much as I can when sending/receiving files (over network as well as optical discs).
I am really glad Sonoris has it implemented natively within the interface.

bests,
Logged
Bastiaan Kuijt  //  BK Audio  //  www.bkaudio.nl

tbridge

  • Newbie
  • *
  • Offline Offline
  • Posts: 13
  • Real Full Name: Tony Bridge
Re: DDP fail
« Reply #19 on: March 30, 2011, 01:33:16 pm »


I have had the occassion that a corrupted .zip archive still unpacked, with a glitch in the audio as result...

Ouch  :o It seems to me that MD5 is pretty secure, and would catch a corrupt file, whereas you say unZipping might not, so you may as well stick with only MD5? Plus, there is a school of thought that questions the integrity of the audio after Zipping, but this seems to be one paranoia too far!
Logged

tbridge

  • Newbie
  • *
  • Offline Offline
  • Posts: 13
  • Real Full Name: Tony Bridge
Re: DDP fail
« Reply #20 on: March 30, 2011, 01:45:12 pm »

My client has received a hefty bill for re-packaging from his broker (though, interestingly, not a bill for re-pressing...hmmm...).

Yesterday, I sent off a detailed email to my client, carefully explaining why the factory seems to have been rather lax in their QC and that I shall need to see a full Report from the factory before I accept any liability. He forwarded that email to the broker, asking for comments, and things have been strangely quiet.

I do not intend to be held responsible for this on the say-so of a broker who clearly has no idea what this is all about - but I fear that in the end, I may have to come to some agreement with my client  ::)
Logged

phonon

  • Newbie
  • *
  • Offline Offline
  • Posts: 43
  • Real Full Name: Andrew Hamilton
Re: DDP fail
« Reply #21 on: April 01, 2011, 09:38:08 am »

yes indeed, but that also includes checking the PQ, which has not been affected by a whacking great silence in Track One.
Thanks for clarifying that the MD5 you sent does not, in fact, pass the master EDL, as you originally wrote, but that it, rather, passes the DDP image that you dumped from the master EDL and then (attempted to) upload(ed).   I get confused sometimes with RISC (reduced instruction set chats).
 (The DDP image is - again, for clarity - _not_ the EDL project files and folders.)   

The way of catching a plunk of silence that is inadvertently dumped along with the intended EDL (to DDP image), is to export an audio CD of the DDP image and audition/test that CD for good audio and cueing.   I assume that you did that, as well.  So, the image on your hdd that was used to beam the data for glass mastering also does not have the missing audio?  However, during the FTP, the connection apparently choked and dropped almost exactly 60 seconds of audio without corrupting anything else in the image?   The other audio is perfect and the PQ marks are perfect, CD-TEXT, EAN, etc...?   


And I agree with everything else you say, except that MD5 still seems to be the industry norm for us audio engineers, maybe the CIA needs something more robust.

My point about MD5 was that it's apparently possible for 2 different files to be hacked to produce the same hexcode.   If a replicator wanted to get out of admitting that they screwed a pooch (and ruined a premaster), they might be able to hire a hacker to fake a looks-right MD5 from a broken DDP image and then allege that the premastering clerk had uploaded a bad premaster (with MD5 intact, to prove it!). 


In your case, fortunately, the MD5 is not valid for the copy of the DDP image they have on their server.   So, however it was maimed, there was no attempt to conceal tampering - just failure to check the checksum (OR to verify that the sender had checked his sent image against his local copy's checksum).   Either way, the plant proceeded without a green light (unless you downloaded only the MD5 and compared that to your still-good DDP image (on your hdd) and then told them it was "a go"). 



The great, Wiki, says MD5's not recommended for electronic signatures or SSL.    Stuff that's way below the NSA's radar is still too sensitive for MD5 in many i-contexts.   Though, perhaps not our t'ing....


Here's a familiar palindrome in SHA-256:
6a6d5e31ecd2c3b0ea77c5466f18fa354c7d28831122da4e87582f9f8cd6b169

and, in SHA-160:

9330dffd86b93c8cb0c0b96c80eb90341ee0249c

and, now, in plain old' MD5:
763098442c7a9d2ab74587160cd6c1c2

Can you decode, or guess, how it's meant to read?    ???


Andrew
Logged
Serif Sound Mastering / Dingbat Lacquer Sound Disk
www.serifsound.com
www.dingbatlsd.com
1 (513) 542-3555

tbridge

  • Newbie
  • *
  • Offline Offline
  • Posts: 13
  • Real Full Name: Tony Bridge
Re: DDP fail
« Reply #22 on: April 01, 2011, 10:17:53 am »

Thanks for clarifying that the MD5 you sent does not, in fact, pass the master EDL, as you originally wrote, but that it, rather, passes the DDP image that you dumped from the master EDL and then (attempted to) upload(ed).   I get confused sometimes with RISC (reduced instruction set chats).
 (The DDP image is - again, for clarity - _not_ the EDL project files and folders.)   
You're right,  I created a DDP image from the master EDL, then burnt a disc from that for my client, which did not, of course, have any additional silence in the first track. The MD5 files were created from that DDP image, and failed on the server, meaning that the uploaded file was different from the DDP file on my local hard drive.

There always was a very small chance that I had somehow managed to make a DDP file containing a minute's worth of silence, but then the MD5 checker would pass this, no?
The way of catching a plunk of silence that is inadvertently dumped along with the intended EDL (to DDP image), is to export an audio CD of the DDP image and audition/test that CD for good audio and cueing.   I assume that you did that, as well.
See above...
The other audio is perfect and the PQ marks are perfect, CD-TEXT, EAN, etc...?
Yup...

Your comments about the MD5 being hacked are, presumably, true, but that's getting a little paranoid, I think... a hacker would probably cost the plant more than they would save in re-pressing costs  8)

I have no idea what your last remarks are about  ???
Logged

tbridge

  • Newbie
  • *
  • Offline Offline
  • Posts: 13
  • Real Full Name: Tony Bridge
Re: DDP fail
« Reply #23 on: April 01, 2011, 10:28:50 am »

Latest developments...

The broker has contacted my client to say that, while checking the MD5 is NOT part of the Replication Plant's normal QC procedure ( ??? :o ), they (presumably) admit that maybe they should have checked it, so are willing to re-press free-of-charge. The broker, however, contends that, as the corrupt file is down to me, they still intend to charge my client for the un-packing/re-packaging of the CDs (not cheap at about 600GBP), which they suggest he passes on to me.

This is an interesting state of affairs: I contend that I should not be held responsible for a file once it leaves my studio, and that, as the MD5 checker fails the file on their server, it is demonstrably NOT the file on my computer. There's a small chance that I uploaded a DDP file with a hole in it, but then the MD5 checker would have PASSED that, surely?

 I would like the broker to demonstrate that their server is not corrupted, as if they are holding me responsible for upload errors, they must surely also be responsible for the integrity of their storage?

Any thoughts before I get back to them again? I'm tempted to take this further (i.e., call my lawyer) as I think my case is pretty watertight, but my client, who has to pay these extra costs to the broker, is happy to spread the damage over several future projects, so maybe I should take it as (yet another) hard lesson.
Logged

phonon

  • Newbie
  • *
  • Offline Offline
  • Posts: 43
  • Real Full Name: Andrew Hamilton
Re: DDP fail
« Reply #24 on: April 01, 2011, 11:30:53 am »


...There always was a very small chance that I had somehow managed to make a DDP file containing a minute's worth of silence, but then the MD5 checker would pass this, no? See above...Yup...

You'd have heard a minute's silence on the ref which you auditioned in toto (or which you at least can Load Back)?.  'T'wasn't missing on the ref of the master premaster.   So couldn't be missing in the master premaster.     

Your comments about the MD5 being hacked are, presumably, true, but that's getting a little paranoid, I think... a hacker would probably cost the plant more than they would save in re-pressing costs  8)

...and I thought the hacker's services - before their rising to cell-lebrity(?) - were typically pro bono [if one could call it that]. 

I have no idea what your last remarks are about  ???

A diversion, merely.  It illustrates the extra complexity and curious charm? of SHA over Message-Digest (burp>)..   j/k

Guess'd the palindrome?


A bull was I air eye saw well baaaah..........  (only spelled, correctly, as the palindrome would require.)

Cheers,
     Andrew
Logged
Serif Sound Mastering / Dingbat Lacquer Sound Disk
www.serifsound.com
www.dingbatlsd.com
1 (513) 542-3555

bblackwood

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 254
Re: DDP fail
« Reply #25 on: April 01, 2011, 12:06:08 pm »

Situations like this are tough.

The few times I've had to deal with this type of situation I've weighed the value of 'winning the fight' vs the value of potentially earning a client for life. Only you know all the details, but sometimes spending a few dollars you aren't required to spend is a good long-term investment...
Logged
Brad Blackwood
euphonic masters

bkuijt

  • Newbie
  • *
  • Offline Offline
  • Posts: 17
Re: DDP fail
« Reply #26 on: April 01, 2011, 02:08:38 pm »

Ouch  :o It seems to me that MD5 is pretty secure, and would catch a corrupt file, whereas you say unZipping might not, so you may as well stick with only MD5?

hmm.... it's weighing pro's and con's:
- MD5 is more secure but voluntarily applied.
- .Zip or .rar is maybe a little less robust but you are FORCED to use it to retrieve the files from the archive.

(My statement is pretty heavy but it only happened twice with .zip and once with .rar. And I did not establish at that time if the error was a corrupted file or an error in the disk writing process after unpacking.)

Many less tech-savvy clients are lost when I try to explain MD5 checksums and getting them to install an extra application for this purpose is a nightmare. Especially since there is no real 'one-step' solution for Mac.
(all the ones I see produce text files in strange places, require manual assembly of the checksum file with copy/paste or do not support nested folders.).

With the people I know and trust I rely on MD5 only. But sending out masters to a client or large factory is usally in .rar format with MD5 checksums included for (voluntary) additional safety.

bests,
Logged
Bastiaan Kuijt  //  BK Audio  //  www.bkaudio.nl

bkuijt

  • Newbie
  • *
  • Offline Offline
  • Posts: 17
Re: DDP fail
« Reply #27 on: April 01, 2011, 02:35:23 pm »

I would like the broker to demonstrate that their server is not corrupted, as if they are holding me responsible for upload errors, they must surely also be responsible for the integrity of their storage?

That's hard to prove, it's more likely a communication error between ftp client and ftp server.
One could make a case saying that communication involves two parties.
Also checking the file could have been done on both ends.

 It's tough indeed and I'm sorry you get burned like this.
 It would be interesting to hear what a lawyer will say about the case.
 But I agree with Brad that your willingness to come to a solution might strengthen your position with the client.
 Maybe sharing the cost with the broker is an acceptable proposition?
Logged
Bastiaan Kuijt  //  BK Audio  //  www.bkaudio.nl

Jerry Tubb

  • Full Member
  • ***
  • Offline Offline
  • Posts: 172
  • Real Full Name: Jerry Tubb
Re: DDP fail
« Reply #28 on: April 02, 2011, 03:28:01 am »

Quote
I've weighed the value of 'winning the fight' vs the value of potentially earning a client for life.

Works in personal relationships as well,

sometimes winning ~all~ the battles isn't the best strategy.

You don't always have to be "right" to win in the long run.

JT
Logged
Terra Nova Mastering
Celebrating 25 years of Mastering!

Pieter Stenekes

  • Newbie
  • *
  • Offline Offline
  • Posts: 5
  • Real Full Name: Pieter Stenekes
Re: DDP fail
« Reply #29 on: April 05, 2011, 11:27:47 am »

Added a verification function to the DDP Creator that optionally reads back all files and test them against the original files with md5. That way you know for sure that the files were uploaded fine. What happens after that is in the hands of the operator...

Also added a resume function on the upload and download. The update will be available soon.
Logged
Pages: 1 [2] 3  All   Go Up