Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[chrisj]

George, can I post a set of files to demonstrate my findings?
I ran off some experiments today and found that while white-noise didn't contain enough extreme high frequency stuff to be ABXable (when you SRCed it to .00001 away from what it was), it was possible to make pathological cases that were obvious.
The demo files I made are actually audible changes being produced through UPsampling. It works just as well to upsample from 44.099K. The reason is- DAW output can easily contain data flat-out illegal by the rules of the Nyquist sampling theorem.
The files are white noise 24db down, filtered in four ways- +1.75db/oct, +2.25db/oct, 3db/oct, and 6db/oct. The latter three are allowed to clip. Each of these are then copied, 'hacked' so they are supposedly 44.099K, and upsampled to 44.1K using two Audacity settings for sinc interpolation- the highest quality setting, and the lowest.
There's 12 files in all, and the 1.75db/oct ones should all be indistinguishable. As the level of clipping increases, and the amount of illegal content increases, the files become increasingly distinguishable. Note this isn't downsampling, and that the SRC routine is about as artifact-free as you can get (try it against sine sweeps, it's pretty flawless). What you'll be hearing is files containing increasingly illegal content, clipping the hell out of your DACs, and then the SRCed files lacking any such clipping or illegal content.
Be careful with levels! This is about as loud as sound files get, and it'll fry your tweeters if you crank it. Turn it right down so you can hear what it's doing to the DAC without other parts of the system being overloaded in turn.
So, George, can I start posting the files? Should I make a big zip maybe, or do 12 posts each with a file attached?

[Erik]

chrisj wrote on Thu, 19 August 2004 16:24

The reason is- DAW output can easily contain data flat-out illegal by the rules of the Nyquist sampling theorem.

Really, how?

Quote:

white-noise didn't contain enough extreme high frequency stuff

Really, how?

Quote:

copied, 'hacked' so they are supposedly 44.099K, and upsampled to 44.1K

Not sure how this on-disk masturbation relates to any real-world scenario.  What are you trying to demonstrate or prove?

--Erik

[chrisj]

How does almost 12 db RMS level difference grab you? People have masturbated over far far less, my dear Erik.
Here's the executive summary, and I have the files handy for when I figure out what to do with them.

Noise at -24db with a 6db/octave treble boost, grossly clipped to -0.002db RMS on average: after upsampling from 44.099K to 44.1K at high quality, -5.1db RMS on average. At low quality, -11.3db RMS on average. In each case, the difference was screamingly obvious and 16/16 ABX double-blind tested. This was the worst case, flat-out distorted and loaded with extreme high frequencies.

Noise at -24db with a 3db/octave treble boost, still clipped to -0.9db RMS:
After upsampling from 44.099K to 44.1K at high quality, signal averaged -4.1db RMS. A medium quality test was done and averaged -6.3db RMS. In both cases the sounds were still distinguishable double-blind 16 out of 16 times. Interestingly, the low quality, which averaged -9.8db RMS, was more difficult to distinguish- 14 out of 16, ABX testing.

Noise at -24db with a 2.25db/octave treble boost was still clipping, but averaged -4.89db RMS. In this case the high quality 'upsampling' was distinguishable from the original 15 out of 16 times- it was -6.6db RMS after the upsampling from "44.099" to 44.1 (note, these are exactly the same file, but the metadata was altered to make a copy pretend to be 44.099K and be SRCed up to 44.1). The medium and the low quality SRCs (this refers to the steepness of the filter- all have an ideal stopband) were -9.2db and -13.8db RMS, but neither could be distinguished from the original by ear- I got nine out of sixteen each time, proving I couldn't hear 'em in spite of the gross difference in amplitude, and in spite of the fact that I provably could hear the much closer 'high quality' version of the same thing.

Noise at -24db with a 1.75db/octave treble boost, upsampled at low and high quality SRC, could not be distinguished from the original at all. Seven out of sixteen on the ABX comparator for each. In spite of this, the original was -12.1db RMS, the high quality conversion was -13.2db RMS, and the low quality conversion was -19.3db RMS. These are major changes, but they are happening at too high a frequency to hear. The 1.75db treble boost version barely clips ever- I believe if you tuned it so it never quite clipped, and used an arbitrarily high-quality sinc interpolation SRC, the RMS loudness would be entirely unchanged.

Program content averaging -4.89db RMS loudness is extreme but not unheard of. My point is just this: program content rich in high frequencies and averaging -12db RMS is still hot enough to be illegal content that is distorting the DAC and would be eliminated by a high quality SRC (even one from the 'same' sampling rate upsampled to a microscopically higher one). High-frequency program content that's as high as -4.89db RMS, at ANY moment, is hot enough to be audibly distorted and obviously different when you filter out information that's beyond the Nyquist limit. High-frequency program content as hot as -0.9db RMS is just blatantly, obviously trashing the DAC and can be distinguished every time from the same thing filtered to let through only frequencies below Nyquist- hell, you can't HAVE high frequency content hotter than about -5db RMS without trashing the DAC in a provably obvious way. Just because the samples can be represented as 16 bit words doesn't mean squat.

So be warned.

How's that for 'prove'?

Files will be coming up as soon as I figure out what to do with 'em- there's about 3500K but I might just throw out the 'medium' interpolation examples and the 6db/octave ones, which are so obvious they're a waste of space.

[George Massenburg]

Chris,

It turns out that we can't do file sharing on this site.  Do you have an FTP site that you can upload fiiles to?

George

[chrisj]

Well, all right then-
The files are in a zip at
http://www.airwindows.com/analysis/MicroUpsampling.zip
I figure that's as good a name as any, for taking a file, cheating the metadata so it claims to be 44.099K and 'upsampling' it to 44.1K.
The files are 16 bit AIFFs.

[Erik]

Ok, now you've posted files on a website.  Impressive.

Can you please describe what you're proposing, or point me to a description of what you're trying to demonstrate?  There seems to be quite a few variables here and I'm not sure what your point is.

Beyond that, this relates to 456 and tape machines how?

--Erik

[chrisj]

Yeah. Gawrsh, this internot thingy sure is complimakated!
I know how to work e-mail too
Okay. The specific aspect of the 456 thread that I branched off with was a bit where I said, you could 'fix' certain things about certain over-hot DAW mixes by resampling- even if you were just 'bumping' the resampling a bit. Specifically, you can even upsample and kill a bunch of information IF it isn't really supposed to be there in the first place.
In other words- if you mix on an analog console, or master through analog gear, AND resample at pretty much your final output level, you are guaranteed to not have this type of distortion, because a properly working sampler WILL NOT encode information beyond the Nyquist limit. If it's working right, no matter what you put into it, you're not going to get anything out that'll clip the DACs. I think this is part of the reason Brad Blackwood can get really hot and lively treble without it being grating- he may be doing some of that in analog and resampling at 44.1K. In so doing, it guarantees he's not generating illegal data.
If you work in a DAW or otherwise all-digital, nothing is protecting you from putting on treble or gain sufficient to overload the DACs. Certain types of sound will be
(a) grating and obnoxious
(b) technically not legal within the constraints of a Nyquist-observing sampling system.
That's in spite of the fact that the actual samples can be represented within that system. The files that I submitted? Every one could be stepped down one LSB less than full scale, which would mean that technically the data would not 'clip'. Yet the result is still totally illegal within the theoretical system- represents values that can not be arrived at through correct sampling of an input waveform, no matter what input waveform you chose. It could only represent a waveform way hotter than the maximum theoretical limit of the sampler- and on playback, this of course distorts horribly.
I'm demonstrating that the danger point for this, when handling really hot high-frequency sound, is at least 4db down from full scale. That's worse than I expected, though I am using pathologically difficult sounds akin to hi-hats cranked through raw digital clipping- but these types of very high frequency transients are common in DAW mixes, I just made examples where it's impossible not to notice what happens.

I suppose I can turn around and ask a question in return. Within your understanding of digital audio, and given that it is possible to upsample digital audio with arbitrary degree of accuracy (assume perfect upsampling) and given that you have sample values that are not actually in excess of full scale- can you have information in a waveform that goes away when you UPSAMPLE? If so, what is it?

I have just proved that you can have information, easily generated through DAW processing, that does go away. I am proposing that this information is 'illegal' and could not be arrived at through direct sampling- and for that reason, it breaks the playback system of the DAC and reconstruction filter- and that this is undesirable, sounds awful and needs to be avoided.

If you're not cranking the treble and pushing levels to upwards of -5db RMS level in your mixing and mastering, you don't have to do a thing, as only serious DAW abuse causes this. However, I think that is more common than uncommon these days... you yourself may be doing it.

If you do- you could fix it completely, just by changing the sample rate of your stereo master to 44.099 and using high quality (such as sinc interpolation) SRC to resample it to 44.1K, which SHOULD be an imperceptibly insignificant change, not even a downsampling, but can clear the glare right up if you're causing it.

And that's where I came in- that is exactly the assertion I made in the other thread. It happens to be true- and the difference can be as extreme as 12db of 'illegal' energy removed, if you're really over the top with it. This can be measured. I measured it.

Another way this relates to tape machines is, tape machines also guarantee you won't have this problem- both by not having it in the first place, and by tending to roll the extreme highs where the problem would manifest. That's not a given, though- you can get illegal 44.1K content even off _vinyl_ if you sample at a very quiet level and then amplify. There's plenty of vinyl records that have information that would have to be restricted to fit onto CD. If you sampled them directly, the input filters get it- but if you sample them at greatly reduced volume, and normalize, you can end up with illegal data that overdrives the DACs even without having individual samples clipping.

[Erik]

I think you're mixing up a bunch of stuff here.  There's nothing at all "illegal" about any 44.1 file.  And there's nothing you can do in a DAW to create energy in that file that's "illegal."

If you process audio through a non-linear process, it's not going to generate higher frequency stuff than is supported by the sample rate in the file.  It will simply alias, and that energy is reflected downward, not upward.  Because, well, there's nowhere 'up' to go.

Likewise what you're talking about isn't a 'treble' phenomenon anyway.  Generate a 21kHz sine wave full scale.  Do your upsample trick.  Check for level and distortion, also check post DAC if you have the gear.  Having removed some variables, what does that tell you?

--Erik

[chrisj]

You are mistaken, at least three times.
It's trivial to create an 'illegal' 44.1 file, and to do it through DAW processing of a 'legal' file. Sampling theory assumes a finite peak amplitude, normally considered as digital full scale. If that is 32767 to -32768, you can take a long series of samples at -32767 and insert one that is 32766 in the middle, and the reconstructed waveform implied by this data is wildly beyond full scale and cannot be directly sampled by a properly designed converter. Hence, 'illegal'.
It's also trivially easy to produce this stuff in a DAW, like it or not, by treble boosting and amplification, particularly if you are permitting digital clipping.
Your one good point here is that doing this does not generate higher frequency stuff- if you tried to represent the DATA accurately it'd force you to go to a higher sampling rate, but within the context of a sampling system the illegal data do not represent higher frequency content- it represents higher AMPLITUDE than permissible.
Here's a question for you about the sine wave. When you generate a full scale 21 or 22K sine wave- where are the actual data points, the sample values? Because the sine is full scale, does that mean the 16 bit data are at 32767 and -32768? I submit that in a properly designed signal generator they are not, and if you have a wave done properly (with the data points REPRESENTING a correct waveform post-reconstruction) there will be no change at all from a SRC 'bump'.
Doesn't change the fact that there's loads of stuff out there which would be changed by such a process, that you can generate it trivially easy by EQ and amplification, and that it breaks the sampling system and contributes to distortion and bad sound.
I actually have not once said that processing could generate frequencies higher than Nyquist within a sampling system- just looked over my previous comments to be sure. I am saying that there can be illegal INFORMATION. The fact that applying a brickwall filter to 22.050K removes this information does not mean that the information is over 22.050K. It happens to be information at or just below 22.050K... that has an amplitude you can't have in a Nyquist sampling system. It's just good fortune that the same process is capable of addressing both problems.

[Erik]

When you use the term "bumping" in relation to tape machines and tape, I assume you were talking about head bump.

Likewise with all the references to "Nyquist" I thought you were talking about frequency.

There's nothing illegal about any of the data you mention.  And you're oversimplifying the DAC reconstruction "issue" as well.

What you're really talking about is amplitude, not frequency, and that has nothing to do with Nyquist/Shannon.

You're really talking about a Fourier transform.  Because you're reconstructing the signal it's generating new signals.  Variations in amplitude caused by this are both simpler and more complex that what you're talking about above.  I assume you're generally interested in this stuff (which is why I bother) so I suggest you read up on Fourier and write a little code, with various filters, to explore and learn.

But your current scheme is basically to use SRC as a gain drop and a smoothing filter without understanding all the tradeoffs involved.

To the extent the goal is to reduce the level of the file to prevent the alleged "DAC clipping" problem, I'd say you're much better off just applying trim than going through these gymnastics.

And, repeat: there's nothing illegal in the files or the data you mention.

--Erik

[chrisj]

*sigh* and heavily limited high frequency material can be loaded with 'illegal' data that clips DACs because it represents a legitimate waveform that is considerably higher in amplitude than digital full scale.
I'm honestly sorry if I've erred in interpersonal ways here- thought I was being very nice. I'm quite sick this week and my fiance has been so sick that she cannot eat, causing me to be very worried. It's possible that the force of this tangent is driven by such personal reasons, and in other situations I might have entirely let it slide.
The content stands. I'll get a page about it added to my website (still desperately in need of updating) and be done with it.

[Erik]

Paging Nika... cleanup aisle three.

--Erik

[chrisj]

Nika will only confirm the existence of 'illegal' data in a 16 bit file that's supposed to be used for digital audio reconstruction. So has Pricey- I've just established differences of up to around 12db RMS, not less than a db. This is not MY idea, it's well known in the field- you'd find discussions about it under the term 'Gibb Effect'. This refers to a reconstructed waveform going beyond the power supply rails. All I've done is quantify it and ABX double-blind test the audibility of the effect using a pathological case or three. That's _all_ I've done.
Did come up with a page about it (linking to the files) at http://www.airwindows.com/analysis/MicroUpsampling.html. It says nothing that hasn't been said in this thread.

[bobkatz]

I think the problem here is Chris's definition of "illegal data"! As Eric is trying to point out, there's nothing illegal about the data in Chris's examples. The fact is that you can put out a CD of square waves at full scale and it will be legal. Anything you can put onto a redbook CD is by definition legal. If it fits in 16 numbers, it's legal, by definition.

However, what Chris is trying to talk about, and if he didn't use the words "are illegal", he'd be fine, is the fact that music can contain levels which "should" be illegal. Levels which when fed through a reconstruction filter such as in a DAC, or nearly any kind of filter, such as an SRC, or and MP3 encoder, or a broadcast chain (get my point!!!!)----will produce overload distortion because they produce output levels (generally, in the analog domain) which are higher than 0 dBFS.

It's as simple as that, and I do intend to cover this area in more detail in my book's second edition, though Paul Frindle covers it to a reasonable extent in his recent post: "When tracking or mixing, If all you have is a standard digital peak meter, it is advisable to keep your levels below -3 dBFS." This is a very safe level. Leave it to the mastering engineers to screw it up beyond that.

And Thomas Lund covers it in great detail at http://www.tcelectronic.com/media/nielsen_lund_2000_0dbfs_le .pdf

Hope this post helps!

BK

[Nika Aldrich]

Erik wrote on Sat, 21 August 2004 17:23

Paging Nika... cleanup aisle three. --Erik

Erik,

Sorry, I'm so lost on what is being said here that I don't know how to contribute.

Does this have to do with waveforms that can be constructed in a digital environment that are meant to represent waveforms that exceed full scale?  It sounds as though this may be the case?

To Bob Katz - yes, the definition of the word "legal" in this context can get difficult, but we often use that term to describe a waveform that fits within the bounds of the Nyquist requirements and thus will be accurately reconstructed.  A waveform that contains content above <.5sample rate has "illegal content," and a waveform that exceeds full scale, despite no individual samples exceeding full scale, also has "illegal content."  I assume this is how the term was being used?

Nika.

[chrisj]

Yup! Thanks, Nika- much appreciated
Basically, I've just done some experimentation with waveforms with illegal content, removed the content by UPsampling a miniscule amount, and ABX double-blind tested them to see if the difference is audible.
By the time you're reducing the RMS loudness of the illegal content 4db or so, it does become an audible difference. Less than that, _I_ couldn't hear.
That also establishes that the potential exceeding of full scale is a lot more than half a db or so. Instruments like tambourines and shakers and hi-hats are capable of producing worst-case scenarios for this situation, and treble boosting and amplification complete the picture.

[Nika Aldrich]

chrisj wrote on Mon, 23 August 2004 16:54

Basically, I've just done some experimentation with waveforms with illegal content, removed the content by UPsampling a miniscule amount, and ABX double-blind tested them to see if the difference is audible. By the time you're reducing the RMS loudness of the illegal content 4db or so, it does become an audible difference. Less than that, _I_ couldn't hear.

I didn't get that.

Quote:

That also establishes that the potential exceeding of full scale is a lot more than half a db or so.

Yes, of course.  This is a probability issue, though.  What is the likelihood that a given waveform exceeds 0dBFS?  What is the likelihood that it exceeds by a given amount?  The probability curve is heavily tilted towards very low excursion over 0dBFS, with a very drastic curve such that after around .5dB or so over the chance on a given non-test signal is very low.  On specific test signals you can easiliy exceed FS by 6dB or more, depending on the reconstruction filter in the D/A conversion process.  I don't think anyone said you can't exceed FS by more than half a dB, just that the likelihood of doing so with samples at FS is very, very small.  

You might check out the paper I wrote at tllabs.com on the subject.

Quote:

Instruments like tambourines and shakers and hi-hats are capable of producing worst-case scenarios for this situation, and treble boosting and amplification complete the picture.

Yes.  The worst case is definitely test signals, but in real world instruments the problem is most noticable with high frequency transients that come from percussion instruments.

Nika.

[chrisj]

Oh, I quite agree. Anyone whose mastered output is not hotter than -4db or so (and that's REALLY HOT) won't ever have a problem with this at all.
That said- if you make stuff hot, bright, and if you use peak limiting or simple digital clipping, this becomes totally relevant.
And how many modern-day major label releases are hotter than -4db RMS more than 50% of the time? Maybe this shouldn't be a common problem, maybe it should be very hypothetical and nobody should ever do this, but seeing as they DO, it might be nice to stop that one kind of 'bad distortion'. The highs sound nicer when you do, even though it's still distorted as hell.
I know that I'm going to be remembering this trick for if I ever get something mix-mastered and damaged that heavily (or bright and clipped). Being able to excise all the illegal content and neatly leave all the legitimate overbright overdistorted stuff is nice and since it's UPsampling, if there's nothing illegal, you lose nothing (also established by ABX). Real win/win situation.
It would be great if I could run that routine 'converting' from 44.1 to 44.1. There seems no theoretical reason why you couldn't- and I think you'd get the same effect. That would be very interesting. I need to look at the first instants of the files to see if it's really having an effect when the interpolation could be essentially nothing...

[Erik]

Nika is talking -0.5 dBfs peak and Chris counters with -4.0dBfs RMS.  Back to Nika.

Nika, I think you need to clarify your whitepaper into a simple bullet point.  A 'mastering peak levels for dummies' if you will.  

What can the baboons do to avoid this problem, and, if they're truly neurotic and/or want to spend some money, how can they be sure they avoid this problem on their precious mixes?

Thanks,
--Erik

[chrisj]

No, I countered with nearly 12db RMS measured difference on a particularly heinous realworld clip-fest sample. Not a theory, a real file. Nika is of course also entirely correct here, he is just referring to something else. We're not contradicting each other at all. I'm not really sure why this is so hard to understand. Oh, right, this is the Erik show. I remember this from all those Bomb Factory threads. Never mind. (Is everybody enjoying the spectacle, or is it getting boring?)
4db is simply the point where it becomes obvious to the ear. Less than that, you won't really hear the distortion but it will still be there- and I am talking _RMS_ level of the change between source file and upsampled file. Nika refers to the amount that a peak is likely to exceed full scale on NORMAL music. Normal music doesn't have this problem, except imperceptibly, and I would defy anyone to ABX something that clipped on reconstruction but was say average 6db from full scale. You won't hear it.

That's the best answer. Don't smash levels, and don't intentionally digitally clip bright stuff to get level out of it. How's that for a bullet point?

For those who feel they must do all these things because the next guy is doing them- and that's too many people and might even include you, Erik, so far as I know- there is something that you can do to clean up the highs. (It works better than simply reducing overall volume by .01db to stop the pressing plant from reporting clipped samples)

[Nika Aldrich]

chrisj wrote on Mon, 23 August 2004 17:21

Oh, I quite agree. Anyone whose mastered output is not hotter than -4db or so (and that's REALLY HOT) won't ever have a problem with this at all. That said- if you make stuff hot, bright, and if you use peak limiting or simple digital clipping, this becomes totally relevant.

OK, Just to clarify, the maximum amount of excursion above FS that you can achieve is dependant upon the reconstruction filter in use and its coefficients.  Dan Lavry and I worked out a simple scenario that gave around 8dB of potential excess signal above FS.  TC Electronic came up with 15dBFS by using a different reconstruction filter design.  The important thing is to try to simulate the reconstruction filter in use in your D/As when trying to calculate how much of a problem you will have.  This is what we tried to do with the Trillium Lane plugin.

Quote:

And how many modern-day major label releases are hotter than -4db RMS more than 50% of the time? Maybe this shouldn't be a common problem, maybe it should be very hypothetical and nobody should ever do this, but seeing as they DO, it might be nice to stop that one kind of 'bad distortion'.

I'm not really comfortable with the 4dB line you are drawing in the sand.  I guess a rule of thumb is helpful for "the baboons," but I think it is more relavent to tell people that just because their samples don't exceed full scale does not mean that the signal won't when it is reconstructed rather than giving them a set amount not to exceed, especially when it is really more fluid than that.  I guess this is where I really wish people were more engineers than baboons about this stuff - that they understood what they were doing and why rather than just following overly-simple rules of thumb.

Quote:

The highs sound nicer when you do, even though it's still distorted as hell. I know that I'm going to be remembering this trick for if I ever get something mix-mastered and damaged that heavily (or bright and clipped). Being able to excise all the illegal content and neatly leave all the legitimate overbright overdistorted stuff is nice and since it's UPsampling, if there's nothing illegal, you lose nothing (also established by ABX). Real win/win situation.

I don't follow your solution to the problem, here.

Nika

[Nika Aldrich]

Erik wrote on Mon, 23 August 2004 17:49

Nika is talking -0.5 dBfs peak and Chris counters with -4.0dBfs RMS.  Back to Nika.

Do I hear -6?  How about -8.  Do I have any takers?  Will anyone go -12?  I'm taking -12.  Do I have a -14? ...

Quote:

Nika, I think you need to clarify your whitepaper into a simple bullet point.  A 'mastering peak levels for dummies' if you will.

Just because your samples peak at less than full scale does not mean that the reconstructed signal will stay under full scale, and since your converters can't pass anything above full scale, upon reconstruction your signal (which "looks" safe to you) may still clip when played back.

Quote:

What can the baboons do to avoid this problem, and, if they're truly neurotic and/or want to spend some money, how can they be sure they avoid this problem on their precious mixes?

They can do one of two things:

1.  Simply turn down their master outputs to allow a little bit of headroom so that their samples are not just below full scale, but below full scale by a little bit extra so that if the signal DOES exceed the level of the individual samples it has some room to reconstruct.

2.  Buy a peak level meter that is more sophisticated than one that just reads the level of the samples.  Buy one that tries to reconstruct the signal the way the converters do before it tells you your peaks.  Then, if your signal exceeds FS even though your samples don't the meter will tell you that.  This type of peak meter is called an "oversampled peak meter" and ones are available from TC Electronic in the System 6000 amongst others.  I personally endorse (and helped design) the plugin called "Mastermeter" sold by Trillium Lane.  It does as much as 8x oversampling for Protools users, at a very reasonable price.

Cool?

Nika.

[chrisj]

Hmm-
Nika's meter should be an ideal preventative solution to the problem. 'Just don't distort', easy as that. I'll endorse that course of action, too.
Nika, I was pointing out that if you want to fix a file that's already damaged in this way, without attenuating anything in the normal range of hearing, you can cheat the metadata of the file so it claims to be 44.099K and then UPsample back to 44.1K using a really high quality SRC such as the Audacity program's sinc interpolation at a high quality level (actually, their low quality level is perhaps even subjectively better).
You shouldn't lose information in a high quality upsample- but you do lose all the illegal data that way. Nothing else is touched. In the absence of illegal data, this operation is imperceptible (from my ABX tests- anyone may try and better my results). If there's illegal data, the operation removes it, sometimes in an extremely obvious way. (from my ABX tests, 16/16)
Hence, a 'fix' for the problem that doesn't require turning the gain down on the rest of the audio. I thought that was relevant, because if someone making such a file was willing to turn the gain down, they wouldn't be having that problem in the first place, and so they can be assumed unwilling to not smash the levels and brightness. I think it's useful to be able to produce 'good' smash rather than just ugly smash. After all, it's fairly hard to hear clipped very high frequency transients unless they're doing something like aliasing or clipping the DAC. Excise the DAC-clipping stuff and you get simple digital clipping, which won't sound quite as nasty, just very loud.
Aside: isn't it fascinating how technology continues to provide uglier and uglier ways of clipping? First tube overdrive. Then transistors. Then TL071s driven beyond their power supply rails. And now, Gibb effect digital DAC-smashing. I guess it'll be DSD next. How much output can you get from hyperclipping SACD recorders?

[Nika Aldrich]

chrisj wrote on Mon, 23 August 2004 20:02

Nika, I was pointing out that if you want to fix a file that's already damaged in this way,

Hmm.  I'm a little leary, for this file isn't really "damaged" per se.  It is just too hot - it represents data that exceeds the bounds of the system (full scale).  The only solution is to reduce the signal level (by reducing the sample levels) until the signal is once again in the "legal" range.

Quote:

without attenuating anything in the normal range of hearing, you can cheat the metadata of the file so it claims to be 44.099K and then UPsample back to 44.1K using a really high quality SRC such as the Audacity program's sinc interpolation at a high quality level (actually, their low quality level is perhaps even subjectively better). You shouldn't lose information in a high quality upsample- but you do lose all the illegal data that way. Nothing else is touched. In the absence of illegal data, this operation is imperceptible (from my ABX tests- anyone may try and better my results). If there's illegal data, the operation removes it, sometimes in an extremely obvious way. (from my ABX tests, 16/16)

Ahh, I am suddenly understanding what you were going at.  Indeed a high quality SRC should be able to work around inter-sample overs, but only if it can deal with information in excess of full scale - I'm sure there are some that do so but I wouldn't count on a random, arbitrary SRC being able to do so.  But just because it can do so does not mean that it will adjust the signal level accordingly so that the file is once again legit.  It will more likely simply expose that you clipped (despite your meters) and remain a clipped file.  I wouldn't really call this a "solution."  I think it is better to simply drop the level accordingly so that you have enough headroom to play with upon reconstruction.  That seems to me to be the only solution to this problem.

Quote:

Hence, a 'fix' for the problem that doesn't require turning the gain down on the rest of the audio. I thought that was relevant, because if someone making such a file was willing to turn the gain down, they wouldn't be having that problem in the first place,

Hmm.  No, I think that people look at their digital non-oversampled peak meters and when they show no overs they assume that the file is clean, legit, and will be reproduced accurately.  I'll be that if people had a reconstruction, oversampling meter and saw that their signal was distorting and understood that, they would turn the signal down appropriately.  I don't think the problem here is people's stubbornness as much as a lack of the proper tools for the job and a lack of understanding.

Quote:

After all, it's fairly hard to hear clipped very high frequency transients unless they're doing something like aliasing or clipping the DAC.

Which is what happens...

Quote:

Aside: isn't it fascinating how technology continues to provide uglier and uglier ways of clipping? First tube overdrive. Then transistors. Then TL071s driven beyond their power supply rails. And now, Gibb effect digital DAC-smashing. I guess it'll be DSD next. How much output can you get from hyperclipping SACD recorders?

Actually, that's one of the cool things about SACD - you can't clip it like that.  The more you compress the lower you have to turn down your signal or you get modulator overload, which the SACD format doesn't accept.  So people that try to compress their mixes find they have to lower the level consequentially, which kind of defeats the purpose of the compression!  It is for this reason specifically that a lot of SACD mixes (masters) have much less compression and sound so much better - the mastering engineers intentionally give them more dynamic range because reducing the dynamic range is self-defeating.

Having said this, I don't want to ignore that DSD is essentially always in a state of clipping, but this aside, what I said above is pretty significant.

Nika.

[Geetar]

Erik wrote on Mon, 23 August 2004 12:49

What can the baboons do to avoid this problem, and, if they're truly neurotic and/or want to spend some money, how can they be sure they avoid this problem on their precious mixes? --Erik

I doubt that I shall ever tire of your contributions, Erik. To paraphrase Sir John Gielgud in "Arthur":

"Usually one must go to a bowling alley to meet a man of your stature."

You may know quite a bit, but I doubt anyone has ever so successfully undermined their own standing in such spectacular acts of internet-based social hara-kiri; you've become quite the Zappa-esque "Posting Fool."

Long may you run.

[dcollins]

Erik wrote on Mon, 23 August 2004 09:49

What can the baboons do to avoid this problem, and, if they're truly neurotic and/or want to spend some money, how can they be sure they avoid this problem on their precious mixes? Thanks, --Erik

I think baboons everywhere will have to drop the level to "resamplers" if you don't want more than you started with.  

But how much?

You can create weird test signals, but for "music" -0.5dB should be plenty.

Won't -3.01dB be enough for full-scale white noise? Which could include todays "top" mastering jobs...

The real problem is post-conversion headroom in "your" D/A which you can't predict.  Except in a device where all you have is 2AA batteries.

Is "Gibbs effect" really from bandlimiting?

DC
 

[Erik]

This whole thread goes back to an argument on rec.audio.pro between Chris and Arny Krueger 2+ years ago.  

If you want more background, surf there; if you want more background on Chris, hit his website.  If the first page doesn't put it all in perspective, nothing will.

--Erik

[Nika Aldrich]

dcollins wrote on Tue, 24 August 2004 07:58

Won't -3.01dB be enough for full-scale white noise?

No.  I've gotten signals to exceed full scale by more than 6dB (8 or 9, I believe) and TC Electronic claimed 15dB.  It's rather easy to calculate once you come up with your coefficients for your filter.  The higher the summed absolute value of the coefficients the higher you can exceed full scale.

Quote:

The real problem is post-conversion headroom in "your" D/A which you can't predict.  Except in a device where all you have is 2AA batteries.

I don't think you'll find an off-the-shelf D/A conversion chip by Sony, AKM, Burr Brown, Crystal, or Analog Devices (did I miss one?) that has any headroom in the D/A digital reconstruction filter above FS.  I also don't think you'll find a custom designed converter that does this.  In other words, I think nothing will give you the headroom, including the device with 2 AA batteries.  I'm willing to be shown wrong, here.

Quote:

Is "Gibbs effect" really from bandlimiting?

No, not necessarily.

Nika.

[magicchord]

Erik wrote on Tue, 24 August 2004 00:36

...if you want more background on Chris, hit his website.  If the first page doesn't put it all in perspective, nothing will...

What are you implying?

[malice]

magicchord wrote on Tue, 24 August 2004 21:00

Erik wrote on Tue, 24 August 2004 00:36 ...if you want more background on Chris, hit his website.  If the first page doesn't put it all in perspective, nothing will... What are you implying?

YOu do know Erik don't you ?

malice

[dcollins]

Nika Aldrich wrote on Tue, 24 August 2004 05:45

No.  I've gotten signals to exceed full scale by more than 6dB (8 or 9, I believe) and TC Electronic claimed 15dB.  It's rather easy to calculate once you come up with your coefficients for your filter.  The higher the summed absolute value of the coefficients the higher you can exceed full scale.

OK, but those signals are digitally generated, and might require knowledge of the filter coefficients.  I use white noise as an example that's a little more like music.  How far will that overshoot?

Quote:

Is "Gibbs effect" really from bandlimiting? No, not necessarily.

There was a series of posts from JJ a while ago about Gibbs, and Iirc them term is widely misused.

DC

[Nika Aldrich]

dcollins wrote on Tue, 24 August 2004 22:57

Nika Aldrich wrote on Tue, 24 August 2004 05:45 No.  I've gotten signals to exceed full scale by more than 6dB (8 or 9, I believe) and TC Electronic claimed 15dB.  It's rather easy to calculate once you come up with your coefficients for your filter.  The higher the summed absolute value of the coefficients the higher you can exceed full scale. OK, but those signals are digitally generated, and might require knowledge of the filter coefficients.  I use white noise as an example that's a little more like music.  How far will that overshoot?

It's a matter of probability.  White noise, being a random signal, certainly has a change of being numerically identical at a given point in time to a specific test signal.  White noise will theoretically hit ALL possible peak values at some point.  White noise, played long enough, should theoretically hit the maximum values possible.  Ergo, you can indeed get 6dB, 8dB, or even 15dB over with white noise, pending the filter coefficients used.

If you plot, however, the overs present due to white noise played at max=FS, you will find that there is a determinable curve representing its peaks, and that curve has exponentially high weighting at very close to 0dBFS and a significantly decreasing quantity of peaks beyond a few tenths of a decibel.  The chance of hitting the maximum excursion allowed by the filter coefficients is something like 16*65536! or so - not very likely.  As you go higher and higher above FS you have exponentially lower chances of hitting those peaks.  Music, being like white noise (as you pointed out) does hold to the same probabilities.

Does that make sense?

Quote:

and iirc them term is widely misused.

Yes, and if I understand correctly, this is one of those situations.

Nika.

[Erik]

Nika Aldrich wrote on Tue, 24 August 2004 18:26

... pending the filter coefficients used.

Likewise many DSP operations can potentially generate similar artifacts, depending on the filter design and/or windowing function chosen.  But you don't see fancy metering on them.  You've got to use your ears.

You really can't even get a mix that hot without using a limiter.  The damage done by a limiter or compressor is so much worse than what we're talking about here... and that happens every time you cross threshold.

--Erik

[chrisj]

Buh, I picked a fine time to finally start overhauling my website. Now all that's there is an elaborate panoramic pic of my studio, and I spent all day working on scans of photos covering everything from the instruments shown to a series depicting the construction of the studio mains shown.
Anyone who's confused by Erik's references? Most likely he's referring to what USED to be there- a mini-autobiographical blurb going back more than ten years, which explains how I have the form of autism known as 'Asperger's Syndrome', became homeless, landed briefly in a psych ward (they called me depressed. No shit..) etc etc.
Etc. etc, includes writing several novels, coming up with a peculiar handicraft best described as prosthetic tails for weird animal people (don't laugh too loud, it's paid for a lot of moist gear), and most relevantly, making the slow and painful transition from an enthusiastic crank to a half decent scientist able to examine his own conjectures instead of issuing PR blasts about them first.
I'm going to ask this once, and then go back to ignoring ad hominem stuff:
Do you expect me to be ashamed of surviving and rising above all that crap?
No way. I stand or fall as exactly who I am. If that's not good enough for you, Erik, that's just too bad.
As for Arny, he's an interesting guy, and the ABX software I used is found on his website. I ported it to MacOS (8 or 9) for him. For free.

[Erik]

Whoa buddy... not at all what I was talking about.

I wasn't talking about your life story, nor was I talking about how you raised money for an 8 track digital studio by selling fetishwear.

To be perfectly blunt, this page is perhaps one of my favorites on the entire Internet:

http://www.airwindows.com/costuming/index.html

Aside from the obvious, the MBA in me is tempted by the obvious business opportunity.  Specifically, the five-tailed kitsune.  Since they're all 50 smackers, the five-tailed version is like getting four tails for free.

Methinks I could buy five of those, set up my own website, then sell 25 mono-tailed kitsunes (or fox) for a tidy profit.  Imagine what kind of digital studio I could slap together with that kinda dough.

--Erik

[Nika Aldrich]

Erik wrote on Wed, 25 August 2004 01:42

You really can't even get a mix that hot without using a limiter.

You can exceed FS and clip your D/As without a limiter.  You can do it with gain change, normalizing, or even without any alteration at all, straight off the A/D, though this would be rare.

Nika.

[chrisj]

Hmmm... I think with a properly designed ADC (note the caveat!), even if you distort it to hell, you'll still have a very tough time getting anything out of it that would reconstruct significantly ABOVE full scale. At full scale, yes- but above it?
Wouldn't it depend on how good the anti-aliasing filters are? If they're really good, you wouldn't have any illegal content. Just content that sits at FS because you're overloading the ADC.
I thought this was how some guys get such extremely treble-rich but ungrating masterings- go into analog, fry the treble up good, hit the ADC and it prevents anything excessive from getting into the data stream...

[PP]

[chrisj]

Hey now, credit where credit is due. You wouldn't have heard anything about this if Erik hadn't been good enough to, er, mention it. (still wouldn't have, except that I had been revising my website at the time)
Maybe I still have some learning to do, since I seem to like such highfalutin plaudits a little TOO much. All that, and if my stuff sounds crap, it STILL sounds crap no matter how heroically I may have arrived at it. 44.1K files and ABX tests don't care if you're brave or cowardly. Let me stick to those for the time being.
Can we go back to talking about Gibb effect now? (what effect DO really lame hairstyles have on a close-harmony singing group?)

[magicchord]

Yeah, let's hand it to Bombguy for pointing out the people who are "different".

If it weren't for the "different" folks, we'd all still be recording on rocks.






Patrick Bryant
MagicChord Music

[Nika Aldrich]

magicchord wrote on Thu, 26 August 2004 17:45

Yeah, let's hand it to Bombguy for pointing out the people who are "different". If it weren't for the "different" folks, we'd all still be recording on rocks.

"Bombguy" may be case in point.