Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Nika Aldrich]

Zoesch, Bob Katz, Steve Parker, et al,

The discussion came up with Bob Cain in respect to analyzing how we hear.  We do not really hear in the Fourier Transform method that is the very basic model of hearing.  We have a non-linear system that decomposes frequencies much like a Fourier Transform, but not as linearly as FT would imply.  For this reason we can modify our model in one of two obvious ways - we can add all sorts of qualifiers to the FT to try to incorporate Fletcher Munson and other non-linearities in the hearing system, or we can use another tool that might be a modified FT that breaks the frequency spectrum, not into sine waves, but into some other waveform such that a simple analysis yields the complex results that are more akin to human hearing.

Again, we can either take a transform tool of simple waveforms (sine waves) and make it very complex with all kinds of qualifiers, or we can use a simple transform tool of a complex waveform (wavelets).  Either way we get the same results, though through different means.  For various reasons one tool might be more appropriate at one time and another tool might be appropriate at another time.

Just for an example, we have been discussing on the pro-audio mailing list about this window of analysis in the ear's hearing that affects what we hear in special situations.  Creating a FT based model of the ear would get very complex if we add this characteristic in.  Using a wavelet based model may give a more accurate result with a simple analysis - so long as the wavelet model chosen was accurate for the situation.  There are many other situations wherein the ear is non-linear and simple FT analysis doesn't really represent the ear's hearing well at all.  On the other hand, the vast majority of our hearing IS fairly Fourier based, so it is still an effective tool for a lot of purposes when discussing an auditory model.

I like Bob Cain's idea of looking at human hearing from a wavelet model to perhaps uncover several areas of suspicion, including Bob Katz's theories about the benefit of more gentle filters.  This issue clearly gets lost on basic Fourier analysis models of the ear but may become more obvious with a good wavelet model.  

I have been following the other threads for some time and never saw anyone explain where Bob Cain's approach to the problem was "wrong."  I certainly don't see any evidence that this idea he puts forth is worthy of being called a "crackpot."  It is far more valid from my assessment than many of the other theories about audibility beyond the "audible range."

Zoesch and Steve Parker - this sounds like something up your alley.  Your thoughts?  Should we start a new thread?

Nika.

[steve parker]

" Back to wavelets for a second, it is possible to prove also that a signal can be mathematically composed of lots of little signals called wavelets, the same signal that can be shown to be composed of lots of sins.  That it can be shown to be composable from either shows that this is not a physical property but a mathematical one."


hi - i follow a lot of what you're saying....
the above is where i get stuck!

to me it seems a strange way to think about fourier (or any other) transforms.
in essence a fourier transform is not showing that a signal is composed of sines - rather it is taking amplitude and (infinite) time information and *extracting* frequency components.
short time fourier transform takes "stationary" sections to (kind of) get frequency/amplitude information over time - leading to bandwidth limitation.
wavelets extract the same info but without the bandwidth/time trade-off.

this to me doesn't seem a million miles from reality!
and it doesn't seem to me to be just a case of mathematical trickery.

you have got my interest up in terms of *how* the ear/brain actually does it.
i'd never really thought well enough about it and just assumed that the maths was to extract information that the ear just "heard" by physical stimulation of different bits?

steve parker.

[steve parker]

"Zoesch and Steve Parker - this sounds like something up your alley.  Your thoughts?  Should we start a new thread?"

hi nika,
i'm not sure that i'll contribute much...but i don't remember wavelets being discussed in relation to audio before (only EEGs etc) and i've wondered for a while if there is any advantage - there certainly would seem to be.
it'd be nice to learn a bit about what circumstances wavelet decomposition may be appropriate in.

so...i say yay....lets start a thread!

steve parker.

[Nika Aldrich]

fuze wrote on Tue, 06 July 2004 18:54

to me it seems a strange way to think about fourier (or any other) transforms. in essence a fourier transform is not showing that a signal is composed of sines - rather it is taking amplitude and (infinite) time information and *extracting* frequency components.

Any waveform can be shown to be derived from sines.  We can transform any waveform to its sine composition.  Just the same, however, we can prove that any waveform can be composed of any number of other types of signals.  We could show that any waveform is composed of triangle waves, or other waveforms and then we can use a specialized transform to find this composition.  

In many respects this indeed does not represent reality.  Indeed waveforms in the natural world are created because of the elasticity (resonance) of natural devices, and elasticity manifests itself in a sinusoidal capacity.  Things really DO resonate in combinations of movements that are sinusoidal.  For this reason, Fourier is indeed very closely tied to reality.

But while devices that CREATE sound are certainly based in physical laws that are steeped in sinusoidal activity, not all RECEIVING devices are so elegantly the same.  The ear, for example, is a very complex organism that has a lot of digital language going on inside of it and is based in a lot more than just elasticity and resonance.  It indeed does break waveforms into individual frequencies, kind of like a FT, but it doesn't do it as linearly as a FT.  The idea of wavelet analysis with respect to the ear is to allow a simple formula (like FT) to analyze what the ear hears, but to give more accurate results.  Rather than start with a sine wave as the basis we start with a more complex wave as the root of all waveforms such that the simple formula, when applied, more accurately reflects the complex transform that happens within the ear.

Call me at the office and I'll see if I can do this more justice.

Nika.

800 222 4700 x 1357

[PP]

[steve parker]

hi nika.

i realise that fourier transform does break the waveform into sines (which might seem out of touch with the ear).
but the meaning behind the transform is to extract frequency information from amplitude/time info (which doesn't seem so out of touch with reality).
our entire audio system is surely trying to model sound creation rather than reception.
i get that a model of the ear would benefit from wavelet analysis - but is there any benefit to (say) adc?

sorry if i ain't making that much sense - i have a nagging feeling that i've not got something straight somewhere.....
i understand fourier, wavelet and other transforms mathematically - i think bob's post has highlighted that i'm a bit fuzzy in terms of the *meaning* of these things for audio.
but i'm not quite sure even how i'm fuzzy?

sorry for such a vague post.
(i would call you at the office but i'm in england - i may still if i work out the time difference!)

steve parker.

[Nika Aldrich]

fuze wrote on Tue, 06 July 2004 21:37

hi nika. i realise that fourier transform does break the waveform into sines (which might seem out of touch with the ear). but the meaning behind the transform is to extract frequency information from amplitude/time info (which doesn't seem so out of touch with reality).

Sure, and this is how we like to think that the ear hears as well.  Our basilar membrane and IHCs divide the waveform into individual frequencies and transmit the presence of these frequencies to the brain.  It's a beautiful mechanism.  So the ear performs something very much like a Fourier analysis, but not quite.  Wavelet analysis helps us construct a potentially more accurate indication of what the ear does - it makes it closer to the ear's mechanisms than a straight Fourier analysis.

Quote:

our entire audio system is surely trying to model sound creation rather than reception.

"Entire audio system"?  You mean our recording equipment or our ears?  Our audio systems - being our equipment - do indeed need to more closely parallel the sound creation.  Our audio systems - being our ears - may more accurately reflect a wavelet type of analysis.

Quote:

i get that a model of the ear would benefit from wavelet analysis - but is there any benefit to (say) adc?

Not that I can think of.  But a wavelet analysis of the ear, for example, may poke a hole in the 20Hz to 20kHz model that we have.  We may find out that the high end of our hearing can touch over the 20kHz in specific situations because of non-linearities in the ear.  This is all unsubstantiated and pure speculation right now, but more helpful models of human hearing may indicate changes we need to make in ADCs, which sort of answers your question.

One thing we've been discussing on the mailing list (the start of the conversation that Bob Cain and Bob Katz have brought here) is the ear's response to certain types of waveforms - namely impulse responses from steep filters.  While all of that material is above 20kHz, because the ear is not linear there is a theory that some of that information becomes audible in certain ways.  Again, all speculation, but difficult to assemble with only a straight Fourier look at the ear.  If we use a wavelet model to discuss human audibility we might say "Hmm, this particular waveform creates an audible reaction in the ear, even though a straight and simple Fourier analysis of this waveform shows up as inaudible."  

Did that make any sense?

Nika.

[steve parker]

""Entire audio system"?  You mean our recording equipment or our ears?"

yep...i meant the recording equipment - how easy it is to be unclear!

"Did that make any sense?"

yep...got that straight now.
this was pretty much what i thought - i must look at the research on the ear more closely though.

i have some bandwidth questions wrt wavelets but i need to work out exactly what the questions are....

thanks!

steve parker.

[Zoesch]

The whole argument for Wavelet decomposition is based in three premises:

1)Audio signals are not fully stationary (They contain frequency components that vary in time)
2)The ear does resolve signals by frequency and time
3)Fourier analysis only provides the frequency-related information from the signal but can't provide time-related information.

These have also been some of the reasons why static additive synthesis can't generate exact replicas of sounds, there's a time-related component (Not just the envelope) which is lost.

Does wavelet theory have any use in audio? I think so, but I remain unconvinced for two reasons.

1)Wavelet decomposition does not contradict Nyquist even though you can sample below the Nyquist limit and still retain all the properties of the waveform.
2)To implement a system that uses wavelet decomposition principles you need windowed variable bitrate sampling, something that's not really easy to implement in real time (You need higher sample rates for HF signals, lower sample rates for LF signals and a bank of quadrature filters to split the incoming signal into frequency bands)

Once you achieve this, which format will you use to represent the signal? How will you process the signal? And so on ad nauseum... just like with DSD, the principle is "sort of" right but the implementation (Where all the issues lie) isn't crystal clear.

[bblackwood]

Hope I got it the way you wanted, guys. Have at it...

[Bob Cain]

bblackwood wrote on Tue, 06 July 2004 22:12

Hope I got it the way you wanted, guys. Have at it...

Wow that was quick!  Thanks.


Bob Cain

[Bob Cain]

Johnny B wrote on Tue, 06 July 2004 01:57

Now, Bob Cain has something he'd like to test, so I have a suggestion: Since Bob is in the Santa Cruz area, he might want to see if he could arrange for such a test to be set up and performed in conjunction with the University of California at Santa Cruz. I hear they have a pretty good engineering program there, so maybe they'd be interested in this.

This is something I would dearly love to do if I had any idea who up there to approach.

Quote:

I also have a goofy question of my own: Would a 256Khz/32-bit chip make any sense at all?

I would be _very_ surprised if anything remotely near that high could make it through the ear.


Bob

[Bob Cain]

fuze wrote on Tue, 06 July 2004 10:54

to me it seems a strange way to think about fourier (or any other) transforms. in essence a fourier transform is not showing that a signal is composed of sines - rather it is taking amplitude and (infinite) time information and *extracting* frequency components.

The Fourier transform says that if you compose a signal by adding up sin waves weighted by amplitude/delay coeficients they correspond to you will get the signal.

Quote:

you have got my interest up in terms of *how* the ear/brain actually does it. i'd never really thought well enough about it and just assumed that the maths was to extract information that the ear just "heard" by physical stimulation of different bits?

I don't know how it does it either but I am persuaded that it is not by finding complex Fourier coeficients.  That is too precisely defined an operation for nature to have stumbled upon it.  This is where I think a lot of people step over the edge.  As I said earlier I think that many of us have come to think of Fourier analysis as having some physical signifigance when in fact it is just one of many ways that a signal can be decomposed, all of them valid and all of them just a mathematical abstraction.

Yes, it has some specific and useful properties but there is no reason those properties are particularly advantageous to perception.  Those properties are mainly simplifications that make analysis and synthesis easy for us and easy to implement using simple, idealized components.  I don't think nature's components are such idealizations.  Sure there are similarities but if nature could find a way to capitalize on some of the non-ideal properties such as non-linearity I am certain it would.

I think that the ear/brain's feature extraction mechanisms are likely to be considerably more ad hoc and fitted to the information that evolution has decided is critical to our survival and reproduction using whatever biological components it can build.

All the above is just reasoning as to why sinusoidal testing of the ear's bandwidth might not tell the whole story.


Bob Cain

[Bob Cain]

Nika Aldrich wrote on Tue, 06 July 2004 10:39

I like Bob Cain's idea of looking at human hearing from a wavelet model to perhaps uncover several areas of suspicion, including Bob Katz's theories about the benefit of more gentle filters.  This issue clearly gets lost on basic Fourier analysis models of the ear but may become more obvious with a good wavelet model.

Actually, I'm not sure how productive looking at it from that angle can be either.  I'm not sure how in the hell you would probe a wavelet based system to discover what wavelet basis it was using.  That sounds like a horrendous search problem to me.  The search space is enormous.

The only reason I mentioned wavelets was to illustrate that there are many ways to skin the cat of extracting features from signals and that many of them are formally equivalent.  I don't think the ear/brain is any more likely to have evolved into a textbook wavelet processor than it is a Fourier processor.  It is just that I could build a feature extraction system using them which would reach perception thresholds on features which contain Fourier coeficients that in isolation would not reach perception thresholds.  It is a straw man for illustration of the basic idea I am trying to put forth that simple sinusoidal testing of the ear/brain capability may not tell us the bandwidth we need in reproduction systems to convey what we hear from direct sources.


Bob

[Johnny B]

Bob Cain,

Since you'd love to pursue a test at UC Santa Cruz,
why don't you arrange a tour and meet some people, then pitch them on your idea. You could bang the phones too.

I've been told that UC Santa Cruz, pound for pound, makes more significant long-term contributions than all the others in the California system, thus, something of this importance may be well received by willing ears...so to speak. Sorry, couldn't help that last bit.