Ideally you'd first learn some general electronics, because audio electronics is only a specific application. You sound like a fairly hands-on person however and most text-books immediately tend to go into solid state theory that you really don't need to get a feel for how a circuit works.
Instead, work your way into the subject by building circuits that are easy to understand functionally. Things like op amp based circuits, for instance. If you want to build a simple preamp, you need only the crudest insight in what an op amp does in order to understand the circuit well enough to get sound. Then move towards more complicated circuits, or circuits that require a deeper understanding to make work, like small power amps built around op amps. Valve amplifiers are also a good entry point if you want to build something first and learn to understand it later.
An alternative is repairing things. Preferably items that you don't care f---ing up entirely when things go wrong (which they will). Get stuff from dumps and charity shops and try to get it working. Once you've got experience with how things fail, you will probably have an insight in how they work as well.
Either way, experiment a lot and try to work out why things do what they do. And whenever you think you have improved your understanding, try it out. Don't just go for fancy projects, most things you learn when you don't have to worry about messing up. I have been extremely lucky in this regard. Before manufacturing headed east and before environmental law we had a local junk yard (a former army dump with the unlikely name "Jerka") where several assembly plants dumped their unused components and rejected boards. Scavengers like me and my dad would then descend on the place and sift through it, paying for our pickings by weight. I still have literally tons of components from those days. This allowed me to tinker without ever worrying about money. It may be hard to to the same thing again these days but whatever you do: make sure you collect an assortment of readily accessible parts.
Once you find yourself able to build simple amplifiers etc, start expanding your theoretical horizons. Trying to do this before you start is perfectly possible but probably demotivating. Learn to use circuit simulators (spice). Learn to use the laplace transform (not necessarily the theoretical basis, learn to use it first) to calculate things like the frequency response of EQ circuits. Once you get to do that you will find yourself able to understand relatively complicated things like discrete amplifiers.
My own career is best described as "blundering", learning stuff wherever the fancy took me and with utter disregard for the path set out by traditional course materials. You will at some point need all that theory, mind you, but only when you want to do really fancy stuff. So here is roughly the order in which I learned the trade (starting from age 12):
Programming. BASIC, then assembly, then Pascal. Never fancied C.
Digital audio basics: FIR filters and how they related to discrete fourier transform. DSP experiments on .wav files that I replayed afterwards to hear the result.
Basic op amp circuits.
D/A converters (using ICs).
Basic noise shapers. DSP programming.
Basic class D power stages. Basic digital PWM.
Basic class D feedback circuits.
Advanced class D power stages. Device physics
Sigmadelta conversion. IIR filters
Discrete A/D and D/A converters
Solid-state amplifiers and discrete op amps.
Nonlinear control theory
To any teacher of electronics this looks like the course book from hell. But for me it worked! At every stage I had something that played music, something to be proud of, while theoretical insights only coalesced around it afterwards. In fact, much of it was paid work! Some people are able to study theory for 4 years before buying their first soldering iron but judging from your question, like me, you're not one of them.
Looking back it's absolutely frightening how far I got before I really had to dig deep into the maths. And I only realised recently that the folks who went to university usually forget all those things by the time they need them and often never get to re-learn them and end up descending into trial&error thus missing the benefits of theory altogether. It truly is dangerous to do too much theory before practice.
Well that's my take on the matter.