Technically speaking, the loading of the mic is supposed to be at a minimum when interfacing to a pre; especially earlier models with an o/p circuit less hefty than modern ones exhibit an increase of distortion as they are loaded down more. A practical relation of 1:5 (mic:pre) is specified as minimum load impedance, absolute values being around 200 ohms R.source (mic) and 1K or higher for R.load (pre). Some vintage pre's have an R.load of only 600 ohms, others do go up to 2K (e.g. V72) or 4K (e.g. V72a).
For modern transformerless high output condenser mics with an R.source as low as 50 ohms, a variation of R.load will basically make no difference, since these mics will perform equally well into both, high and low loads.
Transformer-based condensers can exhibit a divergence in frequency response, due to different interaction with the frequency-dependent impedance components of the transformer, resulting in increase of resonance peaks or damping thereof at either the lower or upper end of the audio band -- all depending on the construction of the individual transformer. This, in turn, may affect the sound to some degree.
Dynamic mics have a source impedance that varies with frequency, so a change of R.load will definitely make a difference here.
But, what it all comes down to in the end is the judgement by your ears, isn't it? Even if freq. response and/or distortion characteristic should not measure right...