build-up

So, if you use the same mic over and over to build up layers of sound, say, vocals, what can we do to prevent that "buildup" that occurs as the mic's particular eccentricities become emphasized?

All of my tracks are composed of synths, VSTi's, vox and guitar, and always ITB. There is always a harshness to my completed work that I have trouble avoiding. I'm aware of it, always, and almost never boost any of my eq's.