I've noticed several daws are implementing a form of hybrid buffer (reaper, studio one, cubase?, Possibly others) where there is a separate buffer for the audio playback tracks and the tracks record enabled. This allows you to have a low buffer on the record enabled tracks, for low latency, and high buffer on the playback tracks for maximum DSP power.