there is always the tendency that when you are doing something intensive in one area, you let other areas slide.
I always tell my students that keeping the feet going is what keeps your fills from speeding up (or slowing down)
besides if you are doing something polyrhythmic, you need to have something to be polyrhythmic against.