What advantages are there in recording w/96 sample rate?
I would say the biggest and most noticable difference is that using plugin EQ's sounds a lot better with 88 or 96K. I personally prefer 88.2k because it's simple math to get it to 44.1 for CD's (as opposed to "fuzzy math").

You get less digital artifacts that way. Anyway, the high frequency EQ sounds much more like an analog EQ when boosting. When boosting at 88 or 96 the high frequencies that you're trying to boost don't get chopped off by the anti-aliasing filter, and therefore it sounds better.
I just got a digi002, and did some comparisons with 44.1 and 88.2, and it is a noticable difference when using high frequency eqs. I think overall, aside from the EQ, it sounds slightly more open, but nothing that will change your world. It all gets chopped off on CD anyway. I still pray for the day that DVD audio catches on.
Jimbo