Yeah, for some reason, our Xterras are designed to shut off the recirculate feature almost every time you manipulate the controls. Shut off the blower motor and turn it back on? You'll have to re-engage the recirculate button. I've always left my vehicles on recirculate all the time - it's much better than smelling diesel exhaust everytime I'm sitting in traffic, and I have a very sensative nose to smells so it would always bother me a lot. It took me a while to get used to having to press that button several times during a single road trip.
That being said, I've also noticed that even with the recirculate feature on, I can always still smell a little bit of exhaust from other vehicles when in traffic, and I always end up with a lot of dust still inside the rig from the trails. I believe I have a leak in the weather stripping or the floor of my rear hatch area somewhere, because I actually ended up with LOTS of dust built up on my pillows in the very back of the rig during the DV trip.