Wash Mitt every time. For all the previous mentioned reasons, plus your hand is insulated against the cold. (rather like a wet suit)
Make sure you keep it clean and before use I always check for any trapped foreign objects.
Good luck with your choice.
I use Meguairs products which were recommended to me, and here I am six months later recommending them.
Take care,
John