There is a school of thought that says 'do not fold them unless you really have to' The wires that run in them are thin and weak and break. Some people fold their mirrors for years without problems, some not so lucky.
On my car the L/H mirror did not defrost, I removed it and immediately found a broken wire. A pretty easy fix if you have all the correct tools and a soldering iron...not so cheap and easy if you take it to a garage.
No car I have ever driven with folding mirrors has ever seen the mirror 'rub' on anything, strange one that . Photos ?