One of the many problems is that a proper national network of cycle paths doesn't exist. There are just bits and pieces of cycle path here and there. Some of these 'bits' might be useful to say local children at a local school, but depending on where you are travelling from and to, it's sometimes more dangerous for the cyclist to leave a path at it's end join say a major road, butting out from "nowhere" into traffic. As I say, lots of other problems with compelling cyclists to stick to cycle paths (which is why The Highway Code no longer suggests this) - so the road is for everyone to share in harmony.
The one thing I don't tend to get is...on say an hours journey in a car there are loads of other things that slow you down. Things like traffic lights, roundabouts, speed humps, speed cameras, speed restrictions, pedestrian crossings, slower drivers, drivers waiting to turn right etc. A person on a bike is just another one of these things. How much time does it 'cost' to slow down a little and give a cyclist wide berth and then speed up again. 5 seconds? 10 seconds. No time at all in relative terms.