I think that the issue is that the percentage of time when drivers hold a mobile phone while not using it for absolutely any purpose is miniscule. The photo or footage taken by the police will clearly show how the phone was being used, and my guess is that if the driver indeed wasn't looking at the phone or holding it to their ear, then he or she would need to produce phone records etc to convince the prosecution (or a jury) that they were just holding the phone for no obvious reason. And so, for all practical purposes holding the phone is a no-no, regardless of the exact wording of the legislation.
Additionally, while there may not be a specific law against holding the phone while it's not being used, the driver should keep both hands on the steering wheel at all times and only remove one hand if required for operating the vehicle, and so a driver holding a mobile phone in one hand can still be prosecuted for driving without due care and attention (which is what drivers were being prosecuted for before the specific legislation for mobile phones was passed), though I'd agree that this is only likely to be the case in the event of a crash, while in normal circumstances the driver would either be fined for using a mobile phone, or be let off.
Well , I did list the very practical purpose for which I might find myself with the phone in my hand , albeit only for a few seconds ; and I did qualify that to say it needed not to interfere with one's control of the car . It matters not how often or how seldom someone may happen to have a phone in their hand , the exception still exists , and as far as the law is concerned I could ( not that I would ) drive around with the thing in my hand all day long and no offence would be committed . Oh , unless something more had happened than simply being snapped by a camera , such a charge would be most unlikely to be brought in a solemn trial ( with a jury ) , being more akin to getting a speeding ticket , is would be far more likely to come up before a JP or magistrate in a summary proceeding , or indeed if not responded to , under the single justice procedure .
Lots of people take one hand off the wheel and remain competently in control ; indeed I knew a one-armed-man who had his driving licence and a very capable chap he was ; he was a grocer and the way he could handle boxes in and out of his Volvo estate had to be seen to be believed ; despite his physical handicap , he was an active member of our golf club and had a very low handicap ( pardon the pun ) .
While , if the circumstances demand it I do of course drive with both hands on the wheel , there can be times , when driving along a clear and straight road where one might have one hand on the wheel and the other not , perhaps with an arm on an arm rest , yet still be in full control . As long as your attention remains on the road , it is perfectly OK to take one hand away to adjust the radio volume or change channels , and of course , smokers often use one of their hands for something other than driving .
Phone records would not reveal whether the phone was being used for something other than telephony ( selecting a music track for example ) or for nothing at all , so that would be completely inconclusive , and if I were using a CB , PMR ( private mobile radio ) or amateur radio transciever , which can look very much like a carphone , then no offence would be committed ( with the usual proviso of being in control ) .
I have three or four of these Binatone sets , as well as two of the Audioline ones , amongst numerous other two way radios ; all of which are perfectly legal to use whilst driving .