It isn't quite as simple as that.
If the LPG is derived from gas associated with oil extraction, that gas would normally be flared off. If that LPG is instead used to fuel a car, it will produce just the same amount of CO2 as if it were flared off, but it will save the CO2 that would have produced by the petrol that would otherwise have been used to fuel the car.
In other words, LPG that would have been flared off saves the CO2 that would have been produced by burning petrol, and doesn't emit any more CO2 than would have been produced by flaring it off.
Now the problem is that LPG became too popular. The demand could not be fully satisfied from only the sources associated with oil extraction, and the additional LPG had to come from gas suppliers. This definitely wasn't zero carbon.
So the Government had to try to balance the amount of subsidised LPG with the amount being produced from oil fields. This is why the subsidy for converting vehicles to dual fuel was withdrawn, and why the reduction in vehicle excise duty was phased out, bringing the demand for autogas LPG more into line with the supply from sources that would otherwise have been flared off.
The good news is that LPG itself is still cheap. But now, the only saving comes from the lower fuel cost.