I agree about the torx bolts usually not being a problem. The manifold gasket isn't a hideously expensive part - I tend to change it every time the manifold's off, but, I can see that it's one of those gaskets that you probably could get away with.
While the manifold is off, you might also consider;
a) replace any broken high pressure pipe clamps
b) replace the rubber crankcase breather push in connectors - these typically become brittle, and break as you remove them to take the manifold off [you must remove at least one of them!]
c) check and note down the glow plug resistances for later reference (assuming they are all working OK now!)
d) lubricate the accelerator linkage pivot on the pump, with a drop or two of ATF.
e) check all of the vacuum connections
f) check the resonance flaps are free to move, and that their vacuum actuators work - i.e., make sure their diaphragms are not split.
g) plug the vacuum pipes, check to make sure nothing's loose near the inlet ports, and briefly run the engine. This will allow you to see if the inlet valves are seating well. My engine has had, and a few others that I've heard about and discussed on various fora have had problem with inlet valve seat recession which causes compression loss, and rough running - especially when cold. If there's a problem you'll see chuffs of exhaust smoke coming out of the inlet ports!
Points a to g are all quick, easy, cheap, and IMO well worth doing while you're there.
For points a and b, unless this area has been worked on quite recently, you *will* need some new parts, and so, it's worth ordering them in advance.