Think you will find it's a pressure relief valve. If pressure into the reg from the bottle exceeds a set value then it lifts to protect the reg/gauge. could be it's not seating correctly. May seat with a gentle tap to the side of that area with SOME pressure from bottle applied.
It could also be that the diaphragm has failed and the pressure relief valve is doing its job and venting the gas rather than allowing cylinder pressure to pass straight through the reg,
You need to confirm the cylinder fill pressure and the max working pressure of the reg.
Is it old or designed for a non uk market, Pic of front may help.
i have never had any problems with old regulators, every old reg i've used that would be at least 20 yrs old for the most of them probaly nearing 50 have all functioned perfectly even with haveing hard life and lots of use. but at work most of the new reguators don't even make it to there offical replace date. they normaly ether start makeing funky squeeking noises, start leacking so they blow a bit of gas out the relief valve, but that adds up over a day if you don't notice they'll leack off more £'s worth of gas than another new rubbish reg