Contact adhesive releases with heat. I would suggest you mechanically fasten the insulation to the walls in addition to adhesive.
We used heat guns to remove laminated plastic from the wood core of counter tops without damaging either the plastic or wood. It did not take a lot of heat to break the bond.