Yes, edges and bits and pieces of bare foam only.
I guess if you literally skinned the entire thing with PVA it would contract and risk warping. But along the edges and in dibs and dabs here and there, it acts as an excellent sealer and barrier to take aerosol enamel or primer...