TO BE ON THE SAFE SIDE

{adv. phr.} To take extra precautions; reduce or eliminate the possibility of a mistake, an error, or even danger. * /Dad always keeps his valuables in a bank's safe deposit box, just to be on the safe side./ Compare: JUST IN CASE.