between the system and the external field can also give rise to a geometric phase, even when the field is in the vacuum state or any other Fock state. Here we demonstrate the appearance of a vacuum-induced Berry phase in an artificial atom, a superconducting transmon, interacting with a single mode of a microwave cavity. As we vary the phase of the interaction, the artificial atom acquires a geometric phase determined by the path traced out in the combined Hilbert space of the atom and the quantum field. Our ability to control this phase opens new possibilities for the geometric manipulation of atom-cavity systems also in the context of quantum information processing.