which the qubits can interact, causing irreversible decay of the quantum state. As a practical matter, however, there exists a high density of excitations out of the superconducting ground state even at ultralow temperature; these are known as quasiparticles. Observed quasiparticle densities are of order 1~μm−3, tens of orders of magnitude larger than the equilibrium density expected from theory. Nonequilibrium quasiparticles extract energy from the qubit mode and induce discrete changes in qubit offset charge, a potential source of dephasing. Here we show that a dominant mechanism for quasiparticle poisoning in superconducting qubits is direct absorption of high-energy photons at the qubit junction. We use a Josephson junction-based photon source to controllably dose qubit circuits with millimeter-wave radiation, and we use an interferometric quantum gate sequence to reconstruct the charge parity on the qubit island. We find that the structure of the qubit itself acts as a resonant antenna for millimeter-wave radiation, providing an efficient path for photons to generate quasiparticle excitations. A deep understanding of this physics will pave the way to realization of next-generation superconducting qubits that are robust against quasiparticle poisoning and could enable a new class of quantum sensors for dark matter detection.