basic research and for potential applications in quantum information processing. We present measurements of the dispersive interaction between an ensemble of helium atoms in the 37s Rydberg state and a single resonator mode by extracting the amplitude and phase change of a weak microwave probe tone transmitted through the cavity. The results are in quantitative agreement with predictions made on the basis of the dispersive Tavis-Cummings Hamiltonian. We study this system with the goal of realizing a hybrid between superconducting circuits and Rydberg atoms. We measure maximal collective coupling strengths of 1 MHz, corresponding to 3*10^3 Rydberg atoms coupled to the cavity. As expected, the dispersive shift is found to be inversely proportional to the atom-cavity detuning and proportional to the number of Rydberg atoms. This possibility of measuring the number of Rydberg atoms in a nondestructive manner is relevant for quantitatively evaluating scattering cross sections in experiments with Rydberg atoms.