Naturally occurring background radiation is a source of correlated decoherence events in superconducting qubits that will challenge error-correction schemes. To characterize the radiationenvironment in an unshielded laboratory, we performed broadband, spectroscopic measurements of background events in silicon substrates located inside a millikelvin refrigerator, an environment representative of superconducting qubit systems. We measured the background spectra in silicon substrates of two thicknesses, 0.5 mm and 1.5 mm, and obtained the average event rate and the integrated power deposition. In a 25 mm^2 area and the thinner substrate, these values are 0.023 events per second and 4.9 keV/s, counting events that deposit at least 40 keV. We find the background spectrum to be nearly featureless. Its intensity decreases by a factor of 40,000 between 100 keV and 3 MeV for silicon substrates 0.5 mm thick. We find the cryogenic measurements to be in good agreement with predictions based on measurements of the terrestrial gamma-ray flux, published models of cosmic-ray fluxes, a crude model of the cryostat, and radiation-transport simulations. No free parameters are required to predict the background spectra in the silicon substrates. The good agreement between measurements and predictions allow assessment of the relative contributions of terrestrial and cosmic background sources and their dependence on substrate thickness. Our spectroscopic measurements are performed with superconducting microresonators that transduce deposited energy to a readily detectable electrical signal. We find that gamma-ray emissions from radioisotopes are responsible for the majority of events depositing E<1.5 MeV, while nucleons among the cosmic-ray secondary particles cause most events that deposit more energy. These results suggest several paths to reducing the impact of background radiation on quantum circuits.[/expand]