factors (Qi) of about one million at the single-photon level. These devices are essential components in superconducting quantum processors; they also serve as proxies for understanding the energy loss of superconducting qubits. By systematically varying several fabrication steps, we identify the relative importance of reducing loss at the substrate-metal and the substrate-air interfaces. We find that it is essential to clean the silicon substrate in hydrogen fluoride (HF) prior to aluminum deposition. A post-fabrication removal of the oxides on the surface of the silicon substrate and the aluminum film by immersion in HF further improves the Qi. We observe a small, but noticeable, adverse effect on the loss by omitting either standard cleaning (SC1), pre-deposition heating of the substrate to 300°C, or in-situ post-deposition oxidation of the film’s top surface. We find no improvement due to excessive pumping meant to reach a background pressure below 6×10−8 mbar. We correlate the measured loss with microscopic properties of the substrate-metal interface through characterization with X-ray photoelectron spectroscopy (XPS), time-of-flight secondary ion mass spectroscopy (ToF-SIMS), transmission electron microscopy (TEM), energy-dispersive X-ray spectroscopy (EDS), and atomic force microscopy (AFM).