Motivated by recent advancements highlighting Ta as a promising material in low-loss superconducting circuits and showing long coherence times in superconducting qubits, we have exploredthe effect of cryogenic temperatures on the growth of Ta and its integration in superconducting circuits. Cryogenic growth of Ta using a low temperature molecular beam epitaxy (MBE) system is found to stabilize single phase α-Ta on several different substrates, which include Al2O3(0001), Si(001), Si(111), SiNx, and GaAs(001). The substrates are actively cooled down to cryogenic temperatures and remain < 20 K during the Ta deposition. X-ray θ-2θ diffraction after warming to room temperature indicates the formation of polycrystalline α-Ta. The 50 nm α-Ta films grown on Al2O3(0001) at a substrate manipulator temperature of 7 K have a room temperature resistivity (ρ300K) of 13.4 μΩcm, a residual resistivity ratio (RRR) of 17.3 and a superconducting transition temperature (TC) of 4.14 K, which are comparable to bulk values. In addition, atomic force microscopy (AFM) indicates that the film grown at 7 K with an RMS roughness of 0.45 nm was significantly smoother than the one grown at room temperature. Similar properties are found for films grown on other substrates. Results for films grown at higher substrate manipulator temperatures show higher ρ300K, lower RRR and Tc, and increased β-Ta content. Coplanar waveguide resonators with a gap width of 3 μm fabricated from cryogenically grown Ta on Si(111) and Al2O3(0001) show low power Qi of 1.9 million and 0.7 million, respectively, indicating polycrystalline α-Ta films may be promising for superconducting qubit applications even though they are not fully epitaxial.[/expand]