We find that an inhomogeneously coupled array forbids a complete single-photon transfer between the external cavities when the central one performs a Jaynes-Cummings dynamics. This is not the case in the ultrastrong coupling regime, where the system exhibits singularities in the photon transfer time as a function of the cavity-qubit coupling strength. Our model can be implemented within the state-of-the-art circuit quantum electrodynamics technology and it represents a building block for studying photon state transfer through scalable cavity arrays.