manifests experimentally through Coulomb oscillations such as charge dispersion. In superconducting circuits, the link to the environment has typically been formed from tunnel junctions where the charge dispersion can be tuned by changing the ratio between the Josephson energy EJ and the charging energy Ec. If, instead, a transparent ballistic junction forms the link between the superconducting island and the environment, the charge dispersion is predicted to be suppressed far beyond the dependence on the EJ/Ec ratio due to imaginary-time Landau-Zener tunneling between Andreev bound states. Here we investigate the charge dispersion of a nanowire transmon hosting a quantum dot in the junction. We observe rapid suppression of the charge dispersion consistent with the predicted scaling law incorporating diabatic transitions between Andreev bound states. We also observe greatly improved qubit coherence times at the point of highest charge dispersion suppression. Our observations further our fundamental understanding of charging effects in superconductors and suggest novel approaches for building charge-insensitive qubits.