for implementing these operations is to couple these resonators via a nonlinear converter and actuate parametric processes with RF drives. It can be challenging to make these processes simultaneously fast and high fidelity, since this requires introducing strong drives without activating parasitic processes or introducing additional decoherence channels. We show that in addition to a careful management of drive frequencies and the spectrum of environmental noise, leveraging the inbuilt symmetries of the converter Hamiltonian can suppress unwanted nonlinear interactions, preventing converter-induced decoherence. We demonstrate these principles using a differentially-driven DC-SQUID as our converter, coupled to two high-Q microwave cavities. Using this architecture, we engineer a highly-coherent beamsplitter and fast (∼ 100 ns) swaps between the cavities, limited primarily by their intrinsic single-photon loss. We characterize this beamsplitter in the cavities‘ joint single-photon subspace, and show that we can detect and post-select photon loss events to achieve a beamsplitter gate fidelity exceeding 99.98%, which to our knowledge far surpasses the current state of the art.