mechanics. While spectacularly successful at controlling natural and artificial quantum systems, certain limitations may arise (e.g., the achievable gate speed) due to non-idealities like the counter-rotating term. Here, we explore a complementary approach to quantum control based on non-resonant, non-adiabatic driving of a longitudinal parameter in the presence of a fixed transverse coupling. We introduce a superconducting composite qubit (CQB), formed from two capacitively coupled transmon qubits, which features a small avoided crossing — smaller than the environmental temperature — between two energy levels. We control this low-frequency CQB using solely baseband pulses, non-adiabatic transitions, and coherent Landau-Zener interference to achieve fast, high-fidelity, single-qubit operations with Clifford fidelities exceeding 99.7%. We also perform coupled qubit operations between two low-frequency CQBs. This work demonstrates that universal non-adiabatic control of low-frequency qubits is feasible using solely baseband pulses.