Conversion between signals in the microwave and optical domains is of great interest both for classical telecommunication, as well as for connecting future superconducting quantum computersinto a global quantum network. For quantum applications, the conversion has to be both efficient, as well as operate in a regime of minimal added classical noise. While efficient conversion has been demonstrated with several approaches using mechanical transducers, they have so far all operated with a substantial thermal noise background. Here, we overcome this limitation and demonstrate coherent conversion between GHz microwave signals and the optical telecom band with a thermal background of less than one phonon. We use an electro-opto-mechanical device, that couples surface acoustic waves driven by a resonant microwave signal to an optomechanical crystal featuring a 2.7 GHz mechanical mode. By operating at Millikelvin temperatures, we can initialize the mechanical mode in its quantum groundstate, which allows us to perform the transduction process with less than one quantum of added thermal noise. We further verify the preservation of the coherence of the microwave signal throughout the transduction process.