...... Regular drill tips can drift off just a bit especially on longer (depth:diameter) holes & deviation can feel like a slightly tighter fit to the length of a valve stem. A reamer will basically follow this pilot hole so you are chasing your tail. ....
To drill deep holes without the drill wandering, rotate the work piece and keep the drill stationary - if the drill starts to wander, the drilling forces push it back on to the centre of rotation of the work piece
The easiest way to set this up is to mount the work piece on a faceplate in the lathe and drill from the tailstock. Holding the work in a four-jaw independent chuck is an option, but the faceplate is usually quicker and easier
All the best,
Ian