Personally I wouldn't be overly concerned about breathing a bit of graphite dust. Of course you don't want to be choking in the stuff, but a little floating around in the air isn't likely to do much to you. It's just carbon and the particulate size produced by machining would be far larger than those produced from a campfire or even a burning candle. To keep the mess down I mount the nozzle of a shop-vac close to the chuck and run it when I'm machining. It does a pretty good job of containing the mess.
https://www.tedpella.com/MSDS_html/Carbon and Graphite Products msds.pdf