This came up about six months to a year ago when someone wanted ground deer antler to use as leavening for some special bread recipe. I can't remember if anyone came up with a solution. Otis' sanding suggestion might work if you used one of the carbide Dura-Grit discs. The grit is brazed onto the backing I believe and wouldn't likely come off on something as soft as deer antler.
Someone I consider a master woodworker once told me that a master woodworker is not someone who never makes mistakes. He is someone who is able to cover them up so that no one can tell.