Yes - that's been my experience as well, I'm not keen on router bits in the drill press.
I've run them in a mill/drill which elimated vibration, but still ran too slow.
This thing I've bought is a very heavy duty drill stand which has as much adjustment possibilities as a radial arm saw, plus an (admittedly lightweight but still rigid) adjustable table which tilts, swivelsand raises.
It's obviously commercial, but no maker's name, and takes a standard black and decker type electric drill.
I want to use a small router in it - and I'm wondering is there a router which will fit straight into a 43mm drill press collar?