I realize I didn't read your original post closely enough. Make sure the bit you are using is meant to be plunged. Not all straight bits are. Unless you are plunging a small distance I would drill a starter hole at the least and possibly hog as much waste out by drilling as possible. Most drill bits will remove waste as fast as a router and much more cheaply.
Someone I consider a master woodworker once told me that a master woodworker is not someone who never makes mistakes. He is someone who is able to cover them up so that no one can tell.