I had this dilemma one time and decided against it,in that the holding power of the chucks surface area is diminished tremendously and is relying on the chuck to clamp on the cutters which are soldered onto the tool. They weren't made for that type of pressure and might chip or damage or come loose in operation.
That said, I solved the problem with drilling a hole in a piece of plywood and clamping it on the router table to raise the feed of the stock up high enough to guide the bearing on the template.
Herb