Take less wood off with each pass. Reserve the last pass for maybe 1/32" to be removed.
Do your best to read the grain but sometimes you must route against the grain on a piece. Backrouting can help but if it can be dangerous to you and to your workpiece. Be very, very careful and only do it as a last resort.
Besides being sharp, a clean bit cuts better. Visual inspection for pitch and burns on the carbide. Clean with 409 or similar household cleaner and a stiff nylon bristle brush.
As to sharp, if you see the carbide is chipped, it isn't sharp. If you can see a flat edge where it should really be a crisp line, it isn't sharp.
And finally, I mention carbide, I assume you are using carbide tipped cutters? You can hone them lightly with a diamond hone. If you are trying to use HSS cutters, don't. They dull too quickly and are more trouble than they are worth unless you have no other choice.