Clamping down is the same as adding weight. Adding wt. frustrates the slip & slide alright but it is not the way to stop stuff from pivoting!
So what? There are 12 degrees of freedom of a work piece on a flat surface!
In/out, l/r, up/dn and 3 rotational components (cw/ccw) relating to the 3 axes of the work or fence, adds up to 12.
So there is a lot of opportunity for the work to move.
A couple of sticks clamped against the back of your guide would help.
But & notwithstanding, clamp down sticks and routers zipping along them provide a lousy means of steering the cutter from point A to ponit B.
Cleats, clamps, stops, fences, high friction surfaces will add & abet.
But be advised, clamping down, by itself, is a recipe for slip/slide city.