You can clamp an auxiliary fence on top of your existing fence. Rockler has a couple of small clamps just for that purpose. You drill holes in the top edge of the auxiliary fence and put the post part of those clamps into the holes. The clamp portion goes behind the fence. You don't have to remove anything from the existing fence. That should solve your problem. You should also be using a really wide, expendable push block to hold it down and against the fence at the same time.
In the future I'd cut the thickness first, then come back to cut the angle on the smaller pieces. with the cutoff on the side of the blade AWAY from the fence. That way you're pressing a flat side against the fence for all cuts.
When I make such cuts, I use two sticks, one to push the piece forward through the saw, and a second to push the piece against the fence. You could also use a feather board, but for this kind of cut, I like to press a longish piece of stock sideways against the work piece to guide it all the way up to the blade. Makes for a nice, stable angled cut.
This is one of those times you could also use a half inch thick push block with a hook on the back end to press the piece forward.
Below is the kind of push stick I'd use. You can make them quickly in the shop. The hook on the back end is needed for slender parts like what you're making.
Glad you asked.