I'm not sure if this would work for you but, at one time I was asked to make the number signs for the camping spots in our RV park.
Nothing fancy, just some 1X6, cut into a rectangle then a 22.5° miter on each end.
My method was to line print what ever number I wanted, you can do this with Notepad by setting your font to print "outlines".
Once I had the numbers printed, I used a utility knife to stab/transfer the pattern to the wood, then went over the small marks in the wood with a "Sharpie" ultra fine point felt tip marker.
From there I used a simple ski about the same as you did.
It worked well and easily, although there was a bit of a learning curve to make the router do as I wished.... we have 32 sites, plus 10 of the sites are pull through from either end, so I ended up with 42 signs. By the time I had the third site number done, I had most of the process under control.
Just my way, YMMMV.