The smaller and lower priced scroll saws all seem to have this same excessive vibration problem, especially when cutting at higher speeds, and it doesn't seem to matter which brand or model saw. They all seem to be prone to this. The only exception that I know of are the Hegner saws. The larger saws like the DeWalt , Delta, Porter Cable, and Excalibur 18-24" saws vibrate much less and are much easier to use at all speeds.
I own a Delta Q3 and a Dewalt DW788 saw and am very satisfied with their low vibration and how well they will cut at any speed. I have used several of my friend's Excaliber and Hegner saws and I like how well they cut, and I have tried to help some friends with the smaller saws without much success. The best improvement that we made was to bolt a Dremel 16" saw to an 24" square steel plate 3/8" thick. It made quite a difference in reducing the vibration, but it also made the saw too heavy to move easily.
Look online for PGT (Precision Ground Teeth) scroll saw blades for the best cuts. Most of the standard Olson blades that I've tried had more tooth set on one side of the blade than the other and this caused them to cut at an angle and produce a rougher cut. No two of their standard blades ever seemed to cut the same way for me either. Their PGT blades cost a bit more, but produce a much higher quality cut higher quality cut that far outweighs their slightly higher cost. I now use Flying Dutchman PGT blades because I've found their quality to be even higher than the Olsen PGT blades.
Take one of your Ryobi thumb screws from your saw to one of the larger hardware stores (if you have one nearby) that have the specialty parts section. They may have a knob in stock with the same metric thread, but with a wing type handle, but be careful that the new knob doesn't interfere with the operation of your saw. It might also be possible to use socket head cap screws in place of your knobs and then use an Allen wrench to loosen and tighten them.
Central North Carolina