It all depends on what you're doing, what you're doing it on and how you're holding it.
The tools for cutting plastic are worlds apart from tools for cutting metal. The blunt angles on metal cutting tools will chip and break plastic, and the sharp angles of plastic cutting tools will wear very quickly in metal. That's not to say you can't use metal cutting tools on plastic, but you'd be hard pressed to have success cutting metal parts with plastic cutting tools.
Typically you would want to use 2 flutes on alum and always use cutting fluid or some kind of lubricant. (Lubing the cutter to prevent welding is more important than heat removal on this kind of machine.) Using 4 flutes often provides better finishes but it's difficult to rough with them. Light cuts on a router might lend itself nicely to using 4 flutes. Hi spiral bits would help reduce tool pressure, provide better chip removal and leave a nice finish even under stressful conditions.
The (lack of) rigidity of the router is going to make it difficult to get a good finish and hold close tolerances so a good understanding of toolpath approach and the difference between conventional and climb cutting will go a long ways. I have not tried to cut metal on my router yet but once I get a better table/bench for it I will then I will be able to give you more informed advice.
For alum the surface speed is much more important. Too much and you get chatter and a bad finish, too little and you break the tool or keep picking your work up off the floor (depending on your workholding method). Again, the light cuts you'll be taking on a router will help negate some of this effect.
Give a little more info on what you're trying to do and I'll give you what ever advice I can.
Instead of seeing things and asking "why", envision things and ask "why not"