Have no trouble outside-wasting thick stock
The dynamics on inside cutting are indeed different.
One thing for sure I would not plunge holes; that is a drill press operation.
If you're drilling with a router bit in Al. you're killing the bit before it has a chance to do your excavational work.
See if a new cutter will waste the inside.
I would definitely use spiral ground solid carbide; in your case (unless CNC) something short and with a 1/2" cutting diameter. A slow spiral up or down.
You may need a consultant you're right on the edge of needing it.
Why? Because this stuff is dangerous. The utmost in router control and rigid fixturing is essential. Moreover, with an excavation as large as yours, in my view, should be done on a mill. Not only for the sake of safety but it will be faster and easier. Routing aluminum is very tricky stuff.
Mill through with a milling tool? Possible if it has bottom cutting capability but, again, in my view, holes should bored on the drill press, not routed.