The Military Overview and your Energy reading at the top of the screen conceal the upkeep for your units. Unit maintenance is very low if you have a small standing army. Your Headquarters provides a base amount of supply, and then your cities and population add more to it. As long as your units don't demand more supply than you have, maintenance stays very low. Once you have more units than you can supply, the Energy cost of your force rises substantially.
Stay with a powerful but small defensive force when you need to save money and don't have a rugged economy to back things up. Once you start to have substantial income from trade, generators, and economic buildings, you can afford to field a larger military without gutting your economy.
Note that you must still maintain non-military units. Explorers, workers, and even trade units count toward your supply limit. Everything has a cost, right?