Four armory stats are upgradeable: Armor, Sword, Boots, and Rocket. Upgrading increases your chances of getting farther up the tower. When upgraded, Armor increases health, Sword increases damage inflicted, Boots increase your attack speed, and Rocket allows you to launch faster and lose less speed when falling. The Sword stat is the first thing you should upgrade in the armory.
The accessories upgrades increase your chances of finding more (and better) loot, help you gain more speed when defeating monsters and lose less speed when falling, and help with speed boost generation.
Potions are unlocked once you rescue the third princess. Monsters may drop potions when defeated. Every upgrade to the potion adds another very specific stat to it—instead of increasing one particular stats number like you find with armory and accessories.
Each princess you rescue unlocks new items, which from then on are dropped by defeated monsters.