Might is increased by upgrading buildings/fields, increasing your number of troops, and completing certain quests.
Losing troops in combat will take their bonus out of your might total (you will never lose levels that you have gained).
Defensive units do not add to might.
Eventually Might will also increase some with successful battles.