I find it confusing that the mage armor spell is named that when it doesn’t count as armor.

Some spells and class features have figurative, not literal, names. The text of the spell or class feature explains what it does. In this case, mage armor (PH, 256) surrounds the target with “protective magical force”; the spell doesn’t provide armor.