This occurred to me right after I woke up this morning, for some reason, and I'm surprised it took so long.
Up to now, the way I've been doing armor has been on the border of being fiddly without, for me, quite going overboard. At present, it works like this: Armor is rated the same way skills are, and is divided into three categories. Light armor is Fair (+2), Medium armor is Good (+3), and Heavy armor is Great (+4). Although it's outside the skill pyramid, you can choose to defend with armor against most melee or ranged attacks (there are exceptions on a case-by-case basis for those times when armor wouldn't logically be of any help) as if it were a skill, like Melee or Athletics. When defending with armor, you can reroll a number of minus dice up to the value of the armor -- e.g., Good armor would let you reroll up to 3 minus dice. If you do, right afterwards (preferably while the GM is dealing with someone else, so as not to slow things down) roll Endurance against a target number equal to the number of dice you rerolled. If that roll fails, you gain a temporary aspect of "Fatigued" until the end of the scene.
It works fine, but the sheer amount of space alone it takes to explain it makes it feel to fiddly to me. Instead, I'd rather take a page out of the swashbuckling game and let armor add Fudge dice to your Melee or Athletics defense up to the armor's rating: +2dF for Light, +3dF for Medium, and +4dF for Heavy. Take the best four dice, and that's your die roll. If the pool of spare dice you don't use contains at least as many minus dice as your Endurance rating, you're "Fatigued," as above. Done.
I like this first and foremost because it means not having to make a second die roll -- that's a central precept I always try to adhere to -- but also because it gives those spare dice a purpose.