Forgive the lateness of my reply. I don't check this site regularly anymore. I found this because once or twice a year I do a google search on Lejendary Adventure.
Here's the best way I know how to explain LA.
Skip the players book (Lejendary Rules for All Players, aka LR4AP) and go straight to the monster book (Beasts of Lejend, aka BoL).
Go back to basics. You have a table top and a bunch of guys are going to fight. The very first thing you need to know is how much you can move your guy so that your opponent is within your attack range or weapon's reach. Once within range, the second thing you need to know is what is your probability of hitting. Then third thing you need to know is how many hits it takes to put your opponent down.
And so to represent these three primordial needs of an action-oriented role-playing game, Lejendary Adventure skips all the hoopla and gimmicks and fanfare and simply gives you what you need. Health, Precision, and Speed. The monsters are then rounded out with two more stats, Attack and Defense, that answer the likely two first details questions that come up from that schema. Aren't some people going to hit harder than others? Aren't some going to defend better than others? There are additional stats that some monsters possess--abilities, powers, anti-power checks, and invulnerabilities. But the five that are common to all are Health, Precision, Speed, Attack, and Defense.
Avatars (player characters) are focused a little differently but still fit the mold. They all have abilities. Some will also have powers. They generally do not have invulnerabilities or anti-power checks. Those are for things like evil spirits and deital minions. And of course attack and defense modes of human and human-like creatures are going to be mainly dependent upon tools (weapons and armor) and their skills (abilities to use them). This is what is covered in LR4AP.
Everything else is filled in from there according to the needs of play. So questions like "Can I fight with two weapons?" "How does parrying work?" "Can I dodge?" "How does morale work?" "What if I want to prospect for gold?" "What if I want to climb a tree, swing from a vine, and dive into water below?" These are handled in the game master's book (Lejend Masters Lore, aka LML).
So question. what does the armor speed penalty affect?
Answer? Whatever makes sense.
It affects movement speed. It affects attacking speed (which may be used in initiative). Does it affect avoidance checks? Depends. If you're avoiding some sort of physical damage attack, then most likely yes. If you're avoiding a mental attack, such as an attempted mind control, then no. Does it affect your abilities? Generally, no. But if it makes sense to the ability, the penalty may apply. Or even a much harsher one. Stealth is a Precision-based Ability, so a Speed penalty would in no way hamper it. But sneaking around in mail will certainly carry a penalty.