Interesting playtest - but why on earth would a fighter be exhausted after one short charge, even wearing heavy armour?
My understanding from talking to re-enactors is that armour doesn't slow you down, but running around in it for an extended period exhausts you.
Also, close combat is very, very exhausting - much more so than movement. You can only do it for a short while before you need to rest.
So in your example, I'd think the Murmillo should arrive fresh in the combat, both fighters should tire quickly when actually fighting, but the armoured guy should tire quicker.