There was a good convention game at Enfilade on about a 6'x8' table using The Sword and the Flame rules and 28mm figures. I believe the units were 10 men instead of the usual 20. I didn't play in the game, but a friend of mine had a good time playing. Looked like about 50 to 60 men on each side in the scenario. I believer there was some gold involved, and multiple factions. The Spanish town looked great for that game.
As for loading times, that's where Bernard Cornwell's literary license comes in handy. The idea of having a mixed light infantry company with some muskets and some rifles allows the muskets to cover the slower loading rifles.
I'd say use Gloire for situations that are more like a duel, with 5 or fewer figures per side. Use TSATF or one of the other skirmish orieted rules for games with 20 to 100 figures per side. You can run a campaign that alternates between the different sized encounters.