As you once pointed out, we play a lot of
FFoL, both the core rules and the
Bigger Battle rules.
And it does depend....

Our self-imposed rule of thumb is to have a limit of about 10 (or so) combat die per unit. For horse-and-musket area stuff, that means infantry units with 10 (or so) figures. We call them companies. For modern-era (i.s., WW2, Vietnam, modern, etc), that means fire teams of 4-6 men.
Of course, there are exceptions. We've played Anglo-Zulu War games with the British having 5-12 men per unit, with 12-18 men per unit for the Zulus. That mix gave the British some incentive to whittle away at the Zulus at range with their firepower, while it gave the Zulus incentive too quickly close with the British and get into melee...
We've not played medieval games. Yet. So while you get what you paid for with this suggestion, since your units will be armed similarly, I'd go with 10 (or so) men per unit. It's enough to look the part while they are in close order (in a shield wall, for example), without devolving into a dice-rolling contest (too much, anyway

).
With any luck, that suggestion is helpful...