Army sizes (in the series, never read the books) never made all that much sense. I'd guess most armies will follow early mideval examples from real life. About 10.000 to 20.000 for a really large force, smaller armies of a few thousand being more common.
I'll preface my answer with a mild Spoiler warning for those who would still like to read the books, I have no idea what's going to be in the show and what isn't...
Agreed. I think the numbers that often get thrown out represent the maximum number of fighting men (of any quality or capability, be it knights or poorly trained commoners) that all Houses in that realm. They would never be mustered all at once, and I think in the books some mention is made of Houses in the Reach that did not begin mustering and contributing any troops until the Iron Islanders take the Shield Islands and threaten to raid up the Mander. Rodrik Cassel is able to raise a decent force (that small army number you mention) on short notice when Theon takes Winterfell, and this is after Robb takes ~18-20 thousand men south with him.
As you say, real medieval armies were rarely that large, or if they were they didn't operate or stay that way for long. Partially because supplying that many men at that period in history for any long stretch of time was nigh on impossible, and partially because knights and the odd mercenaries aside, these guys were not professional soldiers. An army of 20,000 means there are an awful lot of crops not being tended to, shoes not being cobbled, etc... which in a way would just circle back and negatively impact the supply problem even further.