I use a 6'x5' made out of two nice fold-out tables (meaning I can carry this table stacked up in my Volkswagen easily)...I think the draw to 6'x4' and 8'x4' is quite easy to explain.
Simple fact is you can always play smaller, but you can't play bigger. So when possible you'll build a larger table and then play whatever you want on it. I wouldn't be opposed to buying some nice 4'x4' game mats for smaller games, but I'd still be using the same table. You can run most games on a 6'x4' short of huge 28mm games or massive battalion games. The size lets me build good dungeons and has enough depth to be played in both directions depending on the games.
My buddy's game-room has perhaps a 8'x5'...but a good two feet of it is always covered in rulebooks, models and snacks...so the gaming surface is again around 6'x5' and we often game on smaller 3'x3' mats/sub-tables when needed. At the same time we can set up the whole table if needed for a bigger game. It's about versatility.