I think one big "issue" when it comes to scale, is that wargaming buildings are inherently tiny compared to what a real-world structure would be - as a necessity. It allows you more buildings on the table, easier to store, cheaper, etc.
When you get to model railroad buildings you may have more realistic scaled buildings...which in turn leads them to being more suitable for "scaling up", etc.