Also walls and roads can be made to be scale agnostic.
Building height seems to be key. too high and it just doesn't looks 'right' in my brain....
The height of a building varies, what really definets its scale are doors and windows.
-avoid oblong doors- they automatically cause scale assumption. For scifi make rectangular and remote controlled doors- this way they can be a simple personal entry for a bigger scale and a vehicle entrance for a smaller one.
-avoid "lookout" windows, they tell a lot about scale, and not just their size but their positions too. "Lighting" windows do not cause those assumptions, they are often placed so that you even are unable to actually look through it. Go for small windows either in vertical lines or just below the top of the building.
The exceptions are the churches, they can have really big doors and windows. I use this one both for 1/35 and 28mm:

And when you are done with scale agnosticization and all your terrain are unisex, you can smuggle back the scale with some scatter terrain. 3-4 of them per scale collected is not a big deal, but they shall clearly refer the scale: bodies and scattered military equipment is petty obvious if you go into the violent direction, but stuff like barrels, a sunbed or a pair of boots in front of the door also do the job. For smaller scales, 1-2 parked or wrecked car.