Well, I followed my own advice and got cracking!
Here's the front of the structure structure with a tiny balcony:

And here's the back, with a start on the bits-of-paper-rolled-up-in-my-mouth-and-mixed-with-spit technique:

Just kidding, it's only clay

They look a bit brick-like at the moment, but I hacked into them with a scribing tool to make them rounder. Doesn't seem to show on the photo!
The last big part of it will be the balcony on the top. Basically a flat wooden platform with some sort of thatched roof.
Still a fair bit to go!