I built a similar layout using EVA Foam floor mats for bases, with 2D PDF wall sections printed and glued to chipboard, then slotted into shallow cuts made in the foam mats. It is modular for rectangular builds, but not so much for round. For round, I would need to create tabs which could be slotted into the foam mats, at certain points. Never got that far with it, however:
album of photos.
I used this approach to create the full upper level of the
G1: Steading of the Hill Giants model, in 25mm scale/size.
Album of images of the full model.

The slotted foam mats, with chipboard walls, works, but setting up a dungeon is time-consuming and complex. Laying out 2D dungeon tiles is much easier and usually much faster (see above example).
Alternatively, I've used modular cavern wall pieces set atop ground mats/cloths, as well: very modular, much faster to build out a design, and it gives full 3D walls. Some examples:
G2 Glacial Rift of the Frost Giant Jarl album (concept, not an actual scene from the module); and a
2e BattleSystem Skirmishes game,
album of PC's against a Red Dragon and her cohorts in her caverns. I like fast, easy terrain builds, but the more complex, tedious-to-set-up games are fun, as well, just more work. Cheers!