If you declare the import and pay the duty up front then the Post office will not hold your parcel or charge their admin fee.
You *should* pay customs duty on goods imported from outside the EU. If you don't pay that then (unless it is a gift) her majesty's government is entitled to stop your goods at the border and demand payment of the import duty.
This applies to any goods coming in to the country - it's what customs barriers exist to do.
If you tried to take a container off a lorry without paying the import duty you'd be stopped at customs and ordered to pay the duty, plus a fine and some costs.
So think of the post Office charge as the postal equivalent of that.
By not paying that duty up front you are attempting to avoid legitimate (according to the law) charges - so you don't pay your money and you take your chances.
Most sellers don't seem to have a mechanism whereby you can pay such duty up front and the onus is on you as the purchaser to declare it. It's probably not worth the hassle for small US miniature manufacturers to make that a part of the ordering process, so it's a bit of a catch 22 - even if you wanted to pay the duty it's hard to do so.
The PO only stop and check a % of post coming in so you may think the risk worthwhile, (I don't so I never order from outside the EU) but if your package is stopped I'm not sure you really have grounds for complaint - you're attempting to avoid taxes.