And then I found these boxes...
Since the box lids stuck out a little further than the walls of each box, I covered the edges of the lids with sections of pink insulation foam carved to look like rows of stone blocks. I also cut sections of dowels to fit in the joints between the cork wall sections.
I glued some old plaster window frames I had into holes I cut in one of the walls on each floor. On the top floor I build the framework of an old roof using balsa wood and shingles cut from cardboard.
Inside the tower, I cut cardboard flagstones for the ground floor, and used balsa strips to make floor planks for the other three floors.
And here is the end result, shown with a Reaper 28mm figure for scale: