Hywel Sele Lodge stands on (or near) the site of at least one of the earlier Nannau houses, before at least two at the current location.
A bard of the time wrote that house had “…a hundred lofts and sixty doors, the white washed clock tower with 300 steps, the oaks of the roofs covered with slates and lead, forming 12 great ridges or crests and the skilful carving of saddle pommels on the ridges, the glass showing towards Cader Idris, and parlours and the chambers, and the portals.”
In 1778, Thomas Pennant wrote that ‘in his time’ the ruins of the old house could be seen in Nannau Park, “a mere compost of cinders and ashes”.