The sloping main house rests with its west front on the inner city wall, which ran from the Schurkenturm tower down the steep castle slope to the mill canal. The house was bought in 1706 by the widowed Maria Johanna von Stuben, who came from the Hohenberg family, which was the result of the morganatic marriage between Archduke Ferdinand II of Austria-Tyrol and Philippine Welser, a burgher's daughter from Augsburg.
The alliance coat of arms of the von Stuben family and Johann Franz von Barille, who married into the family, can be seen above the entrance gate to the arcaded courtyard. Around 1760, the Geßler cloth merchant family set up a silk factory and dye works in the building, which was followed in 1880 by the sculptor's studio of Peter Paul Hausch. In 1984, the Stubenschen Schlößchen was renovated and converted into owner-occupied flats.
