The Museum is an 1835 brick purpose-built building in the grounds of Walden Castle, a 12th century ruined keep. The small and diverse art collection was acquired by the Museum's trustees, mainly in the 19th century. Most are portraits of local people and scenes, including 'The Withered Elm, Audley End Park' by local artist John Mallows Youngman. The collection includes the only known self-portrait of Henry Winstanley, who was born in Saffron Walden and is best known for building the first Eddystone Lighthouse. The Museum also owns the six grisaille paintings on panel by Giovanni Battista Cipriani, commissioned for the Robert Adams library at Audley End, where they are currently displayed. Other notable pictures in the collection include 'Portrait of a Lady with a Lace Collar Holding a Tulip', which has been attributed to Gilbert Jackson, and 'Adoration of the Magi' by Philip Reinagle, after Rubens, which is on long-term loan to a local church.