The MANOR HOUSE was built between 1736 and 1738 for Elizabeth Hervey, wife of John Hervey, first Earl of Bristol; Elizabeth paid for the building herself.

The architect was James Burrough (later Sir), son of Dr James Burrough, a physician of Bury St Edmunds. He was educated in the town and at Cambridge University and was involved in building projects at several Cambridge colleges, most significantly at Peterhouse and Clare College chapel.

Lady Bristol left the house to her husband in her will, and upon his death it was to go to their youngest son Felton and then to his issue. Each person inheriting the house had to sign an inventory of the goods inside and be accountable for them. Additionally they each had to agree to add Lady Bristol's maiden name Felton to their other christian names "in remembrance of my family".

By 1782 the house had been sold and thereafter had several owners until it was sold to St Edmundsbury Borough Council in 1988 to be converted into the MANOR HOUSE MUSEUM.

The museum opened in January 1993 and houses costume, fine art, horology and temporary exhibitions. There is also a "hands on" gallery for children. The Courtyard Café provides hot snacks, sandwiches and drinks in pleasant surroundings and is also accessible to people not visiting the museum.