Speculative developer Charles Richardson built the houses around St James’s Gardens and established the gardens in the late 1840s for the residents of St James's Square, part of the Norland Estate. He donated the site for St James’s Church to the Church Commissioners as the houses would be more saleable if they had access to their own church. The historic church was designed by architect Lewis Vulliamy and was built between 1844 and 1855. The private communal gardens are laid out in an informal woodland style. Chestnut and lime trees, dating back to the 19th century, dominate the garden.