Nilmoni Thakur was the first person who in 1784, during the time of Warren Hastings, purchased a plot of land on the east side of Chitpur Road and constructed a building there. It was known as Mechhuabazar at that time, later it was renamed as Jorasanko.
Dwarakanath constructed another building by the side of his paternal house where he used to arrange dinner parties for his European friends, ladies and gentlemen. Ganganendranath and Abaninidranth used to stay there. The main portion of this building was damaged and the rest was utilized as a campus of the Viswa Bharati University.
The old house was not constructed all at a time, rather it was extended as and when required without having a definite plan. As a result it was built at various levels and made to bend according to necessity with staircases here and there, coming to look a bit like a jigsaw puzzle. It was a great wonder to Rabindranath when he was a child.