John Rutherford was born in a manse in Yarrow, Selkirkshire. He studied philosophy and mathematics at the University of Edinburgh. He then served as an apprentice to Mr Nisbit, an Edinburgh surgeon. After studying anatomy, surgery and materia medica in London, he travelled to Paris, Leyden and Rheims, where he graduated with an MD in 1719. On his return to Edinburgh, John Rutherford along with Andrew St Clair, Andrew Plummer and John Innes successfully petitioned the Town Council to use a house they had purchased as a chemical laboratory and a place of instruction for students. Two years later, in 1726, the Council elected Rutherford and St Clair Professors of Medicine and Chemistry, all without salary. They also petitioned the Town Council for permission to use a piece of ground adjoining their house, which was in a state of neglect, for the purpose of growing plants, herbs, etc.