Peter Rouw was the son of sculptor Hendrick Rouw (c.1744–1818) and Elizabeth Clemmes (also 1744–1818). Hendrick may have been born in the Netherlands; Elizabeth was born in Westminster to Theophilus Clemmes, a gunmaker. Hendrick and Elizabeth married in 1768. A witness to the wedding was fellow sculptor Peter Matthias Van Gelder (1739–1809), who became a distinguished sculptor and architect. Hendrick Rouw’s brother, Peter Rouw (1754–1808), was also a sculptor. Hendrick and Elizabeth had six known children: Peter (1770–1852), sculptor; Theophilus (1774–1842), sculptor; Henry (1777–1855), sculptor; Sarah (b.1778); John (b.1780); Caroline (1784–1785). Both Hendrick and Elizabeth are buried at St Marylebone Church, London.