Riverside was established in the nineteenth century on the site of the famous 17th-century town of Mianus Neck. The Continental Mower and Reaper Company, which established a business on the western side of the Mianus River in 1865, chose Mianus Neck as the site for their operations. As a result of its failure, it shuttered its doors after only two years.

Jeremiah Atwater, a real estate broker from New York, relocated to Greenwich in the same year and, after seeing the potential of the region within a few years, began purchasing property in Mianus Neck and building residences. The successful rebranding of Mianus Neck to "Riverside" in 1869, in collaboration with Luke Vincent Lockwood, a New York attorney and resident of Mianus Neck, helped to bring more summer renters and homeowners to the island. They built a new Riverside railroad stop and train station in order to better improve their communication. Lockwood served as administrator of the Episcopal Society Sunday School and, with the help of a property grant from Jeremiah Atwater, he established and built today's St. Paul's Church on Riverside Avenue in 1876, which is still standing.