From my understanding of history, public trust and respect for "modern" (college-educated, doctorate-holding) physicians didn't really become widespread until the late 19th century - I'm fuzzy on the exact dates (post American Civil War to be sure) and how it varied between the US and the UK.
Given that, would it be considered necessary/polite for someone to address a physician as "Dr. So-And-So," or had society not yet placed much importance on doctorates? Would calling a doctor "Mr. So-And-So" even be considered an insult?
I've searched variations on "Victorian doctor term of address," and (unsurprisingly) come up with nothing.