First off for everyone's reference, I found a book Letters to a Young Lady on a Variety of Useful and Interesting Subjects By John Bennett which has a whole heap of specific advice on specific books which I have (mostly) listed but I don't know how reliable it is.
The first, Mary(*), is about 20, very serious minded, studious, and moralising. I've also added an interest in natural history because my muse insisted :)
The second, Anne, is about 30, less concerned with being strictly moral and more interested in fashionable and cutting edge thought. She would be willing to read things not considered entirely proper for an unmarried woman, but nothing really shocking.
So, according to the regency encyclopedia:
Girls were taught needlework, simple arithmetic, fine handwriting, music, drawing, history, geography, French fables to recite, reading the Bible, Shakespeare, poetry and some respectable novels.
'Learning' or knowledge proper to male education and including classical and Biblical languages, analytical and scientific discourses, controversial writing, theology and mathematics was condemned by female conduct books, satirized by male and female writers and excluded from most females' education.
Botany, for example, was rewritten for young women with the sexual classification of plants expurgated. Natural history subjects were offered for their moral and religious analogies as much as scientific potential.
I googled "recommended books woman regency" and "recommended books woman 18th century" and found Advice to Young Women from the Jane Austen centre.
Summarising other stuff I have found:
The Bible (duh :))
Fordyce's Sermons to Young Women
Mentoria, Or, The Young Ladies Instructor By Ann Murry
Condemned as unsuitable by many but women read them anyway. Good for Anne but not Mary?
From "Northanger Abbey", Mysteries of Udolpho (actually I don't think either of them would read this :))
Popular but I'm not sure if suitable for women: Pope, Gray, Thomson and Cowper, Byron, Scott, Burns, Crabbe, Wordsworth, Southey, Blake and Coleridge. (Byron wouldn't be, I'm sure)
I've heard of "Gibbon's decline and fall of the roman empire", but would it be ok for women?
(*)Yes, the same women as from my last post, Mary Bennet and Anne de Bourgh respectively, in case you're wondering :)