'Farnham's Freehold' is a book from the middle of Heinlein's career, which means that it is in that transition period between good science fiction and meaningless mystical junk. It is a fairly good book overall, with lots of interesting characters and thought-provoking situations, but it is still more fantasy than science and has plenty of plot holes. But you can see the start of the downward trend of making the plot simply a vehicle for author filibusters on social and moral issues.
The main character, Hugh Farnham, is clearly an idealized representation of the author, just like Dagny Taggart is for Ayn Rand. This, in combination with Heinlein's taste for unconventional sexual morality, leads to several 'squick' moments. These are annoying mainly because of their gratuitous pointlessness.
Most of the meat of the book comes after the unexpected and bizarre plot twist at the middle of the book. I won't spoil it, except to say that the main antagonist, Ponse, is the most interesting, well-developed, complex character in the entire book.
Heinlein has been accused of racism for writing this book, but I believe that the opposite is true. The whole point of the book is to show how people are corrupted by both unjust power and improper subjugation. If you know anything about Heinlein, you know that his entire philosophy is about personal freedom and responsibility, and he fights anything that opposes those beliefs. This is one of the best treatments of racism that I have ever seen, and I highly recommend it if you like challenging and thought-provoking books.