Hey, go for the gusto. If you have the means, and wish to indulge yourself, why not? I didn't buy the house, but rented one, to throw play parties in, worked just fine.
And don't listen to all the naysayers. A private residence for your slaves certainly WILL get you laid. I hear that all the nouveau slaves are requiring that these days.[:D]