I had similar situation. I told my wife clearly that I can not live with your parents and siblings for long time. Eventually she got it ( even though she got mad in the beginning) and they left. Life is short and do not live miserably. Talk to your wife clearly what you want. Never ever live with extended family, many problems come on the way.