s001, I disagree with your logic. There are many factors. Just like your 401K, your house is also your investment. You can also reduce your tax bracket with your interest. In this economy, many people are getting more than 10% return on their 401K, why would you rush to pay off your mortgage which will save you only 3-4%. That means you would lose money.
Paying off your house with very low interest rate will gives you peace of mind but may not be good financial decision when you are getting high interest rate in today's market+ you will lose employer's contribution.