You should be worry about English if you are planning after high school. Children will catch English and culture faster than you have thought.
How raise children is personal choice and you may have your own logic.