Many people done that. Just do it!!! No harm.

But if you get full time position then no issue but what would you show for your primary employer if you get contract position?. One option is you can create your own one man company and be a contractor from that company. Many people have done that. Even I am thinking to do so. But for that you need to have either Green card or citizenship