I think Kumai were very smarter than the others, that is the reason of hate. Most of them did not participated in pujaa aajaa and other bramin pushed them away from practicing it.
But they were engaged government job, there is a saying " kumaaiko Ghumaai"

Think how smart they were and are.