if you leave the smaller team then someone from the larger team will be swapped into the gap.If you do that, wait a few minutes before rejoining, otherwise the logic will see you abusing the balance logic. It can't tell the difference between you being an ass abusing the balancer and you disconnecting/being kicked.
Prevent all players from joining the winning team is the best solution in my opinion.