Is there a definite reason for Buzz Droids taking up the most party slots (3) out of all security and military droids? I mean, they are little and were originally used in swarms in the clone wars, and the four you can party up with isn't really a swarm.

Take a look at their weapons. They are fairly well armed (2 weapons with up to 19 attacks each) and deal a crapton of damage (maximum potential of 152 damage) compared to most droids. Even if they're a close range attacker, they can tear through enemies quite quickly. Compare that to droidekas (arguably one of the best droids) which has a maximum potential damage of 80.

What Ulrike said, they can each do 152 if they do max damage each turn

I see, but wouldn't it make more sense to nerf that damage, so they can be treated as many little attack droids instead of as a few heavy hitters?
(Disclaimer: I don't want to make this a suggestion, I can live with it if they stay as they are and it's probably even a restricted topic, but it would be more fitting canonically IMO if they would be more like a swarm than the Rambo droids they are.)

Not really. Canonly buzzdroids were only designed to tear through metals on ships. It makes sense that they're extremely powerful but limited to small numbers.