This is why I included a whole section in both my Player and GM system books on player motivations and expressly call out that it’s a team game.
So, even if you have the actor motivation where you enjoy getting into your PC’s head, you shouldn’t pick a personality and motives designed to be disruptive. Or if you’re an instigator, you should make things happen when the game starts to drag, not when the party is already in the middle of the action.
GM side it’s mostly about how to engage these motivations in play, but I probably do need a more explicit sidebar about problem players; starting with talking to them.
My personal experience as a GM is that what motivates problem behavior is most typically a mismatch of player motivation and GM presentation. A hardline GM can certainly say take a hike, a more compromising one might try to add few elements that appeal more to that player’s motivations (ex. Include some NPCs that will allow an Actor PC’s personality and backstory matter in the outcome or including things that experimentation to solve for the instigator... make sure there are some opportunities to demonstrate their PC’s capabilities for the challenger motivation, etc.).
There are just some sociopaths out there, but they’re pretty rare. Most of the really disruptive ones from experience are just people who for one reason or another feel a need for attention they’re not getting elsewhere and will take negative attention over no attention. Some people may not want to bother with playing amateur therapist, but I’ve found a lot of these types can be redirected fairly easily with a bit of positive out-of-character attention (like just a few minutes to listen) and once they see they can get that positive attention by playing along with the group, the disruptive behavior drops off dramatically.