The original humans on the isle could be Irendi, who showed up on the island and then were enslaved by the Kopru. The wall could have been built by them (the Irendi) in a futile attempt to resist the Kopru.
The Kopru civilization could have underwent a rise and fall possibly some great plague wiped out most of them, and the few that survived went into some sort of hibernation.
The Fall of the Kopru Empire (which spanned several island) allowed their human slaves to win their freedom. They rebuilt the wall and lived on the southern peninsula.
The island is later conquered by a different tribe from Atruaghin, they absorb the conquered Irendi into their society and the lore of the Kopru is forgotten. The wall stays maintained to protect the villages from the dangerous inhabitants on the greater part of the island (neanderthals, lizard folk, etc).
Meanwhile, a group of Irendi who escaped the conquest, made their way to the central plateau, hoping to throw themselves on the mercy of their former Kopru overlords. They settle on the lake shore with the central island and it’s Kopru-wrought structures in sight, but no Kopru come to dominate them. There they eke out a tenuous existence fishing and hunting. Being led by their Chief, and “talking chief” for countless generations.
Recently a bunch of Kopru have awake ed from their stasis and have been mentally summoning the lake villagers to resume their status as slaves. Their psionic powers are much diminished from the old days of their glorious empire and the best they can muster is the Charm Person ability (noted in their stats).