they changed the story completely from the book anyway, so why not just make a different story with different characters and just do that?
Because the vast majority of people have no idea what the original story is about and they just think "I like Tom Clancy movies". Of course, it doesn't take too many of these for it to turn into "Tom Clancy movies suck".
The real question is why do IP holders seem to care so little if their IP get attached to crappy projects?
Well, for starters, Clancy died in 2013. Death of the primary creator is usually when things start to unravel unless someone steps into the role of wrangling the idiots. The Tolkien estate is facing this with Christopher Tolkien's demise, and I strongly believe Good Omens avoided sucking because Neil Gaiman was there with Pratchett's notes to step on the necks of the retards.
Also, Hollywood is by and large a sausage-making machine. By which I mean whatever you get out of the meetings may only bear a passing resemblance to what you fed into it. Well before wokeist crap was a thing, executive meddling and lawyers could fuck up a production in fascinating ways.
Hollywood loves waving fat gobs of cash at people for their IPs, and they don't always even make it into movies. That's a heck of an inducement if you're not J.K. Rowling and can wipe your ass with $100 bills. Clive Cussler optioned out 'Raise The Titanic' disliked the result, avoided doing it again for the longest time, and then I guess he needed to pay some bills and optioned out the rights for 'Sahara'... which was a disaster. Cussler then committed a very stupid move and sued over it, lost the suit, and pretty much spent the rest of his days cranking out potboilers to pay off the judgement. Sometimes the wisest decision is to walk away.