Haven't read Hostile, can't help you there. I can address the rules side.
Learn, teach and embrace the skill system and especially the rules on task chains early. This is central to the whole system.
Set a limit on the number of terms in character creation, so PCs are all roughly in the same ballpark. OR you can rely on death in char-gen as a balancing mechanism, but that really only works if everyone's rolling completely honestly in person so they have to stop while they can. As soon as someone nudges a die, or the GM has mercy and lets "death" mean stop rolling and play the character, you end up with the stereotypical geriatric admiral with all the skills alongside more reasonable characters.
Run a trial combat before it counts. Traveller combat can be deadly - not quite as deadly as it's sometimes made out to be, since if you win, or even lose in civilized country, you may be able to be patched up. But still more deadly than D&D.
Speaking of which, when you say "older, experienced players" - what system(s)? I've had better luck teaching Trav to brand new players than I have to players who've only played D&D. It's just... different. Your optimal moves are different, your attitude in combat has to be different, your reflexes are different. People who try to play it like D&D in space get very frustrated.
House rules to consider - the book modifier for unskilled checks is -3. I ran that out of the box my first time out and I didn't like how it discouraged players from even trying unless they had to. Next time out I'm using either a -1 (Stars Without Number does this) or a - equal to each character's highest skill rank (abstract, but works at a meta-game level).
I've also tried rolling initiative in combat only once at the start, and having the various dodge/react penalties change your count until the end of combat instead of end of round. I really liked that one but only got to see it a couple of times.