Kitchen sink vernacular fantasy (which D&D exemplifies) has always been weird because it is a hasty marriage of numerous, often discordant, elements. However, I do see a relatively recent change in the core assumptions of the default setting. From 1e through early 5e an attempt was made to present a relatively coherent assumed world. Settings and situations were weird to degree they deviated from the implied base-line of a sanitized, modernized, fantasy vision of Lancastrian England.
Later-day 5e seems to be largely unmoored from any coherent setting. You have Starbucks, magical robots, anthropomorphized animals, and none of the vaguely Medieval population thinks any of this is remotely odd.* Even the hyper-detailed Forgotten Realms has become increasingly strange in the way it fits together, or doesn't. I get the impression that a lot of new players aren't even aware that the Forgotten Realms is supposed to be a coherent world.
As to why, I think that part of the shift is due to media. Older players first experienced fantasy through books. For younger players the primary vehicle of fiction is video games. Video games tend to have shallow worldbuilding and a lot of ludonarrative dissonance. As a result you have a large group of players with a very high tolerance for things that just don't make any sense.**
While some people love 'OMG so random' I have found many (most?) younger, newer players like and enjoy tighter, more consistent worldbuilding, they've just never been exposed to it.
*Yes, similar things were around in D&D from day 1, but there was the implicit and explicit understanding that these things were unusual.
**Granted, Spelljammers and plane hopping makes no sense, but they make sense within the context of the universe presented.