The real question is, do you want to do Savage Worlds fantasy, or do you want to do Savage D&D? The reason I say this is that many automatically assume D&D-like conditions will exist in fantasy worlds. Archetypes like (spellcasting, armor-wearing, and hammer/mace-swinging) clerics and all of the other classes are pretty strong across many game lines. Savage Worlds can very easily do fantasy without such limits. Whether this is a boon or a bane depends on the answer to the question I asked in the first sentence.
False dichotomy! (I'm teasing but a little serious).
Savage Worlds "fantasy" is setting-specific, but the rules themselves are setting neutral. As someone that by default runs "D&D" using Savage Worlds as my system - the approach is simple:
Make whatever *you* think is "D&D Fantasy" expressed in Savage Worlds. That's what I do. I run Forgotten Realms, Spelljammer, Planescape using Savage Worlds Core - with bits of stuff from Shaintar, Fantasy Core, Hellfrost, 50-Fathoms, Beasts and Barbarians, and I use Deadlands and Rippers to cover all my "black powder" and steampunk stuff. ALL the rules work interchangeably.
So that said, the main differences:
D&D RAW has a much slower potential progression speed. I find that Savage Worlds is more high-octane in this regard, and you may want to moderate XP or do away with it entirely.
The nifty thing about SW<>D&D is that SW's tiers do a good job of conforming to the assumed tiers of play based on level. Novice - is about 1-3, Seasoned is about 4-6, Veteran is about 7-9, Heroic is about 10-14th and Legendary is 15+... where the + could scale WAAAAYYYYYY beyond anything D&D can handle.
Spellcasting is naturally different - but there are options available to go full Vancian if you want. Or point-based, or skill-based, etc etc. If you wanna make it magic HEAVY - you can take the magic rules from Rifts and overlay them... then you're talking insanely God-mode levels of play.
Combat is a lot faster. You can set your own setting conceits to dial the "grittiness" factor up or down.