I have said many times that I don't plan out plots so much as create situations that drive decisions. As part of this, I require a fairly large cast of antagonists, potential allies, and other persons of interest who interfere in its development. What makes these people relevant to the situation is that they have some sort of agenda and are pursuing it, and that these tie into the situation in a way that influences a decision.
Because of this, I don't have a ton of time or use for the sort of huge backstories written in long text blocks that many published modules use when they try to set up their villains that go into their elaborate backstories in great detail but culminate in a guy sitting in a dank room waiting for the PCs to show up and stab him in the face.
So rather than writing this sort of stuff, I primarily concern myself with three characteristics of NPCs, and I suggest that if you want to write interesting NPCs who engage meaningfully with the players, you might want to try this yourself.
These three characteristics are their motive, their means, and the opportunity for them to put their plan into action.
Motives are what a NPC wants. I recommend writing it out as one short phrase.
NPCs can have more than one motive, but I suggest you concentrate on one good one instead of a bunch of crappy ones. A good one should either be extremely concrete, or it should be a broad abstraction - stuff in the middle tends to be lacks the benefits of either.
A very concrete motive should have an object it is directed at, an emotional tone or affect that adheres to it, and a clear sense of the outcome. Something like "I want to release the vampire I love from undeath by killing him permanently" is what I have in mind. For "clear sense of the outcome" I suggest you think about what the change the person wants to happen is - that's the outcome that's at stake for them and that drives their participation.
The change here, can, perhaps counter-intuitively, include preserving the status-quo antebellum, though in that case you should think about what threatens the status-quo and phrase the goal as the removal or neutralisation of what they think of as threatening it.
Broad abstract goals work best for recurring NPCs who crop up in a lot of different situations as forces or constraints but aren't necessarily direct actors challenging the PCs. Abstractions work best when they motivate lots of different actions instead of just one action. e.g. A particular character might lust after power, and constantly crop up at the periphery of different situations angling to see what they can get out of it, without directly being one of the antagonists opposing the PCs.
You also should consider carefully whether a NPC actually should just have a progression of concrete motives instead of a single broad abstract goal. Typically, the former works better than the latter.
Means are the tools, skills, resources, allies, etc. that a character brings to bear on accomplishing their motive. A common error at this stage is to phrase their means as a set of desires - they want to gather an army, build their death ray, etc. Those ideas are motives. Means are what they have in hand to progress towards realising their motives.
I recommend that you create 1-3 means that NPCs have to realise their agendas. A good split is at least one internal means (a skill or ability) and one external (a set of helpers, a magic item, a fortified location, etc.), with the third means as a back-up or contingency in case one of the first two fail.
Means are the mediating elements between PCs and NPCs. A NPC with no means has no way to effect their motive. For this reason, they often serve as key differences between NPCs. You can actually get away with recycling motives a fair bit between NPCs so long as the means by which they try to achieve them are different. Everyone can want the Gem of Ultimate Power, but if one NPC is intending to get it by sending his minions to retrieve it, while another is going to use her ninja powers to steal it from whomever has it, the PCs should have no trouble telling them apart.
An opportunity is the moment or set of moments when the NPC uses their means to try to accomplish their motives. Theoretically they are working towards their goals the entire time, even when they are not in conscious focus by the group, but in practice what you want is clear sense of when they enter the narration as agents trying to accomplish their goals.
I like to phrase these as IF-THEN statements, but you might something else useful. I try to avoid focusing specifically on PC actions, and use someone, everyone, anyone, or no one as qualifiers to push myself away from doing so.
I also like to phrase these so that there is a clear sense of what is not the opportunity. You want to be able to make a clear snap decisions that now is or is not the time for things to happen.
e.g. The opportunity for a NPC ninja assassin might be: If anyone retrieves the Gem of Ultimate Power (from inside the Dank Dungeon), then she will try to steal it from them the next time they are asleep without guards.
In that example, you know that until the PCs (or whoever retrieves the gem if they fail) are asleep without guards, the ninja assassin will bide her time.
Bringing This Together
The great advantage of this method is it steers you away from writing fan fiction about the NPCs that details their elaborate backstories and instead focuses on their decisions and interactions with other characters. It does so in a concise way that clearly lays out their future decisions and courses of actions, which are the most important parts of playing them at the table.
I also recommend doing this iteratively, and updating NPCs as you go along, rather than trying to frontload it all when you're coming up with them. Get a motive, a single means, and an opportunity written out on an index card or in an Excel spreadsheet, and then add more of each, or change entries, as inspiration strikes or the game develops in some direction.
I'm not overly rigid about things and prefer some loose improvisation as I go along, but I tend to keep a couple of big questions or decisions in mind that I'm driving towards, and what I'll often do is take a sheet of paper, write each question or decision out on it, and then write out the NPCs under each who I think bear on that problem.
From there, I can figure out whether the problem is sufficiently interesting (i.e. whether it has enough NPCs competing over it), or whether it needs more NPCs (or if there's some interaction between them that doesn't make sense or trivialises the problem accidentally).
Anyhow, I suggest trying this in your own adventure games to facilitate producing "dramatic" situations that are "character driven" without having to lapse into railroading or constraints on PC agency to make "the plot" happen (I'm putting all of these in quotes because these are the superficial impressions produced by what is ultimately a substructure of PC agency interacting with modular components).