Groundtruth, the annotation tool I built last year, started because I was annoyed. I needed labeled data for a research project, every existing tool was wrong in some way, and I'd spent 3 hours on setup before deciding to just build it myself.
I told exactly one person about it while building it. Shipped it, put it on GitHub, moved on.
Six months later two labs I'd never talked to were using it for their own projects. One of them emailed me to say it had saved them "embarrassing amounts of time."
This pattern has repeated enough times that I've started to trust it.
The annoyed-builder advantage
When you build something to solve your own problem, you have an unusually clear success criterion: does this make your life better? You're the most motivated user. You know exactly what the failure modes are. You can test it in real conditions because you're living in them.
The alternative — building for imagined users, based on assumptions about what they might want — requires you to be right about things you can't directly observe. Sometimes you are. Often you aren't.
I'm not saying don't talk to users or don't do market research. I'm saying that starting with a real problem you have personally is a legitimate strategy that doesn't get enough credit.
What you give up
The tradeoff is scope. Your problems aren't everyone's problems. The annotation tool I built works great for the NLP workflows I care about and is probably not the right choice for someone doing image labeling at scale.
That's fine. Not everything needs to serve everyone. Tools that serve a specific person or team very well are genuinely valuable, even if they're never a business.
The things I keep
I keep building tools that annoy me into existence. Some of them are useful to one person (me). Some turn out to be useful to more people. I've learned to not need to know which category I'm building for at the start.
The discipline is finishing them enough to actually use.