There are two very different stages in the life cycle of any software product. The first one is when you're just getting the product off the ground. The features and UI change regularly and it doesn't bother anyone because no one is used to the software working any one way. As long as you're improving things, you're doing a good job.
As soon as you build up a respectable user base, that all changes. Once your users rely on your software to perform a task, it's important to make sure that you don't disrupt that. Adding new features is great, but it's always better to keep existing customers than to attract new ones (or it should be anyway).