There are those times when it is okay to gracefully say NO. Let's face it, you can't always pleased everyone and why pleased those people who always seek to meet their own agenda without contributing to yours in any way?
Building HEALTHY boundaries and saying no is okay, and it will likely reduce your stress on the long run, among other emotional, spiritual and even physical benefits. Your time is valuable, choose who you spend it with and save the "yes to.." for those that value you and care for you.
Boundaries are not only healthy but necessary. It means you respect and honor yourself.