Normally, if you change it at the bank you have account, you don't need to pay any charges. But if you don't have bank account in HK, you can also ask friends or relatives who have HK bank account or credit card account to change it.
I have the same situation of you too. Most of time I not live in HK, when I change HK dollar, I just ask my friend who has HSBC account to change for me, it doesn't have any charges..ha ha !!