Do I need to grant permission for my baby to travel without me?

i do agree with GerdP that it sorta depends on which country you're traveling to.
But bringing your baby's birth certificate would help, so that it shows your wife as the baby's legal mother.

You should tell us to which country they plan to travel since each country has different procedures concerning minors not travelling with both parents. You also should tell us which passport your wife holds instaed of which she does not hold, that would ease up the determination of the visa requirement for both..

No.
The passport is the key.

In the US, for example, the one parent needs the other parent's permission in order to apply for a passport. Once a passport has been issued the passport holder is free to travel, unless a count order tells otherwise.

No. My DIL is Vietnamese and does not have a duel passport just ILR. She took her baby son (with his British passport) to Vietnam. Dad stayed behind. Nobody asked for anything at either end.