How remove XT2 eye cup?

I'm trying to replace my XT2 eye cup with the extended one but can't get the old one off. I've pushed and pulled and pinched and nothing works and I'm afraid to apply more force. The manual says to press the bottom in...Does that mean towards the body or inwards?

Thanks, Phil