I know what you mean. I know who you are...lol just kidding.
If you find there's no option to accept a challenge, try opening the game using the notification link. That usually works. Go to your notifications, and accept the challenge there.
What's that? You cleared your notifications!
How do you accept a challenge?