"I'm still really on the fence about giving a book for young readers to someone in his fifties. I'm sure the book is great but he's not really a YA or children's literature fan and I don't want him to feel that I'm patronizing him."

Age and reading preferences are two separate issues (says the person in their fifties). If he actively dislikes kidlit/YAlit, then I'd agree that such a gift would be risky. But if he simply doesn't seek out such books himself, then you could just tell him the truth: This is a book by an author you really enjoy reading, on a topic you thought he'd enjoy. No need to make a big deal over a book's age categories, which, quite frankly, often come down to marketing decisions on the part of the author or publisher.