totally normal. There isn't any insulation in the door, and no door is air tight. Do a google search for "cold air coming in around car door handles" and you'll see a billion results from every car brand where someone is complaining.
Funny. Noticed this the other day .Although I had the sunroof venting so I figured the negative pressure was drawing cold air through the handle.
