Should I be upset for living in New York?

Interestingly, I am also a PA native who lived in NY (Lower Hudson Valley) for 34 years. I experienced the same frustration you described. When you say you live in NY most people immediately equate that with NYC forgetting that os only a tiny portion of NY. Instead of just saying "NY" when asked where I lived I solved most of the problem by altering my answer to "The southern part of NY" or "About an hour north of NYC" which caused them to understand I was not in the city. Try that and I think your frustrations and explanations will ease.