Well I wouldn't be welcome there, but I'm not insulted, that's like worrying about exclusive dress shops selling clothes I can't afford. It's a waste of time.
I am sure if I could afford to live in that area I would also be able to afford to use the bank branch there, but I am very happy with the service my local branch of HSBC gives me. The cashiers are great, really friendly, and they never refuse to take my money, so I have no complaints.