Seriously though. You're saying you are considering having the ip restriction implemented because of people being hacked/scammed? I have always learned that: 1. It's the internet and 2. It's the internet and 3. It's the internet.
So if you decide to share your info. It is your fault and that's that.
If you decide to go through with this 'idea' you are definitely going to be losing more people, yes.