Restricting access to SSH to only allow specified IPs is a good practice for securing your server. Also it is very simple to do. All it take is a change to one file and you are done. Be careful doing this if you have a dynamic IP. You don’t to get locked out of your server if your IP changes.