By default CentOS allows ssh access to all users who can authenticate with the server. This can be a security risk especially when you have setup the server to authenticate against an Active Directory domain. In this case all the users on the domain can login via ssh to your CentOS server. You can, however, very easily restrict logins to specific users, computers, or even users on specific computers.
To do this, edit /etc/ssh/sshd_config by adding the AllowUsers directive in the following format.
AllowUsers user@host
This allows the user ‘user’ to login at the host named ‘host’. Multiple users can listed by separating each with a space. You can also use * to specify wildcards. You can also specify IP addresses and ranges using *.
AllowUsers *@192.168.1.* johndoe@192.168.1.3
This will allow all users to log into all computers with address starting with 192.168.1 and the user johndoe to log only into the with IP address 192.168.1.3.
This will work for other Linux OSes as well.