Everyone likes to think of "Secure Shell" or SSH as being... well, secure. Unfortunately nothing can be further from the truth if you use the out of box configuration for a lot of distributions. I learned this the hard way many years ago when my home box had someone brute-force an account on it despite having a reasonably secure password.
SSH is secure, so what's the fuss? Out of the box it does a couple of things wrong, at least on Ubuntu. Here's a couple of settings you should consider. These get added to:
So, let's start with Root access:
There's usually no reason you should be shelling into a server or desktop directly as the root user. Connect as yourself or a special admin account that is in the sudoers list and then use sudo to gain admin privileges. If you are going to permit direct shell access to the root user because you have to, make sure to set your firewall to only allow connections from the machines that will be doing that.
So this is more security through obscurity, but move the port. If you're a home user you may need to do that anyway because your ISP probably blocks inbound connections to port 22. Try something random, higher than 1024. Oh, and remember it because you have to configure your SSH client to use the new port. And don't forget about tweaking your firewall to allow it.
RSAAuthentication yes PubkeyAuthentication yes PasswordAuthentication no
Do it. Stop allowing password login to SSH. Generate an encryption key pair and install your public key on the server in the authorized_keys files in your /home/user/.ssh/ folder. You can't brute force your way into a password if the account doesn't allow password logins.
This one is up for debate I guess. The last time I used X11Forwarding was probably 8 years ago. If you aren't using it, set it to no. If you don't know what it is, set it to no.
If you have some sort of network domain with centralized accounts, put your domain administrators group here. This expects a Windows compatible "Domain Admins" group name, and since you can't put the space in the setting, you put a question mark as a wildcard. If you are managing accounts locally, maybe you put your own group name here (mine might be "tony".) With this setting, only users within that group are allowed to login over SSH.
These few simple settings go a long way to hardening your OpenSSH server.