SSH Host Key Management

Aside from the SSH authentication keys for users and root which must be distributed to a new system, the one item that is key to providing a unified operating environment is the known_hosts file located in each user's .ssh directory. This must be maintained with the host keys for all current hosts -- both by hostname and by IP address -- in order that automated access between hosts is possible.

Currently, aslan runs a job that uses the master hosts file located at http://config/config/hosts as input to the ssh-keyscan program to retrieve the host keys for all systems and put them in http://config/config/ssh/known_hosts.

Whenever a new system is added to the environment, its hostname and all its IP addresses must be added to the master hosts file. Each system should (either automatically or on-demand) retrieve the hosts and known_hosts file using commands such as:

curl -s http://config/config/ssh/known_hosts -o $HOME/.ssh/known_hosts
sudo curl -s http://config/config/ssh/known_hosts -o /root/.ssh/known_hosts
sudo curl -s http://config/config/hosts -o /etc/hosts

When a system is rebuilt with new keys, that system will have issues until the new host keys propagate: all systems will see that it has a new host key and SSH will not allow connections to be made. Until the update job runs, individual known_hosts files can be updated, but the automated process will overwrite any changes the next time it runs.