[root@vps ~]# /etc/init.d/sshd restart
Stopping sshd:                                             [FAILED]
Generating SSH1 RSA host key:                              [FAILED]

I delved a little further and discovered that ssh-keygen was failing:

[root@vps ~]# ssh-keygen
cannot read from /dev/urandom, No such file or directory

And finally, from there, I generated the urandom file:

/sbin/MAKEDEV urandom

Also, to ensure on restarts the urandom file (as well as pty and tty) are created, I added the following to /etc/rc.d/rc.local:

/sbin/MAKEDEV pty
/sbin/MAKEDEV tty
/sbin/MAKEDEV urandom

Hopefully this helps if you run into a similar issue down the line!