Thanks to Kevin Hilton:
How to uninstall/install cygwin under Vista:

Uninstall Steps
(From previous attempted failed installations)

In Administrator shell
1. net stop sshd
2. cygrunsrv -R sshd
3. net user

Delete any user listed like sshd or sshd_server.  
The following syntax is to be used:
  net user sshd /DELETE

4. rm -R /etc/ssh*
5. mkpasswd -cl > /etc/passwd
6. mkgroup --local > /etc/group

In User shell
1. cd ~
2. rm -R .ssh

***********************************************

Installation Steps

In Administrator shell
1. ssh-host-config -y
2. net start sshd

In User shell
1. ssh-user-config (Answer questions as appropriate to your situation)
2. Test connection:
   ssh -vvv localhost