Strange Remote Desktop Problems

Before I start my tale of woe, I just want to point out that I've tried disabling the autotuning feature on Vista machines, but this doesn't fix the problem.

I have a relatively clean desktop machine. The desktop is a host for several virtual PCs, and runs Vista.

Remote desktop connections to the virtual PCs hosted on this desktop are solid, and the connections never drop. The virtual PCs run XP and 2003 Server.

But ... remote desktop connections to the desktop itself stall every 5 minutes. Sometimes the RDP connection drops entirely - other times it's just a matter of waiting 10-20 seconds for the connection to reconnect.

Each time this happens I lose a little more hair.

Even stranger - while a connection to the desktop is stalled, a connection to a virtual PC hosted on the desktop is still working great. I can even ping the host machine and see a 2ms response time.

This behavior leads me to believe that:

1. There is nothing wrong with the RDP client
2. There is nothing wrong in the networking stack or hardware of the desktop.
3. There is something wrong with the terminal services service on Vista.

I'm hoping SP1 will fix the issue, and do so before I go bald.

Anyone else see similar behavior?