For the longest time, Windows Media Player would crash every time I plugged in a USB device. This occurred with both Vista Ultimate x64 and later Windows 7 Ultimate, and a solution eluded me.
When I tried installing the Zune client software, the same thing happened. Fortunately, this time I found the solution: delete the %SystemDrive%\ProgramData\Microsoft\Windows\DRM folder. In my case, a running application (MusicMatch Jukebox) had the folder locked but once I learned its identity I was able to kill the process and delete the directory, after which both Zune and Windows Media Player worked flawlessly.