Windows 10 Update Stuck Fix Remove USB Devices

As Microsoft released the Anniversary update to Windows 10 yesterday, many users started to download it.
As a result many have had their computers stuck in starting mode with the spinning circle and no boot.
I for one had this problem, even left the computer running all night and it still was spinning in the morning.
The fix as I read online was to remove a USB adapter for my wireless mouse.
I had already turned off and on the computer, and as soon as I removed the USB adapter, the computer booted.