Complete Windows XP Melt Down
03-04-08

On Friday (02-29-08), my completely up to date Windows XP machine underwent a complete meltdown ... for about half an hour ... then it magically fixed itself.

On Tuesday (03-04-08), the system went down again ... it fixed itself the next day.

This was a major disaster, on Tuesday

Friday's symptoms were different.

Eventually, the system started working again and no specific cause was found.

Symptoms - Friday Tuesday Wednesday Analysis | Context Menu Handlers | Group Policies | Conclusion | References


Symptoms

I experienced two different sets of symptoms - those on Friday and those on Tuesday.

On both days, the first symptom was observed while using Internet Explorer (IE) - when I tried to right click and open a link in another page ... the right click menu would not display.


On Friday, about 6:00 pm

Of course, the first thing I thought was - Virus.

On that machine, I normally keep the System Internals program Process Explorer running ... but I could not find it. So I started it again.

Right click worked ok ... but

The System Event Viewer (Control Panel / Administrator Tools) indicated that a number of Security Policy changes occurred at the same time as the failure. This included deleting, and then re-adding, the ports (137, 138, & 139) used for Windows file sharing.

Based on this information, I assumed that the IT department was simply making some kind of security update. The fact that the machine had been identified as critical (because it collects experimental data 24/7) is a major issue. If changing a critical machine at 6:00 pm on a Friday night broke something, then all the data for Saturday and Sunday would be lost. (Actually, on another machine related to the same experiment, the program crashed at 10:40 pm the same night, and ... in fact ... all data for the weekend WAS lost. Both systems had been up for more than 2 months without a problem.)

After about half an hour, the system started working again. At that point, I noticed that someone had changed the computer's security settings. Specifically, ports 137, 138, & 139 were disabled and then about 2 minutes later, they were re-enabled. (These are the ports Windows uses to share files between machines.) There were numerous other security and policy changes ... but details were lacking.

Based on the data collected, whatever happened to the machine crashed Process Explorer - that is why I could not find a running copy.


On Tuesday, about 7:00 pm

Tuesday, at about 7:00 pm (an hour later than Friday ... but still after hours), the system crashed again. As before, I noticed this because right click no longer worked in IE.

This time, Process Explorer continued to work ... showing data on the properties pages. And there were no crashes. However, the menu options (at the top of the application) quit working.

I tried to start notepad so I could make notes ... but there was no way to start it. I have shortcuts in the Start menu, but could not navigate to them. I was able to get to the Accessories menu and click on notepad ... but it would not start.

I normally use Alt-Tab to switch applications, but this key combination was working like Alt-Esc - directly switching applications instead of showing application icons and letting me step through those before changing the applications.

In several open applications,

Basically, the system was dead.

Note: The IE Favorites menu still worked ... but none of the other IE menus.

I still had an open System Event Viewer (left open since Friday) and, sure enough, there were new security changes at the same time as the failures. (F5 - refresh - was still working.)

Also, browsing the internet still worked as long as I did not try to right click a link ... regular left clicks still worked fine. (Though the menus did not work, I did not think to try and open a new window with Ctrl-N.)

After an hour and a half (about 8:45 pm) I left for the day - the system was still broken.


On Wednesday

Perhaps this was really a keyboard failure ... that would explain why it would come and go. The fact that Alt-Tab performed the same function as Alt-Esc implied a keyboard failure. However, testing on another system did not support this position - I could not find a key that, if stuck, would cause these symptoms.

On Wednesday (the next day) I unplugged the keyboard - the mouse still failed. I plugged it back in (it is USB) ... no difference.

About 5:oo pm, the system magically fixed itself ... again.

The best I've come up with is that too many windows were open. At any rate, after closing about 3 IE windows (using the mouse), the system started working normally again. I tried to recreate the problem by opening more windows and returning to the same sites. It had no effect.


Analysis

During the attack, I check the System Events on several other systems and they did not show similar entries ... to me, this implied that my machine was specifically targeted and NOT a part of a general security update. This suggests some kind of virus. Further analysis suggests that the questionable entries may have actually been caused by how I investigated the problem and therefore are not related to the cause of the problem.

I was still bothered by the continuous security policy changes, but these were explained as normal for our installation.

The Event logs apparently showed nothing of interest, I was able to use them to simply mislead myself.

A virus scan did find some bad files, but none were running.

A check of available memory, resources, handles, and the like did not reveal a problem.

It was determined that the Symantec software had a problem - so it was uninstalled and then reinstalled. (Attempts to repair and update it actually failed.)

I must admit they I am uncomfortable with security changes being made every 90 minutes. I understand the need for security, but I was taught that changes should always be fully tested (usually for several weeks) before being made to a live production system. Continuous, untested, updates is a disaster waiting to happen.


Context Menu Handlers

If a right click is too slow (or simply does not work), you should check the Context Menu Handlers. You can shotgun these (like some other pages suggest) or use a tool like RegMon to see exactly which programs are being run. (Process Monitor replaces RegMon, the Windows XP Process Explorer does not provide a good interface for this type of troubleshooting.)

Note: I do not suggest deleting anything from the registry (especially while debugging a problem). Instead, you should comment out the entries.


Group Policies

One way to disable the context (right click) menus is to set a policy. (Searching the Microsoft site for this information is not fruitful.) The following is from Group policy registry entries for xp home.

Disable Context menu

Remove Windows Explorer default context menu

Note: There were no entries for disabling notepad's context menu.


Conclusion

The system is now working. I tried everything I currently know and was not able to determine what caused this problem.

I was told that the security policies had not changed and, therefore, even though they were continuously "updated", these actions should not have caused the problems.

So ... I have to assume that Windows XP simply lost its mind and that there is a rather small limit on the number of windows that can be open. However, most XP crashes require rebooting the machine or killing an application, not just closing a few windows.


References


Author: Robert Clemenzi
URL: http:// mc-computing.com / Parasites / XP_Meltdown.html