The problem:- 16-bit MS-DOS Subsystem path to the program that you are trying to start or install C:\Windows\System32\config.nt The system file is not suitable for running MS-DOS and Microsoft Windows applications. Choose 'Close' to terminate the application.
Error message when you install or start an MS-DOS or 16-bit Windows-based program http://support.microsoft.com/default.aspx?scid=kb;en-us;324767
Alternatively one can put back the standard version of autoexec.nt/config.nt by copying a backup copy held in %windir%\repair\ to %windir%\system32\ (%windir% is typically C:\Windows) instead of using the procedure in the KB article above.
It could be caused a trojan or something similar, e.g. TrojanDownloader.Win32.Dia.a, more about this here: http://tinyurl.com/5mw6l
Also, AVAST antivirus software can be the cause if one has WinXP SP2 installed: