Follow Us

We use cookies to provide you with a better experience. If you continue to use this site, we'll assume you're happy with this. Alternatively, click here to find out how to manage these cookies

hide cookie message

Microsoft server crash nearly causes 800-plane pile-up

Failure to restart system caused data overload.

Article comments

A major breakdown in Southern California's air traffic control system last week was partly due to a "design anomaly" in the way Microsoft Windows servers were integrated into the system, according to a report in the Los Angeles Times.

The radio system shutdown, which lasted more than three hours, left 800 planes in the air without contact to air traffic control, and led to at least five cases where planes came too close to one another, according to comments by the Federal Aviation Administration reported in the LA Times and The New York Times. Air traffic controllers were reduced to using personal mobile phones to pass on warnings to controllers at other facilities, and watched close calls without being able to alert pilots, according to the LA Times report.

The failure was ultimately down to a combination of human error and a design glitch in the Windows servers brought in over the past three years to replace the radio system's original Unix servers, according to the FAA.

The servers are timed to shut down after 49.7 days of use in order to prevent a data overload, a union official told the LA Times. To avoid this automatic shutdown, technicians are required to restart the system manually every 30 days. An improperly trained employee failed to reset the system, leading it to shut down without warning, the official said. Backup systems failed because of a software failure, according to a report in The New York Times.

The contract for designing the system, called Voice Switching and Control System (VSCS), was awarded to Harris Corporation in 1992 and the system was installed in the late 1990s, initially using Unix servers, according to Harris. In 2001, the company completed testing of the VSCS Control Subsystem Upgrade (VCSU), which replaced the original servers with off-the-shelf Dell hardware running Microsoft Windows 2000 Advanced Server. The upgrade was installed in California last year, according to the FAA.

Soon after installation, however, the FAA discovered that the system design could lead to a radio system shutdown, and put the maintenance procedure into place as a workaround, the LA Times said. The FAA reportedly said it has been working on a permanent fix but has only eliminated the problem in Seattle. The FAA is now planning to institute a second workaround - an alert that will warn controllers well before the software shuts down.

The shutdown is intended to keep the system from becoming overloaded with data and potentially giving controllers wrong information about flights, according to a software analyst cited by the LA Times.

Microsoft told Techworld it was aware of the reports but was not immediately able to comment.




Share:

More from Techworld

More relevant IT news

Comments

Michael said: There is a reason NASA Norad and other such do not run on MicrosoftI make my living on Microsoft servers but for any truly mission critical system I turn back to UNIX VAX or Mainframe Microsoft is simply too buggy to trust with peoples livesOld Engineer -- no doubt Any idiot who trust to a manual process like have people reboot every 30 days or the critical system will go down needs to be fired That is not engineering that is pulling one out the rear end

Jeremy said: I thought in Microsofts Terms of service they give us when we install thier Operating systems says that thier software is not to be used for life critical systems including Air Traffic Control so why is California using it

Chris said: For a Linux Systems Engineer the reboot of systems to make it more stable does not make sense Our Linux servers uptime are typically from months to years If no hardware failure hardware upgrade and no power failure I cannot see why a server should ever be shut ever And when you have hardware redundancy even upgrading of hardware can not render a service useless Also after the installation of new software no need to reboot - the system just carries on and on without interruptionAs to the security side there are more or less 60 viruses for Linux versus millions for WindowsCostwise Sql web server open office etc etc are all included in LinuxPreviously we had servers from Novell - 8 in total to serve 4000 employees very well Now management decided to go for Microsoft servers - the last count was 168 servers and growing Microsoft dictates on the amount of servers So be very carefull when going for Microsoft - it can cost you an arm and a leg

Northshore Process Service Cen said: Northshore Process Service HQ1560 Sherman Ave Ste 301Evanston IL 60201 USATel 8473738972Fax 8665542485Email infonpslawyercom

Hugh said: Diego said Ive worked on both unix and windows and both have their place Unix is only a super version of DOS 1 Mate if you think that UNIX is just a super version of DOS I can only conclude that the only work you have done with it is to have dusted the server2You are quite correct in saying that both UNIX and windows have their place The difference is that UNIXs place is not in the toilet

Criminal said: For a system such as this the contract should have included huge penalties for failures like this Then if those companies had any sense they wouldnt risk it so long as the contract prohibited insurance on such failures This would hopefully force them to use more stable off the shelf X86 solution like GNULinux BSD Solaris or something else

Ray said: FAA need to move to Linux system

JAKUB SZYPULKA (real one) said: ID LIKE TO SAY THAT THE PREVIOUS COMMENTS HAVE NOT BEEN WRITTEN BY ME AND SHOULD NOT BE CONNECTED WITH MY PERSON THANK YOU

Diego said: Please read the article more carefully It wasnt the technology that failed it was the setup Ive worked on both unix and windows and both have their place Unix is only a super version of DOS and its only file system based with no visual window overhead Windows XP has come a long way and pretty stable if you dont visit porn sites and download a bunch of junk

Derek W said: Gee 497 days seems familiar could it be Microsoft KB article 216641 Maybe they should have stuck with Unix and not switched to Windows 9598 Either way it looks like MS had released a patch for it I cant believe they accepted the procedure of rebooting the systems monthly

byteguru said: Unix rules and thats final Whoever doesnt like it it is because do not know any better

Richard said: Microsoft really are poetry in code One couldnt script a better storyWhat is a data overload You mean a typical registry-type-mishmash-needs-reinstallation

JJ said: Its simply scary that a major US airport would replace Unix servers with a Microsoft Windows product Are the powers that be out of thier minds Unix is the most rock solid stable server platform ever developed Maybe they just cant find any Unix sys-admins and the programmers want the goofy looking windows IconsThis is very scary Remind me to take AMTRAK next time I go to SO-CAL

woody said: How embarrassing for Microsoft to have it blasted out via the media that their servers must be rebooted every 497 days or face data overload Should have stuck with Unix or moved to Linux

kenfoo said: This 497 days sounds familiarI believe its the roll-over of the Windows internal timer GetTickCount function that resets to 0 every 497 daysGooglehttpwwwgooglecommysearc

gmathol said: Here it is Microsoft again Im sick and tired of SOHO software and Virus Bill

Garrick said: MS Windows is fine for personal computers games PDAs and vending machines If you had an implanted heart pacemaker for the sake of argument had to have an OS would you trust one running MS Windows

Brian Hammond said: 497 days you say Data overload you say Its rather that some measure of time in milliseconds is being stored in a 32-bit integer After 497 days such a 32-bit integer will fill up and overflow Go google 497 days in milliseconds and 232 - 1 Use the right data type for job stupid MS engineers

Joe P said: Setting up a re-occuring appointment in Outlook or some such too much to ask

The Truth said: Why replace the Unix system with Windows The answer is the same with everything else that happens in the US aerospace industrylowest bidder Want a nice flash re-usable space vehicle Find the lowest bidder Want to fund a foreign takeover Find a future terrorist who is alsothe lowest bidder Want a brand new ATC system Wow Microsoft Thats really cheap As so it goes on



Send to a friend

Email this article to a friend or colleague:

PLEASE NOTE: Your name is used only to let the recipient know who sent the story, and in case of transmission error. Both your name and the recipient's name and address will not be used for any other purpose.

Techworld White Papers

Choose – and Choose Wisely – the Right MSP for Your SMB

End users need a technology partner that provides transparency, enables productivity, delivers...

Download Whitepaper

10 Effective Habits of Indispensable IT Departments

It’s no secret that responsibilities are growing while budgets continue to shrink. Download this...

Download Whitepaper

Gartner Magic Quadrant for Enterprise Information Archiving

Enterprise information archiving is contributing to organisational needs for e-discovery and...

Download Whitepaper

Advancing the state of virtualised backups

Dell Software’s vRanger is a veteran of the virtualisation specific backup market. It was the...

Download Whitepaper

Techworld UK - Technology - Business

Innovation, productivity, agility and profit

Watch this on demand webinar which explores IT innovation, managed print services and business agility.

Techworld Mobile Site

Access Techworld's content on the move

Get the latest news, product reviews and downloads on your mobile device with Techworld's mobile site.

Find out more...

From Wow to How : Making mobile and cloud work for you

On demand Biztech Briefing - Learn how to effectively deliver mobile work styles and cloud services together.

Watch now...

Site Map

* *