Vista plays hide-and-seek with hackers

By Joris Evers
Staff Writer, CNET News.com

Published: June 1, 2006, 12:42 PM PDT

Microsoft is starting a game of hide-and-seek with malicious code writers.

Windows Vista Beta 2, released last week, includes a new security feature designed to protect against buffer overrun exploits. Called Address Space Layout Randomization (ASLR), the feature loads key system files in different memory locations each time the PC starts, making it harder for malicious code to run, according to Microsoft.

"It is not a panacea, it is not a replacement for insecure code," Michael Howard, a senior security program manager at Microsoft, wrote in a blog post announcing the feature. "But when used in conjunction with other technologies...it is a useful defense, because it makes Windows systems look 'different' to malware, making automated attacks harder."

A buffer overrun exploit is malicious code that seeks to exploit a common error in computer code called a buffer overrun or buffer overflow. In such an attack, data is stored beyond the boundaries of a buffer, with the result that the extra data overwrites adjacent memory locations. This can cause a process to crash, or allow malicious code to run.

ASLR is not a Microsoft invention. Several open-source security systems use it already, including OpenBSD, and the PaX and Exec Shield patches for Linux.

Certain attacks attempt to call Windows system functions, such as the "socket()" function in "wsock32.dll," to open a network socket. The new security feature moves these system files around so they're in unpredictable locations. In Windows Vista Beta 2, a DLL or EXE file could be loaded into any of 256 locations, Howard wrote.

"An attacker has a 1/256 chance of getting the address right," Howard wrote.

Randomization seems to have served open-source systems fairly well, said Russ Cooper, senior scientist at Cybertrust, a security vendor in Herndon, Va. The question is how Microsoft implements ASLR and whether the randomization is predictable at all, he said.

"I suspect this will be the first thing looked for--something which tells you which of the locations has been chosen, or anything that provides you with a pointer," Cooper said.

Attackers could also create malicious software that tries to poke at all 256 memory locations. However, that's more likely to cause the PC to crash, rather than allow a complete compromise, Cooper said. "That's good if all you care about is preventing malware from running, but it might not bode well for keeping systems up and running," he said.

ASLR feedback
Microsoft gets some praise in the security world for its ASLR efforts in Vista. "Remote exploitation of overflows has just got a lot harder," David Litchfield, a researcher at Next Generation Security Software, wrote in an e-mail to the BugTraq mailing list.

But there is also skepticism. Somebody using the alias "c0ntex" wrote in a reply to Litchfield that ASLR has been "trivially circumvented in Linux for years now."

Microsoft has only just added ASLR to a Windows Vista trial release, another sign that the successor to Windows XP is not yet ready for prime time. "We added ASLR pretty late in the game, but we decided that adding it to beta 2 and enabling it by default was important so we can understand how well it performs in the field," Howard wrote.

Together with other enhancements in Vista, ASLR raises the bar in terms of security in the forthcoming operating system, Microsoft says. The company has described Vista, slated to be broadly available in January, as the most secure version of Windows to date.

In addition to ASLR, Howard mentioned a buffer overrun detection option in Visual C++; an exception checker in Vista; function pointer obfuscation; and support for NX, or No-Execute, data execution protection that is included in processors.

"The net of this is, ASLR is seen as just another defense," Howard wrote.