Company founder Bill Gates is leaving the chairman role and becomes its new technology adviser.
Mr Nadella, 46, replaces Steve Ballmer and becomes only the third boss in the 38-year history of the business.
His appointment was widely expected. He was born in India and has been with the firm for 22 years.
He previously worked in some of its fastest-growing and most profitable areas, including the Office suite of products.
Mr Gates’ technology adviser role will be on the board of directors.
Microsoft said the founder will devote more time to the company.
In other management shake-ups, John Thomson was announced as the IT giant’s new chairman.
Shares in Microsoft were up 1.2% in early trading. Here is the latest share price.
The company has seen its traditional PC-centric Windows and Office franchises eaten away and must fight against ground made by Apple and Google in mobile computing.
Started in April 1975 by Mr Gates and Paul Allen, Microsoft created software that powered third-party computers.
Its early bMS-DOS system was replaced by Windows and its Office products in the late 1980s, driven by demand from private and business purchases.
However, it was late developing products in recent years and allowed Google to dominate online search and advertising.
Meanwhile, it saw Apple’s mobile devices such as iPhones and iPads grow. Android devices have also sucked potential sales away from Microsoft.
The company’s move into manufacturing its own devices has been hit with glitches, from the halted Kin phones to the as-yet unprofitable Surface tablet devices.
Wall Street investors have also been pushing for overhead reductions through job cuts and better returns taken from the firm’s large cash reserves.