How to design an operating system?

Do you mean customize, if yes, read this
http://www.microsoft.com/technet/desktopdeployment/articles/080505ds.mspx

Good luck if you want to write a Windows XP or Linux replacement. If you want to learn something about operating systems, an embedded operating system for which the source is available, that will run on many cheap development boards ($100 or less), that is free and has been written about extensively is MicroC/OS. Try http://www.micrium.com/downloads/ports/ucos-ii/ap3232.pdf

Yeah, way beyond the scope of this forum. Modern operating systems are created by thousands of people... in teams, with millions of lines of code. Windows XP itself, has over 35 million lines of code...way beyond the ability of one person to develop.

The only operating systems that can be built on a smaller scale are those that fit in specialized embedded devices. Even the Symbian operating system, found in most of the cell phones in the world, requires an industry consortium to develop.

The days of a sixteen year old kid creating an operating system up in his/her bedroom that would change the world are long gone. Creating a modern operating systems is akin to creating a jet liner...a massive team process involving many people with many different scientific disciplines.

The question one should ask is:
What should I design an operating system for?

An Operating system is usually a layer between the hardware and applications.

There are hundreds of operating systems out there today. OS design is not a simple undertaking. One usually needs to have a very strong compelling reason to go through with this.

The 'how' is going to resemble most 'large' software development projects.

I had to do one for a computer science class in college. Here's some things you'd need to look at:

- Study other operating system designs. There are some open source ones to look at- Minix, Linux, FreeBsd, FreeDos.

- Decide on what kind of kernel you want to have.

- Device Drivers

- the User Interface.