Learn about those things about operating systems, starting from this article
🎓 Do everything, obey the fate. The blogger is studying for a master's degree in Southeast University, loves fitness and basketball, and is willing to share what he sees and gains related to technology. Follow the public account @飞天小牛肉 to get article updates as soon as possible, and we will make progress together on the way to growth.
🎁 This article has been included in the CS-Wiki (Gitee official recommendation project, now 1.0k+ star). We are committed to creating a complete back-end knowledge system and avoid detours on the road of technology. Welcome all friends to come to exchange and learn
It is an indisputable fact that you can write programs without understanding computer systems. But we are not learning operating systems to create an operating system, but to write better programs after understanding the operating system .
The full text of the mind map is as follows:
1. What is an operating system
To understand what an operating system is and what the operating system is for, we also need to start from the origin of the operating system's requirements.
In 1946, the world's first general-purpose computer ENIAC (ENIAC) was born at the University of Pennsylvania, using 18,000 electron tubes, covering an area of 170 square meters and weighing 30 tons. The most frightening thing is that ENIAC, or early computers, did not have an operating system . The operation of this 170-square-meter behemoth was entirely manual operation by engineers. What does it feel like? Please see the picture below:
In the picture, the big machine all over the room is ENIAC. From the photo, we can see an operator is manually adjusting a button, and there are thousands of buttons like this in the whole room! It is conceivable that in an era without an operating system, engineers need to directly face this kind of anti-human hardware panel to operate, and the labor cost to operate and maintain a computer is simply unimaginable.
那么，操作系统到底是何方神圣？我们先来看看早期的操作系统 Disk Operating System，DOS（磁盘操作系统） 长什么样：
有同学就说这不就是我们使用的 cmd 命令行，没有错，但是，它是整个屏幕都是这样的界面，全屏都是！而不是某一个对话框里面是这样。
显然，DOS 这种通过键盘输入命令的操作方式已经比我们手动调整按钮、连接线路要简单得多。但是，这样的命令行界面（CLI）必定无法符合大部分用户的口味，于是后来拥有良好图形界面（GUI）的操作系统比如 Windows 逐渐成为主流，毕竟只需通过鼠标点击就能进行操作，门槛已经足够的低。
这就引出了操作系统的第二个主要功能：为用户提供一个友好、清晰且简单的操作界面（专业的叫法是 壳 Shell），用户通过 Shell 执行该操作系统提供的所有的功能。对于命令行来说，就需要提供足够多的命令；对于图形界面来说，就需要提供足够多数量的按钮。
Manage computer hardware and software resources
Provide users with an interface to interact with the system
2. A brief introduction to computer hardware
To be able to work, the operating system must understand a large amount of hardware, at least how the hardware faces the programmer. For this reason, here we first briefly introduce the hardware in modern personal computers, and then discuss the details of the operating system.
A simple personal computer can be abstracted into a model similar to the following figure:
CPU, memory and I/O devices are all connected by a system bus and communicate with other devices through the bus.
The so-called processor is what we often call the CPU, which is the brain of the computer . The CPU fetches the instruction from the memory and executes it. The execution cycle of a CPU is to fetch an instruction from memory, decode and determine its type and operand, then execute it, and then fetch, decode and execute the next instruction. Repeat the cycle until the program is completed.
To put it simply, everyone remembers this sentence: a program needs to be put in memory and assigned a CPU to execute it .
The second major component in a computer is memory. Ideally , the memory should be very fast , faster than executing an instruction, so that the CPU's execution efficiency will not be affected by the memory, and it is large enough and very cheap .
However, the current technical means cannot meet these three goals at the same time, so different treatment methods have emerged. The memory system adopts a hierarchical structure. As shown in the following figure, the top-level memory has the fastest speed, the smallest capacity, and the highest cost. The lower the memory, the slower the speed, the larger the capacity, and the lower the cost .
Why is it so fast to access registers ? That's because the material of the register is the same as that of the CPU, so there is almost no time delay compared to accessing the CPU.
The next level of registers is the cache , which is no stranger to everyone. Usually the use of cache will bring performance improvements. The operating system has been using caching, such as keeping frequently used files in memory, to avoid repeatedly fetching these files from the disk.
Further down layer is memory , also known as main memory , commonly referred to as a random access memory (the Random Access Memory, the RAM ). All access requests that cannot be satisfied in the cache will be transferred to memory.
In addition to RAM, many computers also have a small amount of non-volatile random access memory (Read Only Memory, ROM ). They are different from RAM. After the power is cut off, the ROM will not lose its content, and its content will not be modified once it is stored. And ROM is very fast and cheap.
The next level is the disk , which has a larger capacity. The only problem with the disk is that the time of random access to data is about three orders of magnitude slower than that of the memory. The reason for its low speed is because the disk is a mechanical device and has a special structure . As shown below:
There are one or more metal platters in a magnetic disk , which rotate at 5400, 7200, 10800 rpm or higher. Starting from the edge, there is a robotic arm hanging across the disc. In this case, everyone can recall the old-fashioned plastic record player. Information will be written on a series of concentric circles on the disk. At any given arm position, each head can read a circular area called a track . Combine all the tracks on a given arm position to form a cylinder .
Each track is divided into several sectors , and the value of the sector is 512 bytes. In modern disks, the outer cylinder has more sectors than the inner cylinder. It takes about 1ms for the robot arm to move from one cylinder to the adjacent cylinder. The typical time to randomly move to a cylinder is 5ms to 10ms, depending on the drive. After the magnetic arm reaches the correct track, the drive must wait for the required sector to rotate below the magnetic head (this takes about 5ms to 10ms delay), and then start reading and writing. The speed of the low-end hard disk is
50MB/sthat of the high-speed disk. The rate is
CPU and memory are not the only resources that the operating system needs to manage, and I/O devices are also inseparable from the operating system. As shown in the figure below, I/O devices generally include two parts: the device controller and the device itself, such as the keyboard controller and the keyboard.
The device controller is actually a chip or a group of chips that can receive instructions from the operating system and control physical devices. For example, read data from the device and complete data processing.
In many cases, the actual process of controlling the equipment is very complicated and has many details. So the job of the controller is to provide a simpler (but still very complex) interface for the operating system.
The other part of the I/O device is the device itself . The device itself has a relatively simple interface. This is because the interface can neither do a lot of work, but has also been standardized . After standardization, any disk controller can adapt to any one. Disk, so standardization is very necessary.
3. Four characteristics of operating system
The operating system has 4 distinct characteristics: concurrency, sharing, virtual and asynchronous. Among them, concurrency and sharing are the most basic features of the operating system . Without concurrency and sharing, virtuality and asynchrony cannot be talked about.
Let's take a brief look at these 4 features, many of the detailed concepts involved will not be explained in this article, and will be more in the future.
The twin brothers of concurrency and parallelism often leave beginners confused.
Concurrency : Concurrency refers to the ability to run multiple programs at the same time within a period of time . Of course, these procedures occur simultaneously at the macro level, but alternate at the micro level. The operating system enables programs to run concurrently by introducing processes and threads.
Parallel : Parallel refers to the ability to run multiple instructions at the same time , which means that two or more events occur at the same time . Parallel requires hardware support, such as multiple pipelines, multi-core processors, or distributed computing systems.
With the understanding of the image below:
Speaking of concurrency and parallelism, I have to mention the CPU. As the brain of the computer, the CPU mainly interacts with the memory, extracting instructions from the memory and executing them. And a program needs to be put into memory and assigned a CPU to it to execute, so the ability of concurrent parallelism is closely related to the performance of the CPU:
A single-core CPU can only execute one program at a time, and each program can only be executed concurrently ;
A multi-core CPU can execute multiple programs at the same time, and multiple programs can be executed in parallel .
For example, Intel's eighth-generation i3 processor is a 4-core CPU, which means that 4 programs can be executed in parallel. Of course, even for a 4-core CPU, as long as there are more than 4 programs that need to run "simultaneously", concurrency is still essential, so concurrency is one of the most basic characteristics of the operating system .
Sharing is resource sharing , which means that the resources in the system can be used by multiple concurrently executing processes in memory .
There are two main ways to share:
1) Mutually exclusive sharing
The so-called mutually exclusive sharing means that although the resource is shared and all processes can use it, the same resource is only allowed to be accessed by one process at a time , which is also called mutually exclusive access. A synchronization mechanism is needed to achieve mutual exclusive access. . The mutually exclusive shared/accessed resources are called critical resources .
Give an example of mutually exclusive sharing:
If we use QQ and WeChat video at the same time, the camera resource can only be allocated to one of the processes in the same time period.
2) Simultaneous sharing
Simultaneous sharing is the opposite of mutually exclusive sharing, allowing multiple processes to "simultaneously" access certain resources in the system within a period of time . Of course, the so-called "simultaneous" is often at the macro level, and at the micro level, these processes may alternately access the resource (that is, time sharing)
Give an example of simultaneous sharing:
Use QQ to send file A on the hard disk, and use WeChat to send file B on the hard disk. From a macro point of view, both sides are reading and sending files at the same time, and both are accessing hard disk resources and reading data from them. From a microscopic point of view, the two processes of QQ and WeChat access hard disk resources alternately.
并发和共享作为操作系统的两大最基本特征，其实是互为存在条件的。为什么这么说呢？这样，我们继续通过上述发送文件的例子来看并发与共享的关系（使用 QQ 发送硬盘上的文件 A，同时使用微信发送硬盘上的文件 B）：
如果失去共享性，则 QQ 和微信不能同时访问硬盘资源，就无法实现同时发送文件，并发也就无从谈起
上文说过，一个程序需要放入内存并给它分配 CPU 才能执行。那比如说 GTA5 需要 4GB 的运行内存，QQ 需要256MB 的内存，Chrome 需要 512MB 的内存，网易云音乐需要 256MB 的内存…… 假设我们的电脑只有 4GB 内存且 CPU 是单核的。那么这里就存在如下两个问题：
问题 1：这些程序同时运行需要的内存远大于 4 GB，那么为什么它们还可以在我的电脑上同时运行呢？
答：这得益于虚拟内存技术的空分复用技术。虽然我们的电脑实际上只有 4GB 的内存，但是却可以完美的同时运行这些远大于 4G 内存的程序，在用户看来似乎我们的电脑内存远远大于 4GB。
问题 2：既然一个程序需要被分配 CPU 才能正常执行，那为什么单核 CPU 的电脑中也能同时运行这么多个程序呢？
Answer: This is due to the time division multiplexing technology of virtual memory technology . Although there is actually only one single-core CPU, it is impossible to execute so many programs in parallel at the same time, but microscopically, the CPU alternately serves each process in each tiny time period. It seems to users that our CPU is multi-core .
Let us explain the above two virtual technologies:
1) Space division multiplexing technology
The principle of the space division multiplexing technology is to use the memory as a cache, only to store the most frequently used part of the program, and put most of the program on the disk. This mechanism requires fast mapping of the memory address in order to convert the address generated by the program into the physical address of the relevant byte in the memory. This image consists of one member of CPU, called a memory management unit (Memory Management Unit, the MMU ) to complete this we have seen above in FIG Recalling again:
2) Time division multiplexing technology
Multiple processes can be executed concurrently on the same CPU because of the use of time-division multiplexing technology, allowing each process to occupy the processor in turn, executing only a small time slice each time and switching quickly.
Obviously, if concurrency is lost, and only one program can run in the system within a period of time, it will also lose the meaning of realizing virtuality. Therefore, there is no virtuality without concurrency
Asynchrony refers to: in a multi-program environment, multiple programs are allowed to execute concurrently, but due to limited resources, the execution of the process is not consistent to the end. Instead, it stops and moves forward at an unpredictable speed . This is the process The asynchronous nature.
For example: Lao Slag wants to date two girls concurrently at 8 o'clock and 12 o'clock, and each girl’s date is 2 hours. Dating with No. 1 and No. 2 is two concurrently executed programs, and Lao Zha’s heart It is limited system resources.
Since concurrently running programs will compete for the use of system resources, and the resources in the system are limited, the two-hour appointments of the same number or the second number of the old scum may not end at once, but intermittently. Maybe 8 to 9 o'clock and date one, 9 to 10 o'clock and number two, then 10 o'clock to 11 o'clock and then date one, and finally 11 o'clock to 12 o'clock and number two.
If concurrency is lost, that is, the system can only run each program serially, then the execution of each program will be consistent to the end. Therefore, only when the system has concurrency can it lead to asynchrony .
4. Learn the core content of the operating system
The core content of this series mainly includes the following:
File system management
I/O device management
① Process management
The concept of process (Process) has appeared many times above. In order to better realize the concurrency and sharing of the operating system, the process is introduced.
A process is an execution process of a program, it is temporary. Program includes not only entities that are running, and includes all the system resources to run this program occupies, such as CPU, memory, network resources . When answering the concept of a process, many small partners often only say that it is a running entity and ignore the resources occupied by the process. For example, if the same program is run twice at the same time, then they are two separate processes.
Speaking of processes, I have to talk about threads. There can be multiple threads in a process, and they share process resources.
For example, QQ and Chrome browsers are two processes. There are many threads in the Chrome process, such as HTTP request thread, event response thread, rendering thread, etc. The concurrent execution of threads makes it possible to click a new link in the browser to initiate When an HTTP request is made, the browser can also respond to other events of the user.
What we call process management actually mainly includes two contents:
As the name implies, process communication is the communication between processes. CPU is the most precious resource of a computer. A process needs to be put into memory and assigned a CPU before it can be executed. A program may contain multiple processes. This requires processes to communicate with each other and synchronize with each other to complete the program together. For example, if process A produces data and process B prints data, then B has to wait before A produces data, then A has to send a message to B, telling B that I have produced the data and you can print it.
Process scheduling means that under normal circumstances, there will be multiple processes or threads competing for the CPU at the same time. If only one CPU is available, the CPU must make a choice for the next running process or thread. This selection process is the process Scheduling.
② Memory management
The cache is the bottom memory except for registers, and its management is done by hardware, so what we learn is focused on the management of memory.
In fact, we talked about this topic when we virtualized this feature in the four features of the operating system above. Memory is generally used to store the program being executed. In a very simple operating system, only one program can run in the memory at a time. , If you want to run the second program, the first program must be removed from the memory, and then the second program is loaded into the memory.
Obviously this is very inefficient. For this reason, virtual memory technology is introduced, and the memory is used as a cache, which is only used to store the most frequently used part of the program, and most of the program is placed on the disk.
So what the memory management does is probably:
Put frequently used part of the program into memory
When the memory is full, replace some parts of the memory
③ File system management
A file is actually a logical unit of information created by a process. A disk may contain thousands or even millions of files. Each file is independent of other files. In fact, it is easier to understand a file by treating each file as an address space. The essence of.
Files are also managed by the operating system. The structure, naming, access, use, protection, implementation, and management methods of files are the contents of the operating system design. This is the theme of the file system.
④ I/O equipment management
The operating system must efficiently manage the I/O device. It needs to send commands to the I/O device, catch interrupts, and handle various errors of the device. It should also provide a simple and easy-to-use interface between the device and other parts of the system. interface.
5. Kernel mode and user mode
The term "kernel" may make everyone a little confused. Explain here. Modern operating systems all adopt the concept of process. In order to better deal with the concurrency and sharing of the system and enable processes to work in a coordinated manner, it is far from enough to rely on the functions provided by computer hardware. For example, process scheduling cannot be implemented by hardware, and a set of basic software must be used to transform hardware resources in order to provide a good operating environment for the execution of the process. This software is the kernel.
Simply put, the kernel is a set of program modules in the operating system , as trusted software to provide basic functions and basic operations that support the concurrent execution of processes, and has access to hardware devices and all memory spaces . It is no exaggeration to say that the kernel is the core of the operating system.
Of course, in addition to the kernel program, the operating system also includes other basic components, such as a text editor, a compiler, and programs used to interact with users, such as a desktop system.
① What is kernel mode and user mode
So since the kernel is a program, it needs to run, it must be assigned a CPU. Therefore, there are two programs running on the CPU, one is the kernel program of the operating system (also called system program), and the other is an application program. The former accomplishes system tasks, and the latter accomplishes application tasks. There is a relationship between control and being controlled. The former has the right to manage and allocate resources, while the latter can only apply to the system for the use of resources.
Obviously, we should distinguish between the two types of programs running on the CPU. This is why the kernel mode and the user mode appear.
Kernel mode (kernel mode): When the CPU is in kernel mode, which is the OS program (i.e. the core) in which the running state. Programs running in the kernel mode can access any resources of the computer without restriction and do whatever they want, such as coordinating CPU resources, allocating memory resources, and providing a stable environment for running applications.
User mode: Applications basically run in user mode, or user mode provides space for applications to run. The program running in the user mode can only access the address space where the program is executing on the current CPU, which effectively prevents the operating system program from being infringed by the application program.
For the operating system, what kind of program should be placed in the kernel mode ? This depends on factors such as the demand for resources, the urgency of time and the level of efficiency. For example, resource manager programs such as CPU, memory, and equipment should run in kernel mode, otherwise security is not guaranteed. For file systems and data, file system data and management must be placed in kernel mode, but user data and management can be placed in user mode.
② Interrupt mechanism
Under appropriate circumstances, the kernel of the operating system will actively transfer the right to use the CPU to the application program, which means that the CPU will transition from the kernel mode to the user mode. If the CPU wants to return from the user mode to the kernel mode, it can only be done through the interrupt mechanism. If there is no interrupt mechanism, once the application program runs on the CPU (user mode), the CPU will always run the application program. In other words, interrupts are the only way for the operating system kernel to regain CPU usage rights . It can be said that the operating system is driven by interrupts .
Of course, the interrupt mechanism here is very broad, including three means, that is to say, there are three means to switch from user mode to kernel mode:
1) The program requests operating system services and executes system calls
2) When the program is running, an external interrupt event (such as the completion of the I/O operation) is generated, the running program is interrupted, and the interrupt program is processed
3) An internal interruption (abnormal) event occurs when the program is running, and the running program is interrupted, and it turns to the exception handler to work
The above three methods all happen through the interrupt mechanism, so let's take a look at the types of interrupts.
According to whether the interrupt signal comes from outside or inside the CPU, the interrupt types are divided into external interrupt and internal interrupt:
External interrupt (also called interrupt, interrupt in a narrow sense)
External interrupt has nothing to do with the currently executed instruction, the interrupt signal comes from outside the CPU. If the I/O complete interruption, it means that the device input/output processing has been completed, and the CPU can send the next input/output request. In addition, there are clock interrupts, console interrupts, etc.
Internal interrupt (also called exception, exception)
The internal interrupt is related to the currently executing instruction, and the interrupt signal comes from the CPU. Such as illegal operation code, address out of range, arithmetic overflow, divisor 0, etc.
Here is a brief explanation of the basic principle of the interrupt mechanism : different interrupt signals must be processed by different interrupt handlers. Then when the CPU detects the interrupt signal, it will query the interrupt vector table according to the type of the interrupt signal to find the storage location of the corresponding interrupt handler in the memory.
③ System call
As mentioned above, the program can also make the CPU switch from user mode to kernel mode by executing system calls. So what is the relationship between system calls and interrupts? Why is this method also a kind of interrupt mechanism? That's because the system call is done through a trap instruction , which causes an internal interrupt.
In more detail, the operating system, as the first layer of software on top of computer hardware, needs to provide some simple and easy-to-use services to the upper layer. This upper layer includes users and applications:
Interface to provide users with a graphical interface GUI and command interface is provided to the application program interface , the program interface consists of a set of system calls is composed of the operating system available to developers to use. It can be understood as a special function that can be called by the application program . The application program can request the service of the operating system kernel through the system call .
The abbreviated version of the system call process is roughly as follows:
1) In user mode, the application passes system call parameters
2) Execute the trapped instruction, trigger an internal interrupt, and make the CPU enter the kernel mode
3) In the kernel mode, execute the corresponding request, and the kernel program processes the system call
4) Return to the application
Think about why the system call is necessary ?
For example: we go to the school print shop to print the paper, you press the "print" option of WPS, and the printer starts to work. Halfway through the printing of your paper, another student pressed the "Print" button of Word to start printing his own paper. Imagine what would happen if two processes could freely and concurrently share printer resources?
Obviously, the two processes run concurrently, causing the printer device to alternately receive print requests from the WPS and Word processes. As a result, the contents of the two papers are mixed.
how to solve this problem? This requires the operating system kernel to perform unified management of shared resources and provide "system calls" to the upper layer. Applications or processes running in user mode who want to use shared resources like printers can only use system calls to the operating system kernel Send a request, and then the kernel will coordinate each request (process scheduling).
Through the above example, we can summarize what functions will use the system call: all operations related to shared resources (such as memory allocation, I/O operations, file management, etc.) must be called to the operating system through system calls The kernel makes a request, which is completed by the operating system kernel . This can ensure the stability and security of the system and prevent users from performing illegal operations. These system calls can be roughly divided into the following categories according to their functions:
Equipment management. Complete the equipment request or release, and equipment startup and other functions.
File management. Complete the functions of reading, writing, creating and deleting files.
Process control. Complete functions such as process creation, cancellation, blocking and wake-up.
Process communication. Complete functions such as message transmission or signal transmission between processes.
Memory management. Complete memory allocation, recycling, and obtain the size and address of the memory area occupied by the job.
The stitching and mending is finally finished. To be honest, this series of review articles is really difficult to write. It is necessary to take care of the integrity of the knowledge system and ensure that each module can be covered, and it must not be too deep or too detailed. It needs to be gradual. , To prevent the sudden appearance of a proper term that has not been introduced.
OK, grab the tail of the Spring Festival, I wish you all a happy new year, the Year of the Ox 🎉, this is the beginning of the operating system series~
"Modern Operating System-3rd Edition"
"Wang Dao Postgraduate Entrance Examination-Computer Network"
Baidu Encyclopedia-Process Management
Zhihu — What is an operating system? https://www.zhihu.com/question/61861692/answer/575287870
🎉 Follow the official account | Feitian Veal, get updates instantly
The blogger is studying for a master's degree in Southeast University. He uses his spare time to operate a public account "Flying Veal" , which will be opened on 2020/12/29, focusing on sharing computer basics (data structure + algorithm + computer network + database + operating system + Linux), Java Good articles on the basics and related original techniques of the interview guide. The purpose of this official account is to allow everyone to quickly grasp key knowledge and have a targeted . I hope everyone will support you, and grow up with Veal😃
And recommend personally maintained open source tutorial projects: CS-Wiki (Gitee recommended project, now 1.0k+ star), committed to building a complete back-end knowledge system, and avoid detours on the road of technology, welcome all friends to come to exchange and learn ~