From 575d48efa6246be939ff2a5682c9dc1e7b455c26 Mon Sep 17 00:00:00 2001 From: yenru0 Date: Sun, 30 Nov 2025 21:19:04 +0900 Subject: [PATCH] complement 9.md 10.md 11.md --- notes/10.md | 218 ++++++++++++++++++++++++++++++++++++++++++++++++++ notes/11.md | 223 ++++++++++++++++++++++++++++++++++++++++++++++++++++ notes/9.md | 59 +++++++++++++- 3 files changed, 496 insertions(+), 4 deletions(-) create mode 100644 notes/10.md create mode 100644 notes/11.md diff --git a/notes/10.md b/notes/10.md new file mode 100644 index 0000000..365d070 --- /dev/null +++ b/notes/10.md @@ -0,0 +1,218 @@ +# Signals and Nonlocal Jumps + +## Shell + +Linux Process Hierachy: all is rooted from `init` or `systemd` (pid = 1). + +A shell is an application that runs programs on behalf of the user. + +* `sh` Original Unix shell +* `csh` BSD Unix C Shell +* `bash` GNU Bourne-Again Shell (default Linux shell) + + +## Signals + +All signal is a small message that notifies a process that an event has occured in the system. + +Signal is sent by kernel. + +Signal is identified by small integer ID (1-30). + +Only information in a signal is **its ID** and **the fact that it arrived**. + +| ID | name | Default Action | Corresponding Event | +| --- | ------- | -------------- | ----------------------------------- | +| 2 | SIGINT | terminate | user types `Ctrl+C` | +| 9 | SIGKILL | terminate | kill program | +| 11 | SIGSEGV | terminate | segmentation violation | +| 14 | SIGALRM | terminate | timer expired | +| 17 | SIGCHLD | ignore | child process stopped or terminated | + +Sending a signal is essentially **updating some state** in the context of the destination process. +Kernel sends a signal for one of the following reasons: + +A destination process receives a signal when it is forced by the kernel and reacts some way to the signal. +* Ignore +* Terminate (with optional core-dump) +* Catch + * executing a user-level handler function called **signal handler**. + +### Pending & Blocking + +A signal is **pending** if sent but not yet received. There can be at most one pending signal of any particular type because signals are **not queued**. Signal is managed by bitmask. So subsequent signals of same type is discarded. +* sets `k` bit when delivered +* clears `k` bit when received + +Blocking signals: a process can block the receipt of certain signals. It is not received until unblocked. It is also managed by bitmask. +* can be set and cleared by using `sigprocmask` + +### Receiving + +Suppose kernel is returning from an exception handler and is ready to pass control to process `p` +Kernel computes `pnb = pending(p) & ~blocked(p)`. +* if `pnb == 0`, no unblocked pending signals for `p`, so just return to `p` normally. +* otherwise, choose least nonzero bit `k` in `pnb` and force process `p` to receive signal `k` + * The receipt of the signal triggers action by `p` + * Repeat for all nonzero `k` + +### Default Action & Signal Handler + +Each signal type has a predefined default action: +* terminates +* stops until restarted by a SIGCONT signal +* ignores + +```c +handler_t * signal(int signum, handler_t *handler); +``` + +* `SIG_IGN` ignore +* `SIG_DFL` revert to default action +* handler function pointer + +### Blocking and Unblocking + +* Implicit blocking +Kernel blocks any pending signals of type currently being handled. +* Explicit blocking `sigprocmask()` + +Supporting functions: +* `sigemptyset()` create empty set +* `sigfillset()` add every signal number to set +* `sigaddset()` add signal number to set +* `sigdelset()` delete signal number from set + +```c +sigset_t mask, prev_mask; +sigemptyset(&mask); +sigaddset(&mask, SIGINT); + +sigprocmaksk(SIG_BLOCK, &mask, &prev_mask); +// critical section +sigprocmask(SIG_SETMASK, &prev_mask, NULL); +``` + +### Safe Signal Handling + +Handlers are concurrent with main program and share global data structures. This can lead to troubles. + +Some guidlines help to avoid troubles: + +0. Keep you handlers as simple as possible +1. Call only **async-signal-safe** functions in your handlers +2. Save and restore `errno` on entry and exit +3. Protect accesses to shared data structure by temporarily blocking all signals +4. Declare global variables as `volatile` +5. Declare global flags as `volatile sig_atomic_t` + +#### Async-Signal-Safety + +A function satisfying either two conditions(below) is **Async-Signal-Safety function**: +* **reentrant**: not modifying global data (or static data), only use local data(stack frame). +* **non-interruptible**: blocking singnals during modifying global data. + +POSIX guarantees that the 117 functions to be async-signal-safe, including: +* `_exit`, `write`, `wait`, `waitpid`, `sleep`, `kill` + +**IMPORTANT**: `printf`, `malloc`, `sprintf`, `eixt` are **NOT** async-signal-safe. + +### Correct Signal Handling Example for reaping multiple childs + +```c +void child_handler(int sig) { + int olderrno = errno; pid_t pid; + while ((pid = wait(NULL)) > 0) { + ccount --; + sio_puts("Handler reaped child "); + sio_putl((long) pid); + sio_puts(" \n"); + } + if (errno != ECHILD) { + sio_error("wait error"); + } + errno = olderrno; +} + +``` + +### Portable Signal Handling + +Different UNIX systems have different signal handling sematnics. + +So `sigaction` is introduced to provide a portable interface for signal handling. + +```c +handler_t * Signal(int signum, handler_t *handler) { + struct sigaction action, old_action; + action.sa_handler = handler; + sigemptyset(&action.sa_mask); // block sigs of type being handled + action.sa_flags = SA_RESTART; // restart syscalls if possible + if (sigaction(signum, &action, &old_action) < 0) + unix_error("Signal error"); + return (old_action.sa_handler); +} +``` + +### Concurrent Flows to lead Races + +```c +void handler(int sig) { + int olderrno = errno; + pid_t pid; + while ((pid = waitpid(-1, NULL, 0)) > 0) { + deletejob(pid); + } + if (errno != ECHILD) sio_error("waitpid error"); + errno = olderrno; +} + +int main() { + while (1) { + if ((pid = Fork()) == 0) { + execve("/bin/date", argv, NULL); + } + } + addjob(pid); +} +``` + +In above example, the ecf flow can be executed before the deletejob; thus it cannot be deleted. +Therefore, we need to synchronize the two concurrent flows. + +```c +void handler(int sig) { + int olderrno = errno; + + sigset_t mask_all, prev_all; + pid_t pid; + + sigfillset(&mask_all); + while ((pid = waitpid(-1, NULL, 0)) > 0) { + sigprocmask(SIG_BLOCK, &mask_all, &prev_all); + deletejob(pid); + sigprocmask(SIG_SETMASK, &prev_all, NULL); + } + if (errno != ECHILD) sio_error("waitpid error"); + errno = olderrno; +} + +int main() { + pid_t pid; + sigset_t mask_all, prev_one; + + sigfillset(&mask_all); + signal(SIGCHLD, handler); + + while (1) { + sigprocmask(SIG_BLOCK, &mask_all, &prev_one); // Block SIGCHLD + if ((pid = Fork()) == 0) { + sigprocmask(SIG_SETMASK, &prev_one, NULL); // Unblock SIGCHLD + execve("/bin/date", argv, NULL); + } + sigprocmask(SIG_BLOCK, &mask_one, NULL); + addjob(pid); + sigprocmask(SIG_SETMASK, &prev_one, NULL); + } +} +``` \ No newline at end of file diff --git a/notes/11.md b/notes/11.md new file mode 100644 index 0000000..7cdfab1 --- /dev/null +++ b/notes/11.md @@ -0,0 +1,223 @@ +# System-Level I/O + +IO is the process of coping data between the main memory and external devices. + +In a Linux, **file** is a sequence of $m$ bytes. + +All I/O devices are represented as files. Even the kernel is represented as a file. + +## Unix IO + +* `open` and `close` +* `read` and `write` +* `lseek` changing **current file position** + +### File Types + +* Regular files +* Directory +* Socket +* ... + +#### Regular Files + +A regular file contains arbitary data. + +For example **text file** is a sequence of text lines. EOL is different in different OS: (`\n` in Unix, `\r\n` in Windows & Internet). + +#### Directories + +Directory contains an array of links. Least two links are `.`(itself) and `..`(parent dir). + +* `ls` +* `mkdir` +* `rmdir` + +All files are orgnaized as a hierarchy anchored by root dir named `/`. + +Kernel maintains curr working dir (cwd) for each process that modified using the `cd` command. + +Path names +* Absolute `/home/yenru0/workspace` +* Relative `../workspace` + +### Open & Close & Read & Write + +```c +int fd; + +if ((fd = open("file.txt", O_RDONLY)) < 0) { + perror("open"); + exit(1); +} +``` + +* `open` returns a non-negative integer called **file descriptor** (fd). + * `fd == -1` indicates an error. + * `0`: stdin, `1`: stdout, `2`: stderr + +```c +int fd; int ret; +if ((ret = close(fd)) < 0) { + perror("close"); + exit(1); +} +``` + +Closing an already closed can lead to a disastrous situation in threaded programs. So always check the return code. + +```c +char buf[512]; + +nbytes = read(fd, buf, sizeof(buf)); + +``` + +```c +ssize_t read(int fd, void *usrbuf, size_t n); +``` + +read returns the number of bytes read from the `fd` into `buf`. +`ssize_t` is signed version of `size_t`. + +If `read` returns negative value, an error occurred. + +```c +ssize_t write(int fd, const void *usrbuf, size_t n); +``` + +If `write` returns negative value, an error occurred. + +### Short Counts + +It means that `read` or `write` transfers fewer bytes than requested. It can occur in these situations: + +* `EOF` on reads +* Reading text lines from an terminal +* Reading from a network socket + +Never occurs: +* Reading from disk files (except for `EOF`) +* Writing to disk files + +## RIO pakcage + +RIO is a set of wrappers efficient and robust I/O functions subject to **short couunts**. + +* unbuffered RIO functions `rio_readn`, `rio_writen` +* buffered RIO functions `rio_readnb`, `rio_readlineb` + * buffered RIO functions are thread-safe and can be interleaved arbitrarily on the same descriptor. + + +### Buffered RIO + +To read efficiently from a file, RIO uses partially cached in an interal memory buffer. (`rio_t` structure) + +For reading from file, Buffer has buffered portion of already read and unread data. It is refilled automatically by `rio_readnb` and `rio_readlineb` as needed. This is **partially cached**. + +```c +typedef struct { + int rio_fd; // Descriptor for this internal buf + int rio_cnt; // Unread bytes in internal buf + char *rio_bufptr; // Next unread byte in internal buf + char rio_buf[RIO_BUFSIZE]; // Internal buffer +} rio_t; +``` + +example: + +```c +int main(int argc, char **argv) { + int n; rio_t rio; char buf[MAXLINE]; + rio_readinitb(&rio, STDIN_FILENO); + while ((n = rio_readlineb(&rio, buf, MAXLINE)) != 0) { + rio_writen(STDOUT_FILENO, buf, n); + } + exit(0); +} + +``` + +## Metadata + +Metadata is data about data. (file access, file size, file type) + +* Per-process metadata + * when a process opens a file, the kernel creates an entry in a per-process table called the **file descriptor table** +* Per-file metadata + * can be accessed using `stat` system call + + +```c +struct stat { + dev_t st_dev; // ID of device containing file + ino_t st_ino; // inode number + mode_t st_mode; // protection + nlink_t st_nlink; // number of hard links + uid_t st_uid; // user ID of owner + gid_t st_gid; // group ID of owner + dev_t st_rdev; // device ID (if special file) + off_t st_size; // total size, in bytes + blksize_t st_blksize; // blocksize for filesystem I/O + blkcnt_t st_blocks; // number of 512B blocks allocated + time_t st_atime; // time of last access + time_t st_mtime; // time of last modification + time_t st_ctime; // time of last status change +}; +``` + +### How to Kernel represents Open Files + +* Descriptor table(per-process) +* Open file table(shared by all processes) +* v-node table(shared by all processes) + +When a process opens a file, the kernel creates an entry in the per-process file descriptor table. Each entry contains a pointer to an entry in the open file table. Each entry in the open file table contains a pointer to an entry in the v-node table. + +When a `fork` calls: the child process inherits copies of the parent's file descriptors. And the entry points to open file table's entry increasing `refcnt`. + +### IO redirection + +for example: `ls > foo.txt` + +Answer: `dup2(oldfd, newfd)` it means copies descriptor table entry `oldfd` to `newfd` +so `dup2(4, 1)` makes `stdout` point to the same open file as descriptor 4. + +## stdio + +The C standard library (`libc.so`) provides a collection of higher-level standard I/O functions. + +* `fopen`, `fclose`, `fread`, `fwrite`, `fgets`, `fputs`, `fscanf`, `fprintf` + +`stdio` models open files as **streams**, which are abstraction for a file descriptor and a buffer in memory. + +```c +extern FILE * stdin; +extern FILE * stdout; +extern FILE * stderr; +``` + +### Buffered I/O + +Application often read and write one char at a time. However, UNIX System calls `read` and `write` calls expensive. So we need buffered read & write; use unix `read` & `write` to **get a block of data into a buffer**. And then user application reads/writes **one char at a time from/to the buffer**; it is efficient because it is simple memory access. + +`stdio` uses buffer. `printf` is not write immediately to the `stdout` file; it is stored in a buffer. And then when `fflush(stdout)`, `exit`, or return from `main`, the buffer is flushed to the file using `write` syscall. + +## Remark + +* UNIX IO +* RIO package +* stdio + + + +When to use +* stdio: disk or terminal files +* unix io: signal handlers, or when you need absolute high performance +* RIO: networking + +### Binary + +DO NOT USE: +* text oriented I/O: `fgets`, `scanf`, `rio_readlineb` +* string functions: `strlen`, `strcpy`, `strcat`, `strcmp` \ No newline at end of file diff --git a/notes/9.md b/notes/9.md index d67d80a..841e1a3 100644 --- a/notes/9.md +++ b/notes/9.md @@ -84,7 +84,7 @@ Single processor executes multiple processes concurrently. process execution int Multicore processor share main memory each can execute a separate process. scheduling of processors onto cores done by kernel. -### Concurrent Processes +#### Concurrent Processes Concurrency is **not at the exact same time**. @@ -94,7 +94,7 @@ Control flows for concurrent processes are pysically disjoint in time. But user * Execution time of instruction may vary because of the Nondeterminism of the System: OS scheduling, Interrupts, Cache miss or Page fault, I/O device delays. -### Context Switching +#### Context Switching Prcess are managed by a shared chunk of memory-resident OS code called the **kernel**. @@ -180,7 +180,7 @@ int main() { } ``` -```sh {cmd} +```sh {cmd hide} while ! [ -r 9_1.out ]; do sleep .1; done; ./9_1.out ``` @@ -197,7 +197,58 @@ Any topological sort of the graph corresponds to a feasible total ordering. ### Reaping Child Processes -When a child process terminates, it becomes a **zombie** until its parent calls `wait` to read its exit status. +When process terminates, it still consumes system resources. It is called a "zombie". + +**"Reaping"** is performed by the parent by using `wait` or `waitpid`. And then parent is given exit status information. Finally kernel then deletes zombie child process. + +If any parent terminates without reaping a child, then the orphaned child will be reaped by the `init` process (pid 1). +However, long-running processes should reap their children to avoid accumulating zombies. + +```c {cmd=gcc, args=[-O2 -x c $input_file -o 9_2.out]} +#include +#include +#include + +int main() { + if(fork() == 0) { + printf("Terminating child process\n"); + exit(0); + } else { + printf("Running Parent\n"); + sleep(1); + printf("Parent exiting\n"); + } +} +``` + +```sh {cmd hide} +while ! [ -r 9_2.out ]; do sleep .1; done; ./9_2.out & ps -ef | grep "9_2.out" | grep -v grep; +``` + + +```c +pid_t wait(int * child_status); +``` + +`wait` suspends current process until **one of its children** terminates. + +* **return value**: pid of terminated child +* **child_status**: if not `NULL`, the integer points will be set to the termination status(reason and exit status) of the child. + * It can be checked by using macros defined in `wait.h` ## execve +```c +int execve(const char *filename, char *const argv[], char *envp[]); +``` + +Load and runs in the current process. + +* `filename`: path of the executable file +* `argv`: argument list +* `envp`: environment variables "name=value" + * `getenv`, `putenv`, `printenv` + +It does **overwrite code, data, stack**. retain only **pid, file descriptors, signal context**. + +Called **once and never returns on success**.