complement 9.md 10.md 11.md

notes/10.md

@@ -0,0 +1,218 @@
+# Signals and Nonlocal Jumps
+
+## Shell
+
+Linux Process Hierachy: all is rooted from `init` or `systemd` (pid = 1).
+
+A shell is an application that runs programs on behalf of the user.
+
+* `sh` Original Unix shell
+* `csh` BSD Unix C Shell
+* `bash` GNU Bourne-Again Shell (default Linux shell)
+
+
+## Signals
+
+All signal is a small message that notifies a process that an event has occured in the system.
+
+Signal is sent by kernel.
+
+Signal is identified by small integer ID (1-30).
+
+Only information in a signal is **its ID** and **the fact that it arrived**.
+
+| ID  | name    | Default Action | Corresponding Event                 |
+| --- | ------- | -------------- | ----------------------------------- |
+| 2   | SIGINT  | terminate      | user types `Ctrl+C`                 |
+| 9   | SIGKILL | terminate      | kill program                        |
+| 11  | SIGSEGV | terminate      | segmentation violation              |
+| 14  | SIGALRM | terminate      | timer expired                       |
+| 17  | SIGCHLD | ignore         | child process stopped or terminated |
+
+Sending a signal is essentially **updating some state** in the context of the destination process.
+Kernel sends a signal for one of the following reasons:
+
+A destination process receives a signal when it is forced by the kernel and reacts some way to the signal.
+* Ignore
+* Terminate (with optional core-dump)
+* Catch
+  * executing a user-level handler function called **signal handler**.
+
+### Pending & Blocking
+
+A signal is **pending** if sent but not yet received. There can be at most one pending signal of any particular type because signals are **not queued**. Signal is managed by bitmask. So subsequent signals of same type is discarded.
+* sets `k` bit when delivered
+* clears `k` bit when received
+ 
+Blocking signals: a process can block the receipt of certain signals. It is not received until unblocked. It is also managed by bitmask.
+* can be set and cleared by using `sigprocmask`
+
+### Receiving
+
+Suppose kernel is returning from an exception handler and is ready to pass control to process `p`
+Kernel computes `pnb = pending(p) & ~blocked(p)`.
+* if `pnb == 0`, no unblocked pending signals for `p`, so just return to `p` normally.
+* otherwise, choose least nonzero bit `k` in `pnb` and force process `p` to receive signal `k`
+  * The receipt of the signal triggers action by `p`
+  * Repeat for all nonzero `k`
+
+### Default Action & Signal Handler
+
+Each signal type has a predefined default action:
+* terminates
+* stops until restarted by a SIGCONT signal
+* ignores
+
+```c
+handler_t * signal(int signum, handler_t *handler);
+```
+
+* `SIG_IGN` ignore
+* `SIG_DFL` revert to default action
+* handler function pointer
+
+### Blocking and Unblocking
+
+* Implicit blocking
+Kernel blocks any pending signals of type currently being handled.
+* Explicit blocking `sigprocmask()`
+
+Supporting functions:
+* `sigemptyset()` create empty set
+* `sigfillset()` add every signal number to set
+* `sigaddset()` add signal number to set
+* `sigdelset()` delete signal number from set
+
+```c
+sigset_t mask, prev_mask;
+sigemptyset(&mask);
+sigaddset(&mask, SIGINT);
+
+sigprocmaksk(SIG_BLOCK, &mask, &prev_mask);
+// critical section
+sigprocmask(SIG_SETMASK, &prev_mask, NULL);
+```
+
+### Safe Signal Handling
+
+Handlers are concurrent with main program and share global data structures. This can lead to troubles.
+
+Some guidlines help to avoid troubles:
+
+0. Keep you handlers as simple as possible
+1. Call only **async-signal-safe** functions in your handlers
+2. Save and restore `errno` on entry and exit
+3. Protect accesses to shared data structure by temporarily blocking all signals
+4. Declare global variables as `volatile`
+5. Declare global flags as `volatile sig_atomic_t`
+
+#### Async-Signal-Safety
+
+A function satisfying either two conditions(below) is **Async-Signal-Safety function**:
+* **reentrant**: not modifying global data (or static data), only use local data(stack frame).
+* **non-interruptible**: blocking singnals during modifying global data.
+
+POSIX guarantees that the 117 functions to be async-signal-safe, including:
+* `_exit`, `write`, `wait`, `waitpid`, `sleep`, `kill`
+
+**IMPORTANT**: `printf`, `malloc`, `sprintf`, `eixt` are **NOT** async-signal-safe.
+
+### Correct Signal Handling Example for reaping multiple childs
+
+```c
+void child_handler(int sig) {
+  int olderrno = errno; pid_t pid;
+  while ((pid = wait(NULL)) > 0) {
+    ccount --;
+    sio_puts("Handler reaped child ");
+    sio_putl((long) pid);
+    sio_puts(" \n");
+  }
+  if (errno != ECHILD) {
+    sio_error("wait error");
+  }
+  errno = olderrno;
+}
+
+```
+
+### Portable Signal Handling
+
+Different UNIX systems have different signal handling sematnics.
+
+So `sigaction` is introduced to provide a portable interface for signal handling.
+
+```c
+handler_t * Signal(int signum, handler_t *handler) {
+  struct sigaction action, old_action;
+  action.sa_handler = handler;
+  sigemptyset(&action.sa_mask); // block sigs of type being handled
+  action.sa_flags = SA_RESTART; // restart syscalls if possible
+  if (sigaction(signum, &action, &old_action) < 0)
+    unix_error("Signal error");
+  return (old_action.sa_handler);
+}
+```
+
+### Concurrent Flows to lead Races
+
+```c
+void handler(int sig) {
+  int olderrno = errno;
+  pid_t pid;
+  while ((pid = waitpid(-1, NULL, 0)) > 0) {
+    deletejob(pid);
+  }
+  if (errno != ECHILD) sio_error("waitpid error");
+  errno = olderrno;
+}
+
+int main() {
+  while (1) {
+    if ((pid = Fork()) == 0) {
+      execve("/bin/date", argv, NULL);
+    }
+  }
+  addjob(pid);
+}
+```
+
+In above example, the ecf flow can be executed before the deletejob; thus it cannot be deleted.
+Therefore, we need to synchronize the two concurrent flows.
+
+```c
+void handler(int sig) {
+  int olderrno = errno;
+
+  sigset_t mask_all, prev_all;
+  pid_t pid;
+
+  sigfillset(&mask_all);
+  while ((pid = waitpid(-1, NULL, 0)) > 0) {
+    sigprocmask(SIG_BLOCK, &mask_all, &prev_all);
+    deletejob(pid);
+    sigprocmask(SIG_SETMASK, &prev_all, NULL);
+  }
+  if (errno != ECHILD) sio_error("waitpid error");
+  errno = olderrno;
+}
+
+int main() {
+  pid_t pid;
+  sigset_t mask_all, prev_one;
+
+  sigfillset(&mask_all);
+  signal(SIGCHLD, handler);
+
+  while (1) {
+    sigprocmask(SIG_BLOCK, &mask_all, &prev_one); // Block SIGCHLD
+    if ((pid = Fork()) == 0) {
+      sigprocmask(SIG_SETMASK, &prev_one, NULL); // Unblock SIGCHLD
+      execve("/bin/date", argv, NULL);
+    }
+    sigprocmask(SIG_BLOCK, &mask_one, NULL);
+    addjob(pid);
+    sigprocmask(SIG_SETMASK, &prev_one, NULL);
+  }
+}
+```

notes/11.md

@@ -0,0 +1,223 @@
+# System-Level I/O
+
+IO is the process of coping data between the main memory and external devices.
+
+In a Linux, **file** is a sequence of $m$ bytes.
+
+All I/O devices are represented as files. Even the kernel is represented as a file.
+
+## Unix IO
+
+* `open` and `close`
+* `read` and `write`
+* `lseek` changing **current file position**
+
+### File Types
+
+* Regular files
+* Directory
+* Socket
+* ...
+
+#### Regular Files
+
+A regular file contains arbitary data.
+
+For example **text file** is a sequence of text lines. EOL is different in different OS: (`\n` in Unix, `\r\n` in Windows & Internet).
+
+#### Directories
+
+Directory contains an array of links. Least two links are `.`(itself) and `..`(parent dir).
+
+* `ls`
+* `mkdir`
+* `rmdir`
+
+All files are orgnaized as a hierarchy anchored by root dir named `/`.
+
+Kernel maintains curr working dir (cwd) for each process that modified using the `cd` command.
+
+Path names
+* Absolute `/home/yenru0/workspace`
+* Relative `../workspace`
+
+### Open & Close & Read & Write
+
+```c
+int fd;
+
+if ((fd = open("file.txt", O_RDONLY)) < 0) {
+    perror("open");
+    exit(1);
+}
+```
+
+* `open` returns a non-negative integer called **file descriptor** (fd).
+  * `fd == -1` indicates an error.
+  * `0`: stdin, `1`: stdout, `2`: stderr
+
+```c
+int fd; int ret;
+if ((ret = close(fd)) < 0) {
+    perror("close");
+    exit(1);
+}
+```
+
+Closing an already closed can lead to a disastrous situation in threaded programs. So always check the return code.
+
+```c
+char buf[512];
+
+nbytes = read(fd, buf, sizeof(buf));
+
+```
+
+```c
+ssize_t read(int fd, void *usrbuf, size_t n);
+```
+
+read returns the number of bytes read from the `fd` into `buf`.
+`ssize_t` is signed version of `size_t`.
+
+If `read` returns negative value, an error occurred.
+
+```c
+ssize_t write(int fd, const void *usrbuf, size_t n);
+```
+
+If `write` returns negative value, an error occurred.
+
+### Short Counts
+
+It means that `read` or `write` transfers fewer bytes than requested. It can occur in these situations:
+
+* `EOF` on reads
+* Reading text lines from an terminal
+* Reading from a network socket
+
+Never occurs:
+* Reading from disk files (except for `EOF`)
+* Writing to disk files
+
+## RIO pakcage
+
+RIO is a set of wrappers efficient and robust I/O functions subject to **short couunts**.
+
+* unbuffered RIO functions `rio_readn`, `rio_writen`
+* buffered RIO functions `rio_readnb`, `rio_readlineb`
+  * buffered RIO functions are thread-safe and can be interleaved arbitrarily on the same descriptor.
+
+
+### Buffered RIO
+
+To read efficiently from a file, RIO uses partially cached in an interal memory buffer. (`rio_t` structure)
+
+For reading from file, Buffer has buffered portion of already read and unread data. It is refilled automatically by `rio_readnb` and `rio_readlineb` as needed. This is **partially cached**.
+
+```c
+typedef struct {
+    int rio_fd;                // Descriptor for this internal buf
+    int rio_cnt;               // Unread bytes in internal buf
+    char *rio_bufptr;          // Next unread byte in internal buf
+    char rio_buf[RIO_BUFSIZE]; // Internal buffer
+} rio_t;
+```
+
+example:
+
+```c
+int main(int argc, char **argv) {
+    int n; rio_t rio; char buf[MAXLINE];
+    rio_readinitb(&rio, STDIN_FILENO);
+    while ((n = rio_readlineb(&rio, buf, MAXLINE)) != 0) {
+        rio_writen(STDOUT_FILENO, buf, n);
+    }
+    exit(0);
+}
+
+```
+
+## Metadata
+
+Metadata is data about data. (file access, file size, file type)
+
+* Per-process metadata
+  * when a process opens a file, the kernel creates an entry in a per-process table called the **file descriptor table**
+* Per-file metadata 
+  * can be accessed using `stat` system call
+
+
+```c
+struct stat {
+    dev_t     st_dev;     // ID of device containing file
+    ino_t     st_ino;     // inode number
+    mode_t    st_mode;    // protection
+    nlink_t   st_nlink;   // number of hard links
+    uid_t     st_uid;     // user ID of owner
+    gid_t     st_gid;     // group ID of owner
+    dev_t     st_rdev;    // device ID (if special file)
+    off_t     st_size;    // total size, in bytes
+    blksize_t st_blksize; // blocksize for filesystem I/O
+    blkcnt_t  st_blocks;  // number of 512B blocks allocated
+    time_t    st_atime;   // time of last access
+    time_t    st_mtime;   // time of last modification
+    time_t    st_ctime;   // time of last status change
+};
+```
+
+### How to Kernel represents Open Files
+
+* Descriptor table(per-process)
+* Open file table(shared by all processes)
+* v-node table(shared by all processes)
+
+When a process opens a file, the kernel creates an entry in the per-process file descriptor table. Each entry contains a pointer to an entry in the open file table. Each entry in the open file table contains a pointer to an entry in the v-node table.
+
+When a `fork` calls: the child process inherits copies of the parent's file descriptors. And the entry points to open file table's entry increasing `refcnt`.
+
+### IO redirection
+
+for example: `ls > foo.txt`
+
+Answer: `dup2(oldfd, newfd)` it means copies descriptor table entry `oldfd` to `newfd`
+so `dup2(4, 1)` makes `stdout` point to the same open file as descriptor 4.
+
+## stdio
+
+The C standard library (`libc.so`) provides a collection of higher-level standard I/O functions.
+
+* `fopen`, `fclose`, `fread`, `fwrite`, `fgets`, `fputs`, `fscanf`, `fprintf`
+
+`stdio` models open files as **streams**, which are abstraction for a file descriptor and a buffer in memory.
+
+```c
+extern FILE * stdin;
+extern FILE * stdout;
+extern FILE * stderr;
+```
+
+### Buffered I/O
+
+Application often read and write one char at a time. However, UNIX System calls `read` and `write` calls expensive. So we need buffered read & write; use unix `read` & `write` to **get a block of data into a buffer**. And then user application reads/writes **one char at a time from/to the buffer**; it is efficient because it is simple memory access.
+
+`stdio` uses buffer. `printf` is not write immediately to the `stdout` file; it is stored in a buffer. And then when `fflush(stdout)`, `exit`, or return from `main`, the buffer is flushed to the file using `write` syscall.
+
+## Remark
+
+* UNIX IO
+* RIO package
+* stdio
+
+
+
+When to use
+* stdio: disk or terminal files
+* unix io: signal handlers, or when you need absolute high performance
+* RIO: networking
+
+### Binary
+
+DO NOT USE:
+* text oriented I/O: `fgets`, `scanf`, `rio_readlineb`
+* string functions: `strlen`, `strcpy`, `strcat`, `strcmp`

notes/9.md

@@ -84,7 +84,7 @@ Single processor executes multiple processes concurrently. process execution int
 
 Multicore processor share main memory each can execute a separate process. scheduling of processors onto cores done by kernel.
 
-### Concurrent Processes
+#### Concurrent Processes
 
 Concurrency is **not at the exact same time**.
 
@@ -94,7 +94,7 @@ Control flows for concurrent processes are pysically disjoint in time. But user
 
 * Execution time of instruction may vary because of the Nondeterminism of the System: OS scheduling, Interrupts, Cache miss or Page fault, I/O device delays.
 
-### Context Switching
+#### Context Switching
 
 Prcess are managed by a shared chunk of memory-resident OS code called the **kernel**.
 
@@ -180,7 +180,7 @@ int main() {
 }
 ```
 
-```sh {cmd}
+```sh {cmd hide}
 while ! [ -r 9_1.out ]; do sleep .1; done; ./9_1.out
 ```
 
@@ -197,7 +197,58 @@ Any topological sort of the graph corresponds to a feasible total ordering.
 
 ### Reaping Child Processes
 
-When a child process terminates, it becomes a **zombie** until its parent calls `wait` to read its exit status.
+When process terminates, it still consumes system resources. It is called a "zombie".
+
+**"Reaping"** is performed by the parent by using `wait` or `waitpid`. And then parent is given exit status information. Finally kernel then deletes zombie child process.
+
+If any parent terminates without reaping a child, then the orphaned child will be reaped by the `init` process (pid 1).
+However, long-running processes should reap their children to avoid accumulating zombies.
+
+```c {cmd=gcc, args=[-O2 -x c $input_file -o 9_2.out]}
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+int main() {
+  if(fork() == 0) {
+    printf("Terminating child process\n");
+    exit(0);
+  } else {
+    printf("Running Parent\n");
+    sleep(1);
+    printf("Parent exiting\n");
+  }
+}
+```
+
+```sh {cmd hide}
+while ! [ -r 9_2.out ]; do sleep .1; done; ./9_2.out & ps -ef | grep "9_2.out" | grep -v grep;
+```
+
+
+```c
+pid_t wait(int * child_status);
+```
+
+`wait` suspends current process until **one of its children** terminates.
+
+* **return value**: pid of terminated child
+* **child_status**: if not `NULL`, the integer points will be set to the termination status(reason and exit status) of the child.
+  * It can be checked by using macros defined in `wait.h`
 
 ## execve
 
+```c
+int execve(const char *filename, char *const argv[], char *envp[]);
+```
+
+Load and runs in the current process.
+
+* `filename`: path of the executable file
+* `argv`: argument list
+* `envp`: environment variables "name=value"
+  * `getenv`, `putenv`, `printenv`
+
+It does **overwrite code, data, stack**. retain only **pid, file descriptors, signal context**.
+
+Called **once and never returns on success**.