complement 9.md 10.md 11.md
This commit is contained in:
223
notes/11.md
Normal file
223
notes/11.md
Normal file
@@ -0,0 +1,223 @@
|
||||
# System-Level I/O
|
||||
|
||||
IO is the process of coping data between the main memory and external devices.
|
||||
|
||||
In a Linux, **file** is a sequence of $m$ bytes.
|
||||
|
||||
All I/O devices are represented as files. Even the kernel is represented as a file.
|
||||
|
||||
## Unix IO
|
||||
|
||||
* `open` and `close`
|
||||
* `read` and `write`
|
||||
* `lseek` changing **current file position**
|
||||
|
||||
### File Types
|
||||
|
||||
* Regular files
|
||||
* Directory
|
||||
* Socket
|
||||
* ...
|
||||
|
||||
#### Regular Files
|
||||
|
||||
A regular file contains arbitary data.
|
||||
|
||||
For example **text file** is a sequence of text lines. EOL is different in different OS: (`\n` in Unix, `\r\n` in Windows & Internet).
|
||||
|
||||
#### Directories
|
||||
|
||||
Directory contains an array of links. Least two links are `.`(itself) and `..`(parent dir).
|
||||
|
||||
* `ls`
|
||||
* `mkdir`
|
||||
* `rmdir`
|
||||
|
||||
All files are orgnaized as a hierarchy anchored by root dir named `/`.
|
||||
|
||||
Kernel maintains curr working dir (cwd) for each process that modified using the `cd` command.
|
||||
|
||||
Path names
|
||||
* Absolute `/home/yenru0/workspace`
|
||||
* Relative `../workspace`
|
||||
|
||||
### Open & Close & Read & Write
|
||||
|
||||
```c
|
||||
int fd;
|
||||
|
||||
if ((fd = open("file.txt", O_RDONLY)) < 0) {
|
||||
perror("open");
|
||||
exit(1);
|
||||
}
|
||||
```
|
||||
|
||||
* `open` returns a non-negative integer called **file descriptor** (fd).
|
||||
* `fd == -1` indicates an error.
|
||||
* `0`: stdin, `1`: stdout, `2`: stderr
|
||||
|
||||
```c
|
||||
int fd; int ret;
|
||||
if ((ret = close(fd)) < 0) {
|
||||
perror("close");
|
||||
exit(1);
|
||||
}
|
||||
```
|
||||
|
||||
Closing an already closed can lead to a disastrous situation in threaded programs. So always check the return code.
|
||||
|
||||
```c
|
||||
char buf[512];
|
||||
|
||||
nbytes = read(fd, buf, sizeof(buf));
|
||||
|
||||
```
|
||||
|
||||
```c
|
||||
ssize_t read(int fd, void *usrbuf, size_t n);
|
||||
```
|
||||
|
||||
read returns the number of bytes read from the `fd` into `buf`.
|
||||
`ssize_t` is signed version of `size_t`.
|
||||
|
||||
If `read` returns negative value, an error occurred.
|
||||
|
||||
```c
|
||||
ssize_t write(int fd, const void *usrbuf, size_t n);
|
||||
```
|
||||
|
||||
If `write` returns negative value, an error occurred.
|
||||
|
||||
### Short Counts
|
||||
|
||||
It means that `read` or `write` transfers fewer bytes than requested. It can occur in these situations:
|
||||
|
||||
* `EOF` on reads
|
||||
* Reading text lines from an terminal
|
||||
* Reading from a network socket
|
||||
|
||||
Never occurs:
|
||||
* Reading from disk files (except for `EOF`)
|
||||
* Writing to disk files
|
||||
|
||||
## RIO pakcage
|
||||
|
||||
RIO is a set of wrappers efficient and robust I/O functions subject to **short couunts**.
|
||||
|
||||
* unbuffered RIO functions `rio_readn`, `rio_writen`
|
||||
* buffered RIO functions `rio_readnb`, `rio_readlineb`
|
||||
* buffered RIO functions are thread-safe and can be interleaved arbitrarily on the same descriptor.
|
||||
|
||||
|
||||
### Buffered RIO
|
||||
|
||||
To read efficiently from a file, RIO uses partially cached in an interal memory buffer. (`rio_t` structure)
|
||||
|
||||
For reading from file, Buffer has buffered portion of already read and unread data. It is refilled automatically by `rio_readnb` and `rio_readlineb` as needed. This is **partially cached**.
|
||||
|
||||
```c
|
||||
typedef struct {
|
||||
int rio_fd; // Descriptor for this internal buf
|
||||
int rio_cnt; // Unread bytes in internal buf
|
||||
char *rio_bufptr; // Next unread byte in internal buf
|
||||
char rio_buf[RIO_BUFSIZE]; // Internal buffer
|
||||
} rio_t;
|
||||
```
|
||||
|
||||
example:
|
||||
|
||||
```c
|
||||
int main(int argc, char **argv) {
|
||||
int n; rio_t rio; char buf[MAXLINE];
|
||||
rio_readinitb(&rio, STDIN_FILENO);
|
||||
while ((n = rio_readlineb(&rio, buf, MAXLINE)) != 0) {
|
||||
rio_writen(STDOUT_FILENO, buf, n);
|
||||
}
|
||||
exit(0);
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
## Metadata
|
||||
|
||||
Metadata is data about data. (file access, file size, file type)
|
||||
|
||||
* Per-process metadata
|
||||
* when a process opens a file, the kernel creates an entry in a per-process table called the **file descriptor table**
|
||||
* Per-file metadata
|
||||
* can be accessed using `stat` system call
|
||||
|
||||
|
||||
```c
|
||||
struct stat {
|
||||
dev_t st_dev; // ID of device containing file
|
||||
ino_t st_ino; // inode number
|
||||
mode_t st_mode; // protection
|
||||
nlink_t st_nlink; // number of hard links
|
||||
uid_t st_uid; // user ID of owner
|
||||
gid_t st_gid; // group ID of owner
|
||||
dev_t st_rdev; // device ID (if special file)
|
||||
off_t st_size; // total size, in bytes
|
||||
blksize_t st_blksize; // blocksize for filesystem I/O
|
||||
blkcnt_t st_blocks; // number of 512B blocks allocated
|
||||
time_t st_atime; // time of last access
|
||||
time_t st_mtime; // time of last modification
|
||||
time_t st_ctime; // time of last status change
|
||||
};
|
||||
```
|
||||
|
||||
### How to Kernel represents Open Files
|
||||
|
||||
* Descriptor table(per-process)
|
||||
* Open file table(shared by all processes)
|
||||
* v-node table(shared by all processes)
|
||||
|
||||
When a process opens a file, the kernel creates an entry in the per-process file descriptor table. Each entry contains a pointer to an entry in the open file table. Each entry in the open file table contains a pointer to an entry in the v-node table.
|
||||
|
||||
When a `fork` calls: the child process inherits copies of the parent's file descriptors. And the entry points to open file table's entry increasing `refcnt`.
|
||||
|
||||
### IO redirection
|
||||
|
||||
for example: `ls > foo.txt`
|
||||
|
||||
Answer: `dup2(oldfd, newfd)` it means copies descriptor table entry `oldfd` to `newfd`
|
||||
so `dup2(4, 1)` makes `stdout` point to the same open file as descriptor 4.
|
||||
|
||||
## stdio
|
||||
|
||||
The C standard library (`libc.so`) provides a collection of higher-level standard I/O functions.
|
||||
|
||||
* `fopen`, `fclose`, `fread`, `fwrite`, `fgets`, `fputs`, `fscanf`, `fprintf`
|
||||
|
||||
`stdio` models open files as **streams**, which are abstraction for a file descriptor and a buffer in memory.
|
||||
|
||||
```c
|
||||
extern FILE * stdin;
|
||||
extern FILE * stdout;
|
||||
extern FILE * stderr;
|
||||
```
|
||||
|
||||
### Buffered I/O
|
||||
|
||||
Application often read and write one char at a time. However, UNIX System calls `read` and `write` calls expensive. So we need buffered read & write; use unix `read` & `write` to **get a block of data into a buffer**. And then user application reads/writes **one char at a time from/to the buffer**; it is efficient because it is simple memory access.
|
||||
|
||||
`stdio` uses buffer. `printf` is not write immediately to the `stdout` file; it is stored in a buffer. And then when `fflush(stdout)`, `exit`, or return from `main`, the buffer is flushed to the file using `write` syscall.
|
||||
|
||||
## Remark
|
||||
|
||||
* UNIX IO
|
||||
* RIO package
|
||||
* stdio
|
||||
|
||||
|
||||
|
||||
When to use
|
||||
* stdio: disk or terminal files
|
||||
* unix io: signal handlers, or when you need absolute high performance
|
||||
* RIO: networking
|
||||
|
||||
### Binary
|
||||
|
||||
DO NOT USE:
|
||||
* text oriented I/O: `fgets`, `scanf`, `rio_readlineb`
|
||||
* string functions: `strlen`, `strcpy`, `strcat`, `strcmp`
|
||||
Reference in New Issue
Block a user