add and fix some

notes/8.ipynb

@@ -0,0 +1,70 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "62ac3700",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import sys\n",
+    "sys.path.append(\"../modules\")\n",
+    "import numpy as np"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3d7375ae",
+   "metadata": {},
+   "source": [
+    "# Data Fitting\n",
+    "\n",
+    "It is approximate fit, which allows some errors depending on noise model.\n",
+    "\n",
+    "## Least-Square Data Fitting\n",
+    "\n",
+    "Given $N$ data points and a models has $M$ adjustable parameters.\n",
+    "\n",
+    "find $\\mathbf{a} = \\begin{bmatrix}a_1, a_2, \\cdots, a_M\\end{bmatrix}$ that minimizes $$S = \\sum_{i=1}^N{[y_i - y(x_i; \\mathbf{a})]^2}$$\n",
+    "\n",
+    "It is also equivalent to **maximum likelihood estimation** if $e_i$ is **independently distributed Gaussian**.\n",
+    "\n",
+    "\n",
+    "### Line Fitting\n",
+    "\n",
+    "$y = a + bx, \\quad e_i = y_i - (a + bx)$\n",
+    "\n",
+    "$S(a, b) = \\sum_{i = 1}^{N}e_i^2$\n",
+    "\n",
+    "$$\\begin{align*}\n",
+    "\\frac{\\partial S}{\\partial a} &= 2\\sum_{i = 1}^N [y_i-a-bx_i](-1) = 0\\\\\n",
+    "\\frac{\\partial S}{\\partial b} &= 2\\sum_{i = 1}^N [y_i =a - bx_i](-x_i) = 0\\\\\n",
+    "\\end{align*}$$\n",
+    "\n",
+    "$$\\begin{align*}\n",
+    "a\\sum 1 + b\\sum x_i &= \\sum y_i\\\\\n",
+    "a\\sum x_i + b \\sum x_i^2 &= \\sum x_iy_i\n",
+    "\\end{align*}$$\n",
+    "\n",
+    "$$\\therefore \\begin{bmatrix}\n",
+    "a \\\\ b\n",
+    "\\end{bmatrix} = \\begin{bmatrix}\n",
+    "\\sum 1 & \\sum x_i \\\\\n",
+    "\\sum x_i & \\sum x_i^2 \\\\\n",
+    "\\end{bmatrix}^{-1} \\begin{bmatrix}\n",
+    "    \\sum y_i \\\\ \\sum x_i y_i\n",
+    "\\end{bmatrix}$$\n",
+    "\n",
+    "\n",
+    "\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "name": "python"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}