{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "62ac3700",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append(\"../modules\")\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d7375ae",
   "metadata": {},
   "source": [
    "# Data Fitting\n",
    "\n",
    "It is approximate fit, which allows some errors depending on noise model.\n",
    "\n",
    "## Least-Square Data Fitting\n",
    "\n",
    "Given $N$ data points and a models has $M$ adjustable parameters.\n",
    "\n",
    "find $\\mathbf{a} = \\begin{bmatrix}a_1, a_2, \\cdots, a_M\\end{bmatrix}$ that minimizes $$S = \\sum_{i=1}^N{[y_i - y(x_i; \\mathbf{a})]^2}$$\n",
    "\n",
    "It is also equivalent to **maximum likelihood estimation** if $e_i$ is **independently distributed Gaussian**.\n",
    "\n",
    "\n",
    "### Line Fitting\n",
    "\n",
    "$y = a + bx, \\quad e_i = y_i - (a + bx)$\n",
    "\n",
    "$S(a, b) = \\sum_{i = 1}^{N}e_i^2$\n",
    "\n",
    "$$\\begin{align*}\n",
    "\\frac{\\partial S}{\\partial a} &= 2\\sum_{i = 1}^N [y_i-a-bx_i](-1) = 0\\\\\n",
    "\\frac{\\partial S}{\\partial b} &= 2\\sum_{i = 1}^N [y_i =a - bx_i](-x_i) = 0\\\\\n",
    "\\end{align*}$$\n",
    "\n",
    "$$\\begin{align*}\n",
    "a\\sum 1 + b\\sum x_i &= \\sum y_i\\\\\n",
    "a\\sum x_i + b \\sum x_i^2 &= \\sum x_iy_i\n",
    "\\end{align*}$$\n",
    "\n",
    "$$\\therefore \\begin{bmatrix}\n",
    "a \\\\ b\n",
    "\\end{bmatrix} = \\begin{bmatrix}\n",
    "\\sum 1 & \\sum x_i \\\\\n",
    "\\sum x_i & \\sum x_i^2 \\\\\n",
    "\\end{bmatrix}^{-1} \\begin{bmatrix}\n",
    "    \\sum y_i \\\\ \\sum x_i y_i\n",
    "\\end{bmatrix}$$\n",
    "\n",
    "\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}