{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "129d4b8c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d35579ac",
   "metadata": {},
   "source": [
    "# Study of algorithm\n",
    "\n",
    "* Improve Performance, $P$\n",
    "* At some Task, $T$\n",
    "* With Experience, $E$\n",
    "\n",
    "Learning: **Improving $P$ at $T$ with $E$**\n",
    "\n",
    "* $T$: Output\n",
    "* $E$: Training Data\n",
    "* $P$: Target (or Loss) function\n",
    "  * If you cannot measure it, you cannot improve it - Kelvin\n",
    "  * e.g.: Error rate, Euclidean distance, Logproba, Information theoretical measures(Mutual information, KL)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0d742636",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "255ea9b1",
   "metadata": {},
   "source": [
    "# Derivation w.r.t Matrix\n",
    "\n",
    "In the typical $f: \\R \\to \\R$\n",
    "$$f(x) = ax, \\quad df/dx = a$$\n",
    "and\n",
    "\n",
    "$$f(x) = x^2, \\quad df/dx = 2x$$\n",
    "\n",
    "\n",
    "By the way, $f = w^Tx,\\, (w \\in \\R^D, x\\in \\R^D)$\n",
    "\n",
    "$$\\frac{df}{dx} = \\frac{\\partial f}{\\partial x_1}$$\n",
    "\n",
    "\n",
    "In the case $f(x) = x^2 = x^Tx$\n",
    "\n",
    "$$\\frac{df}{dx} = 2x$$\n",
    "\n",
    "for example: $f(x) = w^Tx$\n",
    "\n",
    "$$f(x) = \\sum w_i x_i$$\n",
    "$$\\frac{\\partial f}{\\partial x_i} = w_i$$\n",
    "$$\\therefore \\frac{d f}{d x} = w$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84468a76",
   "metadata": {},
   "source": [
    "## trace \n",
    "$tr[M] = \\sum^{D}_{i=1}{M_{ii}}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8671c90",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "2025-02-AI (3.12.11)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}