{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Describe the SGD for minimizing the empiral risk and implement it"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a set of n observations ${z_i = (x_i, y_i)}_{i=1}^{K}$, the empiral risk is \n",
    "$R(w) = \\frac{1}{K} \\sum_{i=1}^K(y_i - w^tx_i)^2$, and its gradient is \n",
    "$\\nabla R(w) = \\frac{1}{K} \\sum_{i=1}^K -2x_i (y_i - w^tx_i)$. The SGD algorithm proceeds as follows:\n",
    "\n",
    "- Start from a value $w = w_0$\n",
    "- For $N$ iterations, compute the empirical risk and its gradient for the value $w_k$ on a batch of K observations. Then set $w_{k+1} = w_k - \\eta_k \\nabla R(w_k)$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# x, y list of n observations, x belonging to R^d and y to {-1;1}\n",
    "def SGD(x, y, lr = 0.1, N_iter = 500, K = 10):\n",
    "    (n,d) =  x.shape\n",
    "    w = np.random.rand(d)\n",
    "    Rs = []\n",
    "    ws = []\n",
    "    \n",
    "    indices = np.arange(n)\n",
    "    for iter in range(N_iter):\n",
    "        np.random.shuffle(indices)\n",
    "        indices_iter = indices[0:K]\n",
    "        R = 0\n",
    "        grad = np.zeros(d)\n",
    "        dist = y - np.dot(x, w)\n",
    "        for i in indices_iter:  \n",
    "            R += 1/n *dist[i]**2\n",
    "            grad += -2/n * x[i] * dist[i]\n",
    "        ws.append(w)\n",
    "        w = w - lr * grad\n",
    "        Rs.append(R)\n",
    "            \n",
    "    return w, ws, Rs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1.99999539 -0.99999715]\n"
     ]
    }
   ],
   "source": [
    "y = np.array([-1,1,-1])\n",
    "x = np.array([[0,0],[1,1],[0,1]])\n",
    "\n",
    "print(SGD(x, y)[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Sample a set of observations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x19da7bf0550>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEgCAYAAACwxdQWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3dfbRcdX3v8feH8BCDEEsAcRHToKIg8QnSikUpYkyRXo+i13vFVonYgu0BHy69WkltdbkoXeLjFShyRaytYu1VTATUgIKCKBoQJciDCQJGpYTYABIQEr73j98+ycxkzjkzZ2bPfvq81po1Z/bes+e798zZ3/172lsRgZmZ2YSdig7AzMzKxYnBzMzaODGYmVkbJwYzM2vjxGBmZm2cGMzMrI0Tg01K0lGSQtKyomMpC0l3Srqq6DiaRtJVku4sOo6mcGJoEEl7SnqvpBskPShps6SfSjpL0pOLjq8sJL1P0quLjmNQkp4m6XxJt2bf9X9l3/e/SHpp0fFZee1cdAA2GpKeCXwD+H3gy8AFwGPA4cDbgTdLemVEfK+4KEvjH4B/Ab7SZd6zgNKPCpW0GPg26Tv+LHAz8ATgmcArgQeBKwsL0ErNiaEBJM0BvgrsD7wyIi5tmX2+pHOBK4AVkp4TEf9ZRJy9kCRg94j4bRGfHxG/K+JzZ+AfgDnACyLixtYZkk4B9iskKqsEVyU1w1tIZ4of7UgKAETEauB0YB/gf3dbgaRTJd0u6ZHs+dQuyxwi6T8k/VLS7yTdI+lKSX/asdxukk6XdHO2vk2SvirpBR3LbWvjkDQu6afAI8DfSPp3SY9K2rtLHM/K3vexlml/LWlVFtujkn4t6d8kLWxZZqGkidLACdk6omXapG0Mkl4t6buSfps9vivpVV2WuzOrLz9I0qVZld79kv6fpP06lt1L0kclrcv200ZJ10vq+h11OBDY2JkUACLi8Yj4Vcdn/U9JKyXdnX1390n6iqTnTrENz5N0Rba990r6kKSdJc3O/v5lFvd3JB3csY5l2b5dklXd3ZV97k8kvb6H7ZtYz4GS/jX7Ph/NYjtL0u69rsN2JF8rqf4kfRs4EjgwItZOsswcYBPwy4g4IJt2FKm64QbSGeYnSVUQxwN/ALwvIt6fLTsP+Gm2uvOAu4C9gcXArRHx99lyuwCrgD8C/hW4HpgL/CXwFODILFG1fv6PgXnA/wXuAX4BCLgUODUizu7YljNIie7QiPhRNu0O4PvZun4DLAL+ArgfeE5EbMwOJsdlcV0NnD+xzoj4t2w9dwJ3RsRRLZ/318A5wK2kKqgAlgEHASdHxPkty95Jqt7ZA7g4i+d5wMnAFRGxtGXZb5K+t09my83J1rkgItqSbSdJlwLHAq+NiC9PtWy2/NXZfvkhaR8/HTgJ2JW0H3/WsQ1bgD2BfydVUy0l7bsPAoeQqq2+QvoN/A2wHjg4Ih7P1rEMuJD029od+Ey2395Mqq57c0R8puUzrwIWRsTClmmHAd8i/W4vBH5J2pd/Sfpd/XFEPDbdtlsXEeFHzR/ARuCBHpa7ifTP+cTs9VHZ6weB+S3L7Qr8gHSAm59NG8uW/R/TfMY7s+X+pGP6nsDdwFUt0yY+/zfAvh3LzwJ+DfygY7pISeknHdN37xLLy7L1v6tjegCfmST+Ozti/D3gt8BaYM+O7VmX7bsndbx/h/1ESiwBHJS9npu9PneG3/mLgEezddwOfBr4K9LBudvy3fbPwcDvOmNo2YbXdUy/HngcWEF20plNf1vnd05KnJF9V3Nbps/Npv0GeELL9KtICbn1835MSsZ7dEw/Llv3siL/76r8cFVSM+xJOjOezsQyczumfy4i1k+8iIhHgY+S2qhe2fHeV0jac4rP+HPSP/P1kvaeeJCSzeXAiyU9oeM9n42Ie1snRMRW4HPAH0g6qGXWUcAC0pl76/IPAUjaSdLc7DN/nMX9wininc7LSWe8/yciHmj5vAeATwBPBJZ0vOdXEfHFjmnfyp6fkT0/TDoov7C1uqtXkToRHEbaD3NJZ+LnAj+VdLWkp3UsP7F/pNR7bW9gA3Ab3ffPLyPiPzqmXUNKzJ+I7AiduTp7PrDLev45Irb9NrO/zyMl3KMm2z5JzwGeC3we2K3jt3QN8BCpFGMz4MTQDA+QksN0JpbpTCK3dFl2otroaQAR8W1S75dlwH1ZHfv7JT27430Hk6pDNnR5nEgqCXS2G9w+SbwTB/83tUx7EzCRNLaRdHRWHfEQqeph4jPnkg5CM3VA9nxzl3lrsuendUy/o8uyG7PnebAt+b6DVOX1c6X2mE9IelmvgUXETRGxLCKeDCwETiAdpF9M6miw68Sykl4g6RJSCed+tu+f59B9//y8y7T/mmTexPR5Xd4z7W9rEhNtFu9nx9/RvaRk7S7YM+ReSc2wBjhS0jNi6jaGZ5GK6509fro1RKlzQkScIOksUt32i4HTgOWS3hHb2wFEqrL6X1PEu6Hj9eZuC0XETZJuBP5c0nJSvfZrgVURcU/Ltv0BqV1jLfC3pAPXw9l2fYHBTpB22A892NrL+iLiPEkrgD8F/hj478Apkv49InpuoM3WdRfwWUkT7SdHAH8IXCNpAfAd0gnEB0ilhIdI++djpFJPP9sw2bxu+6qn39YUy3wY+Poky/zXJNNtGk4MzfBlUiPmX5AOjN28iVSd062hsvOsH7afsbWd/UbEGlIi+qCkJwHXAf8k6ZyseuFnpN5P34qsIXJA/0Kq1nopqfF6DzqqkYA3kEoir4iIbWezWWPzIKUFSO0IkBpcv9kxb2K/dSsh9CQifg18CviUpFmkhvHjJX04In44g/WFpOtIiWH/bPJxpIP/WES0jW3IOhXk2UX32cDKjmldf1sdJhrDt0bEFUOPquFcldQMnyKdLb9T0jGdMyUdCpxJOlM/q8v7/0zS/JbldyU1Im8FLsmm7SWp7fcUEZtIZ+dzgNnZ5M+Sejh1LTGo/xHYnyf1kHlT9rif1PjZauIMtvNM9HS6/w/8Ftirx8+/nHR2faqkPSYmZn+fmq3r8h7XtY2kOVkpbpusXeUn2csp45P0ckk7nPhl7TcTde8TVTZd94+kvyT/8Q5/JWlbm1b291tJ1X3fnuJ9PyKdgLy1s70kW8/Oknr9Dq2DSwwNEBEPSRojFbkvlfQlUi+PLaTqhDeSDmCvbq2CaXE7cJ2k80h10G8gdVf9QET8IlvmTaTEczEpCT1Gqv74E+CLEfFwttzHSQ22Z0k6mtTo+gCpwfhlpHEKPV+uISLulfQ1UjXLbOCCiHikY7GLSYnsMknnk3rrvJzUeHlfl9V+H1gi6d2knlIREV+Y5PM3SXoXqVfRdZI+k81aRmpIPrm1cbUPzwS+ne3PNaRqkYNJPYt+zvYG3cl8FJgnaSWp6m4z8FTSd/dMUoP+TdmyX8vm/6uks7PPOoJUJbiOfI8T95H226dJienNpN/CX0RE1ypE2FbyeSPp9/OT7P03k05CngG8BngPqRus9avoblF+jO5Bamj9e+BGUiJ4mNRD6EPAfl2WP4rtffLfRiq+/y57fnvHss8nVeGsJZ1BP0Dq9XMasFvHsjtn6/thtuxD2To/Byzt9vnTbNdrs+UCOGKSZV5N6k75EOlg9AXSAehOWrqfZsseSGqTeGBivS3zdlg+m34ccG3L9lxLSrSdy032/rZtJTXUfjT7rjZl39VaUp3/U3r4rpeSktWPs+3dQmrgvpLUyL9Tx/JHknrzPJh93qWkhu+r2LGb6GTb8L5sGxZ2TF+YTX9fy7Rl2bQlpAbku7Pf1hrgDV3WvUMc2fTfJ/ViupOU8Ddm3/OZwFOL/p+r6sMD3Mxs5FoGuL00Iq4qNhrr5DYGMzNr48RgZmZtnBjMzKyN2xjMzKyNSwxmZtamFuMY9t5771i4cGHRYZiZVcb1119/X0Ts021eLRLDwoULWb16ddFhmJlVhqS7JpvnqiQzM2vjxGBmZm2cGMzMrE0t2hjMrFkee+wx1q9fzyOPdF4v0TrNnj2b+fPns8suu/T8HicGM6uc9evXs8cee7Bw4UKkmdwrqRkigo0bN7J+/XoOOOCA6d+QcVWSmVXOI488wrx585wUpiGJefPm9V2ycmIwq5GVK+GUU9Jz3Tkp9GYm+8mJwawmVq6E44+Hc85Jz01IDpYPJwazmli1CjZn9zzbvDm9NpsJJwazmli6FOZkd4meMye9tmo58cQT2XfffVm0aFGhcTgxmNXE2BhcdBGMj6fnsbGiI7J+LVu2jK9//etFh+HEYFYnY2Nw9tlOCqNw0003ccQRR2x7fcMNN3D00UcPtM4jjzySvfbaa9DQBuZxDGbWCCtXpnaXpUuHkzgPOeQQ1q1bx9atW5k1axannXYaH/7wh7su+5KXvIQHH3xwh+kf+tCHWLJkyeDBDJkTg5nV3kSPrc2b4cILh1PVttNOO3HIIYdw880387Of/YwFCxZw6KGHAvDe976XD3zgA9uWvfrqqwf7sBFzYjCz2uvWY2sYpYbDDz+c7373u5x77rnb2gbuuecetmzZ0racSwxmZiWzdGkqKWzePNweW4cffjjLli1jfHyc/fffH4Af/ehHPP/5z29brmolBjc+m1nt5dVj66CDDmK33Xbj3e9+97ZpN9544w6JoVfHH388L3rRi7jtttuYP38+F1xwwXAC7ZNLDGbWCGNjw++t9fGPf5wzzzyT3Xfffdu0tWvXcuCBB85ofRdddNGwQhuISwxmZn1at24dBx10EA8//DAnnHBC27wLLriAnXaq9qHVJQYzsz49/elP59Zbby06jNyULq1JeqqkKyXdIulmSW8vOiYzsyYpY4lhC3BaRNwgaQ/gekmXR8RPiw7MzKwJSldiiIhfR8QN2d8PArcA+xcb1eg16br6ZlYupUsMrSQtBF4AXNdl3kmSVktavWHDhlGHlitfV9/MilTaxCDpicCXgHdExAOd8yPi/IhYHBGL99lnn9EHmCNfV9/MilTKxCBpF1JS+FxEfLnoeEbN19U3syKVrvFZ6QalFwC3RMRHio6nCBOjNId5JUgzs16VLjEARwBvBG6SdGM27fSIuKzAmEYuj1GawzbsyxibWTmULjFExDWAio7DppbHZYzNrD8nnngil1xyCfvuuy9r1qwZ2npL2cZg5ecG8hpoWp/oTZvg7rvTc03kdStQJwabETeQV1zT+kRv2gR33AH33pueh5Ac8ri1Z7/yuhVo6aqSrBrcQF5xed25pqweeACuvBKuuw5e+ELYe2940pMGWqVv7WnWRRUayG0Sed25pqyuuQb+7u/gkUfgq1+FpzwF3vCGgVaZ5609lyxZwj333LPD9DPOOINXvepVA8XdCycGsyZqWpHv2mtTUoD0fO21AycGyO/WnldcccXAsQ3CicGsqZpU5MuphORbe5qZVVVO9/Yc9q09+5XXrUBdYjCzZsihhDTsW3v2K69bgbrEYGbWJ9/a08zM2vjWnmYV0rTBvGZ5cGKw2mjaYF6zvDgxWG34+k1mw+HEYLUxyPWbXAVVPRFRdAiVMJP95MZnq4WJe0O84x1w//0wd+72EsN0PRR9CfHqmT17Nhs3bmTevHmke3uVzKZN6fpMe+458DWZBhERbNy4kdmzZ/f1PicGq7zWA/ucOSk5fOxjvR/om3Y9uTqYP38+69evZ8OGDUWHsqPNm+G++yACpHTBvomibAFmz57N/Pnz+3qPE4NVXueBfeXK/g70TbueXB3ssssuHHDAAUWH0d0pp6QeEBPGx+Hss4uLZwbcxtBQU9WpV62+fe5c2Dk7xZkzJyWBftoacrpagjVVHW5WEhGVfxx22GFhvVuxImLOnAhIzytW9DavjFrjnTUr4vTTt08fHy9//FZTFfgBAqtjkmOqq5IaaKo69arVt7fGu3VraniGZl041Eqo4j9AVyU10FQl3aqVgqsWr1kVKGrQF3jx4sWxevXqosOolInund3u0TLVvDKqWrxmZSDp+ohY3HWeE4OZWfNMlRhclWRmZm2cGKwRqtYFt9K8syvPicFqz1ddHaE8d7YTzsg4MVjt+aqrI5TXznZ2HyknBgPqfTLmLq0jlNfOdnYfKScGq/3JmC95MUJ57eyqZfeKn2m5u6rV4Zpf1gRVGbDSebnfkp6NVKq7qqRPS7pX0pqiY2mKqp2M9aMMJ25liGEHpQxqGmNj6YylhAfZNnWo9prsIkpFPYAjgUOBNb2+xxfRG1wFrvnVtzJcELAMMeyglEHVSEX2L1NcRK90JYaI+A7wm6LjaJqqnIz1owwnbmWIYQelDKpGatCoVbrEYDYsZagiK0MMOyhlUDVT8TOtUjY+S1oIXBIRi6ZY5iTgJIAFCxYcdtddd40mOKuUMrRXliGGHfQbVCk3wgZRuYvo9ZIYWrlXklmOKtLLxvpTqV5JZlYybpNonNIlBkkXAd8DniVpvaS3FB2TdVfFHo82A26TaJxSViX1y1VJo1eV2gVXjXeY6Q7xjqwdVyXZ0FWhdqH1Uh+veQ0sX150RAUb5NonFe9lY/1xYhiCJlapVKF2oTV5bd0KH/xgs76jHVQhm1spODEMqCoXoJtJ8prqPVUYw7N0Kcyatf31li0NPxZWIZtbOUw2JLpKjyIviTE+nka+TzzGxwsLZVIzGaFfkVH90zr99Iidd07bsdtuEcceW91tGYo6XvvEZoQqXRKjaqpwEjaTGoS61DqccQZ86Utw7LEpdV92WblLdrlzW4H1wIlhQL1WqRTZDjGT5FWFhNersTE44AB49NH0usqJzmwU3F11BMrQtXMmvQ3L3ENxJld0KPo7aAumrDt2KlWN27qaqrtq4e0Dw3iU/bLbVWiHyENe1dkzbf8oRfV6VRtvqhq3TQq3MRSrTtUyvcqzt9ZM2z9KUb1e1cabqsZtM+LEMAJV6No5bHkeRyqdaKsa/LDjbuLgnwpxG4PlIu86/UpXd1c1+GHFXaoGn+aq3GW3++XEMDr9HBuqevyznJ1ySqpjnDA+nur4bKSmSgw7jzoYq67WE70LL5z+RG9szAnBuli6NP2AJkoMValOaxC3MVjP3P5YYWWq029io1vFuMRgPfOJXkX1W9QbBRcnS80lButZnU70ynQCnTsX9axPTgwVUZYDWSnGAgyoKlfEHZqqdpG1wjgxVEDjDmQ5a9wJdJ2KejYSTgwV0LgDWc4aeQJdh6KejYwTQwU08kCWI59Am03NA9wqwoPF8uH9ak3lkc9mXfjKDNZkUyUGVyVZY9W+7aYsXdmscpwYrLFq3Xbjrmw2ACeGmqvbSeMwt6fWjdC1Lw5ZntzGUGN1q0Ov2/bkyjvLpuE2hoaq20lj5/YsX16fktDQ1aE4VLfiboU4MQxZmX7LdatDb90egDVrSlp9XpYfQZUHtbmNpFBODENUtt9yHU4aW01sz6JF26eVriRUth9BVdWtuFsxTgxDVMbfcpVPGiezYAHsumv6u3QloTL+CKqobsXdinFimKFutQX+Ledr4mT8sstAgmOPLWFJKI8fQVmqpkapbsXdqomI0j2AY4DbgLXA3063/GGHHRajtGJFxJw5EZCeV6xonzc+3j7NhmN8PO3zicf4eNERTWKYP4KpfmxmAwBWxyTH1NKVGCTNAs4BXgE8Gzhe0rOLjardVLUFday6KYvKlMiG+SNoQtVUE0tEJddXYpB0gKRvSrpD0kckzW6Z94MhxfSHwNqIuCMiHgW+ALxqSOseisocoGqmkbULdf+xubG+lPotMZwLfBl4HbAX8E1Je2TzdhlSTPsDv2h5vT6b1kbSSZJWS1q9YcOGIX10bxp5gCqJxpXIRvljK+LMPc8SkUsiMzdZHVO3B3BDx+v3AD8A5nbOm+mDlHQ+1fL6jcAnpnrPqNsYqqyubSB13a6RKaotI6/PddvMtBhiG8NuHUnlTOCLwDeBPbq+o3/rgae2vJ4P/GpI6260upba67pdI1VUW0ZeJaImtM3kqN/EcLukl7dOiIgPAZ8Hnj6kmH4IHJi1Z+wKvB7wv/oQ1PWSEj4GDEGRbRl51A/WvW0mZ9MmBkmHtrx8PfCdzmUi4iO0n+XPWERsAU4BvgHcAnwxIm4exrrrYJBq08pcUqJPtTsGFFE3XreGs7ptz6hNVscU2+v47wdeOt1yRT6a0sYwjGrTFSsiFi2qyHiAPtSmjcF14zYiDNjG8HngMkmv7Zwh6cWSrhluqrLJDKPKZGwMzjijfoNza9NbyfViVgLTJoaI+CvgTOALkt4KIOk5kr5Kqlb6vXxDtAnDqjIZdim77I2/RSetvtSuXswqabKiROcDeAvwKPBtYAvwc2AZsFOv68jr0ZSqpIhyVpmU+VIVp58eMWtWxWpmyvglW+0wRVXSzr0kD0l7Ac8EtgIvAa4FjorUUGwjNDZWvuqSpUvhwgu33yysLCe5K1fCBz8IW7em1xM1M2Xbfzso45dsjTJtYpD0D8A7s2U/TLqw3XnAR4C35RqdVcJE1dSqVSkplOWYtmoVbGk5dZk1qzxJy6zMeikxLAc+Bbw/Iv4TQNLdwMWSngz8eUQ8lmOMVgFlOMldubI9ObWWZHbeGd71ruJjNKuCXhLDwRGxrnVCRHxL0kuBy4CvAy/LIzizXk00gG/enJLBRRel6UcdlZ5PPtlJwaxX0yaGzqTQMv0GSS8mDUQzK1RnL89PfhKuump7u8fJJxcanlmlDHQ/hohYC/zRkGKplEp1gWyAzl6e4OEAZjPVU6+kqUy0O1RRZ510P+/rrLZwNUWxOhvAob3E4EZns94NnBiqapCDe7fBqU4MxetsAC+sp9RMzzjKrI7bZJMq3a09R2WQKw94cGo1FHKZjLIPA5+JOm6TTamxiWGQg7sv3GiTqtK1jnptKKvSNtlQNDYxDHpwr81F26w/0x1Mq1Kc7KcUUJVtsqFRumRGtS1evDhWr15ddBhWZsOoI29tmJozZ/IziirUx59ySkoKE8bH05nOZKqwTdYXSddHxOJu8xrb+GwNMqxuZL32OijDMPDp9HuBqypskw1NY6uSrEGGVUdepyqVKjSUebBQYVyVZPXXaxXQVO9vHSDhKpX8Dfqd2bSmqkpyicHqb5Cz485GWhhNr4NezpbrfEY9aCmvzvtmFCa7UUOVHk26UY+NWB53IZruRjy93Pe57veGHmT76r5vhoQB7/lsNmOVP3EbdrtCL91EpztbXrkSli+v99iCQUp5HncxMCcGy00tBswOu5G2l4PWVMloYqeuWbN9WtUbwicz08FCdeokUBAnBstNbU7cejlA9Vo06uWgNVUyat2pAIsWuWG2UxV6XJWceyVZbhrTsaTfDR1ksFhjdqrlzQPcrBBlvRf00PV7ud1BBos1ZqdakVxiMBuUz+KtgjyOwbYZRS+hyvdE6pfrtK1mXGJokFGc2Ob5Gb6Om/XFP5gpucRgwGh6CeX1GbXo+mqj4x/MQJwYGmQU3bvz+ozadH210fAPZiClSgySXifpZkmPS+paxLGZG0VVeF6fMWnCaVyDhvXEg9wGUqo2BkkHA48DnwT+JiJ6ajhwG0Mz7FBl7N5ANhW3MUypMuMYIuIWAElFh2IltEP3/37HD1iz+OZCM1aqqqR+SDpJ0mpJqzds2FB0OFYEVxeY5WLkJQZJVwD7dZm1PCJW9LqeiDgfOB9SVdKQwrOC9VX69yhgs1yMPDFExJJRf6ZVw4xuzezqArOhq2xVktWPexhOwj2vbMRKlRgkHSdpPfAi4FJJ3yg6JhudXJsMqnpw9UAtK0CpEkNEXBwR8yNit4h4ckT8SdExWbs8j6+5jbOo8sHVxSgrQKkSg5XbKI6vM71p15SqfHB1zysrgBOD9ayyx9cqH1x95VYrQKkGuFm5LV2aegtNDDSuzPG16t1a3fPKRqxUl8SYKV8SY3R8lQGzeqjMJTGs/HzyalZ/bmMwM7M2TgyWu6oOITArrZz/qZwYLFdVHkJgVkoj+KdyYrBcVbaLq1lZjeCfyonBclXlIQRmpTSCfyr3SrJcVX0IgVnpjOCfyuMYzMwaaKpxDK5KMjOzNk4MZmbWxonBzMzaODGYmVkbJwYz8/B0a+PEYNZ0Hp5uHZwYzJrOw9OtgxODWdN5eLp18Mhns6bz8HTr4MRgZr4Dk7VxVZKZmbVxYsA99czMWjU+MbinnplZu8YnBvfUMzNr1/jE4J56Zn1y3WvtNb5XknvqmfVhou5182a48ML0z+N/mtppfGIA99Qz61m3ulf/89RO46uSbIRcBVF9rntthFKVGCSdBbwSeBRYB7w5IjYVG5UNhasg6qHuda8rV9Z32/pQthLD5cCiiHgucDvwnoLjsWFx96/6GBuDs8+u34HTfde3KVViiIhVEbEle/l9YH6R8dgQuQrCys4nL9uUKjF0OBH4WtFB2JBMVEGMj7saycrJJy/bKCJG+4HSFcB+XWYtj4gV2TLLgcXAa2KSACWdBJwEsGDBgsPuuuuunCI2s8ZoUBuDpOsjYnHXeaNODNORdALwVuBlEbG5l/csXrw4Vq9enW9gZmY1MlViKFVVkqRjgHcDY70mBTOrAHdVrpRSJQbgbGAP4HJJN0o6r+iAzGxA7u1TOaUaxxARzyg6BjMbMo+WrpyylRjMrG7c26dySlViMLMaqvto6RpyYrDhaVBXP+uTr1RZKa5KsuFwA6NZbTgx2HD4cgJmteHEYMPhBkaz2nAbgw2HGxjNasOJwYbHDYy9cSO9lZyrksxGyY30VgFODGaj5EZ6qwAnBrNRciO9VYDbGMxGyY30VgFODGaj5kZ6KzlXJZmZWRsnBjMza+PEYGZmbZwYbEe+DaNZozkxWDsPwDJrPCcGa+cBWGaN58Rg7TwAy6zxPI7B2nkAllnjOTHYjjwAy6zRXJVkZmZtnBjMzKyNE4OZmbVxYjAzszZODGZm1saJwczM2jgxmJlZG0VE0TEMTNIG4K6i42ixN3Bf0UGMgLezPpqwjeDtbPX7EbFPtxm1SAxlI2l1RCwuOo68eTvrownbCN7OXrkqyczM2jgxmJlZGyeGfJxfdAAj4u2sjyZsI3g7e+I2BjMza+MSg5mZtXFiMDOzNk4MOZF0lqRbJf1E0sWSnlR0THmQ9DpJN0t6XFKtulB8DykAAAOhSURBVAFKOkbSbZLWSvrbouPJg6RPS7pX0pqiY8mTpKdKulLSLdnv9e1Fx5QHSbMl/UDSj7PtfP9M1uPEkJ/LgUUR8VzgduA9BceTlzXAa4DvFB3IMEmaBZwDvAJ4NnC8pGcXG1UuPgMcU3QQI7AFOC0iDgYOB8Zr+n3+Djg6Ip4HPB84RtLh/a7EiSEnEbEqIrZkL78PzC8ynrxExC0RcVvRceTgD4G1EXFHRDwKfAF4VcExDV1EfAf4TdFx5C0ifh0RN2R/PwjcAuxfbFTDF8lvs5e7ZI++exg5MYzGicDXig7C+rI/8IuW1+up4YGkiSQtBF4AXFdsJPmQNEvSjcC9wOUR0fd2+p7PA5B0BbBfl1nLI2JFtsxyUjH2c6OMbZh62c4aUpdp7ttdcZKeCHwJeEdEPFB0PHmIiK3A87N2zYslLYqIvtqQnBgGEBFLppov6QTgvwEviwoPGJluO2tqPfDUltfzgV8VFIsNgaRdSEnhcxHx5aLjyVtEbJJ0FakNqa/E4KqknEg6Bng3MBYRm4uOx/r2Q+BASQdI2hV4PbCy4JhshiQJuAC4JSI+UnQ8eZG0z0QPSElPAJYAt/a7HieG/JwN7AFcLulGSecVHVAeJB0naT3wIuBSSd8oOqZhyDoOnAJ8g9RQ+cWIuLnYqIZP0kXA94BnSVov6S1Fx5STI4A3Akdn/483Sjq26KBy8BTgSkk/IZ3cXB4Rl/S7El8Sw8zM2rjEYGZmbZwYzMysjRODmZm1cWIwM7M2TgxmZtbGicHMzNo4MZgNSNIzJD3WeYljSf8s6cG6XY7c6s+JwWxAEbEW+BTwTkl7A0j6e9LFE4+LiNVFxmfWLw9wMxsCSfsB64BzSZcgOB84PiK+WGhgZjPgEoPZEETEPcDHgFOBTwJva00Kkk7P7gb3uKRXFxWnWS+cGMyG52fAbsD3IuKcjnnfBI6lZne6s3pyYjAbAklHk0oK3wOOkPS81vkRcV1ErCskOLM+OTGYDUjSocBXSA3QRwF3A/9YZExmg3BiMBuApGeQbtu6Cjg1uz/0+4FjJR1ZaHBmM+TEYDZDWU+kVaT7NfxZRDyezfosqWfSPxUVm9kgfGtPsxnKeiI9rcv0rcDBo4/IbDg8jsFsBCT9HfBWYB/gQeARYHGWXMxKxYnBzMzauI3BzMzaODGYmVkbJwYzM2vjxGBmZm2cGMzMrI0Tg5mZtXFiMDOzNk4MZmbWxonBzMza/H/gxbR2eAKRaQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "N_pts = 100\n",
    "x = np.random.normal(0,1,size=(N_pts,2))\n",
    "w0 = np.array([-2.0,1])\n",
    "w0 = w0 / np.linalg.norm(w0)\n",
    "y = np.dot(x,w0) > 0\n",
    "y = 2*y -1\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "plt.scatter(*x[y == 1].T, color='b', s=10, label=r'$y_i=1$')\n",
    "plt.scatter(*x[y == -1].T, color='r', s=10, label=r'$y_i=-1$')\n",
    "plt.xlabel(r\"$x_1$\", fontsize=16)\n",
    "plt.ylabel(r\"$x_2$\", fontsize=16)\n",
    "plt.title(\"Observations Sample\", fontsize=18)\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Test the algorithm you wrote at the first question over these observations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w estimated :  [-0.88882056  0.4582554 ]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEICAYAAABWJCMKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO2dd5wdVfn/P8+tu9lsskl20wObkCCEUEJCQHo3QZpIRwS/KKLyld8XFWMBAcGKgCIiIIIgGCkiQQKhhCIJhPTeNn3Te7L9luf3x8yZe2bm3LKb3Wx253m/Xvvae6eemTtzPucp5xxiZgiCIAjBI9TeBRAEQRDaBxEAQRCEgCICIAiCEFBEAARBEAKKCIAgCEJAEQEQBEEIKCIAguCBiGqIaEiO9X8mojv38xxnElF1a5arpccVgkukvQsgCIVARGsA9AGQ0hY/w8y3tva5mLlrnvW3tPY5CyFfuQShuYgACB2Ji5j53fYsABGFmTmVf8tWPWeEmZMH8pxCMBAXkNDhIaIbiWgqET1ERLuJaBURnWwvX09EW4noBm37Z2w3zjtEtI+IPiSiQ7X1TERDtW0fI6JJRFQL4Cx72X3a9pcQ0Vwi2ktEK4lorL38a0S0xD7HKiL6ZjOuiYnoO0S0AsAKQ7kuIKLF9rE3ENH3sxznu/Z2A5t3V4UgIAIgdBZOBDAfQC8ALwCYAOAEAEMBfAXAH4lId6FcB+DnAMoBzAXwfI5jXwvgfgClAD7WVxDRGADPAvgBgDIApwNYY6/eCuBCAN0AfA3AQ0R0fDOu6VL7uoYb1j0F4JvMXApgBIAp3g3sOMWNAM5gZokLCD5EAISOxL/tFr76+4a2bjUzP227Z/4JYBCAe5m5kZnfBtAESwwUbzDzR8zcCOAnAD5PRIOynPc1Zp7KzGlmbvCsuwnAX5n5HXv9BmZeCgDM/AYzr2SLDwG8DeC0ZlzvL5l5JzPXG9YlAAwnom7MvIuZZ2vriIgeBPAFAGcx87ZmnFMIECIAQkfiUmYu0/6e1NZt0T7XAwAze5fpFsB69YGZawDsBNA/y3nXZ1kOWEKz0rSCiMYR0adEtJOIdgO4AJbFUSi5zvtl+3hrbRfW57V1ZQBuhiUge5pxPiFgiAAIQcVp7duuoZ4ANmbZNteQuesBHOZdSERxAK8AeABAH2YuAzAJADWjjFnPy8wzmPkSAL0B/BvAi9rqXbBcT08T0SnNOJ8QMEQAhKByARGdSkQxWLGA6cycq8WdjacAfI2IziGiEBENIKIjAMQAxAFsA5AkonEAzm+NghNRjIiuI6LuzJwAsBfu9Fgw8wew4hyvEtGJrXFeofMhAiB0JF63O0Opv1f341gvAPgZLNfPKFiVZbNh5s9gB3gB7AHwIYBDmXkfgO/CapnvghVInrgf5fVyPYA1RLQXwC2wAt3esr1jl20iEY1qxXMLnQSSCWGEoEFEzwCoZuaftndZBKE9EQtAEAQhoIgACIIgBBRxAQmCIAQUsQAEQRACSocaDK68vJwrKyvbuxiCIAgdilmzZm1n5grv8g4lAJWVlZg5c2Z7F0MQBKFDQURrTcvFBSQIghBQRAAEQRACigiAIAhCQBEBEARBCCgiAIIgCAFFBEAQBCGgiAAIgiAElEAJwLSV27FqW017F0MQBOGgoEN1BGspDYkUkmnGtU9OBwCs+dUX27lEgiAI7U8gBOCWv8/Crtqm9i6GIAjCQUUgXECRECGZllFPBUEQdAIiACEkUyIAgiAIOsEQgDAhkU63dzEEQRAOKgIhANFwCClxAQmCILgIhACEQyQuIEEQBA+BEIBomJBIiQtIEARBJxACEAmFJAtIEATBQyAEwHIBiQUgCIKgEwgBiIalH4AgCIKXQAhAJCz9AARBELwEQgCiIekHIAiC4CUQAhAOhcBiAAiCILgoSACIaCwRLSOiKiIab1h/OhHNJqIkEV2uLT+LiOZqfw1EdKm97hkiWq2tO671LstNJExtdWhBEIQOS97RQIkoDOBRAOcBqAYwg4gmMvNibbN1AG4E8H19X2Z+H8Bx9nF6AqgC8La2yQ+Y+eX9uYBCiIoACIIg+ChkOOgxAKqYeRUAENEEAJcAcASAmdfY63I52i8H8CYz17W4tC0kEgqEp0sQBKFZFFIzDgCwXvtebS9rLlcD+Idn2f1ENJ+IHiKiuGknIrqZiGYS0cxt27a14LTiAhIEQTBRiACYas9mhVSJqB+AowFM1hb/CMARAE4A0BPAD037MvMTzDyamUdXVFQ057QOYgEIgiD4KaRmrAYwSPs+EMDGZp7nSgCvMnNCLWDmTWzRCOBpWK6mNkEsAEEQBD+FCMAMAMOIaDARxWC5ciY28zzXwOP+sa0CEBEBuBTAwmYes2AiIREAQRAEL3kFgJmTAG6F5b5ZAuBFZl5ERPcS0cUAQEQnEFE1gCsAPE5Ei9T+RFQJy4L40HPo54loAYAFAMoB3Lf/l2MmEhYXkCAIgpeCJoVn5kkAJnmW3aV9ngHLNWTadw0MQWNmPrs5Bd0fomIBCIIg+AhE01gsAEEQBD+BqBklBiAIguAnGAIgWUCCIAg+giEA0g9AEATBRyBqRhkLSBAEwU8gBCAsMQBBEAQfgRCAqGQBCYIg+AhEzShBYEEQBD/BEABxAQmCIPgIiAAE4jIFQRCaRSBqRnEBCYIg+AmEAEgQWBAEwU8gakZvGigzI5HKNXulIAhC5ycQAhD1xAB+/94KDPvJm6hpTLZTiQRBENqfQAiANwbw8LsrAAA7a5raoziCIAgHBYEUAEWKmzW1sSAIQqciGAKQJQ00lRYBEAQhuARCAMIhAhmMABYLQBCEAFOQABDRWCJaRkRVRDTesP50IppNREkiutyzLkVEc+2/idrywUQ0nYhWENE/7Qnn2wxvIBgAkmIBCIIQYPIKABGFATwKYByA4QCuIaLhns3WAbgRwAuGQ9Qz83H238Xa8l8DeIiZhwHYBeCmFpS/YEwjgiZTIgCCIASXQiyAMQCqmHkVMzcBmADgEn0DZl7DzPMBFJRcT0QE4GwAL9uL/gbg0oJL3QJMgeAm6QsgCEKAKUQABgBYr32vtpcVShERzSSiT4lIVfK9AOxmZpWI39xjNhtTb2DpDCYIQpCJFLCNKYeyOb6TQ5h5IxENATCFiBYA2FvoMYnoZgA3A8AhhxzSjNO6MY0IKgIgCEKQKcQCqAYwSPs+EMDGQk/AzBvt/6sAfABgJIDtAMqISAlQ1mMy8xPMPJqZR1dUVBR6Wh8iAIIgCG4KEYAZAIbZWTsxAFcDmJhnHwAAEfUgorj9uRzAKQAWs5V/+T4AlTF0A4DXmlv45hAxuICakhIEFgQhuOQVANtPfyuAyQCWAHiRmRcR0b1EdDEAENEJRFQN4AoAjxPRInv3IwHMJKJ5sCr8XzHzYnvdDwHcTkRVsGICT7XmhXkxBYHFAhAEIcgUEgMAM08CMMmz7C7t8wxYbhzvftMAHJ3lmKtgZRgdEMQFJAiC4CYQPYEB83AQIgCCIASZwAhA1NgPQGIAgiAEl8AIgCkInEiKBSAIQnAJjACYhoIQF5AgCEEmMAJgcgGJAAiCEGQCIwCmILDEAARBCDKBEQCTBfCH91Zg1tpd7VAaQRCE9icwAmCKAQDAmws2HeCSCIIgHBwERgBMWUC5lguCIHR2AlP7RbNYAFkWC4IgdHoCIwDhLBPD76lPHOCSCIIgHBwERgBMQWAA2FXXdIBLIgiCcHAQGAEwjQYKADtqRAAEQQgmwREAgwvoyH7dxAIQBCGwBEgA3BZA79I4jhnQXWIAgiAEluAIgCHdMxYJISG9gQVBCCiBEQBTEDgSJhkPSBCEwBIYASgt8k9+Fg2HRAAEQQgsgRGAa088FGd9rsK1LBomJMUFJAhCQClIAIhoLBEtI6IqIhpvWH86Ec0moiQRXa4tP46IPiGiRUQ0n4iu0tY9Q0SriWiu/Xdc61ySma7xCP5wzUgcVlHiLIuEQkimGS/OWN+WpxYEQTgoySsARBQG8CiAcQCGA7iGiIZ7NlsH4EYAL3iW1wH4KjMfBWAsgIeJqExb/wNmPs7+m9vCayiY0qIofnnZMc73WMS6/DtemY8Nu+vb+vSCIAgHFX7HuJ8xAKqYeRUAENEEAJcAWKw2YOY19jqXQ52Zl2ufNxLRVgAVAHbvd8lbiD4qqJ4aWtuYbI/iCIIgtBuFuIAGANB9JNX2smZBRGMAxACs1Bbfb7uGHiKieJb9biaimUQ0c9u2bc09rQ9dAKJaamhdU2q/jy0IgtCRKEQATGMoNCtySkT9ADwH4GvMrKyEHwE4AsAJAHoC+KFpX2Z+gplHM/PoiooK0ybNIky6AGQ+14sACIIQMAoRgGoAg7TvAwFsLPQERNQNwBsAfsrMn6rlzLyJLRoBPA3L1dTmKAuA4bYA6hPiAhIEIVgUIgAzAAwjosFEFANwNYCJhRzc3v5VAM8y80uedf3s/wTgUgALm1PwluKKAYgLSBCEAJNXAJg5CeBWAJMBLAHwIjMvIqJ7iehiACCiE4ioGsAVAB4nokX27lcCOB3AjYZ0z+eJaAGABQDKAdzXqleWBX1ECHEBCYIQZArJAgIzTwIwybPsLu3zDFiuIe9+fwfw9yzHPLtZJW0lQnYMgOB1AYkACIIQLALTE1ihhoX2xgDEBSQIQtAInADo0wJExAUkCEKACZwA6EHgmLiABEEIMIETgBCZewLXNUkaqCAIwSJwAqD3aotGMpe/fmc95q5vtxEqBEEQDjiBEwCdqBYQ+HD5Nlz66FSZH0AQhMAQWAFgBqIR/ygXVVtr2qE0giAIB57ACUBFaRw3nlyJ524a46SE6izcsKcdSiUIgnDgKagjWGeCiHD3xUcBANbuqPWtFwtAEISgEDgLQEfvCKZoTEoMQBCEYBBoAdA7gik27q7Hlr0N7VAaQRCEA0ugBSBmsADeXrwFJ/7ivXYojSAIwoEl0AIQMQiAIAhCUAh0DRg1uIAEQRCCQrAFwJAGKgiCEBQCXQOGQoTfXH4Mvn7q4PYuiiAIwgEn0AIAAFeOHoTD+5a2dzEEQRAOOIEXAEBiAYIgBBMRAAAEEQBBEIJHQQJARGOJaBkRVRHReMP604loNhEliehyz7obiGiF/XeDtnwUES2wj/kHImq3Wrj9ziwIgtB+5BUAIgoDeBTAOADDAVxDRMM9m60DcCOAFzz79gTwMwAnAhgD4GdE1MNe/RiAmwEMs//GtvgqBEEQhGZTiAUwBkAVM69i5iYAEwBcom/AzGuYeT4A70A6XwDwDjPvZOZdAN4BMJaI+gHoxsyfMDMDeBbApft7MS1leL9u7XVqQRCEdqMQARgAYL32vdpeVgjZ9h1gf857TCK6mYhmEtHMbdu2FXja5jGsTykW3fMFnHtkH9+6aSu3o6ZRposUBKHzUYgAmDzkXODxs+1b8DGZ+QlmHs3MoysqKgo8bfMpiUd82UBb9zbg2ien4//+ObfNzisIgtBeFCIA1QAGad8HAthY4PGz7Vttf27JMduMcMgtALVNKQDA8i372qM4giAIbUohAjADwDAiGkxEMQBXA5hY4PEnAzifiHrYwd/zAUxm5k0A9hHRSXb2z1cBvNaC8rcqoSzpQJIkJAhCZySvADBzEsCtsCrzJQBeZOZFRHQvEV0MAER0AhFVA7gCwONEtMjedyeAn8MSkRkA7rWXAcC3APwFQBWAlQDebNUrawWs+LQ1i5ggCEJno6ApIZl5EoBJnmV3aZ9nwO3S0bf7K4C/GpbPBDCiOYVta7xBiLS9QKp/QRA6I9ITWEO1+BXJtJ3VKgogCEInRAQgC8yMZMp2AbVzWQRBENoCEYAspBlIpCwLQGIAgiB0RkQANHQHUCrNSIgFIAhCJ0YEIAtpZiQdC6CdCyMIgtAGiADoaCZAKs1IpJUFIAogCELnQwQgCymxAARB6OSIAGiwZgKkUpkYgCAIQmdEBCAL33lhtpMFlG2ICEEQhI6MCEAWpq3cgT31CQDiAhIEoXMiAqDh6QiMzXsa2qcggiAIBwARAA2vAGyyBUAsAEEQOiMiADnYtKcegMQABEHonIgAaIwZ3NP1XbmApPoXBKEzIgKg8bVTKnHH2M8535ULSHxAgiB0RkQANIgIh/Yscb7XJ6wpIaX6FwShMyIC4CEsd0QQhIAg1Z2HcMh/S+au343vvTivHUojCILQdogAeMhmAbwyu/rAFkQQBKGNKUgAiGgsES0joioiGm9YHyeif9rrpxNRpb38OiKaq/2lieg4e90H9jHVut6teWEtRVI+BUEICnkFgIjCAB4FMA7AcADXENFwz2Y3AdjFzEMBPATg1wDAzM8z83HMfByA6wGsYea52n7XqfXMvLUVrme/CYdEAARBCAaFWABjAFQx8ypmbgIwAcAlnm0uAfA3+/PLAM4h/zyK1wD4x/4U9kAQzmEBfLpqB8Y+/BEa7OwgQRCEjkwhAjAAwHrte7W9zLgNMycB7AHQy7PNVfALwNO2++dOg2AAAIjoZiKaSUQzt23bVkBx949QDgtgfvVuLN28Dztrm9q8HIIgCG1NIQJgqhG9A+Xn3IaITgRQx8wLtfXXMfPRAE6z/643nZyZn2Dm0cw8uqKiooDi7h8RWwCiYf8l7a6zRgcVC0AQhM5AIQJQDWCQ9n0ggI3ZtiGiCIDuAHZq66+Gp/XPzBvs//sAvADL1dTuKAugT7ci3zrVM7gxmT6gZRIEQWgLChGAGQCGEdFgIorBqswneraZCOAG+/PlAKYwW2NrElEIwBWwYgewl0WIqNz+HAVwIYCFOAhQMYBIiNCrJOZat2GXNTicWACCIHQG8gqA7dO/FcBkAEsAvMjMi4joXiK62N7sKQC9iKgKwO0A9FTR0wFUM/MqbVkcwGQimg9gLoANAJ7c76tpBVQWUDhE+PiHZ+OIvqXOug27LQFoTKZxz+uL8PGK7e1SRkEQhNYgUshGzDwJwCTPsru0zw2wWvmmfT8AcJJnWS2AUc0s6wFB9QMIhwjFsTBikYxGbt5ruYAaEik8PXUNnp66Bmt+9cV2KacgCML+Ij2BPaiJ4ZUQNGn+/lTaWlfbKC4gQRA6PiIAHtJ2fa9cQWpieB01V7AgCEJHRgTAQ9JWgIgjAN6MVxEAQRA6ByIAHtL2xMBiAQiC0NkRAfDQr3sxAGDciH4AgO7FUd82extEAARB6PgUlAUUJPqXFWP+3eejNG7dmqduPAHvLdmCu15b5GwjFoAgCJ0BsQAMdCuKQg1NNKCsGF/9fKUTEwCAvZoASKcwQRA6KiIABVIcDTufdQEQd1B2Eimrw9z2mkYAwLLN+5A0xFQEQWgfRAAKpCiWEQDdBfT01DWoa0q2R5EOet5bsgVPT12De15fjDXba/GFhz/Cb99e1t7FEgTBRgSgQIqimVulC8BjH6zEA5OXt0eRDnpUYz+VTmObbQXMXrurHUt04EinGfZwWB2Orfsa8Lu3lyGd7pjlbwnvLdmCzfZgj0FCBKBAXC6gBneLv6ZR3ED5UHUhGUcO73wM+fEk/ODl+e1djBbxg5fm45EpVZgZELFmZtz0t5m4/M/T2rsoBxwRgAIpsgWge3HUGRJCEQ4R7vz3QgkI54AzChAYXp5V3d5FaBHqOfY+552VpH2d1fZov0FCBKBAlABUlMZ96/7x2Xo89+lavDF/E5qSabwwfZ0EOzWYM7MDBaj+L5hEKo1ZB2Frm33zPnVOTJ09g4IIQIEUR8MgAnprAjCwR7FrGwbw0LvL8eNXF2Dyoi2+YwTNQtAn+eyg7vAW0Vzf/wNvL8OXH5uGxRv3tlGJmofzuwXkN0skA3KhBkQACqQoGkLXeASlRZm+c+/efgaGlJc430MEfLjMmre4JB527b9+Zx2OuPMtvDRzvWt5UzKNU341Be8s9gtGZ0K5E8wzP+dnzfZa7LADyQc7yWa6TlTFv6P24Lg+FacJSrXYmApWw0xHBKBAusajKOsSRUk8IwCxcMglCACweJP1Mntfnq37rAyD5z5d61q+raYRG3bX42evHRQTorUZTfZL1tIg8JkPfIBTfj2lNYvUZnR037kS6aBYbWrAx1AA/ZMiAAXy3XOG4uGrRqKrLQCxcAihEKGrJgB1TZmWRNIzimg0bN1qb6BJPXSp/XzbXpu7AZXj33A6XR1sNCX3zwIAgIZEx/DVNtcCENqXhD3nR0h7OP81u9rXWOuMiAAUyKG9SjDq0B6OBRC3ZworjWcGi9td1+R89gaBVaBpZ22Ta7kSiv2tM574yJpxc9Pugy+XmRlosq9/fwSgo9BZEgCCFgTWn83bX5yHO//dua1yoEABIKKxRLSMiKqIaLxhfZyI/mmvn05ElfbySiKqJ6K59t+ftX1GEdECe58/EHWMqkFZAHG7Y5huAazdUed8TqQZqTQ7nWmasgSaVMXIbG1f39Qyf+TuuoSrXAcbiWTbV4q/e3sZfvrvBc3ej5nxpw+qsHF366QBttQCaAuXy6WPTsW3n5/VrH06xpvYejiNk2a4J99dvAXTqjr+nOB5awsiCgN4FMA4AMMBXENEwz2b3QRgFzMPBfAQgF9r61Yy83H23y3a8scA3AxgmP03tuWXceAosYeEUEKgxwDW7swIQDKVxv88MwP3/mcxgOypZmp5Ks349VtLceRdb2G9dpxCUb2TD6aUtrRWoyVa8JI1l0emVOHvn65r9n6rt9fiN28tw7f+3ryKUoeZUTn+Dfz+3RUFxQDqmpKY2swKZM66XXjRk0SQj7nrd2PSgs3N2scJAgfDAHBiAM0Rvq8/OxPX/mV6G5XowFFIc3EMgCpmXsXMTQAmALjEs80lAP5mf34ZwDm5WvRE1A9AN2b+hK2cuWcBXNrs0rcDygWk/vfvnkkFXbuj1vmcTDGWbd7nVOZqpjEvas7hNAPTV+0AALw6Z0Ozy1XTmHTO29Ys3LAHT9oup1zoFWFLXUDLt+zD3PW7m7dTMxj78Ed4+N0VADL3sCU02r/j799b7hLhdxZvwY1Pf+bb/o6X5+O6v0xvlth/6U/TcMcB6F2sfqN0FgV4Yfo6VI5/A43JzpE9Y3IBBYVCBGAAAL3ZUW0vM27DzEkAewD0stcNJqI5RPQhEZ2mba93kzQd86BEtfiVAIw8pMxZt2VvJgCbSKexq64JCYMLKJVmzK/ejaqt+7Btn7VPmhnlXa0+BnowedKCTXh93kYAVibR0s25c8WzCU1rcuEjH+P+SUvybqeLUVMLXUDnP/QRLn10aov2LYSlm/dhon1/C/FC3v7iXGOFrgQgHCKX8H3j2Zn4YNk2n1WgUj8P5ko0myXzS/u3r2lo/UEQmTmn4M9dvxsn/uJdJ942ZekWX2p1czEFgYNCIQJguiveJyPbNpsAHMLMIwHcDuAFIupW4DGtAxPdTEQziWjmtm3bCihu26IeEuUCGjGgu3G7moYkGpNpJyCotwpXbavBxX+cinMf/Ag3/W0mAGvwsHq7o5jeYezbz8/G//5jDgDgzn8vxNiH/4u3Fm7KWj7THMZtRb5gp6pAmlJpTF5kdkPc+/pinPfgh61arkIr1ZYMdvav2RvwwTL/c9ho/2YhImMM4Ja/z8JFj3wMAPjLf1dh1XZlLWZeBW+L+7YJc/DM1NXNLqOikI6Hr83dgJlrdhrXZYtl7LMtpbZId31pVjUufXRq1ufld28vw5a9jY5I/M8zM/d7zKVMDCB4FCIA1QAGad8HAtiYbRsiigDoDmAnMzcy8w4AYOZZAFYCONzefmCeY8Le7wlmHs3MoysqKgoobtuiKulutiVQFA3jj9eOdL6rQeNUy15V/HrLfFedf/C4NMMoADqrtlmVxjqP24ANvvZ8VO+qQ+X4N/DR8paLakOeVr2qQKYs3YoZa6yhDryt7L9OXY0VW2taXAYThc7YVusZxnt/KgBlATQm03j0/arMMe2DvrN4CxZs2AMAuO+NjPWk/15eAXht7kbc/friFpepkLkqbpswF5f/+RPXMvUb5avg1TVv29eIET+bjPnV5pb7vPW7UTn+DcxZl3+4iyV2P5ps4/Ko37a0yD9Vq0I929mEzUumH4D/CVBlX7mtdZ/RfMxetwt7DPVEa1OIAMwAMIyIBhNRDMDVACZ6tpkI4Ab78+UApjAzE1GFHUQGEQ2BFexdxcybAOwjopPsWMFXAbzWCtfT5pw/vC8uGzkAP77gSGfZhcf0x8hDegAAKu2ewWr4Y/Vw6d3NTZV0itnJc88mAKovQW2je73eUis0BqDGnvlnHvM5lWanE5uXfC1MU9+GAzHE8N4CBcDr89+/PgqZe/Gv2ZkYTjjPQXXXWGvHb/a1wEWzbPM+p1GQL5tJPcfTVm5HTWMSF/9xKuYZ3DdTlm4FALxv/y/kmLGIuWpSApAtPrGnLoEfvGRZBBNmFOYayhUDUAP6FVJ2Lw9MXoZrn/zUt3zSgk245bnsCQcNiRQu+9M03LIfSQmFklcAbJ/+rQAmA1gC4EVmXkRE9xLRxfZmTwHoRURVsFw9KlX0dADziWgerODwLcysZPlbAP4CoAqWZfBmK11Tm1IcC+PBq45D725FruXDencFYE0hCQBb7XiAavk3aZW+6cVMp9mpRLJ1eFLr6z0Vr175tHYW0IPvLMOY+99zLJps5zWRMpSl6QBkKe0usOVUmyXoO3vdLvxy0hL8vRkdgRqzWEPhPN1L9fvREpeK3vfES6FCCGSE6AsPf6SVJ/dvlQnsZ67xEkO8JpsGzlq7y2c1qIZSVLtvHy3fhqPuegv7GhLOb5stpvTdCXPwiZ1Mke3O/+Clea64QcJwHQr1/irB2VXrv9+z1+3CrLV+a+OP71dh2sodvuXffn423lq0OeuYURvsdGRlMbYlBSWNM/MkZj6cmQ9j5vvtZXcx80T7cwMzX8HMQ5l5DDOvspe/wsxHMfOxzHw8M7+uHXMmM4+wj3krd9TZM2xGHWpZAMrsdiyAJOP9ZVvxU61TyT6DaZ7mTB8AbwWvUC1WveJav7MOR9/9tvO90Bx0x9z1bH7V45/gpF+853x/d7HV8jEJQLZKL1dZsgkUM+PFmetbZcC8fC6g65+ajtfnbTQKcU1jEpf9aRoe/2iV6zfLR7a4QyEx7MAAACAASURBVCSfAGj3ULeYcsVXlFBMq9qO0fe9i017zO4S77wVuVi3s9a3LJ9FkkhafVeqd5kzmWobk1ndGBt21+PLj03DNU+4W8jq+VDWLmD5/WubUqjaWuP8tk3JtNGa1DPxvPV5IpXG3oYEXppV7YobNDlBYMM12vcglQaqtu7D8fe9g0Ub3RXzZX+ahi8/9ol/5zxke8+V+6trPGJc35ocnL2GOiBjBvcEAFx9wiBEQpSJAaTTeOyDla5tTS+mHgPYUdtonGZSVfx6ZzHvIHKFWgCZ+t/9Ek1fvROb92ZcPiH7rTCZ3HktAMMLmq1SmbxoC+54eT4embIi5zEVC6r34NU57vH21TWpSmLdjjrf/ahrSuK/K7bjf/8xx+8CAhXcavZWPtmstpCnVvH+ri4B0I6ZK76iWqVV22qQTDP++vFqozXTHAtAz2AzlUehC11TKoXfTF6K37xlnubztN+8j2Pvfdu4TnWi8p5CWRVRzQWkhFG3phqTaaOAZ7MMXpu7AcN+8ib+badYl2hTvJosGYUS4jQzqnfVgxnYor0f+9NuzZZ2rATVO6BkWyAC0Er06hrH6l9egMuOH4hImFwds7x+4J//xxzYUwKwcMNenPIr98Bn6TSjzl6vp4k2eFqehWQBVW3d5xwj3/OrGmLqJdF7P+qVHjPj3tcXu9JUm2MBqCEythoqIhMX/fFj/N8/52HzngZc9qepOPuBD5xrqU+ksK8hgdN/+z5+/C93z2DV2zceCfkqTaLC/ebe+16oBbByq7ul3ZglBpCrR7jaTt2zJ/+72mitqGuJhAgNiVTOyspUcZp+Pz31synJ+O/y7J3ZVPkmfGa5W/Sjqesu8bRyva3xP7y3Ags37PWVpymVxu56/9ArTdo91DsdqsD7e0ssi/bwvqXOukQBFkA6zc47o9+r/cm6y5ZGqyyAWEQEoEOhWhDRUOa2JlOc1w+s0B8sb6ZQfSLlVHB69oq35WlyHSRTadzy3CzMWrsLqTTj3Ac/MnYo0lt87LS6QvZ5rIdf7/3YqFkAW/c14q9TV+OmZ2Y6y0wmejYBaLIr0ELvleK+NxZj9rrdWlqldc+VlfWSHcRbt6MOd09chPU7rZerZ0nMWNnnypzRy65X0G8t3ISPslSE4ZD7Fbvk0Y9d37PFAHJZV6oi1MeVMrlhlLURDYdw5F1v4bYJc13rdUEwCdgf3luBo++e7FqmWyZNqXTWYKyOblE6+zrHce+v94wHgAffycy3vWLLvkx5EylXrEfdR/030ttdyiJX7lf90cxU4v5nzykPs9Ni10Xb2xAoBNUo0JM5GhIpnP/Qh5hatd1pBDXHgmspIgBtQCSceZASKfa5AVqC3lrVKx/vi5swVLrbahrx1qLN+Gz1Tp8LQn9/9ZFEnY5N5D+nQn/4VYWlVypmC8BcYaiYSSEVis6aHX7fdSKV9rXuvzthDp6ZtgZvL7byy8u6xIwmeDaf9ay1u/Cbt5Y633fVJfD7d1egMZnCLX+fjWemrTHu57UAfC4PrTJZv6sO7y2xXHo5BcCulHZoAmByX+hJA8zAxHkbXQKn/xameM7WfY3Y15B0/ab6c5BIplvcFyAzBpZ7uXpmTK5CPShqWQCaANjl1++nKfisRKNeew9UWcwWgO0CSjPqDC7YhgLG7rr52ZkuYVLxjRlamurc9buxfEsNfjN5mfNe7coR4G8tRADagIgWwEqk0lkDgZO+e5pxuQk9sFmrC4DHAjANurajxnqQGpMpX0WuYgCz1+1y+hkAGfeBapHXmQRAO7fjbtCu3RwDMFsAm/dYArCztnmtHlPAN5FiV+v+qY9XO2XJBNjCZr95Fgvgy49Nw5P/zXTKemDyMjz07nK8Miv3sB0mi+bBK491PusV1iNTqpyOgd4AYdJgKeysyVQQpkfMFJfQB7zT+6bkCui7XI5auZpSfgHI1dZJGHqG76htwsPvLvctNz07yhWkttMrcVV+d5aZvzBKNPRr8qaBmhoxac68d/r7t70mfyX99uItWKNZqCrF9d7/LHayoFT/h+H9ujmWdV1Tqs17iosAtAF6Clsylc7q1hhSUWJcrtBfAuUSKo6GXQ++10w0DQWhWopNybTr4QWsFtjSzXtx2Z+m4RotZ1m1jlXL0mgBaJVBRgC0aze8xE1ZLACVyZJvPgOvH3uH4QVMeiyAf82udl5u1apqTKZR4+lPQUQFdyJbW+AYPqbf/ni7zwiQcX3pWJWbJ9VX9zsbXECmTkymLJOahiSe+3Qt5qzb5eqbkksA9HvpTTn2Wmx6A8CLSzy086mxmIBMBW56drbubXAqz6Zk2iVwqiGkvzMmC0D9/iYBUPvq53aOy+zchzrtfmzTntf3l25F5fg3sFxzVSn0DC89w+n6pz7Dss37sMgeGqS8a8z1W7R1ZzARgDbAbQFw1s5A8SydXTL7Zh4E9bJXlMaxZkedU1F6zUSTi0VNpdiUTBuzizbtyfhoy7pYPSxrtAAi4M9eAdyVhhKMSIiwZnstXpq5Hn/+cKVvn2wxgM12GdS0iAs37MFf/usfcM5bMZgsk0Q646/t260I4VAmHKiEqrYxaXSzeAUgW3nV/Y+Gm+/e69k15nw2Vbz1TSlfFpAuCGu216Jy/BtYplU0hQrA5r0NuPPfC3H5nz9BQrcAcricxvziPSf/XT9mYzLt6+yXK+3Vaz3oJDw+fFNDpqYx6aRGNiXTrrI05ZjWUW80qI/686zeGZP7qUYb9qLOYAHo6dGvz7cGMzB1htMt9Zj2zOypT+COl+c5DblEil33qa0nQRIBaANcMYB0dgsg3+BjeuWggny97MrjqsetvGNvsHj9zjo8/uFKPPXxaicIq8Rj1rpd+OIf3EFIBlCntYSPtsc2UsEyxwVkqCDcFoDaPoQzH/jANz5Ledc4bjp1MPbUJ3DJo1N9vYvVi7TbdgFd+MjHriETFIWkuSZTaefFHdSz2Lp++15nfMB+d9iSTXtdrVEAeHPhZlSOf8N3DiUA+cahMcU0SuMR3Hz6EADm7Ju6RNJvAWj32jRYmulRMgmcytoJkbuiy9enY/pqq0NTvafl7K2nc/V8zmYBABkhV1aJKQZQ05hEcTQMIktA9OOd++BHeOpj/7hJ+xoSRmuiIZHW5upIu86p318n8JtIOZ91C1wXAHUcUy9mPXHDu56IXAKoV/otCTI3BxGANkBlAUVCZLU4WhgDnvBZZmz7xz9a5fQyBoCVtr/e67OeMGM9fvnmUvz8P4uxdLPVQlQuoDnr/BUHs9vtMqhnFwCZAb8UJhfQpj0NqBz/Bv4zP9OpKluLOBYmx/Sdt343Zq91jwujzldnp3Bmw2Th3HLGYZ5t0o4FM6hHF+yuSzg/gWrh1yVSWTvi6LyYZTiBQmPVpjFtiAg/GneEU5F5qWtK+Spv3RdsSuc0VeAmAVBDjVf2KnGJaT4BeGPBZpz9wAeue2YKAodzWEQul43nfA2JFGobk45VY4oBpNmymmPhkM8FBAAzVrt74744Yz2OvvttLNvsd8kAGWvGa3XortBaLfNHWQ3ZLABnGAuDG0zP+PFaa+EQOc91UzKNxmTKeY+8Mb7Wpu27mgUQZQGUd41j896GFk8R+Ms3M1kn2/Y14ptnDEF5SRxz1u3Gob3sijpH3np9wlq3I4dfPZFKux7iQT2s46oKVLVqPlq+DWVdYq59P15htSZfnlWN0XZP6PnV5u7r0UjIZfqu3u73oXcrimBvQxJrDOv08noZ2rsrjh3YHfPscydSGX/twJ5dUNOY9LkU6hoLE4C+3YvybtMSiMipyLzUN/nL5gq4G4LXpoC2vk9xNIz6RMoRnMZk2nUvt+5twGtzswe01ZDkeuZUU6qZLqBkbgtAn2MiW4/2WCSEWCSExmTaJ3B6vwD9GJ+u8g/HoM5ZEo+45tLwxZjsxpMlUCo4m7kHm/dmBF5dk8myr21M4omPVtrC5S53SGsIKAuge3EU22uaJAjcEVExgIpS//j++fjh2COyruvbrQhfP20wLjq2v2OumjqTqNaDqgC88xDr7K5rcsUABvW0rAxlWaiW2rzqPb4ObIvtzIVDe3Yx5nq7yxRydfpZvd0/uuIAW3yyDT4HmF0DpUURV9wlmbZcQLFICL3t38A7PlCTIVXURGuPea/Xj6oi87qJ6jzuqeVb9rlE2hT4Nl2Lfgx95rqhvbtiZ22Ty5qaMGO9r5+Aib31bt+5t69Hrn4cuWIA9U0plzWbbRyiaDiEeCSE5Vv24W+frHGt27zH/Nx4J91RsTd1f9SzkUxzVp97YzLzvOiteTWvQ0Rrxd/y91m+6SJrGpP4xaSleODt5di4pwHHDswMI6+7gJRrq1uxFYuTGEAHRGUBqcqnkIpGccHRffHQVcca1/XpVgQiQt9ucfsFdgfCSu0K9hDbjXPdX6Zj9rpdOVPV5lXvwSuzM0MqHNqzBCHKVDKFTOSybMu+vFMxRsMh1/zJpla+cnGZhiVQmCyA0njE5WVLptgJGPYssawWUzZFLmFUvJVlXHovn+tTmn8juM3/eCSEplTaJ2p1TUlXa/n8hz5yDUdsKrc3owlwB2z1zJMR/buhpjHpGw67EHT3XGMy7Wuph4nw7uItuPWF2b599crMm/1Un0ihKJrp+bppTwOen+4fjC9mu4CmrdzhE/WNWQRAZdgonImXbAtZ70+wrzGhbecO1ivXj24BrLHnAfe68342cZHrnF5XrW5Zhl0xAEZj0rIArPOKBdDhUC6gQiwAb0CoOBpGcdTsmVOC0qMkhvpEytcSjNsv0KG9Muml33xuVkEVnaJn1xh6lsSduEC2B7CyVxeMPaovAODTVfnHXY+FyTW4ld5zVzGwhxKA7BaAUQCKoq4gaEITgB6228rkNikkh7tQbj59iDMeVC70cioXkNcCqG9K+Vx7ugDoHcBe+dbnMW5E3ywuoMxvpycmHNXfan1uzWO1mVAVmdXi9Q/IFg4Tvv7sTPxnvn/SolxB4L9NW4PaxhT6d7eytp6fvg4/edU/vEUsHMo6VHS2xspMT7yp3H6PVm6txRMfrXSNqPrrNzPjGumVdEMi5VT8xsyzFOOz1dnfA+8oovp8BqFQJvidsF1E3cUC6Lio1pZjARhaWp/9+Bzn/9M3nuAsL4qFURQ1/yx97CGoe9qVmndiGPWAKgsAsGIH3u1y0aNLFOVdY07lmC04OKxPKf58/SinhX3MwO7o0y2e9bjRcMjlhthe0+gL9ioByOUCMgWBuxZFXH5X1RO4JB5B/zK/D1+5yHbWFjbuUCHEo6Esc9q50csZi4SwZNNeZ7IcRV1Tyte6rdqqWwCZcg/v1x1H9uuG+kQKm/c0uDp66RWu7ppR8aMVW5o/ycneesu1Fo9Y4uWNAZjcU6byeH/HifM24vX5G1EUC+d0I0Uj2QXAhOlY6r387oQ5+MWkpVi7I/N+6NZwWbHHAmjMLgD58Gbr6e9CyOMC8loACzfswd0TF7VIsPMhAtAGqEBYhV1h1xnMczWfQFmXmJN5A1gWQDbVr9AsAMAkANZ51AveEoqjYVSU6haAuSyH2mUusl/GkYPKnAAy4O/kFgkTusbdszjpLx6QuSe5BoRLGHqflneNuV1AdgygNB5B/7JiXyWgRCvNluBl46Qh+Vv0ingk7BtZVVlIOt4YgNc9AVgZSt4Ofqu316K/3SLVK9niWBj97OUn/fI9nKwNIuhyAWljEp0ytBylRRE824z5DhR7GxIojoYRUwJQQL8M0zpTa70pmUaXWNgXSD5G85fnsgAAfzqsmqdDp6/9nOll7xLzD7ymKmHAHQT2WlvKpZQL77wN3TQLIBwix32kAtIZAUhj+ZZ9eGbamhYJTz5EANqASDiEaJicH1GZ7L+67Gjj9iooFQlZqZKqJexF+UjVw7PBTjEcP+4I1/ACh1X4H/p8/N+5h2PNr74IIkKvkhjmrt+NZ6auNr6oFx7TDz8Y+zlXmboWRVwVzunD3NN3RsMh3/jm3uwMVRlvNcw9oFi8aa+rlQtkdwGVxMOIhkOu9FnrPJmWXf8y870GrBFeASs7adwIf2X+wjdOdD4XRUO+1NBjB5X59tFjAD08WVWK+qakr0Pa9pomJzBY15TCEX1LMeV7ZwCA7/qsY6SwZW+jM+xxOET4v3MPx7fPPAwl8QjOPbKPcY4HL2d9zv077q23BGBXXQLPfbrW2EAIUWa45W+deRhW//IC3HTqYFfF2ZhliJQu0YhPsG89a6gj2rEIIZ5jlMyennvqnbgJyPzmuqVtes67a42D2sak83x7BeDIfv74j9cY9FoAuuDoMQAl/BkXUKb/gR5Day1EANqAWDiEsi4xV9rjuBF9ceXoQcbtVSWq5hMeMaA7fn7pCNc26gUAbHcDMr1mPz+kFy47PjPF8lCt1XNE38KCkz1LMg/7Tvthvfv1xcYYwHnD+2ReQvsSS+JuAfD2co55XEADyorxu7eXu7ZRJneuGMAdL8/Hab9537dcH/rXGgoiha62UHotIv1e6u4yL700S+F/zx7mWz+8XzdH1CwLwE1Xw3juugCM1IaE0KneVe9LaQTgCAAA9OtehCG20JtE7PYXrYweNRBhJEy47dxhuMPOMjsszzAkzjV45t7d25DM6qJU9faQiq5OBdatKAoiQkk8gtqmFL79/Cw0JdNoSqaNLfligwVQEo84ufWxcMiXZz9t/NmOtVXqqST7GtySA+wGlm5pm9JOy7T7rYuldziV4f27+fb14u2xr1+7lQVkZ/UZLABHANpgghgRgDZgzOCeOG94H0Q0s7sxmc46Kqh6oYq0VoFuupZ3jeFvXxvjfFcvgHIFeFsGfbVWzxUe0fnt5cfgp188El70HP9Th/ZylduLXrkrM7o0HsEVozLn8r5OXgvg7ouP8uW6q4c+mwWQreIB3KZ/Mm0NBqcqYG8lrwvAoBwCoLZLpt2V1e+uOBaDehajW1HUqfTikZAvhzwe9QuAXs7Th5Ubz/vsJ2tdA58pdJeEnlJr6qug5nxWHYm8rerB5YVZicWee769pjHrhOzXjDnEKk+3ImdCFxVvUb/FpAWbsXjTXjQlU8Zew8XRsK+jVFmXqHP/Y5EQij3umv5lxc6z4bUOTALf33C/vjRygG+Zfr+9lT6QyRLq3z27FalQQWDlrtPFJxzKDOKogv9K7BsTVqfGSIjyDh3TEkQA2oAbTq7EL750tGtWo1yDjKmHtlirMPTP//jGSTha84OqB+GNBVamhbfVowvNl0YOwJw7z3MsgbIuMXzxmH7O+nOP7APA3YP366cOwffOOxyAucerXhmqFMaSeAS3nDEEP/iC5Rry+oajEXc/AN2vq9BnQNIracVDVx7nL4yNXmn8d8V2bK9pdATHawH00o6tv+Re1HapNLtevi+PGoj/3nE2QiFyKtZ4NOQTPVOPUL1qO3loOW49a2jW83vR/cYlscy9LPIITSrN+Jz9e//oAqvF721VDy73WwBHGVqyUc817K5LoHdp3JiqrLbVW/Fqmf7bL9m0F02ptLHXcJdY2OfrLusSc+6/tz+J99zxaMjplAgAR/T1X1NZl6jjohpcXoL/3nEWHrzyWFzmEYEueVrcT351NGb99Fwnw0p/Zr2ZXcoFpGJj67X5G9KcSSFViRHF0TBi4RAakpYLyJvo0FoUJABENJaIlhFRFRGNN6yPE9E/7fXTiajSXn4eEc0iogX2/7O1fT6wjznX/uvdWhd1sKC/c3pQ7+oT3K3yaJgQIo8AaK0c7wOvt3KOP6QMFbavuldJzBnMLbNvGD1KYs4DWRwNo1/3Ynz7zMPws4uGo6I0Zp9Py0oIEY4x+K9N51cVfde49YCqF9X7AkTD5BKO3qVxV0UMAF20MlQYAmtHG0TjsuOtl9b0bpQ4/SLclV3Pksyxiw2tdIWKASQ9AqCjhCceCTti2cvxV/v38VqBIwa4r+kOO7YCAF8/dTAuPra/871bceb+dPG4l+68cLjzua4pibqmFE4dWu64h7wWwCGGRIGXbvk8/nnzSa5lXgEAgN7d4rj4WH+LWVX6JbGws5/6r1t/P/rXAqzfWY9Th5b7UmeLY2FfJ7GyYrcFYHKtqcZWPBLCy9862Vk+rI/f0ulWFHWsmOJoGIN6dgEROckVirhJwLXbWBKPoFfXuBNg1+9xtnkSVIPrtKEZ6y+RyvTKVrsVRcOIR0KWBaANgtfa5D0qEYUBPArgPADVAGYQ0URm1ruF3gRgFzMPJaKrAfwawFUAtgO4iJk3EtEIAJMB6E/Odcw8E50UPZVPWQBrfvVF33ZEhKJo2OUC0ismlbam0CuWS0cOcFoGn/zoHF8mimqFqgeyOGZ9V77gfQ0JHN6n1OeO0APRFaVxlw/UZQFoAgAAFx/XHxNmrMeNJ1fi6alrMteotX2tAb0I5V3jrpz2omgIRJbV0atrDHBPd4ySWASPXz8K33xuFgDgoauOdSoiU+somwWgWxped4KOCtIymytzIFOhxyMZC+DP14/CUf27OUNluLb3lNP7Yg/vl2mx9isrxg/Gfg4T7WEYslkAAHDTqYORTjPun7QEU5ZuxZx1u3De8D6OZRfxzEymn/fBK48FkSXA3saGaWynitIiY3ql6o3dJR7JJDaElSj4q5qyLlH88drj8Z3nZ2PK0q2oT6RQHA37Ks8usbDzPsQiIePIp9FQRogBqzW+s7YJA3v4ha5bcdRym+51//5ekY8Yrr1bUdR5l5XgqXuhu7SyzX39ub6lqLp/nKvnekMi5ZsoqDgaRjxqWwANbScAhVgAYwBUMfMqZm4CMAHAJZ5tLgHwN/vzywDOISJi5jnMvNFevghAERHlz5nqJOhZHrmmGgQsxdf9rbpZ7x1jXa+MirTWuJWf7a7QVMWotyx0Soui+Nopg30VqB5H8GaZ6K4N1WVfxSF6lxbh3dvPcHVGAzLDRky89RS8//0zfdehyqpedNMDHw67/aA9S+LOy2cyjnUBCIfISdfzDpHw/871B3jVOueaswiAeumJgB+POwKH9OyCo/p3Q5dYxGmVnjasHN8/33KpeetN78TfekyiZ0nU9Xvq7iqvBQAA5bY1d9uEuUizu9LNlVt/ytByfGmklUTgrfSMFkCp+RWOhv0WQMzgAlLEwtY1PHrd8bjddjmaBscjIseyjXlcQC9+8/Oucqrn41/fOhnPfO0E43XHI5mEBL2h5X13epf6YwV62rA6p7pn+qn069DfpXgk5HufTQMtlnWxfvu2tgAKEYABAPQhEavhbsW7tmHmJIA9AHp5tvkygDnMrEf4nrbdP3dSFgcXEd1MRDOJaOa2bdsKKO7Bw6nDyvGKbY6eN9yfRqgTj4SyuoBM2zqfcwRGdVSryisA2dBfsgGetNRcFoDOb758DH5+yVEAMl3/jxlY5gQtTa1L5QYyXX805BY4/X6Znh51jC6xCF74+omYdNup+N55h+OGUyqdbXqXFuH/nXu4E9B77qYxjp9W3efvnHWY0Z8PAL+4bASGVJSgV0kcJw7phY/uOMu5BuVCSKYYZ36ut13O3BaAHlDUXVWAOwvI1KL29iAviUeg6qFcg7Tplbx3u+YIgKpsi6O5XUAKl0vQztbJNiGQctnEwhkX0IXH9HNcSBEtBgAAleUlzj33QkQZF5D2nOlJBo9ddzxOGeqtwtwB94jHutJjGmoMqXsuPgpXjs5k6CnR0zEFmLsXRxGPhrBuZy2mrdxhFNDWoJCjmp4cr32TcxsiOgqWW+h8bf11zLyBiEoBvALgegDP+g7C/ASAJwBg9OjRLZuAtB0ZdWgPfPaTc3IGGwHrZdVT7oqytDgB94uTKydaRwlANFSYYOgM9FoAhiwg0wt+pR3r6FESc7k2FKbZo1R+dHE0jDvGfg7JFDsTg0c8cQSXABjKrQ+le+IQ62X+33PcrX1VmSkhG9G/O/p3L8aqbbWIRUJGl53O2Uf0wdlH9DGuO2FwT1wxaiBuPXuok/HktwDc902vhLwxkm6aRWLquOS1JrrEw46FZnJnKPR1XleRyfIp9wjA4PISDOrZJTMcciTkWD+OC8guW4isczSl0q5e2qq1nW10W7V/TEsm0N0sKuU6m1B7MVsA1r6l8QjGHd3PuN9Npw7Bp6t2oqxL1IlTqWvUXUDq946EyRVMjkb8v4OaYSwSIuc57G5bAKqXeC4Lbn8o5G5VA9CjlgMBbMy2DRFFAHQHsNP+PhDAqwC+yszOFFHMvMH+vw/AC7BcTZ2S3qVFeSvqX3/5GJcrQlWOpk4+eistW2qkt4JQLazmdKNXeCt3c+WTvS1x4TH9nZx1HbMFYB27KBrGt88cijMOr9C2D7msHxXPAMwzYulB02yo+zJuRD/7mGH84ZqRePDKY31urOYSDYfw2yus46iK1VtO733TLQSVVaJe/m5Z0kAV3t+layziVCjeil0/rv48+V1AZj+4zpTvnYFn/2eMk8se8wz9DWSeIVX5A8AQLRX1hMoeuO7EQ3DPxUf5zgdkrjeZZudYeu5+xh2T/fke1LPYGe12gKEzmLKO9fTdJ786Gg9ckcl4OnFIT7zwjRPx7u1nOOdS98iU5h0Jkeu30gVKWZ3KAlDbhcj67XTBX7rJnxbcGhRiAcwAMIyIBgPYAOBqANd6tpkI4AYAnwC4HMAUZmYiKgPwBoAfMfNUtbEtEmXMvJ2IogAuBPDufl9NB2bUof5OQS/f8nlUGtL19Eoim0vns5+c68rEefz6Ufhg6bZmjW//kwuOxPwNe3yiocc2/vGNk/Da3I1GUciHqVLKuG0yPVh1TC8s4HcBXXfiIfiCYSgGL8pd8/NLjsL3zz/cCsZHw66OdYpYJITrTzo07zFNqErCW0WU5LhvSgCKo2HUNCbdMQDDfl4XUHEsE1A1tSBf+84peHXOBlcr2PubhA2/kTftWD2PqjdtNBxyKmQ9TRiwrAU16Ys+XEgkHML9X3L3lL//jFo2cwAAC1FJREFUSyNwzAArG62r/TvVNiad30wfMlpVxrkayi98/SQnxqISA/T07LjTlyBzzecN7+MaiK80HsHJh7kTJtQ9MvVriIRC0L11ukvt4x+eje/+Yw7eXmxlO5TEwthTn0D34ihCIcJR/btjuj3A3GGGIS1ag7wCwMxJIroVVgZPGMBfmXkREd0LYCYzTwTwFIDniKgKVsv/anv3WwEMBXAnEd1pLzsfQC2AyXblH4ZV+T/ZitfVKRhdmX8smmwC4G219y4tclwyhfINe9rCp6e6p9rTz3nsoDLjkAeFYPIvO7nk9jm8LVLdh+pO4XRvp2dHFUIkHHLSPrOx/L5xBR/PS8gJFrvLFLE7yPUujeNrWmwC0HqIx8K+QKDJAvC6gJJpRi87jmAaHmTEgO6+NFTv/a5tTOL975+JrXsbcNUT1kxZqhx3XzTcNZCd6jQYi2QEQLmFSuIRPHLNSJw4uCfO+K01s1hfwzANOucN7+O4hpQbpa4pM1uW2wJwZ+SY0J+3Stu60zsdKivdG1fT77vpmYpq2UAXHdvfmTwHsO5nkcHNBFi/r96JU12jWlZZbonU8H7d8Mdrjs96XftDQZEFZp4EYJJn2V3a5wYAVxj2uw/AfVkOO6rwYgrZyNU7trVoK/9jzOAPVadSloA3KKm/nHoAz/temoKkOscOKkNNnsys1kRZY6YQzMJ7vpBz39vOGYaf/nuhMx80kMUC8CxrTKRx6rByPPs/Y3DKUHOvYy/eGNGuuiYMLi9xBiLUz33jKYNx4ymDM+ezA/0xzQLQx9i5yO7TMOm207BmR23WnvGZ82R+QxX4rWlMOlaKHgNQ58sl+rq4qf4RuzxpyIA7sw6Ak4F0/5fcw7MowpoAPHLNSDxyzUic/9CHWL6lBpGQuwe815rWe9ora7CLbcmdZo+nNX7cEa5xiVoTmRKyg+N9WDsSJheQ6i+gWk3ebVwtKO3ava99vrS5175zSnOK2moUErS/6Nj+rgHvvnLSofiKx/Vk6sDWqySOcSP64s2F1iQ2aqjm0w+v8G2bDW/vXNWXRfddZ6tkVQwgHg1jdGUPvDK72jgy7eDyEmNPZC/6Napxk04bVu5UuPpQ1KpIuTRFF7dDenbB5aMGuu6rsi69FkA8Es6ZDKDcT3pDSblJw54YgNfq1ScqGlLRFfOq9zgussHlJXmTEPYXEYAOTqFpnftD27T/zZkp6kX2xgBU60xvQektSG+d1BYjJ+4Ph/Tsgm+deRiuyjIgoM4j14zMuu7N207DH9+vMnZwCocIj31lFPY1JHDP64txk9Y6LxSvxXX5KCsWYgoGe2nSLICLThiEMYN7tmhkWoVeoR7epxSL7vkCSuIRJFJpXDFqIL6jDaOhJqYxJQMo9OctHCJXcFenuWPuZMQnc3wVr0kzu9JmvRaAPkjcF4/uh3nVu/HFLBlIbcHB9ZYIzaYtBojy0QZjkADmlFR1KtX6U5W8ngduwp9dc3BZRkSUc77nQjmyXzc8em1uf3BpUTRr5ZYPXQD01mch8ZQfX3AkkinG6YeXg4j2q/I3oVrSKrtKR3mDcgmAKeako1xYzW1UpQ2BdpWxtbc+4RIAr8DqowXUNiUx5XtnNuvc+4sIQAenI1sAppxohbqulP1mqwlosqX5qff+66dard5C+0cIbvYn3nNorxI8pc1udyBRI7HmFoDc16Y6PHrnssiHKdNKWQB7GxKuZ9YrpHoWUltM+JKPA9B8FNqSA2EB6M9srvHzm4upRaZeYNXyP6RXF9xz8VF47Cu5cwbUPLfjju6Ln2oDownNg4hw+3mH4/VbT223Mpx7ZG9fqmk+Uo4LKPs2+ayYI/p2w9TxZ/uysQo9t54Get2Jh6A0HsHYo3K7c3552dGoKI3jxpMrjUNStzViAXRw8mVStAYqMDukosQZe6U1yGWS62Pr33ByZd5jfeuMw3Dq0PIWp6QKGb57jnlspAPFX25ovhWhMkJN78PDVx2Hf3y2rqDjmDpe5uNQO6B9kTZy65CKrliQJ7tL7aPvd6ARARDyotLgvnh0v4LmPy2UiCGIqyyA5o75EQqRVP5tzCXH9cfxWWYxa2/UKLgmF9ClIwfg0jZsXQ8oK8ay+8bmHIbihW+ciMWG+Z/bGxEAIS/jRvTFby8/Bhcf17otFd03qsTg/i+NwEPvrMDJh/kH4hLal99fnT07qb3pY3cY8w5ceKDIF3M6+bByXw/igwERACEvROSbWrI10DsXqZnEBvbogt9d2bIMFiG4XHb8AJQWRZwJV4TCEAHooEwbf3bOaSY7AteOOQSREOHyUQPzpujp/P7q41zjEQkCEeH8AsZ+EtyQdyLrg5nRo0fzzJmddgIxQRCENoGIZjHzaO9ySQMVBEEIKCIAgiAIAUUEQBAEIaCIAAiCIAQUEQBBEISAIgIgCIIQUEQABEEQAooIgCAIQkDpUB3BiGgbgLUt3L0cwPZWLE5HQK45GMg1B4P9ueZDmdk30UGHEoD9gYhmmnrCdWbkmoOBXHMwaItrFheQIAhCQBEBEARBCChBEoAn2rsA7YBcczCQaw4GrX7NgYkBCIIgCG6CZAEIgiAIGiIAgiAIASUQAkBEY4loGRFVEdH49i5Pa0FEfyWirUS0UFvWk4jeIaIV9v8e9nIioj/Y92A+ER3ffiVvGUQ0iIjeJ6IlRLSIiG6zl3fmay4ios+IaJ59zffYywcT0XT7mv9JRDF7edz+XmWvr2zP8u8PRBQmojlE9B/7e6e+ZiJaQ0QLiGguEc20l7Xps93pBYCIwgAeBTAOwHAA1xDR8PYtVavxDICxnmXjAbzHzMMAvGd/B6zrH2b/3QzgsQNUxtYkCeB7zHwkgJMAfMf+LTvzNTcCOJuZjwVwHICxRHQSgF8DeMi+5l0AbrK3vwnALmYeCuAhe7uOym0Almjfg3DNZzHzcVq+f9s+28zcqf8AfB7AZO37jwD8qL3L1YrXVwlgofZ9GYB+9ud+AJbZnx8HcI1pu476B+A1AOcF5ZoBdAEwG8CJsHqERuzlzjMOYDKAz9ufI/Z21N5lb8G1DrQrvLMB/AcABeCa1wAo9yxr02e701sAAAYAWK99r7aXdVb6MPMmALD/97aXd6r7YJv5IwFMRye/ZtsVMhfAVgDvAFgJYDczJ+1N9OtyrtlevwdArwNb4lbhYQB3AEjb33uh818zA3ibiGYR0c32sjZ9tiP7UdiOAhmWBTH3tdPcByLqCuAVAP+PmfcSmS7N2tSwrMNdMzOnABxHRGUAXgVwpGkz+3+Hv2YiuhDAVmaeRURnqsWGTTvNNducwswbiag3gHeIaGmObVvlmoNgAVQDGKR9HwhgYzuV5UCwhYj6AYD9f6u9vFPcByKKwqr8n2fmf9mLO/U1K5h5N4APYMU/yohINeD063Ku2V7fHcDOA1vS/eYUABcT0RoAE2C5gR5G575mMPNG+/9WWEI/Bm38bAdBAGYAGGZnEMQAXA1gYjuXqS2ZCOAG+/MNsPzkavlX7eyBkwDsUaZlR4Gspv5TAJYw84Paqs58zRV2yx9EVAzgXFiB0fcBXG5v5r1mdS8uBzCFbSdxR4GZf8TMA5m5Etb7OoWZr0MnvmYiKiGiUvUZwPkAFqKtn+32DnwcoODKBQCWw/Kd/qS9y9OK1/UPAJsAJGC1CG6C5ft8D8AK+39Pe1uClQ21EsACAKPbu/wtuN5TYZm58wHMtf8u6OTXfAyAOfY1LwRwl718CIDPAFQBeAlA3F5eZH+vstcPae9r2M/rPxPAfzr7NdvXNs/+W6TqqbZ+tmUoCEEQhIASBBeQIAiCYEAEQBAEIaCIAAiCIAQUEQBBEISAIgIgCIIQUEQABEEQAooIgCAIQkD5/yK3FeR2WWGWAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXyVZ5338c/vZN8TSAiQAIFCW2jLUiJdsLbV2sWli1Ytjlqd+jBqq+PMPDNP+zgvW6vzuI51nDrWqp26dpnaKtrW7pt2I7RQCHtZQwIJhCxA9vyeP84NPQ2BBHLCSe7zfb9e55VzX/d9zvld4fA9V65zL+buiIhIeEUSXYCIiAwvBb2ISMgp6EVEQk5BLyIScgp6EZGQU9CLiIScgl5EJOQU9DLqmdktZnbLkZZHMjOrMLMtR1oWiQcFvcgwC8L7GTM7YGZrzeyiRNckyUVBLzL87gFeB8YCXwEeMLOSxJYkyURBL6FmZp8xsz/GLG80s/tjlreb2dzhel4zOxk4E7jZ3dvc/XfASuDDx9snkWOloJewew44z8wiZjYBSAMWApjZNCAXeGMYn/c0YJO7t8Y8dkXQLnJCKOgl1Nx9E9AKzAXOBx4DdpjZqcHyC+7eO4zPmws093l4M5B3fD0SOXYKekkGzwEXAO8K7j9LNIzPD5YPY2Z3mNm+4PZ/h/C8+4D8Po/LJ/ohIXJCKOglGRwM5POC+88xQNC7++fcPTe4/b8hPG81MM3MYkfwc4J2kRNCQS/J4DngQiDL3WuAF4BLie4F8/pwPq+7rweWAzebWaaZXQXMBn43hNcVOSapiS5AZLi5+3oz20c0iHH3FjPbBDS4e88JeN5rgLuBvcA24Gp3bzje1xU5Vgp6SQruPqHPcuWJel5330J0ikckITR1IyISchrRSxg8O8DySNYE/OAoyyJDZro4uIhIuI3IEX1xcbFXVFQkugwRkVFj2bJlu92933Mojcigr6iooKqqKtFliIiMGma29Ujr9GWsiEjIKehFREJOQS8iEnIKehGRkFPQi4iEnIJeRCTkFPQiIiEXmqBv7+rhp89v4sU3dye6FBGRESU0QZ8aMX76wibu+svmRJciIjKihCfoUyJcPb+cp9fWU9/anuhyRERGjNAEPcBlp0+g1+HlTY2JLkVEZMQYMOjN7C4zqzezVUdY/89mtjy4rTKzHjMbE6zbYmYrg3XDfvKamRPyyM1I5dXNe4b7pURERo3BjOjvJnodzH65+3fdfa67zwVuAp5z99gh9YXB+rhc0edoUlMizJ9SxKubNaIXETlowKB39+eBwSbnIuCeIVU0RAumjmH9rn007u9MZBkiIiNG3ObozSyb6Mg/9ur2DjxuZsvMbPEAj19sZlVmVtXQcPzXTT5r6hgATd+IiATi+WXsB4G/9pm2WejuZwKXAdeb2buO9GB3v9PdK929sqSk33PnD8qcSYUUZKXxePWu434OEZEwiWfQX0OfaRt3rw1+1gMPAQvi+Hr9SkuJcPGsUp5Ys4uO7p7hfjkRkREvLkFvZgXA+cAfYtpyzCzv4H3gYqDfPXfi7X1nTKC1vZu/btRRsiIiA15K0MzuAS4Ais2sBrgZSANw9zuCza4CHnf3/TEPLQUeMrODr/Nbd/9z/Eo/soXTi8nLSOWJ1fW8+9TSE/GSIiIj1oBB7+6LBrHN3UR3w4xt2wTMOd7ChiI9NcLcyYUs396UiJcXERlRQnVkbKy5kwpZv6uVtk7N04tIcgtt0M8pL6Sn16mubU50KSIiCRXaoJ89qQBA0zcikvRCG/Tj8jKZWJDJihqN6EUkuYU26AFOLyvQ1I2IJL1QB/20kly2Nx6gu6c30aWIiCRMqIN+anE2XT1ObZMuRCIiySvUQV8xNgeAzXv2D7CliEh4hTrop5ZEg35Tw74EVyIikjihDvqS3AyKc9Oprm1JdCkiIgkT6qA3M04vK2DVDu15IyLJK9RBD3D6xAI21O+jvUunQhCR5BT+oC8roKfXWVOn6RsRSU6hD/ozyqOnQlileXoRSVKhD/qJBZkUZadRrXl6EUlSoQ96M2NaSS5btC+9iCSp0Ac9wKSiLLY3tiW6DBGRhEiOoB+Tzc6Wdp3zRkSS0oBBb2Z3mVm9mfV7YW8zu8DMms1seXD7asy6S81snZltNLMb41n4sSgvyqKn16lr1jlvRCT5DGZEfzdw6QDbvODuc4PbrQBmlgL8CLgMmAUsMrNZQyn2eE0qygZge+OBRLy8iEhCDRj07v480Hgcz70A2Ojum9y9E7gXuOI4nmfIJo0Jgn6vgl5Ekk+85ujPMbMVZvaomZ0WtJUB22O2qQnaTrgJBZmkRExfyIpIUkqNw3O8Bkxx931m9j7g98AMwPrZ1o/0JGa2GFgMMHny5DiU9ZbUlAjj8zOp0YheRJLQkEf07t7i7vuC+48AaWZWTHQEPylm03Kg9ijPc6e7V7p7ZUlJyVDLOsykMVls36sRvYgknyEHvZmNNzML7i8InnMPsBSYYWZTzSwduAZYMtTXO15TxuSwZbcOmhKR5DPg1I2Z3QNcABSbWQ1wM5AG4O53AFcDnzezbqANuMbdHeg2sxuAx4AU4C53rx6WXgzCyePzuK9qO7v3dVCcm5GoMkRETrgBg97dFw2w/nbg9iOsewR45PhKi69TSvMAWL+zleLpCnoRSR5JcWQswCnjo0G/bldrgisRETmxkiboi3PTyUiN6OhYEUk6SRP0Zsa4/AzqWxT0IpJckiboAcblZVLf2pHoMkRETqikCvqS3AwaFPQikmSSKujH5WdoRC8iSSe5gj4vg+a2Ltq7ehJdiojICZNUQX/wQKnG/Z0JrkRE5MRJqqAvzE4HYO8BBb2IJI+kCvqi7DQA9u7vSnAlIiInTlIF/ZgcjehFJPkkVdAfnLppUtCLSBJJsqCPTt00aupGRJJIUgV9WkqEvMxUTd2ISFJJqqAHKMpO19SNiCSVJAz6NBoPaOpGRJJH8gV9jkb0IpJcki/os9M1Ry8iSSXpgr4wO00HTIlIUhkw6M3sLjOrN7NVR1j/N2b2RnB70czmxKzbYmYrzWy5mVXFs/DjNSY7nX0d3XR29ya6FBGRE2IwI/q7gUuPsn4zcL67zwa+DtzZZ/2F7j7X3SuPr8T4KgyOjm1q0/SNiCSHAYPe3Z8HGo+y/kV33xssvgyUx6m2YaHz3YhIson3HP11wKMxyw48bmbLzGzx0R5oZovNrMrMqhoaGuJc1luKdAZLEUkyqfF6IjO7kGjQvzOmeaG715rZOOAJM1sb/IVwGHe/k2Dap7Ky0uNVV18Hz0mvK02JSLKIy4jezGYDPwOucPc9B9vdvTb4WQ88BCyIx+sNRXlRFgDbGw8kuBIRkRNjyEFvZpOBB4FPuvv6mPYcM8s7eB+4GOh3z50TKScjleLcdGr2KuhFJDkMOHVjZvcAFwDFZlYD3AykAbj7HcBXgbHAf5kZQHewh00p8FDQlgr81t3/PAx9OGblRdls04heRJLEgEHv7osGWP9Z4LP9tG8C5hz+iMSbPCab17fvHXhDEZEQSLojYyEa9LVN7XT36KApEQm/pAz6SWOy6Ol16prbE12KiMiwS9KgzwbQPL2IJIXkDPoiBb2IJI+kDPoJBZmkRIwde9sSXYqIyLBLyqBPTYmQn5mqE5uJSFJIyqAHyM9Ko6WtO9FliIgMu6QN+oKsNFradQZLEQm/pA36/Mw0WtoU9CISfkkb9AVZaTQr6EUkCSRt0OdnpdLSrjl6EQm/5A16Td2ISJJI3qDPSqOju5f2rp5ElyIiMqySOugB7XkjIqGXtEE/Nid67dgGXVJQREIuaYP+4CUFdRoEEQm7pA36ssIg6JsU9CISbkkb9GNy0slKS6FGI3oRCblBBb2Z3WVm9WbW78W9LeqHZrbRzN4wszNj1l1rZhuC27XxKnyozIyyoixN3YhI6A12RH83cOlR1l8GzAhui4EfA5jZGKIXEz8LWADcbGZFx1tsvJUXZVHTpHPSi0i4DSro3f15oPEom1wB/NKjXgYKzWwCcAnwhLs3uvte4AmO/oFxQpUVakQvIuEXrzn6MmB7zHJN0Hak9sOY2WIzqzKzqoaGhjiVdXTlRdnsPdDF/g6dCkFEwiteQW/9tPlR2g9vdL/T3SvdvbKkpCROZR1dWZH2vBGR8ItX0NcAk2KWy4Hao7SPCAf3pa/Zq3l6EQmveAX9EuBTwd43ZwPN7l4HPAZcbGZFwZewFwdtI0J54cGg14heRMIrdTAbmdk9wAVAsZnVEN2TJg3A3e8AHgHeB2wEDgCfCdY1mtnXgaXBU93q7kf7UveEKs7NID01oi9kRSTUBhX07r5ogPUOXH+EdXcBdx17acMvEjHKCrM0oheRUEvaI2MPiu5Lr6AXkfBK+qAvK8yiplFfxopIeCV90E8fl8ue/Z3Ut7QnuhQRkWGR9EE/b3L0jAyvbWtKcCUiIsMj6YP+9LJ80lMiLN+uoBeRcEr6oM9ITWFqcQ4b61sTXYqIyLBI+qAHmFaSw6bd+xNdhojIsFDQA1OLc9i25wBdPb2JLkVEJO4U9ESDvrvXdeCUiISSgh6YNCYb0IXCRSScFPTAxILoyc1qmxX0IhI+CnqgtCADgLomHTQlIuGjoCe6i2VxbgZ1GtGLSAgp6AMTCzOpbdaIXkTCR0EfmFCQSZ3OYikiIaSgD0woyKJOI3oRCSEFfWBCQSb7Orppae9KdCkiInGloA9MCK4fqz1vRCRsFPSBiQWZgPalF5HwGVTQm9mlZrbOzDaa2Y39rL/NzJYHt/Vm1hSzridm3ZJ4Fh9PGtGLSFgNeHFwM0sBfgS8F6gBlprZEndffXAbd/+HmO2/CMyLeYo2d58bv5KHR2leBhFD+9KLSOgMZkS/ANjo7pvcvRO4F7jiKNsvAu6JR3EnUmpKhHF5mdRqRC8iITOYoC8Dtscs1wRthzGzKcBU4OmY5kwzqzKzl83syiO9iJktDraramhoGERZ8TehMFMjehEJncEEvfXT5kfY9hrgAXfviWmb7O6VwMeBH5jZSf090N3vdPdKd68sKSkZRFnxN1H70otICA0m6GuASTHL5UDtEba9hj7TNu5eG/zcBDzL2+fvR5QJBZnUNrXhfqTPMRGR0WcwQb8UmGFmU80snWiYH7b3jJmdAhQBL8W0FZlZRnC/GFgIrO772JFiQmEWHd297D2gg6ZEJDwGDHp37wZuAB4D1gD3u3u1md1qZpfHbLoIuNffPhyeCVSZ2QrgGeBbsXvrjDSH9qXXOW9EJEQG3L0SwN0fAR7p0/bVPsu39PO4F4EzhlDfCXVoX/rmdk4vK0hwNSIi8aEjY2McHNFrzxsRCRMFfYzi3AzSUkz70otIqCjoY0QiRllhFpt370t0KSIicaOg72Pe5CKWbW3SLpYiEhoK+j4qK4rYva+DrXsOJLoUEZG4UND3UTllDABVW/cmuBIRkfhQ0PcxY1wu+ZmpLNvamOhSRETiQkHfRyRinDmliNe3NQ28sYjIKKCg70fF2Bxq9mpfehEJBwV9PyYWRi8U3qoLhYtICCjo+zG+4K1TIYiIjHYK+n68dSoEBb2IjH4K+n6MPxj0OouliISAgr4f4/MzSU+N8GaDToUgIqOfgr4fqSkRZk7IZ+WO5kSXIiIyZAr6IzijLJ/qHS309uqcNyIyuinoj+CMsgJaO7rZsmd/oksRERkSBf0RnFFWCKDpGxEZ9QYV9GZ2qZmtM7ONZnZjP+s/bWYNZrY8uH02Zt21ZrYhuF0bz+KH04zSXNJTI6xS0IvIKDdg0JtZCvAj4DJgFrDIzGb1s+l97j43uP0seOwY4GbgLGABcLOZFcWt+mGUlhLhzMmFPFa9ix7N04vIKDaYEf0CYKO7b3L3TuBe4IpBPv8lwBPu3ujue4EngEuPr9QT75NnV7Ct8QCvbN6T6FJERI7bYIK+DNges1wTtPX1YTN7w8weMLNJx/hYzGyxmVWZWVVDQ8Mgyhp+76iI/vGxYZf2pxeR0WswQW/9tPWdy/gjUOHus4EngV8cw2Ojje53unulu1eWlJQMoqzhV5KXQU56Cpt3a88bERm9BhP0NcCkmOVyoDZ2A3ff4+4dweJPgfmDfexIZmZMGZujXSxFZFQbTNAvBWaY2VQzSweuAZbEbmBmE2IWLwfWBPcfAy42s6LgS9iLg7ZRY2pxjkb0IjKqpQ60gbt3m9kNRAM6BbjL3avN7Fagyt2XAF8ys8uBbqAR+HTw2EYz+zrRDwuAW919VF2jb9bEfB5eWUfTgU4Ks9MTXY6IyDEbMOgB3P0R4JE+bV+NuX8TcNMRHnsXcNcQakyo+VOiX8i+vq2JC08dl+BqRESOnY6MHcCc8kIiBq9v1zVkRWR0UtAPICs9hcljsnXKYhEZtRT0gzCtJJc36xX0IjI6KegHYVpxdBdLnbJYREYjBf0gTB+XS3tXL9saDyS6FBGRY6agH4S5k6OnLH59+94EVyIicuwU9IMwY1weeRmpLNuqoBeR0UdBPwgpEWPu5EKWbdUuliIy+ijoB2n+lCLW7Wyhtb0r0aWIiBwTBf0gzZ9SRK/Diu264pSIjC4K+kGaO6kQMzRPLyKjjoJ+kPIy0zilNI9l2xT0IjK6KOiPwfwpRSzb0khbZ0+iSxERGTQF/TH4wOyJ7O/s4ZGVdYkuRURk0BT0x+DsaWOoGJvNfUu3D7yxiMgIoaA/BmbGx94xmVe3NOpsliIyaijoj9GH55eRGjGN6kVk1FDQH6NxeZksnF7Ms+vqE12KiMigDCrozexSM1tnZhvN7MZ+1v+jma02szfM7CkzmxKzrsfMlge3JX0fOxrNmpjP5t376erpTXQpIiIDGjDozSwF+BFwGTALWGRms/ps9jpQ6e6zgQeA78Ssa3P3ucHt8jjVnVAnl+bS1eNs2b0/0aWIiAxoMCP6BcBGd9/k7p3AvcAVsRu4+zPufvBk7S8D5fEtc2SZMS4PgPW79IWsiIx8gwn6MiD2m8eaoO1IrgMejVnONLMqM3vZzK48jhpHnGklOQBs2aMRvYiMfKmD2Mb6aev3mnpm9gmgEjg/pnmyu9ea2TTgaTNb6e5v9vPYxcBigMmTJw+irMTJTk+lND9DUzciMioMZkRfA0yKWS4HavtuZGYXAV8BLnf3joPt7l4b/NwEPAvM6+9F3P1Od69098qSkpJBdyBRpozNYeseXVpQREa+wQT9UmCGmU01s3TgGuBte8+Y2TzgJ0RDvj6mvcjMMoL7xcBCYHW8ik+kirHZvLqlkcerdya6FBGRoxow6N29G7gBeAxYA9zv7tVmdquZHdyL5rtALvA/fXajnAlUmdkK4BngW+4eiqA//+RxANz+zEbau3SSMxEZucy93+n2hKqsrPSqqqpElzGg/3hyA7c9uZ5pxTk89U/nY9bf1xkiIsPPzJa5e2V/63Rk7BB86Mzozkebdu/nlc2NCa5GRKR/CvohmDQmmzW3Xkp6SoRn1uqUCCIyMinohygrPYUzyguo0iUGRWSEUtDHwfwpRby2bS9X3P4X7VsvIiOOgj4Orj23gs+cO5VNDfv52h+rE12OiMjbKOjjoKwwi69+cBbXnlvBM+saeHrtrkSXJCJyiII+ji48NXpE79/eXUV1bXOCqxERiVLQx9Gc8kJOL8sH4KYHV/K7ZTUJrujE2Nnczu+W1TASj8kQER0wNSy++egafvLcJgCq/vUiinMzElxR/Lk7T6+t509v1PHwyjo6u3spL8rimndM4voLp7/t4DF3p6W9m4KstARWLBJuRztgSkE/TJZuaeQjd7wEwMfPmkxtUxs7m9v578+8gwkFWQmubmhW1jTzncfW8sKG3WSkRphanMOmhv10Blfc+sf3nsynF1aQl5HKi2/u4aYHV7Kt8QBXzp3IlfPKOHNKEfmZCn2ReFLQJ4C7c9ODK7k3uIh4fmYqLe3dfPmiGXz5opMTXF1UQ2sH3b29R/3geWrNLh6v3sUX3zOd8qJsXtu2l0/+7BW6e51/uvhkPrNwKmkpEdwdd/jsL6t4Ojh4LCc9hf2d0fMApaUYXT3R99pJJTn8/vqF5CnsReJGQZ9AD79RR0d3Dx86s5xP/vwVVu5o5k9ffCflRdkJrav5QBeX/OB5dra0c8XcibR19tDS3sXU4lyumlfG9HG5dPf28p7vPUdrRzcTCzK57rxp3PbEesbmpnP/351DaX7m4c/b1sWS5Tto7ehm1Y5mCrPTuWpeGfMmFfLypkZe3dLI7U9v4FPnVHDL5acloOci4aSgHyGqtjRy9R0vkZ4aYelXLkrInHVXTy979nXylYdW8tTaevIzU8nLTCMzLcKbDW8d7GUG7pCeGuF7H5nDVx5aSWt7NzPG5fLL6xYMafrpXx5YwZIVtfzo42fS0t5F04Eu0lIiVNe2MKEgky+9Z0Y8uiqSVI4W9IO5wpTESWXFGL71oTO48cGV3P70Br7y/r7XWB8e3T29vLGjmfte3c79y7Zz8LP9lg/O4tMLpx7abtnWRvIz07jjuU3sbGnjlU2NfOOK07l8zkTmTylic8N+zpo2hrSUoe2stfhdJ/Hoqp1c94v+P8zHF2Ty0cpJ/a4TkWOnEf0J5u584D//QnVtCz/55HwuOW183F+jcX8n97y6jYde38EtHzyNHzy5/tC5ePIzU1l01mTOnjqWC08dd9TnOdDZTXb68IwF2jp7eG3bXjLTItTsbWPmhHxa27v5lwdW8GbDfm4JDkDTqZ9FBkdTNyNMW2cP533nGU4Zn8tvPnt2XJ/70ZV1fP43rx3W/s+XnMKiBZMZk5Me19eLt87uXhb/qopn1zVw0cxSSvMzmFCQybXnVoTyy9s3G/ZR19TOwuljh+VDbVdLOw8sq+HKeWX86qWtTCjI5COV5aRGIpgx5L/ORqLl25t46c09fHh+GSW5GUkzWFDQj0C3PbGeHz69gVdueg/j+vlS83isrGnmQz/+K109zkUzx/G1K07nliXVnFFWwBffPX3UvOF7e53vPr6OHz/71jXk3z97Ard9dC7pqdFg2ljfyh9X1FHb1MbZ08by/tkTyExL4fHqnXz/ifV88pwpNB3o4mPvmDQijmPo6XVSIm/9/lvau/j3x9bxy5e34g63XnEanzqnIi6v5e785pVtfPvPa2lt7z5sfUFWGp3dvVx2xni+/9G5cXnNROvtdepbO2ho7eCDt//lUPslp5Vy1tSxzC4vYEZpHvmZqYf+H7R39ZCeEiESGR3/LwaioB+BNta3ctH3n2fRgsn825WnH/Zma+/qobvX6erupWiAUXhXTy9pKRE+96tlvLRpD8//84UUZI/+0W9vr/PfL27h24+upbOnl3dOL+aHi+bRdKCTT/zsFepa2kmNRHfbnDQmiyvnlvGfT29823OcUVbAQ184l9QEjFy///g6SvIz+dOKWqq27uXk0jymj8tlTHYav3hpKxGDT5w9hcerd7GzpZ13Ti/mw/PLuGpe+WHP1dvrgwqk5rYubn96Az99YTMQ3cX1U+dWUJiVxlnTxtLc1sXXllSzKTjL6os3vpvuHqcgK+2w98yBzm4yUlPe9gE1Urg7a3e20tHdy9SxOfzdr6t4eVP04j8TCzK5fG4ZT63ZxdY9Bw4d3wEwrTiHsbnp9PQ6K2qamVSUxa1XnM55M4oxM1Zsb2JXSztnTRs76g7wU9CPUP/28Gp++sJmrppXxnXvnEpRTjptnT08/EYd91dtZ0dTG6kR49PnVjAuP+PQPuuxfv3yVr7x8Go+OHsi/7Oshi+9ezr/ePEpCerR8Ojq6eXnf9nMtx5d+7b2n36qknedXMxDr+3g239ey94DXSyoGMPXrjiN+6u2k2LGz/6ymQUVY7j+3dOZXVbAPUu3cd07p5KRmjLkutq7ekiJ2KHjCFbXtXBKaR4RM275YzW/fGkrAOkpEa6aV8Zz6xvY2dJ+6PHfvXo2H6mcxGPVO/n2o2vp6O5lR1MbZYVZfOqcKXx4fjkPv1HHH5bv4LVtTUwrzuHjZ03ms+dNe1sdd/1lMz96ZiNjctLZUL8PiB6k9/UrTqe9q4ecjMO/Z9nUsI+Lb3ue7t7o///TJubT1dNLTkYqH5k/iRc2NPDn6p1MLc7hV9edRVnhiT/Ir76lnaKc9EPveXens6eXe1/dzs1L3jpLbHpqBBwumjWO/Mw0Pnf+SVQU5wDRv5wOdPTwyuY91DW38+BrNTTu76I0P4Ps9BSWbol+dzUuL4OI2aF/n7QU44NzJtK4v5PrL5zOvEmFAw4WjnQE+BOrd/Hvj6/jE2dPoafX2dSwjy9fdPKAA7hjNeSgN7NLgf8AUoCfufu3+qzPAH4JzAf2AB9z9y3BupuA64Ae4Evu/thAr5csQe/u/OiZjXzv8fWD2v6imaV848rTGV+Qyebd+7nht69RXdtyaP0ppXk8dP25w/YFaqI99HoNf161k53N7Vx82ni+cMFJh/4M7+rppb61g/H5mYdGoO7O//plFU+uiR7AlZEaoaO7ly9ccBKfWTiVkrz+p3Tcna4ePzRNBPBGTRNfvnc5//qBmZx/8jh2trTzsZ+8hBksPm8aHd29fOPhNUwryeGcaWP5zSvbgOjUwecvmM7cSYW4O//17JuU5GVwxdyJh33Y9PQ6Dyzbzu9fr+WlTXuO+HvITk/hzMlFTBmbzbbGA7ywYTenTcwnLSXC8u1NAKz46sUD/lX351V1vPjmHtbUtRwKvJK8DBpaO8hMi3DFnDIeWVlHZnoKi8+bxnXvnNrvXxVHmwJxd+5bup15k4t4dfMe3jOzlImD+NDYvHs/F9/2HBMLs/j8+SeRn5XGl+9bTmd3dHSemRbhQ2eW09Xdy86Wdv7+PTOorBgz4PP29ejKOh5YVsNTwUF+4/Mz+Yf3zuD//G4l8NZuxgDf/+gcrpxbxu59HeRnpZGZloK7U13bQmF2Gt98ZC1Pr63n/JNLcJw7PjGf/Z09nPVvTx46cPCg8fmZnDt9LJ8//yRmlOYd+j3W7G1j+rjcY+5HtNYhBL2ZpQDrgfcCNcBSYJG7r47Z5gvAbHf/nJldA1zl7h8zs1nAPcACYCLwJHCyu/f0fZ1YyRL0B63Y3sTm3U8uIicAAAjsSURBVPtZsqKWiEXno8+eNpbx+Zm0dnSTGjF+/fJWvvnoWgqy0rjhwuk8sKyGtTtbuWjmOL7/sbnUNrUxtTgnLiPVMNnf0c2z6xr48XMbWbWj5W3r3lFRRErEaDrQxUkluVQUZ3NyaR7PrmvgyTW7+NQ5U1i3sxV3DgUBQG5GKu7O/s4einPT2b2v87DXnTEulz9/+V3HPe1RtaWRz/36Na49ZwqXnj6eNxv2895Zpdzw29d4em09KRHjQBAeMyfk8+DnzyUrPYXW9i4a93cyZWzOoF+ru6eXW/+0mrrmdr579Wy+9/g6Pr5gCrMm5rO6toWv/bGaVzY3ctvH5nDVvHI27GrlO4+to7mtizfr97FnfyeLFkzi5g+eRk+vs2pHM+3dvZQVZvLff91y6EMPYGxOOr+/fiGr61r4jyc3kJEWYfF502hu66K8KJu5kwvZ2dzODb99jbU7W/ut985Pzuc9M0vjOqW0YVcru1o6OGlcDhMKsnhy9S6KctIpzE7jht++zpq66HsnMy1Ce1cvWWkp5GSksntfxxGf86bLTuXuF7dQ19zObz57FqX5Gfxlw2421O/j/qrtdPU4J5XkcN6MEuZNLuRXL21la+MBnv3fF/T7V9hAhhr05wC3uPslwfJNAO7+zZhtHgu2ecnMUoGdQAlwY+y2sdsd7TWTLegHa2N9K195aNWhC5Hf/vF5fGD2xARXNTrUt7azdPNezj1pLF9/eDUPvrbjmB6fkRqdfjl4SguI7sn0hQtOorq2hbU7W3n/GRNo7+rh0VU7ufi00iF/CdzfvLy7c6Czh7aunuj3OD1OUU76sM4n9/Y67/73Z9nZ0k6KGfs7oyP42LlvgIiB89YIONac8gLOOamYn72w6dB00bSSHNo6e6hrbj/8AcB3rp7NR+aX86c36nh1cyP5Walkp6dy/YXT493FAdW3tPPIyjpW1bZQmJVGw74O/rC8FoC/XTiV0vwMirLTmTUxevba636xlF0tHeRmpHLhqeP44TVz37YzxIHObp5YvYu/v3f5obb0lAjf/9ic4/4/PdSgvxq41N0/Gyx/EjjL3W+I2WZVsE1NsPwmcBZwC/Cyu/86aP858Ki7P9DP6ywGFgNMnjx5/tatW4+1n0mht9e5Z+k2UiPGRysnjZo9aUaauuY2cjJSyUpLoa6pnUljsvj1K9soyEqjuraZj8wvZ/2ufaSnRDi9rIDxBdE9o+pb2tm9r5Ntjfu55LTxSfP7f259Aw+/UUteZhpjctL5aOUkdrW0k5YSoTQ/g1+8uJWmtk4KstIYm5vBnn0dpKdGeO/MUiqKcw7Ns6/a0cyrmxspyErj/bMn0NHVy09f2MT0cbnsaGqjvqWdsqIsLppZyrSS45vCOFG6gg+6/nZR3d54gEdX1XHNgslHPIHfwWmf8qIsnlxTz6nj8zi9rOC46xlq0H8EuKRP0C9w9y/GbFMdbBMb9AuAW4GX+gT9I+7+u6O9pkb0IiLH5mhBP5h9zmqA2OPRy4HaI20TTN0UAI2DfKyIiAyjwQT9UmCGmU01s3TgGmBJn22WANcG968GnvbonwpLgGvMLMPMpgIzgFfjU7qIiAzGgF/tunu3md0APEZ098q73L3azG4Fqtx9CfBz4FdmtpHoSP6a4LHVZnY/sBroBq4faI8bERGJLx0wJSISAkOdoxcRkVFMQS8iEnIKehGRkFPQi4iE3Ij8MtbMGoDjPTS2GNgdx3JGA/U5OajPyeF4+zzF3Uv6WzEig34ozKzqSN88h5X6nBzU5+QwHH3W1I2ISMgp6EVEQi6MQX9nogtIAPU5OajPySHufQ7dHL2IiLxdGEf0IiISQ0EvIhJyoQl6M7vUzNaZ2UYzuzHR9cSLmd1lZvXBVbwOto0xsyfMbEPwsyhoNzP7YfA7eMPMzkxc5cfPzCaZ2TNmtsbMqs3s74P20PbbzDLN7FUzWxH0+WtB+1QzeyXo833BqcIJTv19X9DnV8ysIpH1D4WZpZjZ62b2p2A51H02sy1mttLMlptZVdA2rO/tUAS9RS9g/iPgMmAWsCi4MHkY3A1c2qftRuApd58BPBUsQ7T/M4LbYuDHJ6jGeOsG/sndZwJnA9cH/55h7ncH8G53nwPMBS41s7OBbwO3BX3eC1wXbH8dsNfdpwO3BduNVn8PrIlZToY+X+juc2P2lx/e97a7j/obcA7wWMzyTcBNia4rjv2rAFbFLK8DJgT3JwDrgvs/ARb1t91ovgF/AN6bLP0GsoHXiF53eTeQGrQfep8TvT7EOcH91GA7S3Ttx9HX8iDY3g38CbAk6PMWoLhP27C+t0MxogfKgO0xyzVBW1iVunsdQPBzXNAeut9D8Of5POAVQt7vYApjOVAPPAG8CTS5e3ewSWy/DvU5WN8MjD2xFcfFD4B/AXqD5bGEv88OPG5my8xscdA2rO/tAa8wNUpYP23JuN9oqH4PZpYL/A74sru3mPXXveim/bSNun579Oprc82sEHgImNnfZsHPUd9nM/sAUO/uy8zsgoPN/Wwamj4HFrp7rZmNA54ws7VH2TYufQ7LiD7ZLkK+y8wmAAQ/64P20PwezCyNaMj/xt0fDJpD328Ad28CniX6/UShmR0ckMX261Cfg/UFRC/jOZosBC43sy3AvUSnb35AuPuMu9cGP+uJfqAvYJjf22EJ+sFcwDxMYi/Gfi3ROeyD7Z8Kvqk/G2g++OfgaGLRofvPgTXu/v2YVaHtt5mVBCN5zCwLuIjoF5TPAFcHm/Xt88HfxdXA0x5M4o4W7n6Tu5e7ewXR/7NPu/vfEOI+m1mOmeUdvA9cDKxiuN/bif5iIo5fcLwPWE90XvMria4njv26B6gDuoh+ul9HdF7yKWBD8HNMsK0R3fvoTWAlUJno+o+zz+8k+ufpG8Dy4Pa+MPcbmA28HvR5FfDVoH0a8CqwEfgfICNozwyWNwbrpyW6D0Ps/wXAn8Le56BvK4Jb9cGsGu73tk6BICIScmGZuhERkSNQ0IuIhJyCXkQk5BT0IiIhp6AXEQk5Bb2ISMgp6EVEQu7/A1vWNYOOef4XAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "w, ws, Rs= SGD(x, y)\n",
    "dist = [ np.linalg.norm(ws[iter]/np.linalg.norm(ws[iter]) - w0) for iter in range(len(ws))] \n",
    "\n",
    "print(\"w estimated : \", w/np.linalg.norm(w))\n",
    "\n",
    "plt.plot(Rs)\n",
    "plt.title(\"Empirical risk\")\n",
    "plt.show()\n",
    "\n",
    "plt.plot(dist)\n",
    "plt.title(\"||w - w0||\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Noise your observations with an additive Gaussian noise and perform the optimisation again. Compare with the result of question three."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x19da7ae6278>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEgCAYAAACwxdQWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3debgcdZ3v8feHAIlBiMOi+BAxQUAQUITowKAMQoyIQxAdvOKoBFTAOaA4zB0XrgrXx4szgOIIjHBBUAdBvIqJgLIoIDKChh1kSxQ1KEOIskjYEr73j1+dpLvT56T79FLb5/U8/ZzT1XWqv1Wnu771W0sRgZmZ2ah18g7AzMyKxYnBzMyaODGYmVkTJwYzM2vixGBmZk2cGMzMrIkTg41J0l6SQtK8vGMpCkkPSLom7zjqRtI1kh7IO466cGKoEUkbSfq0pJslPSFpuaRfSTpJ0kvyjq8oJB0v6e15x9ErSVtJOkvSPdn/+s/Z//vrkt6Ud3xWXOvmHYANh6RtgcuBlwPfA84BngN2Az4KHCpp/4j4eX5RFsZnga8D32/z2iuBwo8KlTQLuJb0P/4GcBfwAmBbYH/gCeDq3AK0QnNiqAFJU4EfAFsA+0fEpQ0vnyXpDOAqYL6knSLiv/OIsxOSBGwQEX/J4/0j4pk83ncCPgtMBV4bEbc2viDpKGDzXKKyUnBVUj18gHSl+KWWpABARCwEPgVsBvzPdhuQdLSk+yQ9nf08us06O0j6jqQHJT0j6SFJV0t6W8t6kyV9StJd2fYelfQDSa9tWW9VG4ekEUm/Ap4G/lnStyU9K2nTNnG8Mvu7UxuW/aOkK7LYnpX0R0n/KWlGwzozJI2WBg7JthENy8ZsY5D0dknXS/pL9rhe0gFt1nsgqy/fTtKlWZXeY5L+n6TNW9bdWNKXJC3OjtMySTdJavs/arENsKw1KQBExPMR8YeW9/ofkhZI+l32v3tE0vclvXqcfXiNpKuy/X1Y0smS1pU0Jfv9wSzun0ravmUb87JjOzuruvtt9r63S3p3B/s3up1tJH0z+38+m8V2kqQNOt2GrUmeK6n6JF0L7AlsExGLxlhnKvAo8GBEzMyW7UWqbriZdIV5JqkK4mDgdcDxEXFCtu4mwK+yzX0V+C2wKTALuCciPpOttx5wBfA3wDeBm4BpwIeAlwJ7Zomq8f1vAzYB/i/wEPB7QMClwNERcVrLvnyelOh2iYhbsmW/Bm7ItvUnYEfgg8BjwE4RsSw7mRyYxXUdcNboNiPiP7PtPAA8EBF7NbzfPwKnA/eQqqACmAdsBxwREWc1rPsAqXpnQ+DiLJ7XAEcAV0XEnIZ1f0z6v52ZrTc12+aWEdGUbFtJuhTYD3hnRHxvvHWz9a/LjssvScf4FcDhwPqk43h/yz6sADYCvk2qpppDOnb/BuxAqrb6Pukz8M/AEmD7iHg+28Y84FzSZ2sD4LzsuB1Kqq47NCLOa3jPa4AZETGjYdmuwE9In9tzgQdJx/JDpM/V30bEc2vbd2sjIvyo+ANYBjzewXp3kL6cL8ye75U9fwKY3rDe+sAvSCe46dmyudm671rLe3wsW+8tLcs3An4HXNOwbPT9/wS8uGX9ScAfgV+0LBcpKd3esnyDNrHsk23/X1qWB3DeGPE/0BLjXwF/ARYBG7Xsz+Ls2L2o5e/XOE6kxBLAdtnzadnzMyb4P98deDbbxn3A14APk07O7dZvd3y2B55pjaFhHw5qWX4T8Dwwn+yiM1v+kdb/OSlxRva/mtawfFq27E/ACxqWX0NKyI3vdxspGW/YsvzAbNvz8vzelfnhqqR62Ih0Zbw2o+tMa1l+fkQsGX0SEc8CXyK1Ue3f8rdvlbTROO/xXtKX+SZJm44+SMnmSuANkl7Q8jffiIiHGxdExErgfOB1krZreGkvYEvSlXvj+k8CSFpH0rTsPW/L4v7rceJdmzeTrnj/PSIeb3i/x4GvAC8EZrf8zR8i4qKWZT/Jfm6d/XyKdFL+68bqrk5F6kSwK+k4TCNdiZ8B/ErSdZK2all/9PhIqffapsBS4F7aH58HI+I7Lct+RkrMX4nsDJ25Lvu5TZvt/EdErPpsZr9/lZRw9xpr/yTtBLwa+BYwueWz9DPgSVIpxibAiaEeHiclh7UZXac1idzdZt3RaqOtACLiWlLvl3nAI1kd+wmSXtXyd9uTqkOWtnkcRioJtLYb3DdGvKMn//c3LHs/MJo0VpG0d1Yd8SSp6mH0PaeRTkITNTP7eVeb1+7Mfm7VsvzXbdZdlv3cBFYl32NIVV6/UWqP+YqkfToNLCLuiIh5EfESYAZwCOkk/QZSR4P1R9eV9FpJl5BKOI+x+vjsRPvj85s2y/48xmujyzdp8zdr/WyNYbTN4gTW/Bw9TErW7oI9Qe6VVA93AntK2jrGb2N4Jam43trjp11DlFoXRMQhkk4i1W2/ATgWOE7SMbG6HUCkKqt/GifepS3Pl7dbKSLukHQr8F5Jx5Hqtd8JXBERDzXs2+tI7RqLgE+QTlxPZft1Ib1dIK1xHDqwspPtRcRXJc0H3gb8LfD3wFGSvh0RHTfQZtv6LfANSaPtJ3sArwd+JmlL4KekC4jPkUoJT5KOz6mkUk83+zDWa+2OVUefrXHWOQX40Rjr/HmM5bYWTgz18D1SI+YHSSfGdt5Pqs5p11DZetUPq6/Ymq5+I+JOUiL6N0kvAm4EviDp9Kx64X5S76efRNYQ2aOvk6q13kRqvN6Qlmok4D2kkshbI2LV1WzW2NxLaQFSOwKkBtcft7w2etzalRA6EhF/BM4GzpY0idQwfrCkUyLilxPYXki6kZQYtsgWH0g6+c+NiKaxDVmngkF20X0VsKBlWdvPVovRxvCVEXFV36OqOVcl1cPZpKvlj0nat/VFSbsAJ5Ku1E9q8/f/IGl6w/rrkxqRVwKXZMs2ltT0eYqIR0lX51OBKdnib5B6OLUtMaj7EdjfIvWQeX/2eIzU+Nlo9Aq29Ur0U7T/DvwF2LjD97+SdHV9tKQNRxdmvx+dbevKDre1iqSpWSlulaxd5fbs6bjxSXqzpDUu/LL2m9G699Eqm7bHR9KHGPx4hw9LWtWmlf1+JKm679px/u4W0gXIka3tJdl21pXU6f/QWrjEUAMR8aSkuaQi96WSvkvq5bGCVJ3wPtIJ7O2NVTAN7gNulPRVUh30e0jdVT8XEb/P1nk/KfFcTEpCz5GqP94CXBQRT2XrfZnUYHuSpL1Jja6PkxqM9yGNU+h4uoaIeFjSD0nVLFOAcyLi6ZbVLiYlsssknUXqrfNmUuPlI202ewMwW9LHST2lIiIuHOP9H5X0L6ReRTdKOi97aR6pIfmIxsbVLmwLXJsdzztJ1SLbk3oW/YbVDbpj+RKwiaQFpKq75cDLSP+7bUkN+ndk6/4we/2bkk7L3msPUpXgYgZ7nniEdNy+RkpMh5I+Cx+MiLZViLCq5PM+0ufn9uzv7yJdhGwNvAP4JKkbrHUr725RfgzvQWpo/QxwKykRPEXqIXQysHmb9fdidZ/8j5CK789kPz/asu7OpCqcRaQr6MdJvX6OBSa3rLtutr1fZus+mW3zfGBOu/dfy369M1svgD3GWOftpO6UT5JORheSTkAP0ND9NFt3G1KbxOOj2214bY31s+UHAv/VsD//RUq0reuN9fdN+0pqqP1S9r96NPtfLSLV+b+0g//1HFKyui3b3xWkBu6rSY3867SsvyepN88T2ftdSmr4voY1u4mOtQ/HZ/swo2X5jGz58Q3L5mXLZpMakH+XfbbuBN7TZttrxJEtfzmpF9MDpIS/LPs/nwi8LO/vXFkfHuBmZkPXMMDtTRFxTb7RWCu3MZiZWRMnBjMza+LEYGZmTdzGYGZmTVxiMDOzJpUYx7DpppvGjBkz8g7DzKw0brrppkciYrN2r1UiMcyYMYOFCxfmHYaZWWlI+u1Yr7kqyczMmjgxmJlZEycGMzNr4sRgZmZNnBjMzKyJE4OZmTVxYjCrmAUL4Kij0k+ziXBiMKuQBQvg4IPh9NPTTycHmwgnBrMKueIKWJ7d92z58vTcrFtODGYVMmcOTM3uFD11anpu1q1KTIlhZsncuXDBBamkMGdOem7WLScGs4qZO9cJYagWLKhcJnZVkpnZRFW0td+Jwcxsoira2u/EYGY2URVt7Xcbg5nZRFW0td+JwcysFxVs7S9cVZKkl0m6WtLdku6S9NG8Y7Lh8FQOZsVQxBLDCuDYiLhZ0obATZKujIhf5R2YDc5o547ly+Hcc1PpvGIXYWalUbgSQ0T8MSJuzn5/Argb2CLfqGzQKtq5w6yUCpcYGkmaAbwWuDHfSGzQKtq5w6yUiliVBICkFwLfBY6JiMfbvH44cDjAlltuOeTorN8q2rnDrJQUEXnHsAZJ6wGXAJdHxBfXtv6sWbNi4cKFgw/MzKwiJN0UEbPavVa4qiRJAs4B7u4kKZiZWX8VLjEAewDvA/aWdGv22C/voMzM6qJwbQwR8TNAecdhZlZXRSwxmJlZjpwYzMysiRODmbXnOUpqy4nBzNZU0RvQWGecGMxsTZ6jpNacGMxsTZ6jpNYK113VbJgqeB/3/vAcJbVWyCkxulWnKTF8IkvGOw6dHqPGqb6nTvVU31YvpZoSw8bm9sBkvOPQzTFyNbpZe04MJeITWTLecejmGLka3aw9J4YS8YksGe84dHOMRqvRR0ZcjWTWyG0MJeM2hqQfbQxmdTZeG4MTg5nlwxk8V258NrNicU+KQnNiMLPhc0+KQnNiqDHPkWa5cU+KQvPI55pqHNx17rnulWND5pHVhebEUFPtSvL+btpQzZ3rD11BuSqpplySN7OxuMRQUy7Jm9lYnBhqzCV5qyyPkeiJq5KsMLrpJeUeVTYmj5HomRODFUI33+V+fe+dXCrKYyR65sRghdDNd7kf33tfVFaYe1b0zInBCqGb73I/vve+qKywbqfNddFxDZ5Ezwqjm/bCXtsWffc2A2r9QRhvEj33SrLC6KaXVK89qtxd1wCP9ByDE4PVlrvrGnPmpDlhRksMbo8AnBjMrM5cdGzLicHM6s1FxzW4V5JVjjuZmPXGiaGmqnry9PgEs945MdRQ1U6ejUnO4xPMele4xCDpa5IelnRn3rFUVZVOnq1Jbto0D3o161XhEgNwHrBv3kFUWZFnDOi2iqs1yT322PiDXqtahWbWVxFRuAcwA7iz0/V33XXXsO7Mnx8xMpJ+FsX8+RFTp0ZA+tlJbN38zXjrFvF4mA0SsDDGOKe6u2pNFbGH3kQGoXbTDX2s7fv+12bNiliV1BFJh0taKGnh0qVL8w7H+mCiVVxz58Jpp639ZD7W9qvU5mLWD6VNDBFxVkTMiohZm222Wd7hWB90Oylmv7Zf5DaXwnEjTS0UcnZVSTOASyJix07W9+yqg1WHuyTWYR97VuOZSKtovNlVC1dikHQB8HPglZKWSPpA3jHVWZnHPHRzcdtpdVTflekK3HVutVG4xBARB0fESyNivYiYHhHn5B1TnZX1XFCKhFaKIBu4zq02CpcYrFjKei4oRUIrRZANumkEKlNJyNbgxGDjGkSD8DDOGaVIaP0Kcpgn4U7q3MpWErI1jTXAoUwPD3Arj4kMYuvlvQo/aK3XIId5QDs1MpLiGX2MjOQdkbXBOAPcXGIYMJeomw2z9iS3BuVu9BpktwfUxTXrgBPDAHVboq5DEvE5o8+6OaDDquIZ9IAUGzhPiTFA3UzxUMVpGdqNDfCdFPusH3OCDCou/3NLyyWGAermYq5sHVTWZryL01JU8ZRJr3OCmLVwYhigbkrUVfvOVi3RVYKreKxDhZwSo1tVmRKjStMyePYEs2Ibb0oMJwYbmKInuqLHlwsflNpwYjBrUfoSzSBO4KU/KNaNUk2iZzYMpW4DGVS302EelDr0zS4xJwYbU5W/u/1o7M/t+AzqBD6sHhCeMqPwnBisrap/d3vtoJPr8RnUCXxYvZZKXVyrBycGa6us391h3YMh1+MzyBP4MAaZVK1vdgU5MVhbZfzuDvMqPvfjU+ZRgh5PUXieEsPaKuPUFcOe8aFsx6dQPGVGobm7qlXGIHpbulu/VdV43VVdYrDK6PdVfBUnNjTrhNsYxlDlrppV1s+q97I2wJv1yomhjap31bTO5N7APBZftdiAOTG04StFgz52nunnidxXLTYETgxtFPZK0Yau56qpfp/IfdViQ+DE0Ia7WRdbqWpS+n0ir/tVS6n++eXl7qpWKqWbANR9aPundP/8YvPsqgZU42KrdDUpgyh+lnnUcy9K988vLyeGmqhKm2Upa1LqeiLvt1L+88vJA9xqYpjTRQxSvwax1bU2ptQ8D8nQuI2hJlw9u5qPxRicLWvFbQwV0UsbQZ17WrUeN1dVt1GVukbrC1cllUQ/5u3pdELLKl04tjtu06bBuuvCihWuql6lKnWN1hcuMZTEsK5yq3bh2HrczjwTTj01JYVJk+CYY3z+A9ywa02cGEpiWN/bqlWztB43WL1/K1fCY4/lE9fQdFr/WOe6RltTRHT8AGYCPwZ+DXwRmNLw2i+62dZa3mdf4F5gEfCJta2/6667Rh3Mnx8xMpJ+DvI9pk6NgPRzvPfqNZ5h7E/r+3Szf6VXq521bgELY6xz8FgvtF0ZfgiMALsC5wHXAxtmr93SzbbGeY9JwGJgK2B94DbgVeP9TV0Sw7B0csLu9ZyT5zlrWAkpdyMj6QCPPkZG8o7ICmS8xNBtVdJLIuL0iLgpIuYBlwA/ljQN6Fe/19cDiyLi1xHxLHAhcECftm0d6GQ8Vq9VTnlWWdVmvJnbDWyCuk0MkxufRMSJwEWk6qUN+xTTFsDvG54vyZY1kXS4pIWSFi5durRPb22d6vWc43PWELjdwCaoqwFuki4GzoiIK1uW/xNwckT03Jgt6SDgLRHxwez5+4DXR8TRY/2NB7jlo9durVXqFmt94g/F0Iw3wG2tiUHSLhFxc/b7ZICIeKbNeltExIN9CHZ34PiIeEv2/JPZe5441t84Mdig+Dw1RB6SPlS9jny+WtKbICWEdkkhe63npJD5JbCNpJmS1gfeDZS8N72VUdXGdBRe1fpKl1gnieFbwGWS3tn6gqQ3SPpZPwOKiBXAUcDlwN3ARRFxVz/fw+qrm2lFfJ4aMjc8FcZaE0NEfBg4EbhQ0pEAknaS9APgp8Bf9TuoiLgsIraNiFdExOf7vX2rp25LAD5PDVlrYzmU/wYiJdVRY3FE/G/gSODfJV0L3ALsCBwG7DS48Mz6p9sSgDv15GC0LzG4Hi9HHSUGSRsD2wIrgTcCNwDbRMR5EfH8AOMz65uJlABqM+ahaFyPl6u1JgZJnyVNgTECnEIqJcwiTYlhFVGF236ujUsAJeJ6vFx10l31WeBs4ISI+O9s2d7AxcCPgPdGxHODDnQ87q7aG/cStEJyX+GBGq+7aif3Y9g+IhY3LoiIn2RdWC8jJYd9eg/T8uKp+K2QOr2BiPVdJ72SFo+x/GbgDcCMPsdkQ+ZSu5k16ukObhGxSNLf9CsYy4fvsW5mjXq+tedou4OVm0vtq7lq2+rOd3Aza+BpMMycGMyauPt8gzr0Yba2nBjMGozbEF+nE6WLTrXmxGDWYMxBcHU7UbroVGtODGYt2k6DMd6JcpgliWG9l/sw11rPvZLKzL1PrGNz5sC5564eHj56omwcNn7uuatnBR3EB+u44+ALX4Dnn4ezz4aLLhrcB9d9mOstIkr/2HXXXaNb8+dHTJ0aAenn/Pldb2LM7Y6M9G97ZTLevlfiuLTbiZGR9CEafey33+A+WJMmrfleZhMELIwxzqm1LTEMYhqIdhePdbnQGu/Cedo0OPXUChyXdoM9WksSMJj5Ra64Alau7H07Zh2obRvDIKpQ69xe17rvZ565uq32X/+1wseltbX6iCMGUzc/Zw5Mnrz6+brrpvcyG4DalhgGUYU6VjV0HYx34bxyJUyalH5W8ri0liQGUTc/d25qUzjzzPT8iCNKWuzKuIGv0NY67XYZFGna7Tp/3hv3HeCgg+DZZ9Pv666blpf9fGZ94HneC6HXabetC3Wec6h132fPhssuS7+vWAEzZ9b32FgDz/NeeLVtY7DBG1R1u/Uo7xHcHiNReK5KsoGqc9VaIRWlGscfjNy5KslyU+eqtUIqSjWOPxiF5qoks1F5V7EMg6txrAMuMZhBfUYneqoL64ATgxkUp4plGFyNY2vhqiQzqG4VSx2qx6zvnBgsF4U7X415I4YuFG2n6nYPCesbJwYbusKer9reiKFDRdypPCfvKlqStK44MdjQVXKywSLuVF7VY0VMktYVJwYbuoGerxYsgLe9LT2GeUIqYhtFP6rHJqKISdK64pHPlouBDHxdsADe9S545pn0fP314TvfGd4J0aN5k6KMrrZxlWbks6SDgOOB7YHXR4TP9hU1kB6TV1yxOilAmtp1mN1O3Q008ViJ0itUYgDuBN4BnJl3IFZCc+akeyE3lhiKUKVTF60lJieE0ipUYoiIuwEk5R2KlVHVbmZTJnUZOV4ThUoMVl1Dq34v05Vqldok6jRyvAaG3itJ0lWS7mzzOKDL7RwuaaGkhUuXLh1UuNYH7r3YRtUOShF7ZdmEDT0xRMTsiNixzWN+l9s5KyJmRcSszTbbbFDh2gS0jm0aVu/FUo2pqlqXzry6xtpAeByD9VW7C+FhXEyW7gK8ilfYvYwct+4M+CqoUIlB0oGSlgC7A5dKujzvmKw7Y1U1D/pisnQX4L7CtokawlVQoRJDRFwcEdMjYnJEvCQi3pJ3TNadsS6EB30xWcoLcF9h20QM4SrIvZKsr/Ia2+QxVVYbc+akLsGjI8sHcBXkKTHMzMqmD12dSzMlhpmZdWDA43UK1cZgZmb5c2IwK6JSDcqwqnFisFKpxfmydIMyrGqcGKw0anO+LN2gDKsaJwYrjdqcL0s5KMOqxInBSqM250uPiraceRyDlUqVZqo2y5PHMVhllOl2C2Zl5aokMzNr4sRgZmZNnBjMzKyJE4OZmTVxYuhRLUbimlmtODH0oDYjcc2sVpwYelCbkbhmViuVHcfw3HPPsWTJEp5++umBvcehh8L++0MESLDppnD33QN7u4GZMmUK06dPZ7311ss7FDMrgMomhiVLlrDhhhsyY8YMJA3sfR59FB5/HDbaCF70ooG9zcBEBMuWLWPJkiXMnDkz73CsFx4Wbn1S2aqkp59+mk022WSgSQFSMthyy3ImBQBJbLLJJgMtWdkQuMHL+qiyiQEYeFKoCh+nCnCDl/VRpRODlYT7/PauNlPP2jBUto3BSmK0CmT5cjj3XE8zPVGjU3W7jcH6wInB8tWuCsQntYnx1LPWJ65KKpnDDjuMF7/4xey44455h9IfrgKxIqp59aYTQ8nMmzePH/3oR3mH0T++W5kVjXt4OTE06vdFwh133MEee+yx6vnNN9/M3nvv3dM299xzTzbeeONeQyuWuXPhtNOcFKwY3MPLbQyjBtEGusMOO7B48WJWrlzJpEmTOPbYYznllFParvvGN76RJ554Yo3lJ598MrNnz+4tEDPr3Jw56SSwfHltqzedGDKDaANdZ5112GGHHbjrrru4//772XLLLdlll10A+PSnP83nPve5Veted911vb2ZmfWHe3g5MYwa1EXCbrvtxvXXX88ZZ5yxqm3goYceYsWKFU3rucRgViA17+HlxJAZ1EXCbrvtxrx58xgZGWGLLbYA4JZbbmHnnXduWs8lBjMrCjc+NxhEG+h2223H5MmT+fjHP75q2a233rpGYujUwQcfzO677869997L9OnTOeecc/oVqpkZULASg6STgP2BZ4HFwKER8Wi+UfXmy1/+MieeeCIbbLDBqmWLFi1im222mdD2Lrjggn6FZmbWVtFKDFcCO0bEq4H7gE/mHM+ELV68mO22246nnnqKQw45pOm1c845h3XWKdqhNzNLClViiIjGDsM3AH+fVyy9esUrXsE999yTdxhmZl0r8mXrYcAPx3pR0uGSFkpauHTp0iGGZVZRNZ8GwlYbeolB0lXA5m1eOi4i5mfrHAesAM4fazsRcRZwFsCsWbNiAKGa1YdnubUGQ08METFup3xJhwB/B+wTET7hmw2DZ7m1BoWqSpK0L/BxYG5ELM87HrPa8Cy31qBQjc/AacBk4MrsdpM3RMSR+YZkVgOeBsIaFCoxRMTWecdgVls1nwbCVitUVZJVkHu6mJWOE4MNjm94YlZKTgwVl+utQH3DE7NScmKouFxvBeqeLmvnqjYrICeGRn3+kg7i1p7dyvVWoL6f8/jKVtXmJFYbheqVlKsBjPz0rT1xT5fxlGlQmUdG14oTw6gBfEkHeWvP2bNn89BDD62x/POf/zwHHHBAT3HbkJTp3sJlSmLWMyeGUQP6kg7q1p5XXXVVX+KzHJVpUFmZkpj1zIlh1IC+pL61p42rLFVtZUpi1jM3PjcawL09+31rz271fCtQNzjaqEHc+9YKySWGAev3rT271dOtQN3gaFZLLjEMSCVu7ekBama15BLDgFTi1p5ucDSrJScGG5sbHM1qyYnBxleWXjNm1jclqOieON8ZtDM+TmbWqLKJYcqUKSxbtswnvbWICJYtW8aUKVPyDsXMCqKyVUnTp09nyZIlLF26NO9QCm/KlClMnz497zDMrCAqmxjWW289Zs6cmXcYZmalU9mqJDMzmxgnBjMza+LEYGZmTVSFXjuSlgK/zTuOPtgUeCTvIPrI+1Ns3p9iG/T+vDwiNmv3QiUSQ1VIWhgRs/KOo1+8P8Xm/Sm2PPfHVUlmZtbEicHMzJo4MRTLWXkH0Gfen2Lz/hRbbvvjNgYzM2viEoOZmTVxYjAzsyZODAUj6SRJ90i6XdLFkl6Ud0y9kHSQpLskPS+plF0JJe0r6V5JiyR9Iu94eiXpa5IelnRn3rH0StLLJF0t6e7sc/bRvGPqhaQpkn4h6bZsf07IIw4nhuK5EtgxIl4N3Ad8Mud4enUn8A7gp3kHMhGSJgGnA28FXgUcLOlV+UbVs/OAffMOok9WAMdGxPbAbsBIyf8/zwB7R8RrgJ2BfSXtNuwgnBgKJiKuiIgV2dMbgFLPhx0Rd0fEvXnH0YPXA2YrRfAAAAL8SURBVIsi4tcR8SxwIXBAzjH1JCJ+Cvwp7zj6ISL+GBE3Z78/AdwNbJFvVBMXyV+yp+tlj6H3EHJiKLbDgB/mHUTNbQH8vuH5Ekp84qkySTOA1wI35htJbyRNknQr8DBwZUQMfX8qez+GIpN0FbB5m5eOi4j52TrHkYrJ5w8ztonoZH9KTG2WuY93wUh6IfBd4JiIeDzveHoRESuBnbP2xYsl7RgRQ20PcmLIQUTMHu91SYcAfwfsEyUYaLK2/Sm5JcDLGp5PB/6QUyzWhqT1SEnh/Ij4Xt7x9EtEPCrpGlJ70FATg6uSCkbSvsDHgbkRsTzveIxfAttImilpfeDdwIKcY7KMJAHnAHdHxBfzjqdXkjYb7Yko6QXAbOCeYcfhxFA8pwEbAldKulXSV/MOqBeSDpS0BNgduFTS5XnH1I2sI8BRwOWkhs2LIuKufKPqjaQLgJ8Dr5S0RNIH8o6pB3sA7wP2zr4vt0raL++gevBS4GpJt5MuSq6MiEuGHYSnxDAzsyYuMZiZWRMnBjMza+LEYGZmTZwYzMysiRODmZk1cWIwM7MmTgxmPZK0taTnWqdIlvQfkp4o63TjVl9ODGY9iohFwNnAxyRtCiDpM6RJEA+MiIV5xmfWLQ9wM+sDSZsDi4EzSFMYnAUcHBEX5RqY2QS4xGDWBxHxEHAqcDRwJvCRxqQg6VPZXeCel/T2vOI064QTg1n/3A9MBn4eEae3vPZjYD9Keic7qxcnBrM+kLQ3qaTwc2APSa9pfD0iboyIxbkEZ9YlJwazHknaBfg+qQF6L+B3wP/JMyazXjgxmPVA0tak269eARyd3Rf6BGA/SXvmGpzZBDkxmE1Q1hPpCtJ9Gv4hIp7PXvoGqWfSF/KKzawXvrWn2QRlPZG2arN8JbD98CMy6w+PYzAbAkn/CzgS2Ax4AngamJUlF7NCcWIwM7MmbmMwM7MmTgxmZtbEicHMzJo4MZiZWRMnBjMza+LEYGZmTZwYzMysiRODmZk1cWIwM7Mm/x9sm6EgrEsubgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_noise = x + np.random.normal(scale = 0.2, size = (N_pts,2))\n",
    "\n",
    "plt.scatter(*x_noise[y == 1].T, color='b', s=10, label=r'$y_i=1$')\n",
    "plt.scatter(*x_noise[y == -1].T, color='r', s=10, label=r'$y_i=-1$')\n",
    "plt.xlabel(r\"$x_1$\", fontsize=16)\n",
    "plt.ylabel(r\"$x_2$\", fontsize=16)\n",
    "plt.title(\"Observations Sample\", fontsize=18)\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w estimated :  [-0.8911448   0.45371901]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO2deZgU1dX/P6d7NoZhWIcdZFgUwQUUQYO4K6iJmsRE1OSnicb4vprlNW+ixqiJZjGaqMkbEiUJ0SQal0QNiSgaRVyRfRFxYECWkWUGZt+3+/ujq2qqq6t7eoYZBqrP53nmme6qW1X3dld/69xzzr1XjDEoiqIowSXU0xVQFEVRuhcVekVRlICjQq8oihJwVOgVRVECjgq9oihKwFGhVxRFCTgq9ErKIiLVIjI2wf5HROTOg7zGWSJS1JX16ux5ldQlracroChuRGQ7MARocW1+zBhzc1dfyxiT087+G7v6msnQXr0UpaOo0CuHI58xxvynJysgImFjTEv7Jbv0mmnGmOZDeU0lNVDXjXLEICLXisg7IvKQiJSLyDYR+ZS1fZeIFIvINa7yj1nul1dFpEpElorIUa79RkTGu8r+TkQWiUgNcLa17ceu8peKyFoRqRSRrSIyx9r+FRHZZF1jm4h8vQNtMiJyk4hsAbb41OsiEfnQOvcnIvK/cc7zTavcyI59qkoqoEKvHGnMANYDA4EngaeAU4DxwJeA34iI2/VxNXAvMAhYCzyR4NxXAT8B+gBvu3eIyHTgz8B3gX7AGcB2a3cx8GkgF/gK8JCInNSBNl1mtWuSz74/Al83xvQBjgNe9xaw4gjXAmcaY9Rvr8SgQq8cjrxgWez239dc+z42xvzJcqs8DYwC7jHGNBhjXgEaiYi+zYvGmDeNMQ3AHcBpIjIqznX/aYx5xxjTaoyp9+y7DlhgjHnV2v+JMeYjAGPMi8aYrSbCUuAVYFYH2vszY0ypMabOZ18TMElEco0xZcaY1a59IiIPArOBs40xJR24ppJCqNArhyOXGWP6uf5+79q3z/W6DsAY493mtuh32S+MMdVAKTA8znV3xdkOkQfKVr8dInKhiCwTkVIRKQcuItKDSJZE1/28db4dluvpNNe+fsANRB4UFR24npJiqNArQcex3i2XzgBgd5yyiaZy3QWM824UkUzgH8AvgCHGmH7AIkA6UMe41zXGrDDGXAoMBl4AnnHtLiPiMvqTiMzswPWUFEOFXgk6F4nI6SKSQcRX/74xJpEFHY8/Al8RkXNFJCQiI0RkIpABZAIlQLOIXAhc0BUVF5EMEblaRPoaY5qASqLTTjHGvEEkDvG8iMzoiusqwUOFXjkc+Zc1aMj+e/4gzvUkcDcRl83JRESxwxhjlmMFWoEKYClwlDGmCvgmEUu7jEhAd+FB1NfLl4HtIlIJ3Egk4Oyt26tW3RaKyMldeG0lIIguPKIEFRF5DCgyxvygp+uiKD2JWvSKoigBR4VeURQl4KjrRlEUJeCoRa8oihJwDrtJzQYNGmTGjBnT09VQFEU5oli1atV+Y0ye377DTujHjBnDypUre7oaiqIoRxQisiPePnXdKIqiBBwVekVRlICTlNCLyBwRKRCRQhG5zWf/jSKywZqr+20RmWRtHyMiddb2tSLySFc3QFEURUlMuz56EQkD84DzgSJghYgsNMZ86Cr2pDHmEav8JcCDwBxr31ZjzJSurbaiKIqSLMlY9NOBQmPMNmNMI5GFHi51FzDGVLre9ibxLICKoijKISQZoR9B9HzZRda2KKzl0LYC9xOZ5MkmX0TWWHNp+y7GICI3iMhKEVlZUqJrJyiKonQlyQi937zaMRa7MWaeMWYccCtgTyK1BxhtjJkK3AI8KSK5PsfON8ZMM8ZMy8vzTQNVFEVROkkyQl+Ea/EGYCTxF26AiGvnMgBrebcD1utVRFboObpzVe0cOw7U8PaW/YfykoqiKIcVyQj9CmCCiORbizfMxTPftohMcL29mLbV7POsYC4iMhaYAGzrioony5kPvMGX/vj+obykoijKYUW7WTfGmGYRuRlYDISJLJC8UUTuAVYaYxYCN4vIeUQWMi4DrrEOPwO4R0SaiayMc6MxprQ7GqIoiqL4k9QUCMaYRUTWwXRvu8v1+ltxjvsHkfU0u536phbeKChh0rBcRg/MPhSXVBRFOSIIzMjY6oZmbvzrKt7YXNzTVVEURTmsCIzQhyWSHNTSqin8iqIobgIj9KGQCr2iKIofgRH6sCX0rbpilqIoShTBEXrHddPDFVEURTnMCIzQh6yWqEWvKIoSTWCEXoOxiqIo/gRH6DUYqyiK4ktghF5EEFHXjaIoipfACD1E3Dcq9IqiKNEESuhDIpp1oyiK4iFYQh9S142iKIqXQAl9WESDsYqiKB4CJfShkAq9oiiKl0AJfTikwVhFURQvwRJ6dd0oiqLEECihD6lFryiKEkOghF4tekVRlFiCJfSh+Hn0Ri19RVFSlEAJfaI8etV5RVFSlaSEXkTmiEiBiBSKyG0++28UkQ0islZE3haRSa59t1vHFYjI7K6svJdQgikQVOcVRUlV2hV6EQkD84ALgUnAlW4ht3jSGHO8MWYKcD/woHXsJGAuMBmYA/zWOl+3kMhHr64bRVFSlWQs+ulAoTFmmzGmEXgKuNRdwBhT6XrbmzYD+lLgKWNMgzHmY6DQOl+3kCjrRmO0iqKkKmlJlBkB7HK9LwJmeAuJyE3ALUAGcI7r2GWeY0d0qqZJkNCiV+eNoigpSjIWvfhsi1FNY8w8Y8w44FbgBx05VkRuEJGVIrKypKQkiSr5E0qYddPp0yqKohzRJCP0RcAo1/uRwO4E5Z8CLuvIscaY+caYacaYaXl5eUlUyZ+wzl6pKIoSQzJCvwKYICL5IpJBJLi60F1ARCa43l4MbLFeLwTmikimiOQDE4DlB19tfxIHY7vrqoqiKIc37frojTHNInIzsBgIAwuMMRtF5B5gpTFmIXCziJwHNAFlwDXWsRtF5BngQ6AZuMkY09JNbUkYjFUfvaIoqUoywViMMYuARZ5td7lefyvBsT8BftLZCnaERBa9Zt0oipKqBGxkrObRK4qieAmW0Et8X7zKvKIoqUqghD4cElp0rhtFUZQoAiX0IRGaWw1vFBTHumpU6BVFSVECJfThkLBuVznX/mkFz6/5JGqf5tcripKqBEvopW0g7v7qhqh9KvOKoqQqgRL6UKhN6LPSoyfJ1KwbRVFSlUAJvduiz0yLbprKvKIoqUqwhD6hRX+oa6MoinJ4ECihd7tuYi16VXpFUVKTQAl92DUpcoZX6FXnFUVJUQIl9G6L3osKvaIoqUqwhN4VjP3qYyv5v9e2OO/VdaMoSqoSKKF3Z90AzH9zm/NaLXpFUVKVQAm913Vj4rxWFEVJJQIl9GFPa9yDpHTAlKIoqUqwhN7junFPTa86ryhKqhIooY913bgt+kNdG0VRlMODQAm916J3i7tm3SiKkqoES+i9Fr26bhRFUYIl9OK16N2um0NdGUVRlMOEpIReROaISIGIFIrIbT77bxGRD0VkvYi8JiJHufa1iMha629hV1beS0VdU9T76GCsSr2iKKlJWnsFRCQMzAPOB4qAFSKy0BjzoavYGmCaMaZWRP4LuB+4wtpXZ4yZ0sX19qW4sj7qvVvcW1XnFUVJUZKx6KcDhcaYbcaYRuAp4FJ3AWPMEmNMrfV2GTCya6uZHPuqPEKf4J2iKEqqkIzQjwB2ud4XWdvicR3wkut9loisFJFlInKZ3wEicoNVZmVJSUkSVfKnuNKzfKAGYxVFUZISer8pIX1lU0S+BEwDHnBtHm2MmQZcBTwsIuNiTmbMfGPMNGPMtLy8vCSq5M9DV0xh7KDevvtU5xVFSVWSEfoiYJTr/Uhgt7eQiJwH3AFcYoxxTGtjzG7r/zbgDWDqQdQ3ITPHD+JvN5zqu08tekVRUpVkhH4FMEFE8kUkA5gLRGXPiMhU4FEiIl/s2t5fRDKt14OAmYA7iNvlSJwp6VtV6RVFSVHazboxxjSLyM3AYiAMLDDGbBSRe4CVxpiFRFw1OcCzVi77TmPMJcCxwKMi0krkoXKfJ1unywnFUXrVeUVRUpV2hR7AGLMIWOTZdpfr9XlxjnsXOP5gKthR4q0xpVMgKIqSqgRqZCyoRa8oiuIlZYReURQlVQmc0EucFqlFryhKqhI4oY9n0WvWjaIoqUrghD5+MFZRFCU1CZzQxw/GqtQripKaBE7o48ViVeYVRUlVAif0ml6pKIoSTQCFPt4eVXpFUVKTwAm9dzlBG114RFGUVCVwQh/PolfXjaIoqUrghD6eRa9ZN4qipCqBE/p4qMwripKqpI7Qq9IripKipI7Qq02vKEqKkjpCrzqvKEqKokKvKIoScFJH6NV1oyhKipI6Qq86ryhKipI6Qt/TFVAURekhUkbodeERRVFSlaSEXkTmiEiBiBSKyG0++28RkQ9FZL2IvCYiR7n2XSMiW6y/a7qy8h1CdV5RlBSlXaEXkTAwD7gQmARcKSKTPMXWANOMMScAfwfut44dANwNzACmA3eLSP+uq37yaDBWUZRUJRmLfjpQaIzZZoxpBJ4CLnUXMMYsMcbUWm+XASOt17OBV40xpcaYMuBVYE7XVL1jqOdGUZRUJRmhHwHscr0vsrbF4zrgpY4cKyI3iMhKEVlZUlKSRJU6jgq9oiipSjJC7zcdpK9sisiXgGnAAx051hgz3xgzzRgzLS8vL4kqdRzVeUVRUpVkhL4IGOV6PxLY7S0kIucBdwCXGGMaOnLsoUCzbhRFSVWSEfoVwAQRyReRDGAusNBdQESmAo8SEfli167FwAUi0t8Kwl5gbTvkqM4ripKqpLVXwBjTLCI3ExHoMLDAGLNRRO4BVhpjFhJx1eQAz1oLf+w0xlxijCkVkXuJPCwA7jHGlHZLS9pFlV5RlNSkXaEHMMYsAhZ5tt3len1egmMXAAs6W8GuQi16RVFSlZQZGas6ryhKqpI6Qq9KryhKipIyQq9ZN4qipCopI/Qq84qipCqBFvqZ4wcye/IQAN7aXMI7hft7uEaKoiiHnkAL/W+vPplb50wE4NlVRVz9h/d7uEaKoiiHnkALfUjAyutXFEVJWQIu9OI72Y6iKEoqEXihD3ks+qWbu2d2TEVRlMOVQAu9SOTPzTULlvdMZRRFUXqIQAu915q3MZpTryhKChFwoY+16AEamlsPfWUURVF6iIALvfhm3TQ0qdAripI6BFroRfyXuKpvbjnkdVEURekpAi70sVk3APVNKvSKoqQOgRZ68PfR16vrRlGUFCL4Qu+zrUFdN4qipBCBF3o/pVeLXlGUVCLwQu83CYL66BVFSSUCL/QhzaNXFCXFCbzQ++XRq0WvKEoqkZTQi8gcESkQkUIRuc1n/xkislpEmkXkcs++FhFZa/0t7KqKJ4tvHr0KvaIoKURaewVEJAzMA84HioAVIrLQGPOhq9hO4Frgf31OUWeMmdIFde0UvumV6rpRFCWFaFfogelAoTFmG4CIPAVcCjhCb4zZbu077BTULxjboBa9oigpRDKumxHALtf7ImtbsmSJyEoRWSYil/kVEJEbrDIrS0q6eL54l85/dWY+oMFYRVFSi2SE3s/N3ZF5fkcbY6YBVwEPi8i4mJMZM98YM80YMy0vL68Dp24fd9bN6AG9EFEfvaIoqUUyQl8EjHK9HwnsTvYCxpjd1v9twBvA1A7U76BxZ92EwyGy0sIq9IqipBTJCP0KYIKI5ItIBjAXSCp7RkT6i0im9XoQMBOXb/9Q4O6OpIWEjLQQjeq6URQlhWhX6I0xzcDNwGJgE/CMMWajiNwjIpcAiMgpIlIEfAF4VEQ2WocfC6wUkXXAEuA+T7ZOt+POugmHhHBIaNEVphRFSSGSybrBGLMIWOTZdpfr9QoiLh3vce8Cxx9kHQ8Kd9ZNmi30atAripJCpMDI2LbX4ZAQFqGlNb7S76us56FXN+u6soqiBIaUEvq0UKhdi/4bf1vDr17bwsbdld1fOUVRlENAUq6bI40nrp/BrtJaINp1Y/vo/7G6iPxB2dx8zoSYY2sbmwFoVYteUZSAEEiLfub4QcydPhrwWvRCmpVY/4tXNvsea+u734haRVGUI5FACr0bt1yHw0LIb95iF47Qq84rihIQgi/0Ep11k+YSer+AqzpsFEUJGoEX+pAn6ybkEv46nxGytviH1KRXFCUgBF7ooy36EGnhtvdltU0x5dV1oyhK0Ai80LvxWvTltY0xZYzlvGlpVSeOoijBIKWE3uujL09g0Wt6paIoQSGlhN7Oo7fxFXrrv1r0iqIEhZQS+rRwtNCX1jTElLGDsWrRK4oSFFJK6MMSLfT7Kn2E3vqvE58pihIUUkLoszPCAIQ8rpu9lfWxhS2lV9eNoihBISWE/tSxAwFobTVRwdi9FbFCb8u7um4URQkKgZzUzMuv5k5h0YY9jB+cE5Ve6WfR2z56tegVRQkKKSH0fbLSueKUyCRn7gFT+xJY9LoKlaIoQSElXDdu3BZ9VUMzTZ6oq5NHrxa9oigBIeWEPs0ze6W9ULgxhmXbDjguG3XdKIoSFFLCdePGnqY4OyNMbWMLjc2t9M6E1TvLmTt/mVNOg7GKogSFpCx6EZkjIgUiUigit/nsP0NEVotIs4hc7tl3jYhssf6u6aqKd5aw2EIfecY1Wq6byrroUbKaR68oSlBoV+hFJAzMAy4EJgFXisgkT7GdwLXAk55jBwB3AzOA6cDdItL/4KvdeULSZtEDNDRFFL26oTmqnAZjFUUJCslY9NOBQmPMNmNMI/AUcKm7gDFmuzFmPeC1g2cDrxpjSo0xZcCrwJwuqHensWentIW+sSUyJ71X6DUYqyhKUEhG6EcAu1zvi6xtyZDUsSJyg4isFJGVJSUlSZ764HAseisYW13vsehV6BVFCQjJCL3fEhzJqmBSxxpj5htjphljpuXl5SV56s5he2QcH70l9FXqulEUJaAkI/RFwCjX+5HA7iTPfzDHdiu92rHo1XWjKEpQSEboVwATRCRfRDKAucDCJM+/GLhARPpbQdgLrG09hi3fvS2hr2ts4YuPvMeCdz6OKqcWvaIoQaFdoTfGNAM3ExHoTcAzxpiNInKPiFwCICKniEgR8AXgURHZaB1bCtxL5GGxArjH2tbj9LJcN/urG1i+PbZKatErihIUkhowZYxZBCzybLvL9XoFEbeM37ELgAUHUccupc1HH7Hovdk2NhqMVRQlKKTcFAh2emVWeqTpXt+8TYul83sr6tldXndI6qYoitIdpNwUCLaTPjPNsugb/YXedt2c+rPXANh+38XdXzdFUZRuIOUsepvMtGiL/pvnjOf+z5/g7NdgrKIoQSHlLHpbvjNsobd89CeM7Ef/3ulOufZ89A3NLTS3GHpnptxHqCjKEUYKW/SW68ay6NPTQs42iLhu9lTE980/8HIBV//h/YTXqKht0uwdRVF6nJQVetui31/TGHkfDjnuHIi4bk772esxxz38n82Mue1FPt5fwz7XUoR7K+r54cKNNFvTXpZUNXDiPa/wf68XdmczFEVR2iXlhN5eE9Zef2TdrnIAMtKErPQ2i765xd8Sf2TpVgDK65pocpW54/kNPPbudt7ZegCI5OcDLNqwp2sboCiK0kFSTuhtxDMLT0Y4HGXRV9U34Yc9ZUJVfRPNrW2TddoLldgWfdh6kmhQV1GUniblhD6e7KanSZSPvtInv/5AdYMz4KqqvjnK6k8LRz5K28q3573XgVeKovQ0KZcyYgu1eCbWzAiHyEyPb9GPue1Fx0qP7I9eWDw9HNlnW/mOhd+qS1UpitKzpKxFLwKPfvlkZ3u6JxhbWRdr0but8+qGZppd79NCkWNtK99+CKjOK4rS06Sc0LuZdlTbqoaZaSFEhO33XUyfrDQq4/jo3bS0Gie4mxayLXrj7Iu8V6VXFKVnSTmhv+70fDLCIU4bN9CZkx4iFr1NOCRUxZkDx4vtk0+zXTeWJW9vVx+9oig9Tcr56KeM6sfmn1wIRE9FnOFy24RFkrLoIWKxZxAiHLKDsRGhtwVfhV5RlJ4m5Sx6NyFXcNVt0Yc6YdHbwdj6pmiBP5KFvqK2iR8u3EhDc0tPV0VRlIMgpYXejS3UELHok8W23O1gbF1TRBSbAiD0D75awGPvbuf51Z902zUef3c772870G3nP9z4pLyOlz/Y29PVSDl2ldY68bRURIXeQlzi7k6jbA87+GrPc19vCb39AGhuNUesRWw/rJq68WF198KNXDF/Wbed/3Djs/Pe4ca/rkpp0TnUrNpRyqz7l/DsyqKerkqPoULvQ8j1qcyZPDRh2TaffOSHa1v09gOgobmVY37wMqt3lnVDTbsXu2fTGVFqamnlnF++wX8+3NfV1TqiKa6KTI3RFGeKjfZoaG6hNs4aCoo/hcXVAL5LhqYKKvQ+2AJ31MBsrpg+KmFZW+BtYbd99N65clZ8fOTdZHbHpjMzcB6obmRbSQ3ff35DF9cqGHS2l3fRr95i0l2Lu7g2PUNpTSPPr+l+K9vurbceJr2o+qaWQ97LV6H3wQ7Sjs/LISMc/RFdf3p+1Hs7T9521TiuG0/+fEfcQYcL9g+kM8an8Uw2YYyhxrM+b0ceIDUNzYFyd9hzJnWUrSU17Zb517rdfP5373bq/IeSm55Yzf88vY6istpuvY49xuVwmTJ84p0vM/uhNw/pNVXoEzB+cE5U2iVAbq/0qPdNMRa97aP3Cl131TIS8H1+TVGX3chtM3wehOumOfqY37+1jcl3L3Zm9QRobElO7HYeqGXy3Yv52/JdccsYYw6bB0F9Uws/W7Qp7sLz0HmhT4Zv/G0Nq3aUOcZHe8xbUsi7W/d3W33iseNA5KHV3QkLbRMMtl+2trGZ25/bwF+W7eCTblwrevuB7n24eUlK6EVkjogUiEihiNzmsz9TRJ629r8vImOs7WNEpE5E1lp/j3Rt9bsHezHwcXk5jjVg09cj9Bf+6i2u+v0yx1df39TCHc9v4DlPl7SiLrm8/M7w1pYS/ufpdazqgjjAwnW7yb99EcWV9W2um04IqN01tY98/N0dAJTXNrrKJCdEW/dHfKwvfRB/yuev/2UV+bcv6nA9u5p3C/fz1pb9PPrmNt4pjC+eDU2Ju+5bS6r54JMKtpZUd7ou9Ul8vsYYHlhcwFW/T7yITnfgBPuTfCAdLG5DaOKdL/HDhRtjyjyzYhd/W76TO1/4gC8+8l6XXr+0ppFdpYdW4G3aFXoRCQPzgAuBScCVIjLJU+w6oMwYMx54CPi5a99WY8wU6+/GLqp3l/H4V6fzwOUnRG2z/exTRveLETmv0AO8u/WAY8GX1zXxxPs7eacwOmWwzCVwB0t9UwvFrkVPdpdHXieyIBMxb0khBXurAPjF4gIAdpXVOi6sxk5Yn14RL7Es+UaXpe8+72/fKIxrQSXj9HolQdDXGMOaQxAML66q56o/vM+Nf10FQFlN/O+8vYfcub9cyqf/723O/eXShD2VF9Z8Elc82nuYAJTXHpwBUt/UwpjbXuSvy3Z0+Ng24yjy/8r5y/hSO6u2dQa7191qDMWV9Yy57UXqm1p57N3tMWXdVv9e12+sK7jgoTeZdf+SmO0VdU1s2VfVpdfykoxFPx0oNMZsM8Y0Ak8Bl3rKXAo8br3+O3CuSAeS0XuQM4/O4wvT/AOuEwbnxLhgRg3o5VvWvmk/3F3pu7+8Cy36Be98zJxfveUIgH1DNjS1ice2kmq+/dSadkW6uqGZBxYX8IVHIj7dnZZo1Da2OBZQXRKC4cUbbLLr4T6Xu8z9Lxdw7YLlvudqcyF1uBoAPLf6Ez7723d5OUGPoCvYXxURdtsVUZrg4d6Rh2c8MS6vbeTbT69l1v1LuOdfH8bsT8ai31d1cGJm1+1nizZ1+Nim5ui41nvbDvB2gl5QZ7E/65ZWw+sfFSd9XFfE1RqaW/jM/73N8o9Lo9yWbr74yHuc380++2SEfgTgdo4WWdt8yxhjmoEKYKC1L19E1ojIUhGZ5XcBEblBRFaKyMqSkpIONaA7uPHMcVx3ej4iwvB+0cI+ZmBv32NsazSepVbh+bHWN7Xw3WfXOW4iLxt3V8QNUhVXNlBa0+j8yPZW2NduE85b/7GeF9bubjets7Q6IkaV9dHBzoq6Jqct9U0Hb9HbPLtyl1NPr9jtiGOZ2iaDN8Brs3hj2wAkPzeA/SBcs7M8caUPkgM10T/kjlj0f1m2gwcWf8RLPiuS+VmWLa2GD/e0GRUL3vk4pkx9Eg/ofZX+4pOIf6/fzTMrI5JQY6V61jR23Bho8mSqdYa6xhaeW12EMYbmllZe27QvpgfUaN1vLa2G3RWJH2xuaU8LSZSrsTNs31/Lhk8quCNB9lmBZc13Z6wiGaH3e6x5axSvzB5gtDFmKnAL8KSI5MYUNGa+MWaaMWZaXl5eElXqXm67cCJ3fjrinRo1IJtVPziPW+dM5PGvTic9zf8j21JcTS/XUoQ2j391OqePH8SeirqoL3LNznKeXVXE/zy9luqGZk772Wu8t7XN3XPxr9/m7F+84XutOutHZVtjeypiLXo7ONyef700js+8oq7J1wpPFvtcxhA1b9BTK3Zx/8sR95A3GBvPynUGpbmaYoxhQ1EFxhi+/pdVzva6phYKi6ujfuwDe2cAbTnsHSFezvr2/TUxy0QWe0TzQEKhb/tMK2qbuPOFD5i3ZCv/9cTqmKD6Ph+hb2xuZdOexN395IS+4xb9zU+u4Xt/Xw8Qk0nVEZo8mWo2iQSvYG8Vr3/U5qb76aJN3PLMOt7bdoC/rdjFdY+v5J9rd0cdY9+LrcbENaz8rl3b2MKUe151HmqdwU7aazEmJt73SXkdv35ti/O+phvHRyQj9EWA27cxEtgdr4yIpAF9gVJjTIMx5gCAMWYVsBU4+mArfagZmJPJf501jjOPziM95P+RtbQaRvSPdeucOnYAF0wewtaSGn78Ylv32u3qKdhbxZ6Keu57+SOgbdGTeINqbOG1rTH7x1rvEg/7fnX3JNYXlceIaanLCq10uZciFn3kfPWdsNbcD52txdEBxS3W+4YkLTm7nFvoX/+omM/85u0YPxPyTMIAACAASURBVOv720o578Gl/Omdtu32Z1FS1UBLq+GZFbuSykhZurmESXct9u0VXfDQm/y3JcqlNY0UFlc5cQibhBa9q+1eF0+VRzz9xLihuYXC4mih9363yVjKdqwnK71zCXjuuFBHs57s4vUeN5+3Z+Rm/pvb+O6z6533dk+6ur6Zamt+qlU7It/XJqvHY38uSwpK+PuqxHn7fnGu1zbts85b6usiS4R7FtshuVlR+256YjUPvrq57dpJzq/VGZL5dlcAE0QkX0QygLnAQk+ZhcA11uvLgdeNMUZE8qxgLiIyFpgAbOuaqvcMaeH4fjvbcnSTHgrx/04bw5XTR/Gnd7Y71q2dhVPV0OzMs2PfkO3lSrcJfXyL3v7R2SKyfX8Nl/zmHX7q8aUeqG4TmUeWtn01FXVNjsXt/iF+tLeS7z67znkIFOytomBvVaxP3iWk3vbYVmCy6ZV2OXfvxG7zz62Ho836ooh7Zu2uNjeNbTEWV9Xz5Ps7+N4/1vPE+zvbve4LayJz/NiBar867a6o46R7X+W8B9+McbWVJgh0untPpR5hsx/OP/3s8YC/e6WxuTXqu4NI+6Kv0fadlNe2ZXy4rdb91jkamltjehK7SmsTZov9cOFGahvarvHXJD7T1lbj48aMvnZJgp5XZX0TB2oaY1x0IuLcHztKa3n5g71c+Ku3+Ne63QnvM+95/HooLa2G7/19HZ//3XsseOfjDo1Mtu+9llYT4/P3Jh889u72pFNiO0q7Qm/53G8GFgObgGeMMRtF5B4RucQq9kdgoIgUEnHR2CmYZwDrRWQdkSDtjcaYI2+IqAtv98tNlo/rxs5cmTo6ssiJbTW7g7O2K2bTnkrOe3Cpk9Psl+EDLuGqrKemodmZadPdBbZvetuqtAX/sXe388+1bZOUlbqsTreft7KuyXlw1Lks+vlvbuPZVUXORGezH36T2Q+/yTE/eJkxt73IE+/v4Jan10bVxZsi6Ai9j6tmsyv7YG9FPat2lDrZI24ZssXAa7Xa7pK0kLBieynGGOfBWFXfzK6yyI+r1tWmwuIqvvzH92OWj/zEKuu1dt2W67pdFc7r9UUVUeW8Au7GLcL7PYJtZ2gN7pPJgN4Zvj76hubWmCDtXo//2f3gP+sXbzDr/iWs3lnGuO8vYtWOUquOjVaboLqxmVc27qVgbxWLN+5l1v1LuPOFDyirafQVoMfe3e7UNSs9xIvrozv69U0tvLh+T9Tn9cTynZx4zyssd40Ur29qodZ1v/gJvf1wsu8dO7Bpn7vF6lkBfFJW6/R2Nu6uTJjh5P0M/dwnrQaecc2TYwfdk8G+P4vK6pxEBxvv/Tb/zW384pXNdAdJ9deMMYuMMUcbY8YZY35ibbvLGLPQel1vjPmCMWa8MWa6MWabtf0fxpjJxpgTjTEnGWP+1S2tOISISNRMl24SPQT6ZEam/re7hm43yT7XjV1YXM3ijZGuYkZayDfl0PHRVzZEicDm4moefKWAllbjLIVYWhO5jvvH9q2n1rJuVznfeWYdxVUNhCQ2m+hvy3fxmpWhsKSghMLiKt7esp+8nEwgOgDq5o7nP+C5NZ+4rE3DzgO1DHV1W+0fk98w8LnzlzmW6bm/fIPP/+49xyJb/nGpI1BeUbOxB+H8a/1uvvDIezy1YpfzoKhvanE+/96ZbQ/lX76ymbe27Oe1TcU8tXwn63aVU9vYzIZPIsLtnbLa/ZlvcblPPvL4zMtqkrXo/YW+f+90huRmRaXS2jS2tMak7HpjEEs3l/DShj2s3VXuCNpPX4z06JYWlERdC+Df6/Zww19WcecLHzjZYwV7q5h676v877PrfNthX/O0sQMp2FsVdZ89sLiAm55cHRV72mz1jha83WZUrNxeyp/f2x7zedQ3tbCrtJZpP36Vcd9fxJ/f2+58f3Y8xO4I1DU1O0ZNVX1z1PoQiXLXT/nJf3jFdS9XN8Tek96YQUl1A0Vltfzlve1Oe5taWnnwlYKoHtDqnWUJr+3nWlu8cW+3jOBNuYVHuoK0UIi0UGyQMpFbJyfLEnpLNNzRfDtrxuYjy7dYUtXAzPteZ/t9F0ftd7tuNrisyH+ti1hU0/MHcsCyeMpqGykqq+Wrj62MOsdPFm1i+celDM3NYkhuFlNG9WdXaaQefXulx3TZL/rV2zS2tDq9jKWbS9i+P76LyR302l1RR/6g3o5A7iqto6isNsai/+7sY3hgcQGrd5Qz57ihTiaHe/3evy7byclHDWC36zNLCwl/vX4Gc+cvY5vlJrJ9o08t38mMsZEEsEqXH3dXaS23P7eBey+dTL/sDKeedqD49gsnRvUE3NifE8DD/2kLpnldBNUNzTQ0t5CZFtvT+9M7H3PFtFHsr27g9ueiMzLsh3vfXhkMyc3kk/J67nzhA74yc0zbtZpbY1J2i8pq+ZWrPo+9u53H3t3O2Ly2TLGVlv/adge5HzKvfBgRvMz0kGMx7yiNfJ4vrN3Nw3Onxjyc7YfQSaP7s6SghP3VjeT1iRgD9jncxoqdQeVOc3zBEzx9fs0njM3L4bJ570Rtf271J873Z1v9tiTWNLQ4vbmI0Ee2/9H1QInHDX9ZxbkTB/PLL57o67rxPkD3Vzfw85c+Yvn2Us46ZjDLth3gu1Zwuqy2iXsvOw6Az/2249NQvPztWVHrZHQVKvSdIC0s9EoPO0Jgv04Lx+8g5VgWvR1ocwvpTxdF+5njdTWXbi7hmgXLnd5BYXE13356bUy5f6/f7YjkgZpGvvf39THCPSgnIm57K+s5aXQ/+mS13QpHDcyOcUPYImafp9VE3AHx2OMS4j3l9cwcPyhq/7wlhZw6dmDUtolD+0TqVFEX5Spw++Ft0djjsuhDISHbWhZyj8fSr6xvdnpALa3G6S38/q2IAPy/047ClotXNrZlc2wrqWFQTiaV9U1RvS9v29rjtU3FXHT8MCC6V7V5XzWLN+71zbXfUFSBCIzo14uhuVm8UVDCpj2VrHDNvhhx3UQf672PbHaV1jJ1dL+o9NJCy51WVtPI2LzebCup4WPrwd3g8v97rc4aj8Vri+CJo/pFzltcTVNLK2lhce4p94Oy2hWfSQ+Lb8LBW1sio4u9jB6QzTJr7QKv+NY2NjsPrbqmlrifRTxe+6iYZ1bu8g3GbtoTPTZmQ1EFTa12PK2ap1a0ZeX8ZdkO+vZK55bzO5dz4mcUdAU6100nyAiHooTRXns2PZHrJsaiT34AlS1U85YUAm0Pi20ui7p/dps/3/0jKatpdH7AbtyW3Ij+2c7DAyB/UJsF2CczjUGWu8bm6CE57dbZHq3b1BIR1+H92lw3M/IHULC3KkZAh+RmkREOsaeynuv/HN0DsfmkrI7WVsOu0lrnYRUSHKH3Ul7bGBUv8D4I7v7nRmcaW3cAt6i8llEDepGblU6lS6jqm1p85zX3BuLtyfD++4nVFBZX86U/vB+T8dHY0uobaF2zs5wR/XrRKyPMYJfLy52uuaGoPOmpjptaDOccMzhqm90bK61t5NihkYznHdb8K9X1zb6De/66bIfTU7QprmogOyPMAKv9lfVNfOq+1zn9viVkZ6RZ9W6guKqem55czXOuRWzG5eX4piTHY29lvZPZVFLVwDuF+51BV9UNLTEusI7y5/d2OL3pRPxmSaFz727eVxUTS5v/5raYzKmeRoW+E6SFJWpyM/tmDcdJvQTondlm3TRZ/lW/LB0/bD9qPEv/tgsnOueH6K5yUVltjLhBtOAN75cV9eAa4RokVtXQTL/s6Bt5ZP9sfva54xPW2XatVNQ10WpgWN+2cx49pA+rd5Zz5z+j5xrJSg8xtG8WeyvqHfH1sreynl1ltTQ0tzoB7rAIvTLa6j/3lLZs4LLaJp51CewOz2RSy7eXsmJ7bPrkWktsc7PSooJmjyzd6ozevHrGaGf7uLzoh98Uy8IFeHbVLt4u3O90720amlvZVlLNkNxMBvdpe5gW7KtyBua5Yxvu9DvvZ9ceowdmOzGk7IwwZbWNPLJ0K/VNrRw9pE9U2eqGZg7UNMbEnH7wwgfc8fwHQNssrvsq6+mdmebcf3acorGl1TFQ9lTUs3J7GS+ujx53MCgnM+m0ztPHD2L5x6VOSuZza4q4+g/v855l4dc2NFNV30RuVnwnxfT8AQmvUVRWR2V9Mycf1d/pocSjzDLUCvZWxwj9iP69YowYL4d6MlsV+k6QFgrRJ6vty7WtyfSw8NurT/JdrMR23Xz/+Q1MuOMlVu8s54yjYweHHTciZjwZ33lmHY3NrRTsbbM23JbQrAmDHJfKqWPbbuZhfbOcG9KLW/AywyGnfhnhUIwF7/3B9+uVHiVMfnh7LEP7tpWP1yPICIcZ2jeLPRX1lNU08ukThsWUaWk1Tvf9JEvoQyJRn8flJ4/kimmjuLKdtQTczJ48hM9OHcGXTz0KiIz0HNk/mz5ZaY5Fb4yJatdPPtv2sPOOoXB/D4+60lbdlNc2UlhczbHDcqMetBD57iD6c+vMwDWbvD6ZZFqD/fIH9abVwH0vRdwb/bLTo3qE1Q3N7K9qYPLw2HvRnjhvqFW/orI6cjLTnOD2Q//ZHHUeiPQe/HzfA3MyfDPV/Ojl6bF5H9h/ePtjKuubY3LV3cRLovAyaVguM8dF3IpeI8fG7j3sLK2J6U0O75cVFYD2w91rPhSo0HeCGWMHcJrlX75g0hDni04LCxcdP4xHvnxyzDG9M6J/yC2thhvOGBu17U9fOcWx5Nw3wnvbDnDWA0ui/KVjXPsH9M5w/KCzJrQ9PEb1zwZgeF//m3/6mIgYDe/Xy3lw9ctOZ2BOdE/DDpoOc50n3g/q9e+cGTO1M0QCi351d5OeJgzrm8WOAzXUNLYwYXAf33JvWq6pk0ZHrC63jx5g2pgB/PzyE5h2lL8FN8CnJzVmYG8eumIKc10Ph5H9e5HbK92x6P/n6bXOAK1rPzUm6vgRnqkyTh6T2HqESOrfR3urmHZUf747+xjSQsLbt57Nn649he/OPsY6b3bCc3jXS4jHkNwsMi1R9X7+l5w4POr7LK1ppKqhmTM97h5oy0Bxu5TOPDov5v6GNt/86p3lziAmN7lZ6c7DZ0Z+22/KZtpR/Z3XXpdRPIbGudc7Qm6vNGd+pbM8xthQz32/q7Qupqe9eV813/tHdO/Ny6gB0d/rxScM4+ErpnS2yu2iQt8JHvziFP7rrHG88b9n8esrpzrWRloC100oJI7VDBEL8thh0RbT2ccM5r7Pn8CT18/gS5ZlaWPP0WGLmzuTon92m3BNcp1zpJUymROnO/vzy0/gz1+dzhenjXLK9M/OiHJL/evm0x0fty1uB2oaGZzrb9GPzcthlifwCpCblcajXz6Zx786nZH9/cUrIxxx3dh+67w+mfz887Euore37KdPVprzMBTBEYzzJw1xysWzxvz8wnYvxi14I/r3ilj0Vm/Jzg4ZNaAXP7xkMgAvfWsW/7xpJiMti/6b507gvGMHM8NyE0wd3Y8/XjON2ZMj9frWuROcNtlD9S86fhhzjhtG4U8vYmT/bM6eONgRUndsww+vZejXS7TbZX9GY13HfP+iifTvnREl3DaXTRkeI2w2o11Cdf2sfN/P9D+b9jkZOO6ApU16OOQsbnPesUP42w2nOvt+d/VJPPm1yPvTxg6M6kEnYnCf+J+XeGZquXTKcGZNiL1X+2SlO1k9J7seNhCdlguRaUjsXt5VM0Yza8KghAO+rpoxmpvPHs9lU6KnC5t31UlcNtU7hVjXoVk3B4FtGdlBp/a6hjmZaVQ3NHPG0XnMu+okAP5zyxmc9+CbUWU+NX6QkxXhZcbYgazeWU6Oy4Jyd3/dDw/7x+j239uMy+tN/qDejlDYItAvO50sK/J/waQhHD+yrzML4gWThyIC508aysDe8V03f7hmWszc8H2y0pltubTiLaOWkRZimEtYBvTO4NxjB3PrP6LTDyvqmjhxVD8G5mRyzsTBfG3WWESE5d8/10mVtNviZfbkIU4KphtbkAa4jh/Vvxf9szMor22KSlt0+8rtzzscEtJCwmenjnAyLjb88AKyM9IIhyIxnTc37+fqGaMZnJvFgre3U7Cviqz0UMJufHsC53VpXDV9NOPzcpzBb7+5aiqj+meTk5nmfMejXA9au8dwlMfCPHpIDmPzcnj1ljMoq2nijAfaptfNCIeYMLjN/Tasb6+olMBPjRvIu5brYuyg3lx03FAefy92GuP0NOGYIX34eH9NzG/nuBF9yUgLsfFHs0kPhyiva+TbT611zhuPQX3avr/p+QPYV1nvuHm8k+Ldc+lx9M4IM/6Ol6K252als8dKGx3erxf/ueVMPtxTyTf/tibqtyQSGWj2n037OG5ELj/97PH89xOr8ONLp47mr8t28pVPjWHCkD6dmmPoYFCLvguwu8+JgrHQlnkzeXiuk4o5Po57wp1RcfdnJjFmYLZzLMSfAGmIy9K2g73ebvXbt57N018/zff48YNzmJ4/gP86axw/tvKBzz82Yo0Ozc3ihjPGkT+oN+GQ8Or/nMFtF06MOYeI8Osrp0b1Lty9CncKmXuMQHo4xFBX0HZgTgbpHteELd7jrDosuPYUTrP8qYNzs6LcRt4e1sKbZ/KruVNjxBHaLHq3YI3ol83AnEwO1DT6+p7dHDeiLx/8aHZ0xlJWujPs/ZQxA9h07xzHcrbbkdcnk2Rn9J46OjZAGEkPbWNgTgZ3fWaSEycalJPpBBa/fuY4AI4Z2nbP2bEFd+8IcGZt7ZOVzuiB0Q+BptbWKFeZd2i/29ovrmrg6KH+93hGOMRlU4dH6umJ+fS3793MNDLSQgzuk8UT18/gj9dM4zMnRo65dc5E7r3sOO53rScxdlBv+mSl8ZurpvLU105l6XfP5pEvncyT18+IuX5mWsg3JTq3Vxq3nH80E4f2YcbYgYwfnOMMFMxMC0V9pza2sefnwgL48WXHs/2+i5lgBb6H5GZx/+UncNPZ43jwiyf6HtOVqEXfBdgrCZ3k+iFefPywmC7cyP692FJcHRPs9MPOI7/hjLF8ZWY+n506gv3VDYzsn83npo7g2+cdzdLNJY4fdOyg3mzbXxMlGhMtoZ193FDGDMrmr8sic5EM91hgAGdMyOOHn5nEF08ZRTgk3DqnTcDvvew4vnnehBiBnDCkD2t2+U/9e8mJw+nbK51rrDnme8dJf3STEQ5FxQG8vvRF35zFdY+voJwmxg1uP8Vz8vBcrv3UGMevfsLIyPdj94B+cPGx/NgaKZrnE1zulRF2UjjdeLvzNskGFqHNx52XxL0wc/xAymqamDw8NyoX/t/fOJ3jRvTlcyeNZMxtLwJtQf+HrpjCrDVFUWJ05fTRXDl9dFT+vW2Z264nO6vFOz23G2NI+HAa6QpM76moi8lKyUoP0doKl00dwbi8HF7+9iyOsQTQHpPid7+ICOceG3F55g/qzddm5TtCbc+m2T87gw0/nB113JzjIg+9eW8URm23ezh3f2YSP3JNVtYnK50po/rx8rfPcF277bjsjDBV9c3MyB/AccP7suCdj52EBb/eczy+GGcdjO5Ahb4L+M4FRzP/zW2ce2ybf3je1SfFlJs4LJclBSVJLTpx3Ii+QJvV0C87w3FLPGgFbd697RzsNcj/9Y3TY4JCJ4zsy/I7ziUvJ2I1/uiS46htbPYdeRcKCdfOzI/ZDhGXijfYaJMZZ9pmiHaDeIXhxW+eHjO0PBSSKKH3+ocnDc/l4uOHsfjDvVxiWXWJSAuH+OElk9laUs3ZrsCi7U+++IRhvkJ/2tiBbLemUvC6qKaO7sejX57W7rXbw/aTx5vPyM0T10d81ZX1TWSEw2zeV8VNZ4937hE3ttD3yghz9YyjYvZDxDVhYwvTUVYSwJiB2Ww/UMunj4/OePrenGPITAvzr3W7owTKG0CFaHfTCSP7RbVxUE4GD1x+ImdPbPs+Jg5t6/m9+M3T+WB3ZcIHyfB+veIOSMqOY1H7YV/jKzPzo4Q+UYqmMW1zsudkpjlpw3ZGlP35j+zfi19+4USumL8s6fp0Jyr0XcC1M/PjiqSbG88cx57yOr4wbWTU9qXfPcuZc91m5vhBLP/+ub5BMps+nh+s122emRZmcJ/oLnayQa1kSWTFDvCxhm0mD48VKYi4GmZNGMTlJ4/0tY5+8OlJ/ODT3pUsE/OX66K77b3Sw4gQ1bPq5xIjd1DQm4E0I39gUuLcHnaPpCMLduRmpXPXZxK3PTeJ7zcUEh798slR6ZPHDO3Dn649hVPyB1gD0KI/+/8+azwA153edp9/dO8c31WYPnPicEqqGpg1YRATh+U68w8BrPzB+QnrNjYvEhvoLH5uORs7GPu7q0+K604C/7jIlFH9OGfiYG6dM5HZD0diaiP7ZzuxBXsWz2wrWHvKmAHMGDvQ8eP3NCr0h5C+vdJ5eO7UmO1HxVm1KpHIHy6EE1hebos+WUIhiRHm5XecG5MxcTDkZKUxpE8W6eEQP/3s8by8cW/c+UW8rpuOzrkej1GWe6O9gTUdxZsVEo/ZPmM93FZ2MsR7yPfrlc7/WumhkFyvpauIN0Ia2oKxvTPTYga4ufHrpWalh1lw7SlR244ekuOMgLVjZvZkaxOssSJr7jw/6RHM3YkKfQB55uunRQ2u6ikSWVcdIVHKXGf4xjnjndGzV80YzVWuEa5eBni6SV31oz12WC4Th/bhrg72TuIxuE8mxVUNCedbOlR4H5qHUujjBUOT5Wuz8uO6Kb2MGdSbPdZUH/YoYHuU8RnWeJZ+nTB2ugMV+gAyPX9Au8O9u4NJw2JHUh6OHDWwd9xelJf+2el85/yjKa1t5E/vbKerlrzPSg9HBfsOln9/4/SY+c4PNTPyB/D+x7HLTXS1uzARybhuEn2Hd1zc/oN3wuActhRXkx4OOemcdqr13FNGcdYxeQmD2T2BCr3SJUwc2odnboxN2fzenGOStpAOR0SEb5w7gbrGFhqbW7n57PE9XSVfBudm9bir74nrZ8TEmiA2/bI7SeS6uej4YbxduN8Zfd5ZFt58ujN7ZXZGGn/+6nQnMB4KyWEn8qBCrxwkU0f3IyTwo0smR438tbGDeEc6vTLCUXPbKLGkhUPEm2X3t1efxPgkUmI7S0Y4RGNLa8LZMK+cPorPTh3ha/U/fcOpUQsAJaJXRphetJ0j3mjkwwnpquBSVzFt2jSzcqX/FLVKMCmpaqChuSXu1AiK0h4f7a3k7S37uX7W2PYLBxQRWWWM8c39VYte6XH8BispSkeYODQ3Kh9fiabnQ/SKoihKt6JCryiKEnCSEnoRmSMiBSJSKCK3+ezPFJGnrf3vi8gY177bre0FIjLbe6yiKIrSvbQr9CISBuYBFwKTgCtFxJtseh1QZowZDzwE/Nw6dhIwF5gMzAF+a51PURRFOUQkY9FPBwqNMduMMY3AU8ClnjKXAo9br/8OnCuRGYMuBZ4yxjQYYz4GCq3zKYqiKIeIZIR+BOBeHqbI2uZbxhjTDFQAA5M8FhG5QURWisjKkpKS5GuvKIqitEsyQu83rM2bfB+vTDLHYoyZb4yZZoyZlpd3+A8+UBRFOZJIRuiLAPcM+SOB3fHKiEga0BcoTfJYRVEUpRtpd2SsJdybgXOBT4AVwFXGmI2uMjcBxxtjbhSRucDnjDFfFJHJwJNE/PLDgdeACcaYuJNwi0gJELvAZPIMAvYfxPFHItrm1EDbnBp0ts1HGWN8XSLtjow1xjSLyM3AYiAMLDDGbBSRe4CVxpiFwB+Bv4hIIRFLfq517EYReQb4EGgGbkok8tYxB+W7EZGV8YYBBxVtc2qgbU4NuqPNSU2BYIxZBCzybLvL9boe+EKcY38C/OQg6qgoiqIcBDoyVlEUJeAEUejn93QFegBtc2qgbU4NurzNh900xYqiKErXEkSLXlEURXGhQq8oihJwAiP07c2weaQiIgtEpFhEPnBtGyAir4rIFut/f2u7iMivrc9gvYic1HM17zwiMkpElojIJhHZKCLfsrYHtt0ikiUiy0VkndXmH1nb860ZYbdYM8RmWNvjzhh7pCEiYRFZIyL/tt4Hus0isl1ENojIWhFZaW3r1ns7EEKf5AybRyqPEZn5081twGvGmAlEBqHZD7YLgQnW3w3A7w5RHbuaZuA7xphjgVOBm6zvM8jtbgDOMcacCEwB5ojIqURmgn3IanMZkZliIc6MsUco3wI2ud6nQpvPNsZMceXLd++9bYw54v+A04DFrve3A7f3dL26sH1jgA9c7wuAYdbrYUCB9fpR4Eq/ckfyH/BP4PxUaTeQDawGZhAZIZlmbXfucyIDGE+zXqdZ5aSn696Jto60hO0c4N9E5scKepu3A4M827r13g6ERU+Ss2QGiCHGmD0A1v/B1vbAfQ5W93wq8D4Bb7flwlgLFAOvAluBchOZERai2xVvxtgjjYeB7wGt1vuBBL/NBnhFRFaJyA3Wtm69t4OyOHhSs2SmAIH6HEQkB/gH8G1jTGVkiQP/oj7bjrh2m8j0IFNEpB/wPHCsXzHr/xHfZhH5NFBsjFklImfZm32KBqbNFjONMbtFZDDwqoh8lKBsl7Q5KBZ9qs2SuU9EhgFY/4ut7YH5HEQknYjIP2GMec7aHPh2AxhjyoE3iMQn+lkTC0J0u+LNGHskMRO4RES2E1nQ6BwiFn6Q24wxZrf1v5jIA3063XxvB0XoVwATrGh9BpFJ1Rb2cJ26k4XANdbra4j4sO3t/8+K1J8KVNjdwSMJiZjufwQ2GWMedO0KbLtFJM+y5BGRXsB5RAKUS4DLrWLeNtufxeXA68Zy4h4pGGNuN8aMNMaMIfKbfd0YczUBbrOI9BaRPvZr4ALgA7r73u7pwEQXBjguIjKd8lbgjp6uTxe262/AHqCJyNP9OiJ+ydeALdb/AVZZIZJ9tBXYAEzr6fp3ss2nE+mergfWWn8XBbndwAnAGqvNHwB3WdvHKqw4fwAAAGZJREFUAsuJLMP5LJBpbc+y3hda+8f2dBsOsv1nAf8Oeputtq2z/jbaWtXd97ZOgaAoihJwguK6URRFUeKgQq8oihJwVOgVRVECjgq9oihKwFGhVxRFCTgq9IqiKAFHhV5RFCXg/H85BWxDepHhqwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXyU5b338c8v+56QhRCSYAKC7KBEFtG6K2rVLrZKl1M9Wk4XT/u0fdrqaY/a5dSlT6unPbbWunDaWrdqK3XfUFAUDbIvgUAChASSkJB9z/X8MUMMkJAAEyYz832/Xnll7nuu3PO7wvCdO/dyXeacQ0REAl+YvwsQERHfUKCLiAQJBbqISJBQoIuIBAkFuohIkFCgi4gECQW6iEiQUKBLwDCzO8zsjv6WhzMzyzOz0v6WRXxBgS7iI96QXmpmzWa2xcwu8ndNEloU6CK+8ziwGkgDfgT8zcwy/FuShBIFugQFM7vBzP7Za7nYzJ7qtbzbzGYO1XbNbAJwBnC7c67FOfcMsB747PH2SeRYKdAlWLwNnGNmYWaWBUQC8wHMbCyQAKwbwu1OAXY45xp6/exa73qRk0KBLkHBObcDaABmAucCrwB7zGyid3m5c657CLebANQd9uN1QOLx9Ujk2CnQJZi8DZwHfML7+C08oXuud/kIZvaAmTV6v/7jBLbbCCQd9nNJeD4MRE4KBboEk4PBe4738dsMEOjOua855xK8X784ge1uBMaaWe898hne9SInhQJdgsnbwPlArHOuDFgOLMBz1cnqodyuc24rsAa43cxizOzTwHTgmRN4XZFjEuHvAkR8xTm31cwa8QQuzrl6M9sBVDnnuk7Cdq8DFgO1wC7gGudc1fG+rsixUqBLUHHOZR22XHCytuucK8VzaEbEL3TIRUQkSGgPXQLJWwMsD2cHgPuOsixywkyTRIuIBAe/7aGnp6e7vLw8f728iEhAWrVqVbVzrs8xgvwW6Hl5eRQWFvrr5UVEApKZ7ezvOZ0UFREJEgp0EZEgoUAXEQkSCnQRkSChQBcRCRIKdBGRIKFAFxEJEgMGupk9YmaVZrbhKG3OM7M1ZrbRzPocd9pXtu5r4OfPb6K147gHzxMRCUqD2UNfjGfs5z6ZWQrwO+Aq59wU4HO+Ka1vZbXNPPROCYWltUP5MiIiAWfAQHfOLQNqjtLkC8Czzrld3vaVPqqtT3Py04gMN5YXa5hpEZHefHEMfQIwwszeMrNVZvYvPthmv+KjIyg4JZU3NleigcVERD7mi0CPAGYBVwCXAv9pZhP6amhmi8ys0MwKq6qOfw/7yhmjKa5sZP2ewydZFxEJXb4I9DLgZedck3OuGliGZ3LcIzjnHnTOFTjnCjIy+hwsbFAunzYKgGVbddhFROQgXwT6c8A5ZhZhZnHAHGCzD7bbr5S4KMZmxLNmt/bQRUQOGnD4XDN7HM88ielmVgbcDkQCOOcecM5tNrOXgXVAN/CQc67fSxx9ZWZuCsu2VuOcw8yG+uVERIa9AQPdObdwEG1+CfzSJxUN0vxx6Tz70R5WltQwd2zayXxpEZFhKWDvFL18WhaJMRE8s6rM36WIiAwLARvosVHhnHfaSJYWVdHdrcsXRUQCNtABzj8tg+rGNjZV1Pu7FBERvwvoQP/EBM+lj0u3DOnNqSIiASGgAz09IZoZOcm8pevRRUQCO9ABzj1tJKt31XKgud3fpYiI+FXAB/r5p2XQ7WDZtmp/lyIi4lcBH+jTc1IYERfJWzqOLiIhLuADPTzMOHdCBm9v1eWLIhLaAj7QAc6fOJL9Te2sKTvg71JERPwmKAL9vAkjiQgzXt+0z9+liIj4TVAEenJcJBOzEtlYrhuMRCR0BUWgA4xOjqWirsXfZYiI+E3wBHpKLBUHWv1dhoiI3wRRoMfQ0NZJfWuHv0sREfGLoAn0rORYAO2li0jIGjDQzewRM6s0s6POQmRmZ5pZl5ld47vyBi83NQ6AHVWN/nh5ERG/G8we+mJgwdEamFk4cDfwig9qOi6Ts5KIiQxjZUmNv0oQEfGrAQPdObcMGCgl/x14BvDb/fdREWGcMWaEAl1EQtYJH0M3s2zg08ADg2i7yMwKzaywqsr3Q97OyU9jy9566pp1YlREQo8vToreB/zQOdc1UEPn3IPOuQLnXEFGRoYPXvpQc8am4hx8WKq9dBEJPb4I9ALgCTMrBa4Bfmdmn/LBdo/ZzNwUwsOMNbs1pouIhJ6IE92Acy7/4GMzWww875z7x4lu93jERIaTMyKW0v1N/nh5ERG/GjDQzexx4Dwg3czKgNuBSADn3IDHzU+2U9Li2bm/2d9liIicdAMGunNu4WA35py7/oSq8YG8tDhW76rFOYeZ+bscEZGTJmjuFD0oLy2ehtZOqhrb/F2KiMhJFXSBPj0nGYCPdurEqIiElqAL9Gk5yURFhOnSRREJOUEX6NER4UzOSmJzhSa7EJHQEnSBDpCVHENlg46hi0hoCcpAz0yKYV+9htEVkdASlIE+MimahtZOWtoHHI1ARCRoBGegJ8YAUNmgvXQRCR1BGeiZSdEA7KvXcXQRCR1BGehZyZ499PIDLX6uRETk5AnKQM8ZEYcZGqRLREJKUAZ6TGQ4o5NjKa1WoItI6AjKQAfIS4+jRKMuikgICdpAz0+PZ0dlI13dzt+liIicFEEb6LNOGUFDWydb9moIABEJDQMGupk9YmaVZrahn+e/aGbrvF8rzGyG78s8dnPy0wB4f4cG6RKR0DCYPfTFwIKjPF8CnOucmw78DHjQB3WdsNEpsYxOjmFdmYbRFZHQMJgZi5aZWd5Rnl/Ra/F9IOfEy/KNyaOT2ViuQy4iEhp8fQz9RuCl/p40s0VmVmhmhVVVVT5+6SNNHp3EjqpGjekiIiHBZ4FuZufjCfQf9tfGOfegc67AOVeQkZHhq5fu15TRSXQ72KwToyISAnwS6GY2HXgIuNo5t98X2/SFKaOTAHTYRURCwgkHupmNAZ4Fvuyc23riJflOdkosybGRbFKgi0gIGPCkqJk9DpwHpJtZGXA7EAngnHsAuA1IA35nZgCdzrmCoSr4WJgZU0YnsbG8zt+liIgMucFc5bJwgOdvAm7yWUU+NjU7mcUrSuno6iYyPGjvoxIRCd47RQ+aMjqJ9s5utu1r9HcpIiJDKugDfVp2MgAbdNhFRIJc0Ad6Xlo88VHhbNyjQBeR4Bb0gR4WZkwZncwGXekiIkEu6AMdYEq250qXtk7dMSoiwSskAv3sU9Np7ejmgxKNvCgiwSskAv2scelER4Tx5pZKf5ciIjJkQiLQY6PCmTcujTe3VOKcZjASkeAUEoEOcMHEkezc30yJJo4WkSAVMoF+znjP6I4rtg+bscNERHwqZAI9Ly2OzKRo3t+hQBeR4BQygW5mFJySyprdmpJORIJTyAQ6wITMRPYcaKG5vdPfpYiI+FxIBfr4zAScg+2VOjEqIsEnpAJ9QmYCANsqG/xciYiI74VUoJ+SFk9EmLGtUkPpikjwGTDQzewRM6s0sw39PG9m9hszKzazdWZ2hu/L9I3I8DDy0+PZtk976CISfAazh74YWHCU5y8Dxnu/FgG/P/Gyhs6EzETtoYtIUBow0J1zy4CjjWp1NfAn5/E+kGJmWb4q0NfGZyawq6aZhtYOf5ciIuJTvjiGng3s7rVc5l13BDNbZGaFZlZYVVXlg5c+drPzUnEOjbwoIkHHF4FufazrcwQs59yDzrkC51xBRkaGD1762M3KG0FMZBjvFFf75fVFRIaKLwK9DMjttZwDlPtgu0MiOiKcCZmJFOs4uogEGV8E+hLgX7xXu8wF6pxzFT7Y7pDJGRHLntoWf5chIuJTEQM1MLPHgfOAdDMrA24HIgGccw8ALwKXA8VAM3DDUBXrKzkj4nh9cyXd3Y6wsL6OGImIBJ4BA905t3CA5x3wTZ9VdBJkp8TS3tlNdVMbIxNj/F2OiIhPhNSdogfljIgFYHdNs58rERHxnZAM9AmZiQBsqtAdoyISPEIy0HNGxJISF8mGsjp/lyIi4jMhGehmxrTsZDaUK9BFJHiEZKADjMtIYOf+ZjzndEVEAl/IBnrOiFga2zqpa9GYLiISHEI40OMAKNMNRiISJEI40D2XLpbV6tJFEQkOIRvoud499O1Vml9URIJDyAZ6clwk07KTeWnDsB52RkRk0EI20AGunjmaDXvqddhFRIJCSAf63LFpAKzaWevnSkRETlxIB/rEUYnERYUr0EUkKIR0oEeEhzE7P5WlRZW6wUhEAl5IBzrAZVNHsbumhY3l9f4uRUTkhAwq0M1sgZkVmVmxmd3Sx/NjzGypma02s3VmdrnvSx0a508cCcCK7ZpjVEQC24CBbmbhwP3AZcBkYKGZTT6s2Y+Bp5xzpwPXAb/zdaFDZWRiDHlpcXxQouPoIhLYBrOHPhsods7tcM61A08AVx/WxgFJ3sfJDONJovsyJz+NlSX7aevs8ncpIiLHbTCBng3s7rVc5l3X2x3Al7xzjr4I/HtfGzKzRWZWaGaFVVVVx1Hu0Lh8ehYNrZ28ubnS36WIiBy3wQR6X7MoH35JyEJgsXMuB8+E0X82syO27Zx70DlX4JwryMjIOPZqh8jZp6YTHxXOypIaf5ciInLcBhPoZUBur+UcjjykciPwFIBz7j0gBkj3RYEnQ3iYkZsapzlGRSSgDSbQPwTGm1m+mUXhOem55LA2u4ALAcxsEp5AHz7HVAZhTGocuxToIhLABgx051wncDPwCrAZz9UsG83sp2Z2lbfZ94Cvmtla4HHgehdgd+qMSY1jd61mMBKRwBUxmEbOuRfxnOzsve62Xo83AfN9W9rJNSYtjtaObiob2shMivF3OSIixyzk7xQ96LTMRAA27NHE0SISmBToXtNykgkPM1bvOuDvUkREjosC3SsuKoIJmYmsLVOgi0hgUqD3Mj07mQ176nRiVEQCkgK9l6nZSdQ2d1Be1+rvUkREjpkCvZfJo5MB2KgToyISgBTovUzKSiTMYIPGRheRAKRA7yUuKoJxGQlsKtceuogEHgX6YaZmJ7O2rI7ubp0YFZHAokA/zLkTMqhqaGN5sWYwEpHAokA/zGXTRpEUE8HLGyr8XYqIyDFRoB8mOiKcSVlJFO1t8HcpIiLHRIHeh9NGJbJ1X6NuMBKRgKJA78OEzEQa2zp1g5GIBBQFeh9OG+UZeXGrDruISABRoPdhwkhvoO9ToItI4BhUoJvZAjMrMrNiM7ulnzafN7NNZrbRzP7q2zJPruS4SEYlxVCkQBeRADLgjEVmFg7cD1yMZ8LoD81siXeWooNtxgO3AvOdc7VmNnKoCj5ZJoxK1JUuIhJQBrOHPhsods7tcM61A08AVx/W5qvA/c65WgDnXKVvyzz5TstMYFtlI51d3f4uRURkUAYT6NnA7l7LZd51vU0AJpjZu2b2vpkt6GtDZrbIzArNrLCqqur4Kj5JJo5Kor2zm9L9zf4uRURkUAYT6NbHusMv0I4AxgPnAQuBh8ws5Ygfcu5B51yBc64gIyPjWGs9qQ5e6bJ+j2YwEpHAMJhALwNyey3nAOV9tHnOOdfhnCsBivAEfMCaOCqRvLQ47l+6XTcYiUhAGEygfwiMN7N8M4sCrgOWHNbmH8D5AGaWjucQzA5fFnqyRYSHcePZ+RRXNlJW2+LvckREBjRgoDvnOoGbgVeAzcBTzrmNZvZTM7vK2+wVYL+ZbQKWAt93zu0fqqJPluk5nqNG6zWDkYgEgAEvWwRwzr0IvHjYutt6PXbAd71fQeO0UYlEhBnr99Rx+bQsf5cjInJUulP0KGIiw5mQmcgG7aGLSABQoA9gWnYyG/bU6cSoiAx7CvQBTM1Jpra5QydGRWTYU6APYP64NMIM/ndFqb9LERE5KgX6AMZmJHDljNE8Wbib9k4NAyAiw5cCfRCunD6ahtZO3t8R8FdiikgQU6APwtnj04kKD+Pd4mp/lyIi0i8F+iDERIYzNTuJj3bV+rsUEZF+KdAH6YwxI1hbVkdrR5e/SxER6ZMCfZAumDSS9s5uXtpQ4e9SRET6pEAfpHlj08hLi+PxD3YP3FhExA8U6INkZnz+zFw+KKnh168W+bscEZEjKNCPwRdnn0JSTAR/XF5Cd7eGAhCR4UWBfgyS4yL5/oKJtHR0sa+h1d/liIgcQoF+jMalxwOwo6rJz5WIiBxqUIFuZgvMrMjMis3slqO0u8bMnJkV+K7E4SU/42CgN/q5EhGRQw0Y6GYWDtwPXAZMBhaa2eQ+2iUC3wJW+rrI4WRUUgwj4iJZvUuTR4vI8DKYPfTZQLFzbodzrh14Ari6j3Y/A+4Bgvrgsplx7oQM3tpaRZdOjIrIMDKYQM8Gel98XeZd18PMTgdynXPPH21DZrbIzArNrLCqquqYix0uLpqcSU1TOz97fpPuHBWRYWMwgW59rOvZNTWzMOBe4HsDbcg596BzrsA5V5CRkTH4KoeZiydnYgaLV5Ty9b+s8nc5IiLA4AK9DMjttZwDlPdaTgSmAm+ZWSkwF1gSzCdGoyPCeXLRPOafmsbSoip+/WqRpqgTEb8bTKB/CIw3s3wziwKuA5YcfNI5V+ecS3fO5Tnn8oD3gaucc4VDUvEwMTs/lfu/cAYAv3mzmM0VDX6uSERC3YCB7pzrBG4GXgE2A0855zaa2U/N7KqhLnA4S4mL4r5rZwJQrMsYRcTPIgbTyDn3IvDiYetu66fteSdeVuBYMHUUPAklutFIRPxMd4qeoJjIcLJTYimp1h66iPiXAt0HJmQm8NGuAxqwS0T8SoHuA1fOGM2ummaWFlX6uxQRCWEKdB+4fFoW4zLi+fYTa3hv+35/lyMiIUqB7gMxkeH85aY5JMZE8CtNfiEifqJA95Gs5Fi+OGcMhTtr2bW/2d/liEgIUqD70Gdn5RAVHsYDy7b7uxQRCUEKdB/KSo7liulZvLS+QkMBiMhJp0D3sXlj06ht7mBdWZ1CXUROKgW6jxXkjQDg6vvf5fLfvENdc4efKxKRUKFA97GxGQn88prpXH9WHpsr6nl2dZm/SxKREKFAHwKfK8jljqumMHFUIs+tKdehFxE5KRToQ2jh7DGs2X2AZduq/V2KiIQABfoQWjh7DLmpsdz90haN8yIiQ06BPoSiIsK4+fxT2VRRz8byen+XIyJBblCBbmYLzKzIzIrN7JY+nv+umW0ys3Vm9oaZneL7UgPTRZM884++vVUDd4nI0Bow0M0sHLgfuAyYDCw0s8mHNVsNFDjnpgN/A+7xdaGBKi0hmhk5KfxjTbkOu4jIkBrMHvpsoNg5t8M51w48AVzdu4Fzbqlz7uAAJu/jmUhavG6Yn0dxZaOG1xWRITWYQM8GdvdaLvOu68+NwEsnUlSwuXxaFtkpsTzwtsZ4EZGhM5hAtz7W9XnswMy+BBQAv+zn+UVmVmhmhVVVVYOvMsBFhodx49n5fFhay4//sV6HXkRkSAwm0MuA3F7LOUD54Y3M7CLgR8BVzrm2vjbknHvQOVfgnCvIyMg4nnoD1rVn5hIZbvzl/V3c/fIW2ju7/V2SiASZwQT6h8B4M8s3syjgOmBJ7wZmdjrwBzxhrgPFfYiPjmDt7ZcA8IdlO7j75S1+rkhEgs2Age6c6wRuBl4BNgNPOec2mtlPzewqb7NfAgnA02a2xsyW9LO5kBYXFcFjN80hLT6KR98tYV3ZAX+XJCJBxPw1zkhBQYErLCz0y2v7W31rBxf96m0qG9r4zkUT+PZF4/1dkogECDNb5Zwr6Os53SnqB0kxkfzXp6cBcO/rW9m2r8HPFYlIMFCg+8nFkzNZ/oPziYkM49oH39e46SJywhTofpSbGsej18+mpqmde1/fSle3Y39jG83tneyrb+WjXbUUV2rvXUQGJ8LfBYS6eePS+NysHBavKGXxilLCw4yE6AjqWjx77GEGCdERXD8/n+9ePMHP1YrIcKZAHwbu/Mw0nl7lmdloWnYykeHGiLgoZuSmcKC5nacKy/jtm9u4cOJIEmIiGJeR4OeKRWQ40lUuw8TKHfvZXdvCNbOOHAbnQHM78+58k5aOLk/b/7iQzKSYk12iiAwDusolAMwZm9ZnmAOkxEVxy2UTe5Z/9WpRzyEZEZGDFOgB4itn5VF61xVcf1YeTxWWceVv36G+VaEuIh/TMfQAc/uVkzlrXBpff+wjrvztO3xqZjYzx6Swr66VvPR45o5N83eJIuInCvQAY2ZcMmUUi284k/te38Zv3txG79MgXzt3HD+49DTCwvoaJNNjz4EWYiPDSY2POgkVi8jJokAPUOeMz+Cc8RlUN7ZRXNlIVUMbb2zexwNvb6ets4uvnTuO+pYORiXHkBgT2fNzew60MP+uN8kZEcvyH5yPWf/BLyKBRYEe4NIToklPiAbgk9OzSImL4tF3S3n03dKeNlfOGE1UeBh1Le2sLKkBoKy2heXbqvnEhOEzjLFzjvK6VrJTYv1dikhA0mWLQcY5xysb97KurI6nCj3Xttc0tdF7To2Hv1LAz57fRHiY8fp3z/X7XnprRxd/XLaDx1buYm99KxdOHMnPPjWV0Qp28appaue25zZwxpgRXDc7l7io0N0XPdpliwr0IOacw8woq22muLKR/PR4DjR3MCM3hSc/3MUPn1nPE4vmMj0nmbioCLbta6C6sZ25Y1MxM9o7u9lUUc/IxOijhuvB1xmMrm7HPS9vYXtVE9fMymZOfhpfenglG8vriY4IIzMphsqGVlo7upmdl8ofv1JAcmzkwBsOYKXVTby+eR+fm5VLclxw9/WgJz7YxW/fLGbeuDRuOiefiaOSep7r6nY8/M4OXtu0jzs/M423iqq497WtNLV39bT5w5dncemUUf4ofVCK9jZQ09TOhMwE0rx/QfuKAl2OUNPUzpxfvE5Hl2PiqEQeveFMrv6fd6lsaOPfPjGWaTnJvLpxH0vWeian+u7FE/jWhR8P87u3rpXqxjbe276fPy7fwZ9unH3If8renluzhwPNHZRUN7G9qpHl26oBz7AG3Q6iwsP4/ZfO4MJJmQC8v2M/f3l/J69s3EtHl+NLc8dw3ZljKKttJjU+mtn5qUPyOzn4f2EwH07OOd4qquLM/FQSovveW9xb10pmUnSf2yva28Ctz66jtaOb4spG2ru6iQoP47Ozsvn2hRMYley/G8e6uh1hdujvoa6lg5fWV/Cp07OJiQw/6s9X1reys6aZglNG9Nn3lvYuZv38NZp7BfSMnGR+dMVk9je28dqmfTy7es8hP3PO+HRuvWwSDy7bzj/WlHPRpJE89JUzD2lTtLcBh+vzfdjS3sXtSzaw6BPjOHVkAh/tquWJD3Zx52emE36UCwiOR3tnNxN+/PG0ys99cz4zclN8tn0FuvRpRXE1972xjQ+8x9XBE67tXR9Pj3fZ1FFERYTx3Jpy/vu6mVw9M5uOrm4uvXcZO6qbetpNyvL8JyqpbuSSyaO455rpRIWHcedLm/nj8pIjXnv5D87nV68W8dzacr538QRuvuDIMeH/9F4ptz238Yj1L3zrbKaMTgagurGNvXWtTM1OHlSfW9q7WFd2gNn5qUeEzf99ei3b9jXw2FfnHhHSXd2Ou1/eQnREGJdMHsWj75bw7Oo9XH9WHndcNQWAxrZOKg60kBwXyV0vbeHZj/Ywb2wa91wzndzUONo7u6lpaiclLpKLfv029S0dTMxKYlxGAmNS43h98z427KmjrbObH18xiZvOGXtIDcfyl9Dx2F3TTOHOGh5+p4SIsDDuuWY6pdVNlFQ3cedLnhm2JmUl8d2LJ/Dkh7uJjghjZm4KX553Cj/55yaqGtr4sLSm56a3L84Z0zNM9EEdXd3c8sx6nvmojB9fMYmapnYeeHs7h0+ze9PZ+ZyZn8qDy3ZQ29zOU/82r+dc0c+f38Qj75bwH5dP4okPdzMnP5WVJTUUVzYCcNdnpnH2+HRaO7p5ZeNevjB7DPe8UsTjH+zi9DEp/P0b85nzi9fZV9/Gk4vmMqePS3237K1n9a4DjE2P56NdB5iZm8LU7KRDLjDoz6sb97Loz6t6lnNTY7nnszN6/vI9UScc6Ga2APhvIBx4yDl312HPRwN/AmYB+4FrnXOlR9umAn342LCnjtc372Nmbgqz81N5af1eIiPCMDzD/IaZ8cWH3mf9njr+/o35bCyv5/8+vZZvnj+uZ1yZW55Zf8gHQXiY0dXrf+k1s3K4YX4eV/zmHQBK77oCgIbWDhKiI/p8o3d3O55bu4eIsDBa2rs4NTOBGx79kI6ubuaOTSMvLZ5H3vV8WMRFhfP5glw+c0Y207KTKa9rJTMxms5uR7dzxEVFULS3ge89vYYNe+pZMGUUew60sL+xjdPHjKBoX0NPIKTERdLZ5YiPDucb551KWW1znx9KBy36xFg2ltfxbvH+ftt864JTeWF9BdurPv4Q/OtX53DWuPRD2i3fVsWXH/4AgMunjeKuz06npKqJO1/aTFtnN0//2zwiwsPo6OomIsz6/L0VltawrqyOgrwRjE6J7QlCgOb2Tl5av5dJWUkcaG5n3MgE7l9azOXTsvj9W9t5e2vfk7eHhxlXzxjN65v3Ud/aSUJ0BKnxUeyqaeaUtDh27m8mJjKMy6dmMW5kApsq6nlhXQXXzMrh+rPyej5wf/T39Ty2chf/Oj+f//zkJMyMHVWNbK5ooGhfA/PGpjEyKZqx6fH9hl9TWydX/c87h/wuZ+amcM74dJZvq2bN7v5nAjODr8zLY/GK0p6fu3hyJlfNGE1uahwAJdVNXHrvskPezwBREWH8+vMz+OT00T3rWju6WLWzlhfWV3D1jNHMzk/lzpe2sHhFKevvuIQNe+q5/pEPaGjr5FsXjuc7F40/4VA/oUA3s3BgK3AxngmjPwQWOuc29WrzDWC6c+5rZnYd8Gnn3LVH264CPbBU1rfyyd++Q0dXN51djlHJMbz6nU/0vDn3N7axfk8dv3p1KylxkeSnx/On93YCsOrHF/UcR9xd00x7V/dxDzC2amctty/ZwIY99f22Oe+0DN4qqiIjMZrG1k7SEqL46jlj+fkLm4iP9gxutqm8vmdsHPD8yX9mXiqZSTE8uHwH6QnRbK449DW+f+lprJkbVQsAAAjhSURBVC+ro3BnLX//xlns3N/M1/6yisa2TlLiIlk4ewy7aprZub+J711yGpOzkvj2E6t5f8fHfwFdW5DLezv2c+2ZuXzz/FP7rL+4spGb//oRW/Y2MOuUEWwsr6O1wxMu//nJyRhwzytbGJeRwNfPG8cFE0fy0c4DxEWHU3GglVufXUd9a2fP9saPTOD6+Xl85vQc/t+rRTz8zscfTmnxUexvau9ZXjg7l5svGE91QxtvFVXx0PIdJMRE8Levn0V2SiwflNTw15U7uX5+PtOyk/ns71dQ09TOdy4ez6dP/3joio6ubr76p0LeKqoiPMy45owcKhtaWVpUxQ3z87j9yikD/Esf3QclNfz4H+v5yVVTiY0KZ0ZOMmZGR1c3L6yroHR/EweaO2hp7+LJwt1MHJXIn/51Nlff/y4Vda2MSophZFI068rqADgzbwR3XDWF7VVNfOvx1YDnA7elvYv46AjKalt4bOVONlfU89evzuXhd0p4YV3FEXXNzE2hdH8Tp2Yk8LevnwV4PkS///Q6XlhfgRl87+IJXDF9NPnp8cfV9xMN9HnAHc65S73LtwI45+7s1eYVb5v3zCwC2AtkuKNsXIEeeEqqm/jh39ZRsr+JOz89jYsmZx61fdHeBnbVNHPxAO2OlXOOF9fvJT89nsmjk1i+rYpTRyawetcBlqwp5+WNewEYmRhNVkosa717bGPT43n6a/NIS4imurGN2qZ2uh00tXdyxpgRR7zGw++U8LdVZZw+ZgT/9ampPTdrtXV2ER3hOY7c1NbJ04W7uWpmdp83anV1O1btrKWmqY36lk4+f2buoPv56Lsl/OSfnv2mf958Nr96rYi3ivreg+4tMSaC26+cwmMrd7J61wFGxEVS29xBZLjR0eW4eHImZ4wZwcsbKlhbVsfPPzWVyoY2Hv9gF38+7FxIS3sXYWH09PdwRzvv4Jxj5/5m/mdpMX/zjib6yelZ3PXZ6f2ed/A15xxvbK5kanYyo5JjqGrwHKP/zBnZACxZU05dSwf/9eLmQ37uiulZ3P+FMw5ZV1HXwiX3LqOh14flQfddO5OWji7ufnkL9S0d/OLT07hu9pie58sPtHDWXW/2LH/t3HGHjM90LE400K8BFjjnbvIufxmY45y7uVebDd42Zd7l7d421YdtaxGwCGDMmDGzdu7ceVwdEulPa0cXi1eUkjMitudP4901zWyqqGdOfiopcYFzd+zBS1CTYiM5a1w6TW2dPPHhbqIiwvjcrBx21zSzeW8D723fz/xT04iOCKeru5spo5N7Dh+0dnRhBku3VLJsWzX5afF8ce4Y4qIi6O52bK9qZHxm4pD3Zdu+Bt7eWsW/zs8/6l3M/vLPteU0tHYyJjWOgrwR/Z743bCnjv9dUcoV3ns+Th2ZQJjRcxlle2c3TW2djOjjw31FcTVmRrdzjEmN6/k3OlYnGuifAy49LNBnO+f+vVebjd42vQN9tnOu34OK2kMXETl2Jzp8bhnQ++/EHKC8vzbeQy7JQA0iInLSDCbQPwTGm1m+mUUB1wFLDmuzBPiK9/E1wJtHO34uIiK+N+CZCedcp5ndDLyC57LFR5xzG83sp0Chc24J8DDwZzMrxrNnft1QFi0iIkca1Klm59yLwIuHrbut1+NW4HO+LU1ERI6FZiwSEQkSCnQRkSChQBcRCRIKdBGRIOG30RbNrAo43ltF04HqAVsFF/U5NKjPoeFE+nyKc67Pqcb8FugnwswK+7tTKlipz6FBfQ4NQ9VnHXIREQkSCnQRkSARqIH+oL8L8AP1OTSoz6FhSPockMfQRUTkSIG6hy4iIodRoIuIBImAC3QzW2BmRWZWbGa3+LseXzGzR8ys0jv708F1qWb2mplt834f4V1vZvYb7+9gnZmd0f+Why8zyzWzpWa22cw2mtm3veuDtt9mFmNmH5jZWm+ff+Jdn29mK719ftI7VDVmFu1dLvY+n+fP+o+XmYWb2Woze967HNT9BTCzUjNbb2ZrzKzQu25I39sBFejeCavvBy4DJgMLzWyyf6vymcXAgsPW3QK84ZwbD7zhXQZP/8d7vxYBvz9JNfpaJ/A959wkYC7wTe+/ZzD3uw24wDk3A5gJLDCzucDdwL3ePtcCN3rb3wjUOudOBe71tgtE3wZ6T9wZ7P096Hzn3Mxe15wP7XvbORcwX8A84JVey7cCt/q7Lh/2Lw/Y0Gu5CMjyPs4CiryP/wAs7KtdIH8BzwEXh0q/gTjgI2AOnrsGI7zre97neOYhmOd9HOFtZ/6u/Rj7meMNrwuA5wEL5v726ncpkH7YuiF9bwfUHjqQDezutVzmXResMp1zFQDe7yO964Pu9+D90/p0YCVB3m/v4Yc1QCXwGrAdOOCcOzidfO9+9fTZ+3wdkHZyKz5h9wE/ALq9y2kEd38PcsCrZrbKzBZ51w3pe3tQE1wMI31NFx6K110G1e/BzBKAZ4D/45yrN+t3Vvig6LdzrguYaWYpwN+BSX01834P6D6b2SeBSufcKjM77+DqPpoGRX8PM985V25mI4HXzGzLUdr6pN+Btoc+mAmrg8k+M8sC8H6v9K4Pmt+DmUXiCfPHnHPPelcHfb8BnHMHgLfwnD9I8U6wDof2K9AnYJ8PXGVmpcATeA673Efw9reHc67c+70Szwf3bIb4vR1ogT6YCauDSe/Jt7+C5xjzwfX/4j0zPheoO/hnXCAxz674w8Bm59yvez0VtP02swzvnjlmFgtchOdk4VI8E6zDkX0O2AnYnXO3OudynHN5eP6/vumc+yJB2t+DzCzezBIPPgYuATYw1O9tf584OI4TDZcDW/Ecd/yRv+vxYb8eByqADjyf1jfiOXb4BrDN+z3V29bwXO2zHVgPFPi7/uPs89l4/qxcB6zxfl0ezP0GpgOrvX3eANzmXT8W+AAoBp4Gor3rY7zLxd7nx/q7DyfQ9/OA50Ohv97+rfV+bTyYVUP93tat/yIiQSLQDrmIiEg/FOgiIkFCgS4iEiQU6CIiQUKBLiISJBToIiJBQoEuIhIk/j/KA4qPBO5fJAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "w, ws, Rs= SGD(x_noise, y)\n",
    "dist = [ np.linalg.norm(ws[iter]/np.linalg.norm(ws[iter]) - w0) for iter in range(len(ws))] \n",
    "\n",
    "print(\"w estimated : \", w/np.linalg.norm(w))\n",
    "\n",
    "plt.plot(Rs)\n",
    "plt.title(\"Empirical risk\")\n",
    "plt.show()\n",
    "\n",
    "plt.plot(dist)\n",
    "plt.title(\"||w - w0||\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The results seem less accurate than before, but is still relatively good"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Test the algorithm on the Breast Cancer Wisconsin (Diagnostic) Data Set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "569  observations in the dataset\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>diagnosis</th>\n",
       "      <th>radius_mean</th>\n",
       "      <th>texture_mean</th>\n",
       "      <th>perimeter_mean</th>\n",
       "      <th>area_mean</th>\n",
       "      <th>smoothness_mean</th>\n",
       "      <th>compactness_mean</th>\n",
       "      <th>concavity_mean</th>\n",
       "      <th>concave points_mean</th>\n",
       "      <th>...</th>\n",
       "      <th>texture_worst</th>\n",
       "      <th>perimeter_worst</th>\n",
       "      <th>area_worst</th>\n",
       "      <th>smoothness_worst</th>\n",
       "      <th>compactness_worst</th>\n",
       "      <th>concavity_worst</th>\n",
       "      <th>concave points_worst</th>\n",
       "      <th>symmetry_worst</th>\n",
       "      <th>fractal_dimension_worst</th>\n",
       "      <th>Unnamed: 32</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>842302</td>\n",
       "      <td>M</td>\n",
       "      <td>17.99</td>\n",
       "      <td>10.38</td>\n",
       "      <td>122.80</td>\n",
       "      <td>1001.0</td>\n",
       "      <td>0.11840</td>\n",
       "      <td>0.27760</td>\n",
       "      <td>0.3001</td>\n",
       "      <td>0.14710</td>\n",
       "      <td>...</td>\n",
       "      <td>17.33</td>\n",
       "      <td>184.60</td>\n",
       "      <td>2019.0</td>\n",
       "      <td>0.1622</td>\n",
       "      <td>0.6656</td>\n",
       "      <td>0.7119</td>\n",
       "      <td>0.2654</td>\n",
       "      <td>0.4601</td>\n",
       "      <td>0.11890</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>842517</td>\n",
       "      <td>M</td>\n",
       "      <td>20.57</td>\n",
       "      <td>17.77</td>\n",
       "      <td>132.90</td>\n",
       "      <td>1326.0</td>\n",
       "      <td>0.08474</td>\n",
       "      <td>0.07864</td>\n",
       "      <td>0.0869</td>\n",
       "      <td>0.07017</td>\n",
       "      <td>...</td>\n",
       "      <td>23.41</td>\n",
       "      <td>158.80</td>\n",
       "      <td>1956.0</td>\n",
       "      <td>0.1238</td>\n",
       "      <td>0.1866</td>\n",
       "      <td>0.2416</td>\n",
       "      <td>0.1860</td>\n",
       "      <td>0.2750</td>\n",
       "      <td>0.08902</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>84300903</td>\n",
       "      <td>M</td>\n",
       "      <td>19.69</td>\n",
       "      <td>21.25</td>\n",
       "      <td>130.00</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>0.10960</td>\n",
       "      <td>0.15990</td>\n",
       "      <td>0.1974</td>\n",
       "      <td>0.12790</td>\n",
       "      <td>...</td>\n",
       "      <td>25.53</td>\n",
       "      <td>152.50</td>\n",
       "      <td>1709.0</td>\n",
       "      <td>0.1444</td>\n",
       "      <td>0.4245</td>\n",
       "      <td>0.4504</td>\n",
       "      <td>0.2430</td>\n",
       "      <td>0.3613</td>\n",
       "      <td>0.08758</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>84348301</td>\n",
       "      <td>M</td>\n",
       "      <td>11.42</td>\n",
       "      <td>20.38</td>\n",
       "      <td>77.58</td>\n",
       "      <td>386.1</td>\n",
       "      <td>0.14250</td>\n",
       "      <td>0.28390</td>\n",
       "      <td>0.2414</td>\n",
       "      <td>0.10520</td>\n",
       "      <td>...</td>\n",
       "      <td>26.50</td>\n",
       "      <td>98.87</td>\n",
       "      <td>567.7</td>\n",
       "      <td>0.2098</td>\n",
       "      <td>0.8663</td>\n",
       "      <td>0.6869</td>\n",
       "      <td>0.2575</td>\n",
       "      <td>0.6638</td>\n",
       "      <td>0.17300</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>84358402</td>\n",
       "      <td>M</td>\n",
       "      <td>20.29</td>\n",
       "      <td>14.34</td>\n",
       "      <td>135.10</td>\n",
       "      <td>1297.0</td>\n",
       "      <td>0.10030</td>\n",
       "      <td>0.13280</td>\n",
       "      <td>0.1980</td>\n",
       "      <td>0.10430</td>\n",
       "      <td>...</td>\n",
       "      <td>16.67</td>\n",
       "      <td>152.20</td>\n",
       "      <td>1575.0</td>\n",
       "      <td>0.1374</td>\n",
       "      <td>0.2050</td>\n",
       "      <td>0.4000</td>\n",
       "      <td>0.1625</td>\n",
       "      <td>0.2364</td>\n",
       "      <td>0.07678</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 33 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         id diagnosis  radius_mean  texture_mean  perimeter_mean  area_mean  \\\n",
       "0    842302         M        17.99         10.38          122.80     1001.0   \n",
       "1    842517         M        20.57         17.77          132.90     1326.0   \n",
       "2  84300903         M        19.69         21.25          130.00     1203.0   \n",
       "3  84348301         M        11.42         20.38           77.58      386.1   \n",
       "4  84358402         M        20.29         14.34          135.10     1297.0   \n",
       "\n",
       "   smoothness_mean  compactness_mean  concavity_mean  concave points_mean  \\\n",
       "0          0.11840           0.27760          0.3001              0.14710   \n",
       "1          0.08474           0.07864          0.0869              0.07017   \n",
       "2          0.10960           0.15990          0.1974              0.12790   \n",
       "3          0.14250           0.28390          0.2414              0.10520   \n",
       "4          0.10030           0.13280          0.1980              0.10430   \n",
       "\n",
       "   ...  texture_worst  perimeter_worst  area_worst  smoothness_worst  \\\n",
       "0  ...          17.33           184.60      2019.0            0.1622   \n",
       "1  ...          23.41           158.80      1956.0            0.1238   \n",
       "2  ...          25.53           152.50      1709.0            0.1444   \n",
       "3  ...          26.50            98.87       567.7            0.2098   \n",
       "4  ...          16.67           152.20      1575.0            0.1374   \n",
       "\n",
       "   compactness_worst  concavity_worst  concave points_worst  symmetry_worst  \\\n",
       "0             0.6656           0.7119                0.2654          0.4601   \n",
       "1             0.1866           0.2416                0.1860          0.2750   \n",
       "2             0.4245           0.4504                0.2430          0.3613   \n",
       "3             0.8663           0.6869                0.2575          0.6638   \n",
       "4             0.2050           0.4000                0.1625          0.2364   \n",
       "\n",
       "   fractal_dimension_worst  Unnamed: 32  \n",
       "0                  0.11890          NaN  \n",
       "1                  0.08902          NaN  \n",
       "2                  0.08758          NaN  \n",
       "3                  0.17300          NaN  \n",
       "4                  0.07678          NaN  \n",
       "\n",
       "[5 rows x 33 columns]"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"data.csv\",header = 0)\n",
    "print(df.shape[0], \" observations in the dataset\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.drop(\"Unnamed: 32\",1)\n",
    "x = df.iloc[:,2:] \n",
    "y = df.diagnosis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "le = LabelEncoder()\n",
    "y = le.fit_transform(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = (x - x.mean())/ np.sqrt(x.var())\n",
    "x_np = x.to_numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection  import train_test_split\n",
    "x_train, x_test, y_train, y_test = train_test_split(x_np, y, test_size=0.20, random_state=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w estimated :  [ 0.38127251  0.04831026 -0.18291229 -0.01426605  0.28136352  0.30300141\n",
      " -0.01443595 -0.38641629  0.25410684 -0.11719687 -0.05078908  0.07906987\n",
      " -0.097882   -0.04137002  0.16112217 -0.16299693  0.22066419 -0.22795517\n",
      "  0.12206528 -0.09284573  0.28597531 -0.01611214 -0.13893487  0.1677988\n",
      " -0.08172362  0.00078073  0.25929497 -0.09928938 -0.09400798  0.02985984]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEICAYAAABGaK+TAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3dd3Qc53nv8e+zC4AgQBQCBECwk2KVqQ7LsmRJlmQ5KrYp95aE9tUR48SO7OP4RspVclN9E+feuN3k2JZtFSeypVwrKu6iZUm2VQmKxWxi7yABVrCi7XP/mAG5ggCi7g5m9/c5B2dm3pndeThY/vbFu7Mz5u6IiEj8JKIuQEREhkYBLiISUwpwEZGYUoCLiMSUAlxEJKYU4CIiMaUAl5xjZsfNbNY51n/TzP5qmPt4u5ntHsm6hvq8kr8Koi5A8oOZbQfqgK605gfc/TMjvS93H9fP+k+N9D4Hor+6RAZLAS7Z9G53/2WUBZhZ0t27+t9yRPdZ4O6d2dyn5AcNoUjkzOwTZva8mX3FzI6Y2VYzuzJs32VmzWa2OG37B8JhkKVmdszMnjOz6Wnr3cxmp237DTP7qZmdAK4L2/4hbftFZrbSzFrNbIuZ3RS2f9LM1of72GpmfzSIf5Ob2afNbBOwqZe6bjGzdeFz7zGzL/TxPHeG200Z3FGVfKAAl9HiLcBqoBr4PvAw8GZgNvD7wL+aWfoQxMeBvwcmACuBh87x3B8DvgiUAb9NX2FmlwPfA/47UAlcA2wPVzcD7wLKgU8CXzGzSwfxb7ot/Hed38u67wJ/5O5lwELgVz03CMfpPwFc6+4aF5c3UIBLNj0e9rC7f+5IW7fN3e8PhzceAaYCf+fube7+FNBOEObdfuLuv3b3NuAe4K1mNrWP/T7h7s+7e8rdT/dYdztwn7svDdfvcfcNAO7+E3ff4oHngKeAqwfx7/1Hdz/k7qd6WdcBnG9m5e5+2N1fTVtnZvZl4PeA69y9ZRD7lDyiAJdsus3dK9N+vp22bn/a/CkAd+/Zlt4D39U94+7HgUPApD72u6uPdgjeKLb0tsLMbjazl8zskJkdAW4h6PEP1Ln2+/7w+XaEQ0BvTVtXCSwheAM4Ooj9SZ5RgEtcnelth0MrVcDePrY91yU3dwHn9Ww0szHAo8D/AercvRL4KWCDqLHP/br7MndfBNQCjwP/mbb6MMHQzf1mdtUg9id5RgEucXWLmb3NzIoIxsJfdvdz9Xj78l3gk2Z2g5klzGyymc0HioAxQAvQaWY3A+8cicLNrMjMPm5mFe7eAbTy+tMrcfdnCcb5HzOzt4zEfiX3KMAlm34Ufpml++exYTzX94G/Jhg6uYwg7AbN3V8h/IASOAo8B0x392PAnQQ948MEH4Q+OYx6e/oDYLuZtQKfIvigtmdtS8PanjSzy0Zw35IjTDd0kLgxsweA3e7+l1HXIhIl9cBFRGJKAS4iElMaQhERiSn1wEVEYiqrF7OaMGGCz5gxI5u7FBGJveXLlx9w95qe7VkN8BkzZtDY2JjNXYqIxJ6Z7eitXUMoIiIxpQAXEYkpBbiISEwpwEVEYkoBLiISUwpwEZGYUoCLiMRULAL88RV7+I+Xej0NUkQkb8UiwH+2pon7n98WdRkiIqPKgALczCrN7IdmtsHM1pvZW82sysyWmtmmcDo+U0XOnDCOnYdO0tmVytQuRERiZ6A98K8BP3f3+cBFwHrgbuBpd58DPB0uZ8SsCaV0dDl7jvR2c28RkfzUb4CbWTlwDcG9A3H3dnc/AiwCHgw3exC4LVNFzqwpBWDrgROZ2oWISOwMpAc+i+DGrveb2Qoz+46ZlRLcqbsJIJzW9vZgM1tiZo1m1tjS0jKkImdOCAJ8W4sCXESk20ACvAC4FPiGu18CnGAQwyXufq+7N7h7Q03NG66GOCDVpUWUFRewTT1wEZEzBhLguwluIPtyuPxDgkDfb2b1AOG0OTMlgpkxa0KpAlxEJE2/Ae7u+4BdZjYvbLoBWAc8CSwO2xYDT2SkwtBMBbiIyOsM9IYOfwo8ZGZFwFbgkwTh/59mdjuwE/hgZkoMzJwwjsdX7uV0RxfFhclM7kpEJBYGFODuvhJo6GXVDSNbTt+6z0TZfvAE8yeWZ2u3IiKjViy+iQnBueCgM1FERLrFJsBnTNC54CIi6WIT4OPGFFBbNkYfZIqIhGIT4KAzUURE0sUqwGfVKMBFRLrFKsBnTijl0Il2jpxsj7oUEZHIxSzAxwGoFy4iQuwCPDyVUAEuIhKvAJ9WVULCFOAiIhCzAC8qSDC1qkTngouIELMAh2AYZau+jSkiEr8AP69mHFtbjtOV8qhLERGJVOwCfG7dONo6U+w+fDLqUkREIhW7AJ9TVwbAxv3HI65ERCRasQvw2bXBueAb9x+LuBIRkWjFLsDLiwupryhmc7N64CKS32IX4BD0wtUDF5F8F8sAn1tXxuZmnYkiIvktlgE+p1ZnooiIxDPAwzNRNulMFBHJYzEN8PBMlGaNg4tI/oplgJcXFzKxvJjN6oGLSB6LZYBD0AtXD1xE8ll8A7w2OBMlpTNRRCRPDSjAzWy7mf3OzFaaWWPYVmVmS81sUzgdn9lSX29u3ThOd6TYffhUNncrIjJqDKYHfp27X+zuDeHy3cDT7j4HeDpczpruDzI3aRhFRPLUcIZQFgEPhvMPArcNv5yBm12ri1qJSH4baIA78JSZLTezJWFbnbs3AYTT2t4eaGZLzKzRzBpbWlqGX3GoYmxwJop64CKSrwoGuN1V7r7XzGqBpWa2YaA7cPd7gXsBGhoaRvQTxzl14/RlHhHJWwPqgbv73nDaDDwGXA7sN7N6gHDanKki+zK3roxNzcd0TRQRyUv9BriZlZpZWfc88E5gDfAksDjcbDHwRKaK7MuC+nJOd6R0l3oRyUsDGUKpAx4zs+7tv+/uPzezZcB/mtntwE7gg5krs3fn15cDsL6p9cyNHkRE8kW/Ae7uW4GLemk/CNyQiaIGanbtOAqTxrqmVt590aQoSxERybrYfhMToKggwezaMtbtbY26FBGRrIt1gAMsqC9jfZMCXETyT+wD/Pz6cpqPtXHgeFvUpYiIZFX8A3zS2Q8yRUTySfwDPDwTRePgIpJvYh/glSVFTKooZp164CKSZ2If4BB8oUdDKCKSb3IiwM+fVM6WlhOc7uiKuhQRkazJjQCvL6cr5WzcrysTikj+yIkAX1CvM1FEJP/kRIBPqyqhtCipM1FEJK/kRIAnEsb8+nLWN2kIRUTyR04EOATj4OuaWnWXehHJGzkT4BdMruB4WyfbDura4CKSH3ImwC+cWgHA6t1HIq5ERCQ7cibAZ9eMY2xhklW7jkZdiohIVuRMgBckE1wwuYJV6oGLSJ7ImQAHuHBKBev2ttLRlYq6FBGRjMutAJ9aSVtnitf26XRCEcl9ORXgF03p/iBT4+AikvtyKsCnVZVQWVLIql0aBxeR3JdTAW5mXDilUh9kikheyKkAh2AYZVPzcU6169KyIpLbBhzgZpY0sxVm9uNweaaZvWxmm8zsETMrylyZA3fhlEq6Us7avRoHF5HcNpge+GeB9WnLXwK+4u5zgMPA7SNZ2FB1f5C5UuPgIpLjBhTgZjYFuBX4TrhswPXAD8NNHgRuy0SBg1VbXkx9RbHORBGRnDfQHvhXgT8Hur8hUw0ccffOcHk3MLm3B5rZEjNrNLPGlpaWYRU7UBdOqdA1UUQk5/Ub4Gb2LqDZ3ZenN/eyaa/XcXX3e929wd0bampqhljm4Fw0tZLtB09y+ER7VvYnIhKFgfTArwLeY2bbgYcJhk6+ClSaWUG4zRRgb0YqHILLpo0HYPmOwxFXIiKSOf0GuLv/hbtPcfcZwEeAX7n7x4FngA+Emy0GnshYlYN00dRKCpPGsh2Hoi5FRCRjhnMe+F3A581sM8GY+HdHpqThKy5MsnByBcu3qwcuIrmroP9NznL3Z4Fnw/mtwOUjX9LIePOMKh54fjunO7ooLkxGXY6IyIjLuW9idrts+njau1Ks2aPTCUUkN+VsgDdMDz7IXKZhFBHJUTkb4NXjxjBrQinL9UGmiOSonA1wgIYZ42nccZhUqtdT1EVEYi23A3x6FUdOdrD1wPGoSxERGXG5HeAzNA4uIrkrpwN85oRSqkuLaFSAi0gOyukANzMumz6eRn2QKSI5KKcDHIIv9Ow4eJLm1tNRlyIiMqJyPsCvmFUNwAtbDkZciYjIyMr5AD9/UjkVYwt5YcuBqEsRERlROR/gyYRxxawqnt98EHedDy4iuSPnAxzgqtkT2HPkFLsOnYq6FBGREZMXAX7lecE4+PMaRhGRHJIXAX5ezThqy8bw/GYFuIjkjrwIcDPjyvOqeXGLxsFFJHfkRYADXDl7AgdPtPPa/mNRlyIiMiLyJ8DDcfAXNut8cBHJDXkT4FPGlzC9ukTng4tIzsibAIegF/7y1kN0dqWiLkVEZNjyKsCvmj2BY22drNp9JOpSRESGLa8C/OrZNSQTxjMbWqIuRURk2PIqwCtKCrls2nieea056lJERIYtrwIc4O3za1i7t5X9urysiMRcvwFuZsVm9oqZrTKztWb2t2H7TDN72cw2mdkjZlaU+XKH77p5tQA8q164iMTcQHrgbcD17n4RcDFwk5ldAXwJ+Iq7zwEOA7dnrsyRM39iGfUVxRoHF5HY6zfAPdB9W/fC8MeB64Efhu0PArdlpMIRZma8fV4tv918gPZOnU4oIvE1oDFwM0ua2UqgGVgKbAGOuHtnuMluYHIfj11iZo1m1tjSMjp6vdfPr+V4WyeN23WvTBGJrwEFuLt3ufvFwBTgcmBBb5v18dh73b3B3RtqamqGXukIuvK8aoqSCX61QePgIhJfgzoLxd2PAM8CVwCVZlYQrpoC7B3Z0jKndEwBb5lVpdMJRSTWBnIWSo2ZVYbzY4F3AOuBZ4APhJstBp7IVJGZcN28Wra0nGDnwZNRlyIiMiQD6YHXA8+Y2WpgGbDU3X8M3AV83sw2A9XAdzNX5si78fw6AH6xdl/ElYiIDE1Bfxu4+2rgkl7atxKMh8fS1KoS3jSpnJ+uaeKOa2ZFXY6IyKDl3Tcx0928cCIrdh6h6ahudiwi8ZPXAX7TwnoAfrFGwygiEj95HeCza8cxt24cP1OAi0gM5XWAQ9ALX7b9EAeOt0VdiojIoOR9gN+8cCIph6fW7o+6FBGRQcn7AJ8/sYwZ1SX8bE1T1KWIiAxK3ge4mXHTwnpe3HKQIyfboy5HRGTA8j7AAW69oJ7OlOvDTBGJFQU4sHByObNqSnns1T1RlyIiMmAKcIJhlPddMplXth9i1yFdG0VE4kEBHlp0cXA588dXqBcuIvGgAA9NrSrh8plVPLZiD+69XtpcRGRUUYCned8lk9l64ASrdh+NuhQRkX4pwNPcfEE9RQUJHnt1d9SliIj0SwGepmJsITcuqONHq5vo6NINj0VkdFOA9/DeSyZz6EQ7z+h+mSIyyinAe7h2Xg21ZWP4wSs7oy5FROScFOA9FCYTfOTNU3l2Y4vOCReRUU0B3osPXz4NAx5ZtivqUkRE+qQA78XkyrFcP7+Wh5ft0oeZIjJqKcD78LG3TOPA8TaWrtN1wkVkdFKA9+HaubVMrhzLQy/viLoUEZFeKcD7kEwYH718Ks9vPsjWluNRlyMi8gb9BriZTTWzZ8xsvZmtNbPPhu1VZrbUzDaF0/GZLze7PvTmqRQmje+9qF64iIw+A+mBdwJ/5u4LgCuAT5vZ+cDdwNPuPgd4OlzOKbVlxSy6eDKPLNulu/WIyKjTb4C7e5O7vxrOHwPWA5OBRcCD4WYPArdlqsgo3XH1LE51dPEfL6kXLiKjy6DGwM1sBnAJ8DJQ5+5NEIQ8UNvHY5aYWaOZNba0tAyv2gjMm1jGtXNreOCFHZzu6Iq6HBGRMwYc4GY2DngU+Jy7tw70ce5+r7s3uHtDTU3NUGqM3JJrZnHgeBtPrNTNHkRk9BhQgJtZIUF4P+Tu/xU27zez+nB9PZCzV3+68rxqzq8v59u/2UYqpZs9iMjoMJCzUAz4LrDe3b+ctupJYHE4vxh4YuTLGx3MjCXXzGJz83F+pasUisgoMZAe+FXAHwDXm9nK8OcW4J+AG81sE3BjuJyzbr2wninjx/L1X23SLddEZFQo6G8Dd/8tYH2svmFkyxm9CpMJ7rx+Dn/+6Gp+taGZGxbURV2SiOQ5fRNzEN576WSmVZXw5aUb1QsXkcgpwAehMJngzhvmsHZvK0/pIlciEjEF+CDddvEkZk4o5au/3KQzUkQkUgrwQSpIJrjzhtmsb2rl52v3RV2OiOQxBfgQvOeiycypHceXfr6B9k7d8EFEoqEAH4Jkwrjn1gXsOHiS7724PepyRCRPKcCH6O3zarl2bg1fe3oTh07oSoUikn0K8GH4y1sXcLK9i6/+cmPUpYhIHlKAD8OcujI+dvk0Hnp5J5v2H4u6HBHJMwrwYfrcO+ZQUpTkr59cqy/3iEhWKcCHqXrcGO66aT4vbDnIo6/qcrMikj0K8BHwscuncdn08fzDT9Zx8Hhb1OWISJ5QgI+ARML4x/ddwIm2Tr74k/VRlyMieUIBPkLm1pXxx9eex3+t2MOvN8bv1nEiEj8K8BH0J9fNZlZNKXc9upqjJzuiLkdEcpwCfAQVFyb56ocvpuVYG3/5xBqdlSIiGaUAH2EXTqnkc++Yw49W7eWJlXujLkdEcpgCPAP++O2zaZg+nr96fA27Dp2MuhwRyVEK8AxIJoyvfPhiHLjz4RW6YqGIZIQCPEOmVpXwpfdfyIqdR/j7H6+LuhwRyUEK8Ay69cJ6llwzi39/aQc/XL476nJEJMcowDPsz39vHm+dVc09j/2ONXuORl2OiOQQBXiGFSQT/OvHLqG6tIg7vtfIvqOnoy5JRHKEAjwLqseN4duLG2g91cEnH1jGsdP6ko+IDF+/AW5m95lZs5mtSWurMrOlZrYpnI7PbJnx96ZJFXzj9y9j0/5j/MlDr9LRpTNTRGR4BtIDfwC4qUfb3cDT7j4HeDpcln5cM7eGf3zfBfxm0wHuenQ1qZS+qSkiQ1fQ3wbu/mszm9GjeRHw9nD+QeBZ4K4RrCtnfbBhKvuOnuZflm5kTEGCL952AYmERV2WiMRQvwHehzp3bwJw9yYzq+1rQzNbAiwBmDZt2hB3l1s+c/1s2jpT/Oszm0kmjL9ftBAzhbiIDM5QA3zA3P1e4F6AhoYGjRkAZsafvXMuHV0pvvXrrSTN+Ot3v0k9cREZlKEG+H4zqw973/VA80gWlQ/MjLtvnk9XyvnOb7fRerqTf/7AhRQmdWKQiAzMUNPiSWBxOL8YeGJkyskvZsY9ty7gC++cy2Mr9rDke42cau+KuiwRiYmBnEb4A+BFYJ6Z7Taz24F/Am40s03AjeGyDIGZ8Znr5/C/3nsBz21s4WPfeYmWY7qvpoj0z7J504GGhgZvbGzM2v7i5udr9vG5R1YwvqSIb/9hAwsnV0RdkoiMAma23N0berZrwHUUuWnhRH74qSsx4APffIEfrdINIUSkbwrwUWbh5Aqe+MzbWDipgj/9wQr+6vE1nO7QuLiIvJECfBSqKRvD9++44sylaG/7t+fZ3Hw86rJEZJRRgI9SRQUJ/sctC7j/E2+m+Vgb7/q/v+G+327T1+9F5AwF+Ch33fxafvbZq3nrrGr+7sfr+NC3XmRri3rjIqIAj4W68mLu+8Sb+ZcPXsTG/ce46Wu/4ctLN2psXCTPKcBjwsx4/2VTWPr5a7npTRP5+tObeMeXn+Pna/aRzVNBRWT0UIDHTF15MV//6CX84I4rKC0q4FP/sZwPfetFGrcfiro0EckyBXhMvfW8an5y59v44nsXsuPgST7wzRe5/YFlrNh5OOrSRCRL9E3MHHCqvYv7X9jGt57bytFTHVwxq4pPXXse186t0WVqRXJAX9/EVIDnkONtnTz8yk6+85tt7Gs9zYL6cu64eia3XFBPcWEy6vJEZIgU4HmkvTPF4yv38K3ntrCl5QQVYwt5/6VT+OjlU5lTVxZ1eSIySArwPJRKOS9tPcj3X9nJL9buo6PLaZg+nvdeOpmbF9ZTVVoUdYkiMgAK8Dx38Hgbj766m4eX7WJrywmSCePK86p594WTeOeb6qgsUZiLjFYKcAHA3VnX1MqPVzfx49V72XXoFMmEcdn08Vw3r5br5tcwr65MH36KjCIKcHkDd2f17qM8tW4fz2xoYV1TKwCTKop525wJXDGrmitmVTOpcmzElYrkNwW49Gvf0dM8+1ozz7zWzEtbD3H0VAcAU6vG8paZ1Vw8tZKLplQyb2IZRQX6CoFItijAZVBSKWfDvmO8vO0gL209yLLthzl0oh0IrpS4oL6ci6dUcMGUSuZPLGN27TidqiiSIQpwGRZ3Z/fhU6zafYTVu4+yatcR1uw5yonwJswJg+nVpcypHce8iWXMqStjZnUp06pLqBhbGHH1IvHWV4AXRFGMxI+ZMbWqhKlVJbzrwkkAdKWcbQeO89q+42zcf+zMz9MbmulKu255ZUkh08PHTq8uYXpVKVOqxlJfMZaJ5cWMLVLPXWQoFOAyZMmEMbu2jNm1ZdxK/Zn20x1dbDtwgh0HT7Dz0El2HDzJzkMnWb37KD9bs+914Q5QXlxAfcVY6iqKmVg+hokVY6kpG0N1aRFVpUVnppUlRSQTOjtGpJsCXEZccWGSBfXlLKgvf8O6zq4Ue4+cZtfhk+w7epp9rafZ33r6zPyGplZajrfR28ieGYwvCcK8qrSIqpIiyooLKB9bGEyLC3vMh9PiQkrGJClM6oNXyS0KcMmqgmSCadUlTKsu6XObzq4Uh062c/B4O4dOtHPwRDuHjrednQ+nWw8c59jpTlpPdZwZiz+XwqQxtjBJSVEBJUVJxhYlKS0qYGxR8sxySVGwPtguyZiCBEUF3dPEmWkwH7SPSVtO36YgYTqfXjJqWAFuZjcBXwOSwHfc/Z9GpCrJawXJBLVlxdSWFQ/4MZ1dKY63ddJ6qpPW0x3BT/f8qQ5Otndxsr2LU+2dwXxHF6fauzjZ3smRk+3sPRKu7wjaTnekhv3vMIOiZILCZIKCpFGQCEK9IGkUJhMkE0ZB4ux8Yfc2SQu3S5yZFiYs2D6ZoDBpZx6bSBhJC5bNuuchkTAS4XIwHwx5JSxsTxBOw7bweRJGj+cMHpe08PkTZx/b/WMW/FsNI5EIpmbBB9sQPKeFz929znq0JQyw4HkNzj4vhiV4XVsifFM8u03afJ69YQ45wM0sCfwbcCOwG1hmZk+6+7qRKk5koAqSCSpLikbskgCplHOqo4u2zhTtnSnaOrvCaarXtrPTLtq7UrR1pIJpZ4rOLqczlaIz5XR2dS8HbR1dTlfK6Qjbu1LO6c6us9t0pc5s29nl4fbhfCpFyoNaU+7oftcB6/FGgL3xDcDo8SYSBn93O5x9c+h+I+HMfNr21vf68Gnofku5/xOXn/Mvz6EYTg/8cmCzu28FMLOHgUWAAlxiL5EwSscUUDom6koGzsMQ7zoT6B7OByHf5R6GPWnznrZ98Niey6lw2+Ax4WPDNndIuePh/oNlcM6ug3Cb7nXh/NltgvlU8CRntgnaX799sC7teVPd++6tjmA5FTw4mE+dbfO0/cDZerqfj+7lPtY7Z5+3+/ifXXd2OXwwYwpH/jOY4QT4ZGBX2vJu4C09NzKzJcASgGnTpg1jdyJyLsEQCjpTJ48M5y2ht1fJG/6Ic/d73b3B3RtqamqGsTsREUk3nADfDUxNW54C7B1eOSIiMlDDCfBlwBwzm2lmRcBHgCdHpiwREenPkMfA3b3TzD4D/ILgNML73H3tiFUmIiLnNKzzwN39p8BPR6gWEREZBH23WEQkphTgIiIxpQAXEYmprN7QwcxagB1DfPgE4MAIljNSRmtdMHprU12Do7oGb7TWNtS6prv7G75Ik9UAHw4za+ztjhRRG611weitTXUNjuoavNFa20jXpSEUEZGYUoCLiMRUnAL83qgL6MNorQtGb22qa3BU1+CN1tpGtK7YjIGLiMjrxakHLiIiaRTgIiIxFYsAN7ObzOw1M9tsZndHWMdUM3vGzNab2Voz+2zY/jdmtsfMVoY/t0RQ23Yz+124/8awrcrMlprZpnA6Pss1zUs7JivNrNXMPhfV8TKz+8ys2czWpLX1eows8PXwNbfazC7Ncl3/28w2hPt+zMwqw/YZZnYq7dh9M8t19fm7M7O/CI/Xa2b2e1mu65G0mrab2cqwPZvHq698yNxrLLj90Oj9IbjS4RZgFlAErALOj6iWeuDScL4M2AicD/wN8IWIj9N2YEKPtn8G7g7n7wa+FPHvcR8wParjBVwDXAqs6e8YAbcAPyO4cckVwMtZruudQEE4/6W0umakbxfB8er1dxf+P1gFjAFmhv9nk9mqq8f6fwH+ZwTHq698yNhrLA498DP33nT3dqD73ptZ5+5N7v5qOH8MWE9wa7nRahHwYDj/IHBbhLXcAGxx96F+E3fY3P3XwKEezX0do0XA9zzwElBpZvXZqsvdn3L3znDxJYIbpmRVH8erL4uAh929zd23AZsJ/u9mtS4L7jb8IeAHmdj3uZwjHzL2GotDgPd2783IQ9PMZgCXAC+HTZ8J/wy6L9tDFSEHnjKz5RbchxSgzt2bIHhxAbUR1NXtI7z+P1XUx6tbX8doNL3u/htBT63bTDNbYWbPmdnVEdTT2+9utByvq4H97r4prS3rx6tHPmTsNRaHAB/QvTezyczGAY8Cn3P3VuAbwHnAxUATwZ9w2XaVu18K3Ax82syuiaCGXllwx6b3AP8vbBoNx6s/o+J1Z2b3AJ3AQ2FTEzDN3S8BPg9838zKs1hSX7+7UXG8gI/y+o5C1o9XL/nQ56a9tA3qmMUhwEfVvTfNrJDgl/OQu/8XgLvvd/cud08B3yZDfzqei7vvDafNwGNhDfu7/yQLp83Zrit0M/Cqu+8Pa4z8eKXp6xhF/rozs8XAu4CPezhoGg5RHAznlxOMNc/NVk3n+N2NhuNVALwPeKS7LdvHq7d8IIOvsTgE+Ki592Y4vvZdYL27fzmtPX3c6r3Amp6PzXBdpadRSfEAAAEySURBVGZW1j1P8AHYGoLjtDjcbDHwRDbrSvO6XlHUx6uHvo7Rk8AfhmcKXAEc7f4zOBvM7CbgLuA97n4yrb3GzJLh/CxgDrA1i3X19bt7EviImY0xs5lhXa9kq67QO4AN7r67uyGbx6uvfCCTr7FsfDo7Ap/u3kLwie4W4J4I63gbwZ84q4GV4c8twL8DvwvbnwTqs1zXLIIzAFYBa7uPEVANPA1sCqdVERyzEuAgUJHWFsnxIngTaQI6CHo/t/d1jAj+vP238DX3O6Ahy3VtJhgf7X6dfTPc9v3h73gV8Crw7izX1efvDrgnPF6vATdns66w/QHgUz22zebx6isfMvYa01fpRURiKg5DKCIi0gsFuIhITCnARURiSgEuIhJTCnARkZhSgIuIxJQCXEQkpv4/wL67YBaAM50AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "w, ws, Rs= SGD(x_train,y_train, lr = 0.001, N_iter=200)\n",
    "\n",
    "print(\"w estimated : \", w/np.linalg.norm(w))\n",
    "\n",
    "plt.plot(Rs)\n",
    "plt.title(\"Empirical risk\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = np.array([np.dot(x_test[i], w) >0 for i in range(len(x_test))])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "confusion matrix : \n",
      "[[56 16]\n",
      " [11 31]]\n",
      "----------- \n",
      "f1 score : \n",
      "0.6966292134831461\n",
      "----------- \n",
      "accuracy : \n",
      "0.7631578947368421\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import f1_score, accuracy_score, confusion_matrix\n",
    "\n",
    "def print_confusion(confusion_matrix):\n",
    "    print('Confusion matrix:')\n",
    "    print(\"{0:20} {1:10} {2}\".format(\"GT \\ Pred\", \"0\", \"1\"))\n",
    "    print(\"{0:20} {1:10} {2}\".format(\"0\", str(confusion_matrix[0,0]), str(confusion_matrix[0,1])))\n",
    "    print(\"{0:20} {1:10} {2}\".format(\"1\", str(confusion_matrix[1,0]) , str(confusion_matrix[1,1])))\n",
    "\n",
    "print(\"confusion matrix : \")\n",
    "print(confusion_matrix(y_test, y_pred))\n",
    "print(\"----------- \")\n",
    "print(\"f1 score : \")\n",
    "print(f1_score(y_test,y_pred))\n",
    "print(\"----------- \")\n",
    "print(\"accuracy : \")\n",
    "print(accuracy_score(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This quick method already gives intersting results (see accuracy and f1 score above), although it is obviously possible to improve them."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}