Classification_customdataset.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.metrics import confusion_matrix\n",
    "import xgboost as xgb\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from xgboost import XGBClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.metrics import accuracy_score\n",
    "import warnings\n",
    "import seaborn as sn\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Table of content\n",
    "* [Dataset](#1)\n",
    "* [Data pre-processing](#2)\n",
    "* [Class distribution](#3)\n",
    "* [Train and test split](#4)\n",
    "* [Random Forest Classification](#5)\n",
    "* [Confusion  Matrix](#6)\n",
    "* [XGBOOST](#7)\n",
    "* [Grid Search](#8)\n",
    "* [Class Balancing](#9)\n",
    "    * [RFC](#9.1)\n",
    "    * [XGBOOST](#9.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dataset : <a id=\"1\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time, the [dataset](https://www.kaggle.com/kingburrito666/basic-computer-data-set) is not custom, it comes from Kaggle.com, a very well known site for finding datasets in the Machine and Deeplearning community, we will have to use other datasets from this platform during our projects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"Dataset/Computers.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>price</th>\n",
       "      <th>speed</th>\n",
       "      <th>hd</th>\n",
       "      <th>ram</th>\n",
       "      <th>screen</th>\n",
       "      <th>cd</th>\n",
       "      <th>multi</th>\n",
       "      <th>premium</th>\n",
       "      <th>ads</th>\n",
       "      <th>trend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1499</td>\n",
       "      <td>25</td>\n",
       "      <td>80</td>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1795</td>\n",
       "      <td>33</td>\n",
       "      <td>85</td>\n",
       "      <td>2</td>\n",
       "      <td>14</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1595</td>\n",
       "      <td>25</td>\n",
       "      <td>170</td>\n",
       "      <td>4</td>\n",
       "      <td>15</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1849</td>\n",
       "      <td>25</td>\n",
       "      <td>170</td>\n",
       "      <td>8</td>\n",
       "      <td>14</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>3295</td>\n",
       "      <td>33</td>\n",
       "      <td>340</td>\n",
       "      <td>16</td>\n",
       "      <td>14</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  price  speed   hd  ram  screen  cd multi premium  ads  trend\n",
       "0           1   1499     25   80    4      14  no    no     yes   94      1\n",
       "1           2   1795     33   85    2      14  no    no     yes   94      1\n",
       "2           3   1595     25  170    4      15  no    no     yes   94      1\n",
       "3           4   1849     25  170    8      14  no    no      no   94      1\n",
       "4           5   3295     33  340   16      14  no    no     yes   94      1"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our dataset here consists of 11 columns, it is a dataset around our previous topic, the company that sells computers.\n",
    "Here we can see computer characteristics, price, fan speeds, hard drive capacity, number of gigs of Ram, screen size, whether the computer has a CD drive, whether it has a multi-power supply, whether it has a premium OS etc...\n",
    "The shop manager asked us to classify the PCs sold by the company into 3 categories, entry-level, mid-range and high-end."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data pre-processing : <a id=\"2\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Unnamed: 0', 'price', 'speed', 'hd', 'ram', 'screen', 'cd', 'multi',\n",
       "       'premium', 'ads', 'trend'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.columns = ['ID', 'categorie', 'speed', 'hd', 'ram', 'screen', 'cd', 'multi',\n",
    "       'premium', 'ads', 'trend']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the columns, 'cd', 'multi' and 'premium', the variables take 2 distinct textual values 'yes' and 'no', in general when we want to work on data, we must avoid textual variables as much as possible (except in the case of text analysis with NLP we will come back to this in a future topic). We have therefore done some data prepocessing on our data, in fact we have done a special case of [one-hot-encoding](https://fr.wikipedia.org/wiki/Encodage_one-hot) in order to change our 'yes' and 'no' fields into 1 and 0, to facilitate the work of our model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['no', 'yes'], dtype=object)"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.unique(df['cd'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "housing_map = {'yes': 1, 'no': 0}\n",
    "df['cd'] = df['cd'].map(housing_map)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['no', 'yes'], dtype=object)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.unique(df['multi'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "housing_map = {'yes': 1, 'no': 0}\n",
    "df['multi'] = df['multi'].map(housing_map)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['no', 'yes'], dtype=object)"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.unique(df['premium'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "housing_map = {'yes': 1, 'no': 0}\n",
    "df['premium'] = df['premium'].map(housing_map)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>categorie</th>\n",
       "      <th>speed</th>\n",
       "      <th>hd</th>\n",
       "      <th>ram</th>\n",
       "      <th>screen</th>\n",
       "      <th>cd</th>\n",
       "      <th>multi</th>\n",
       "      <th>premium</th>\n",
       "      <th>ads</th>\n",
       "      <th>trend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1499</td>\n",
       "      <td>25</td>\n",
       "      <td>80</td>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1795</td>\n",
       "      <td>33</td>\n",
       "      <td>85</td>\n",
       "      <td>2</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1595</td>\n",
       "      <td>25</td>\n",
       "      <td>170</td>\n",
       "      <td>4</td>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1849</td>\n",
       "      <td>25</td>\n",
       "      <td>170</td>\n",
       "      <td>8</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>3295</td>\n",
       "      <td>33</td>\n",
       "      <td>340</td>\n",
       "      <td>16</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   ID  categorie  speed   hd  ram  screen  cd  multi  premium  ads  trend\n",
       "0   1       1499     25   80    4      14   0      0        1   94      1\n",
       "1   2       1795     33   85    2      14   0      0        1   94      1\n",
       "2   3       1595     25  170    4      15   0      0        1   94      1\n",
       "3   4       1849     25  170    8      14   0      0        0   94      1\n",
       "4   5       3295     33  340   16      14   0      0        1   94      1"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now have a much cleaner dataset with only numerical data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.drop('ID', axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This exercise will be an exercise in classification, here we will classify our product in three categories according to cost, category 0, corresponds to the entry level of the product (-1400 €), category 1, which corresponds to the middle of the range (between 1400 and 2000 €) and finally category 2, the top of the range (+2000 €)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range (0,len(df)):\n",
    "    if df['categorie'][i]<1400:\n",
    "        df['categorie'].loc[i]=0\n",
    "    elif 1400 <= df['categorie'][i] <= 2000:\n",
    "        df['categorie'].loc[i]=1\n",
    "    else:\n",
    "        df['categorie'].loc[i]=2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>categorie</th>\n",
       "      <th>speed</th>\n",
       "      <th>hd</th>\n",
       "      <th>ram</th>\n",
       "      <th>screen</th>\n",
       "      <th>cd</th>\n",
       "      <th>multi</th>\n",
       "      <th>premium</th>\n",
       "      <th>ads</th>\n",
       "      <th>trend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>80</td>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>33</td>\n",
       "      <td>85</td>\n",
       "      <td>2</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>170</td>\n",
       "      <td>4</td>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>25</td>\n",
       "      <td>170</td>\n",
       "      <td>8</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>33</td>\n",
       "      <td>340</td>\n",
       "      <td>16</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>94</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   categorie  speed   hd  ram  screen  cd  multi  premium  ads  trend\n",
       "0          1     25   80    4      14   0      0        1   94      1\n",
       "1          1     33   85    2      14   0      0        1   94      1\n",
       "2          1     25  170    4      15   0      0        1   94      1\n",
       "3          1     25  170    8      14   0      0        0   94      1\n",
       "4          2     33  340   16      14   0      0        1   94      1"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Class distribution : <a id=\"3\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On this graph, we see the distribution of the number of products per class, we see that the classes are totally unbalanced, class 2 is over-represented compared to class 0 **(we will show after that it is not a problem)**. In spite of this imbalance which will happen in many of your datasets, we will see if our models can generalize and classify our computers in the right categories (low end, mid range or high end)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Categorie')"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfwAAAGJCAYAAABvvYFhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd3SUVf7H8c+EIpKhExDEQgsYMAUlCJEiJSpiY1XKMSggqGtwAQUs7FqI2GhmEYzAclQ0ohQFdBEF0RXpwrpiFgEBxYWERJDENEPu7w9P5udISAYzw5T7fp2Tc8i9z9z5fufR+czzzDwThzHGCAAAhLQwfxcAAAB8j8AHAMACBD4AABYg8AEAsACBDwCABQh8AAAsQOADAGCB6v4uwJeOHftZpaWh/zUDjRo5lZOT5+8yzgpberWlT4leQ5EtfUqB1WtYmEMNGoSfdj6kA7+01FgR+JKs6VOyp1db+pToNRTZ0qcUPL1ySh8AAAsQ+AAAWIDABwDAAgQ+AAAWIPABALAAgQ8AgAUIfAAALEDgAwBgAQIfAAALEPgAAFiAwAcAwAIEPgAAFiDwAQCwAIEPAIAFCHwAACxA4AMAYAECHwAACxD4AABYgMAHAMACBD4AABYg8AEAsACBDwCABQh8AAAsQOADAGCB6v4uwJcaNXL6u4RKFRaVKPdEgb/LAACEuJAO/JEpa5R1LLDDdOX0G5Xr7yIAACGPU/oAAFiAwAcAwAIEPgAAFiDwAQCwAIEPAIAFCHwAACxA4AMAYAECHwAACxD4AABYgMAHAMACBD4AABYg8AEAsACBDwCABQh8AAAsQOADAGABAh8AAAsQ+AAAWIDABwDAAgQ+AAAWIPABALBAwAZ+aWmpUlNT1b17d8XExGjEiBE6ePCgv8sCACAoBWzgv/jii0pPT1dKSooWL16satWqaeTIkSoqKvJ3aQAABJ2ADPzi4mL94x//UHJysnr27Kn27dtr5syZys7O1j//+U9/lwcAQNAJyMDPyMhQfn6+rrjiCteY0+lUVFSUtm3b5sfKAAAITgEZ+JmZmZKkpk2buo03adJEhw8f9kdJAAAEtYAM/IKCAklSzZo13cZr1qyp4uJif5QEAEBQq+7vAspTq1YtSb++l//b0C8uLlbt2rX9VZbPRETUCYg1goUtvdrSp0SvociWPqXg6TUgA79Zs2aSpKysLDmdTtd4VlaW2rRp46+yfObo0dwq3T4iok6V1wgWtvRqS58SvYYiW/qUAqvXsDCHGjVynn7+LNbisfbt28vpdGrLli2usby8PH399deKj4/3Y2UAAASngDzCr1mzpm6//XbNnDlTjRs3VosWLTR9+nQ1bdpUiYmJ/i4PAICgE5CBL0n333+/Tp48qb/97W8qKCjQZZddpvnz55/yQT4AAFC5gA38atWq6cEHH9SDDz7o71IAAAh6AfkePgAA8C4CHwAACxD4AABYgMAHAMACBD4AABYg8AEAsACBDwCABQh8AAAsQOADAGABAh8AAAsQ+AAAWIDABwDAAgQ+AAAWIPABALAAgQ8AgAUIfAAALEDgAwBgAQIfAAALEPgAAFiAwAcAwAIEPgAAFqju7wJ8acHkRH+XUKnCohJ/lwAAsEBIB35OTp5KS42/ywAAwO84pQ8AgAUIfAAALEDgAwBgAQIfAAALEPgAAFiAwAcAwAIEPgAAFiDwAQCwAIEPAIAFCHwAACxA4AMAYAECHwAACxD4AABYgMAHAMACBD4AABYg8AEAsACBDwCABQh8AAAsQOADAGABAh8AAAsQ+AAAWIDABwDAAgQ+AAAWIPABALAAgQ8AgAUIfAAALEDgAwBgAQIfAAALEPgAAFiAwAcAwAIEPgAAFiDwAQCwAIEPAIAFCHwAACxA4AMAYAECHwAACxD4AABYgMAHAMACBD4AABYg8AEAsACBDwCABQh8AAAsQOADAGABAh8AAAsQ+AAAWIDABwDAAtXPZOPMzEzNmTNHGzZsUFZWltLT07Vq1Sp16NBBAwYM8FWNAACgijw+wt+/f79uvPFGrVmzRjExMfrll18kSTk5OZowYYLWrFnjsyIBAEDVeHyE/+yzz6pZs2Z67bXXVKtWLb333nuSpOeee06FhYWaP3++EhMTfVYoAAD44zw+wt+8ebNGjRolp9Mph8PhNnfLLbdo7969Xi8OAAB4h8eBHxYWdkrQlykoKFBYGJ//AwAgUHmc0p07d1ZaWppyc3NdYw6HQydPntTrr7+uyy+/3CcFAgCAqnMYY4wnG+7bt0+DBw9W9erVFR8frzVr1uiaa67R3r179cMPP+iNN95Q+/btfV0vAHhdYVGJck8U+Px+IiLq6OjR3Mo3DHK29CkFVq9hYQ41auQ87bzHH9pr3bq1li5dqtTUVG3atEnVqlXTpk2bFB8frxkzZqht27ZeKdibRqasUdYx3/9PDCC4rZx+owLjKRvwnTO6Dv/CCy/UtGnTfFULAADwkQoDf+PGjbr00kvldDq1cePGShfr2rWr1woDAADeU2HgDx8+XG+88YY6deqk4cOHy+Fw6Pdv+ZeNORwOZWRk+LRYAADwx1QY+K+++qoiIyNd/wYAAMGpwsCPj493/fvNN9/U4MGD3cYAAEBw8Pg6/HXr1qm4uNiXtQAAAB/xOPAvv/xyrV27VqWlpb6sBwAA+IDHl+W1bdtWixYt0rp169S6dWs1btzYbd7hcOjZZ5/1eoEAAKDqPA78Dz74QBEREZKkAwcO6MCBA27zp/uefQAA4H8eB/66det8WQcAAPChM/qmPUk6duyYdu7cqRMnTqhhw4aKiYlR3bp1fVEbAADwkjMK/Dlz5igtLU1FRUX/v0D16ho9erTuv/9+rxcHAAC8w+PAf/vtt5Wamqqbb75ZN998sxo3bqysrCy9++67mjt3rpo3b65bbrnFl7UCAIA/yOPAf+WVV3TbbbfpySefdI21atVKV1xxhWrUqKFFixYR+AAABCiPr8P/7rvv1Ldv33Ln+vTpo/3793utKAAA4F0eB37Tpk31/ffflzt36NAh1alTx2tFAQAA7/I48Pv06aPZs2dr27ZtbuPbtm3T7Nmz1bt3b68XBwAAvMPj9/CTk5O1YcMGJSUlqWnTpmrcuLGys7OVmZmp1q1ba/z48b6sEwAAVIHHge90OrVkyRItXbpUW7du1U8//aQLLrhAnTt31sCBA1WrVi1f1gkAAKrgjK7DP+ecczR06FANHTrUV/UAAAAf8DjwH3744dPOhYWFqXbt2mrZsqX69++v+vXre6W4MmlpaVq/fr3S09O9ui4AALbwOPCPHDmiHTt2qLCwUOeff74aN26snJwcHTp0SNWqVVOjRo30448/au7cuUpPT1eLFi28UuDrr7+umTNnKi4uzivrAQBgI48/pd+3b1+Fh4crPT1da9eu1eLFi/XRRx9p6dKlatKkicaMGaMNGzbo/PPP14wZM6pcWGZmpu655x5NmzZNLVu2rPJ6AADYzOPAX7BggcaOHXvKkXaHDh00ZswYzZ07V/Xq1dMdd9yhTZs2VbmwXbt2KTw8XCtWrFBMTEyV1wMAwGYen9I/duyYGjRoUO5c3bp1lZOTI0lq0KCB8vPzq1xY7969ubYfAAAv8Tjwo6KitHDhQnXv3l3nnHOOa7yoqEgLFy7UJZdcIkn6+uuv1axZM+9XCgA+FBFxdr4t9Gzdj7/Z0qcUPL16HPgPPvighg8frj59+qh79+5q2LChcnJy9K9//Uv5+fmaP3++tmzZolmzZum+++7zZc0A4HVHj+b6/D4iIuqclfvxN1v6lAKr17Awhxo1cp523uPAj4uL07Jly/TSSy/p888/148//qjzzjtPPXv21D333KMLL7xQGzZs0NixYzVixAivFA8AALzjjL54p1WrVnruuedOO5+QkKCEhIQqFwUAALzrjAK/tLRUq1ev1meffaasrCxNnjxZO3fuVHR0tFq1auWrGgEAQBV5HPh5eXkaNWqUduzYoYiICGVnZ+vnn3/W8uXLlZKSokWLFql9+/a+rBUAAPxBHl+HP23aNH377bdKT0/Xxx9/LGOMJGnWrFm64IILNGvWLJ8V+cwzz/C1ugAAVIHHgb9mzRqNGzdOcXFxcjgcrvEGDRpo1KhR2rlzp08KBAAAVedx4P/8888677zzyp0LDw9XQUGB14oCAADe5XHgR0ZG6p133il3bu3atWrXrp3XigIAAN7l8Yf27r33Xt13330aPny4+vbtK4fDoY0bN2rx4sVasmSJXnjhBV/WCQAAqsDjI/zevXtr+vTpOnDggKZMmSJjjKZNm6Z169YpJSVF/fr182WdAACgCs7oOvz+/furf//+2r9/v44dO6a6deuqVatWCgvz+HUDAADwA4+TetiwYdqzZ48kqWXLlurUqZPatGmjsLAw7dq1S9dff73PigQAAFVT4RH+O++8o5KSEknSli1btGrVKl1wwQWnbLd582YdOnTINxUCAIAqqzDwd+/erYULF0qSHA6H0tLSTrvt6NGjvVsZAADwmgoDf9y4cbr99ttljFHfvn31wgsvqGPHjm7bVKtWTU6nU07n6f8kHwAA8K8KA79mzZo6//zzJUmvvvqqoqKiCHYAAIKQx5/Sj4+PV0FBgb744gsVFxe7vkvfGKP8/Hxt375dkyZN8lmhAADgj/M48Ddu3KixY8fqxIkT5c47nU4CHwCAAOVx4L/wwguqX7++pkyZolWrVsnhcGjgwIH65JNP9Oabb+rll1/2ZZ0AAKAKPA783bt364knnlBiYqLy8/OVnp6unj17qmfPniosLNScOXM0b948X9YKAAD+II+/eKe0tFRNmjSR9OsX73zzzTeuuauvvloZGRnerw4AAHiFx4F/4YUXavfu3ZKkiy++WAUFBdq/f78kqaSkRD///LNvKgQAAFXmceBff/31mjlzphYuXKh69eopNjZWTzzxhD788EPNmTNHbdq08WWdAACgCjx+D/+uu+7S8ePH9dVXX0mS/vrXv2rkyJEaM2aM6tSpozlz5visSAAAUDUeBb4xRrm5uZo4caJrrEOHDnrqqacUERGhVq1a8YU8AAAEMIcp+wad09ixY4cmTpyoAQMG6C9/+YtrPDs7W1deeaWaN2+u2bNnKyoqyufFAoAvFBaVKPdEgc/vJyKijo4ezfX5/fibLX1KgdVrWJhDjRqd/uC7wiP8/fv3a+TIkYqIiFBMTIzbXN26dfXMM8/oxRdfVFJSkt599121aNHCO1V7SU5OnkpLK3w9ExIC6T84X7OlV1v6lOzqFfCnCj+09/LLL6tZs2ZaunSpevXq5TZXs2ZN3XTTTXr77bdVr169Cv+SHgAA8K8KA3/Lli268847K3x/vn79+rrjjju0ZcsWrxcHAAC8o8LAz87O9ug0fdu2bXXkyBGvFQUAALyrwsBv1KiRMjMzK10kOztbDRo08FpRAADAuyoM/C5dumjZsmWVLvLuu+/qkksu8VpRAADAuyoM/Ntvv11ffPGFUlJSVFRUdMp8cXGxpk6dqs8//1y33367z4oEAABVU+FleR06dNDkyZM1ZcoUvffee+ratatatGihkydP6ocfftDmzZt1/PhxjR07VgkJCWerZgAAcIYq/aa9wYMHq3379po/f77WrVunwsJCSVJ4eLiuvPJKjRgx4pRr9AEAQGDx6Kt1Y2NjNXv2bEnSjz/+qOrVq6tu3bo+LQwAAHiPx388p0zDhg19UQcAAPAhj/88LgAACF4EPgAAFiDwAQCwAIEPAIAFCHwAACxA4AMAYAECHwAACxD4AABYgMAHAMACBD4AABYg8AEAsACBDwCABQh8AAAsQOADAGABAh8AAAsQ+AAAWIDABwDAAgQ+AAAWIPABALAAgQ8AgAUIfAAALEDgAwBgAQIfAAALEPgAAFiAwAcAwAIEPgAAFiDwAQCwAIEPAIAFCHwAACxA4AMAYAECHwAACxD4AABYgMAHAMACBD4AABYg8AEAsACBDwCABQh8AAAsQOADAGABAh8AAAsQ+AAAWIDABwDAAgQ+AAAWIPABALCAwxhj/F0EAAC2KSwqUe6JAq+tFxbmUKNGztPOV/faPQWgkSlrlHXMew8mAADesnL6jco9i/fHKX0AACxA4AMAYAECHwAACxD4AABYgMAHAMACBD4AABYg8AEAsACBDwCABQh8AAAsQOADAGABAh8AAAsQ+AAAWIDABwDAAgQ+AAAWIPABALAAgQ8AgAUIfAAALEDgAwBgAQIfAAALBGzg5+XlaerUqerdu7fi4uI0cOBArV271t9lAQAQlAI28B9++GGtX79eKSkpeuedd5SYmKjk5GRt3LjR36UBABB0AjLwjx49qjVr1uiRRx5Rt27ddNFFF+mee+5RfHy8lixZ4u/yAAAIOtX9XUB5zj33XM2bN0+dOnVyG3c4HPrpp5/8VBUAAMErII/wnU6nevToIafT6RrbuXOnNm3apF69evmvMAAAglRABv7v7du3T8nJyYqJidGgQYP8XQ4AAEEnIE/p/9bWrVuVnJys5s2bKy0tTTVq1PB3SQAAeEVERJ2zdl8BHfgrVqzQI488ovj4eKWmprqd4gcAINgdPZrrtbXCwhxq1Oj0ORmwp/RXrlypiRMn6tprr1VaWhphDwBAFQTkEf6RI0f017/+VV26dNGECRN0/Phx11yNGjVUv359P1YHAEDwCcjAX7NmjQoKCrRp0yZ1797dba5Tp05KT0/3U2UAAASngAz8YcOGadiwYf4uAwCAkBGw7+EDAADvIfABALAAgQ8AgAUIfAAALEDgAwBgAQIfAAALEPgAAFiAwAcAwAIEPgAAFiDwAQCwAIEPAIAFCHwAACxA4AMAYAECHwAACxD4AABYgMAHAMACBD4AABYg8AEAsACBDwCABQh8AAAsQOADAGABhzHG+LsIAABsU1hUotwTBV5bLyzMoUaNnKedr+61ewpAOTl5Ki0N/dczERF1dPRorr/LOCts6dWWPiV6DUW29CkFV6+c0gcAwAIEPgAAFiDwAQCwAIEPAIAFCHwAACxA4AMAYAECHwAACxD4AABYgMAHAMACBD4AABYg8AEAsACBDwCABQh8AAAsQOADAGABAh8AAAsQ+AAAWIDABwDAAgQ+AAAWIPABALAAgQ8AgAUIfAAALEDgAwBgAQIfAAALEPgAAFigur8L8KWwMIe/Szhr6DX02NKnRK+hyJY+pcDptbI6HMYYc5ZqAQAAfsIpfQAALEDgAwBgAQIfAAALEPgAAFiAwAcAwAIEPgAAFiDwAQCwAIEPAIAFCHwAACwQcoFfWlqq1NRUde/eXTExMRoxYoQOHjzo77LO2Lfffqt27dqd8vP2229LkjIyMpSUlKTY2Fj16tVLCxYscLt9MDwOaWlpGjJkiNvYoUOHdPfdd6tTp07q1q2bnn/+eZWUlLht8/rrr6tPnz6Kjo7WoEGD9OWXX57xGmdbeb1Onz693H3821qDode8vDxNnTpVvXv3VlxcnAYOHKi1a9eeUY3B0KdUea+hsk8lKTMzU+PHj1eXLl0UFxen0aNHa8+ePa55bzwHVbbG2VBZn+PHjz9lf/bo0cM1Hyx9SpJMiElNTTVXXHGFWb9+vcnIyDB33XWX6dOnjyksLPR3aWfk/fffN506dTJZWVluPwUFBSYnJ8fEx8ebRx991Ozdu9csW7bMREdHm8WLF7tuH+iPw6JFi0y7du3M4MGDXWNFRUUmMTHR3H333Wb37t3mo48+MvHx8WbGjBmubZYuXWpiYmLMihUrzJ49e8ykSZNM586dTXZ2tsdrnG3l9WqMMaNGjTKTJ08+ZR+XCZZek5OTTb9+/cyGDRvMgQMHzNy5c0379u3N559/HnL7tKJejQmdfVpaWmquv/56M2TIEPOf//zH7N2714wZM8Z069bN5OXleeU5yJM1/N2nMcb079/fzJ49221/5uTkBFWfZUIq8IuKikxsbKxZtGiRayw3N9fExMSY5cuX+7GyMzdr1ixz6623ljs3d+5ck5CQYH755RfX2MyZM02fPn2MMYH9OBw5csTcfffdJjY21lxzzTVuIbhixQrToUMHc/z4cdfYW2+9ZWJjY01BQYExxpjExETz7LPPuuZLSkrMVVddZWbPnu3xGmdLRb0aY0zPnj3NG2+8cdrbB0OvWVlZJjIy0nz88cdu48OGDTPjx48PqX1aWa/GhMY+NebXXseOHWu+/fZb11hGRoaJjIw0X3zxhVeegypb42yorM+ioiITFRVlPvnkk3JvHyx9lgmpU/oZGRnKz8/XFVdc4RpzOp2KiorStm3b/FjZmdu9e7dat25d7ty2bdt0+eWXq3r1//9jh126dNH333+vzMzMgH4cdu3apfDwcK1YsUIxMTFuc9u2bdMll1yievXquca6dOmi/Px87dq1S9nZ2Tpw4IC6dOnimq9WrZouu+wyV1+VrXE2VdTriRMndPjwYbVp06bc2wZLr+eee67mzZunyy+/3G3c4XDop59+Cql9WlmvobJPJSkiIkIzZ85Uy5YtXbUvWLBATZo0UWRkpFeegypbIxD63Ldvn0pKSk67T4OlzzIhFfhlD17Tpk3dxps0aaLDhw/7o6Q/7JtvvlFWVpYGDx6sbt26aejQofrss88k/drneeed57Z9kyZNJEmHDx8O6Mehd+/emj59ui644IJT5irq68iRI66+ytumrK/K1jibKur1m2++kSStXLlSiYmJuuqqqzRp0iRlZWVJUtD06nQ61aNHDzmdTtfYzp07tWnTJvXq1Suk9mllvYbKPv29hx56SAkJCVq9erWefvpphYeHe+U5qLI1zrby+ty9e7eqV6+utLQ09e7dW/369VNKSopyc3NdPUjB02dIBX5BQYEkqWbNmm7jNWvWVHFxsT9K+kPy8/N16NAh5ebmaty4cXr55ZfVsWNH3XXXXfr8889VWFhYbo+SVFRUFLSPgzf6qmyNQFEWDk6nU6mpqXryySe1Z88eJSUlqaCgIGh73bdvn5KTkxUTE6NBgwaF9D79fa+huk9HjhypJUuWaMCAAbrvvvv01VdfheR+La/Psg/vtWjRQi+99JImTpyo9evX695771VpaWnQ9Vm98k2CR61atSRJxcXFbg9wcXGxateu7a+yzljt2rW1fft21ahRw9VHx44dtW/fPs2fP1+1atU6JbjLfq9du3bQPg5n0tfvtynrq7I1AsWQIUN03XXXuU7dtm/fXpGRkerZs6c++ugj1ynGYOp169atSk5OVvPmzZWWlqYaNWqE7D4tr9dQ3KeS1LZtW0nSU089pX//+9967bXXvPIcFGi9ltfn008/rbvvvlt169aVJEVGRqpx48YaPHiwdu7cGXR9htQRfrNmzSTJdQqtTFZW1imnXAJdeHj4Ka8KIyMj9b///U/nnXdeuT1Kv54uDNbHobK+mjdv7jb2223K+qpsjUDhcDjc3qeVfj0tWL9+fR0+fDjoel2xYoWGDx+uDh066LXXXlP9+vU9qjHY+pRO32so7dOsrCytXLlSxhjXWFhYmNq0aeM6RV3V56BA6LWyPsPCwlxhX6Zdu3aSfj0dHyx9lgmpwG/fvr2cTqe2bNniGsvLy9PXX3+t+Ph4P1Z2Znbs2KG4uLhTrs/96quv1LZtW3Xu3Fnbt293uzZ306ZNuvjiixURERG0j0Pnzp2VkZGhEydOuMY2b96s8PBwRUVFqWHDhmrZsqVbXydPntT27dtdfVW2RqBISUnRTTfd5Db2/fff69ixY2rTpk1Q9bpy5UpNnDhR1157rdLS0tze4w61fVpRr6G0Tw8fPqwHH3xQ27dvd4398ssv+vrrr9W6dWuvPAdVtsbZUFmf9913n+69916325Q9L7dp0yZo+nQ569cF+NiMGTNMfHy8+fDDD13XRCYmJpqioiJ/l+ax4uJiM2DAAHPDDTeYbdu2mb1795opU6aYDh06mF27dpns7GzTuXNnM2HCBLNnzx6zfPlyEx0dbZYuXepaIxgeh0mTJrldqlZYWGj69u1rRowYYTIyMszatWtNly5dTGpqqmubxYsXm0svvdQsXbrU7Nmzxzz00EMmPj7edV2sJ2v4w+973bp1q4mKijJTpkwx+/fvN5s3bzY33HCDueWWW8zJkyeNMcHR6+HDh01MTIwZNmyYyczMdLtW+dixYyG1TyvrNVT2qTHGnDx50iQlJZlrr73WbN261ezevduMGzfOXHbZZea7777zynOQJ2v4u89Vq1aZyMhIk5aWZg4ePGg+/vhjc9VVV5kxY8YEVZ9lQi7wS0pKzPPPP2+6du1qYmNjzciRI813333n77LO2JEjR8yECRNMt27dTMeOHc2gQYPM5s2bXfNffvmlGTRokOnYsaO56qqrzCuvvOJ2+2B4HH4fgsYYc+DAATNixAhz6aWXmoSEBDNjxgzXk2WZBQsWmB49epjo6GgzZMgQ89VXX53xGmdbeb1++umn5rbbbjOxsbEmPj7ePPzww+bHH3902ybQe33llVdMZGRkuT9l/YbKPvWk11DYp2WOHz9uJk+ebBISEkx0dLQZMWKE+e9//+ua98ZzUGVrnA2V9blixQpzww03mOjoaJOQkGCmTp3q9p0IwdKnMcY4jPnNmxcAACAkhdR7+AAAoHwEPgAAFiDwAQCwAIEPAIAFCHwAACxA4AMAYAECHwAACxD4AABYgMAHAMACBD4AABYg8AEAsACBDwCABQh8AAAsQOADAGABAh8AAAsQ+AAAWIDAB0JcUlKSoqKitHPnznLnhwwZoqSkJJ/XcejQIbVr105vv/22z+8LwKkIfMACJ0+e1KRJk1RYWOjvUgD4CYEPWKBOnTo6cOCAZsyY4e9SAPgJgQ9YoG3btrr11lv16quvauvWrafdLikpSUOGDHEbO3jwoNq1a6dly5ZJkjZv3qx27dpp48aNuuOOOxQTE6NevXpp8eLFysnJ0fjx4xUXF6crr7xS06ZNkzHGbb2jR4/q3nvvVXR0tHr06KFZs2appKTEbZt169bplltuUXR0tLp27arHHntMubm5rvlly5YpKipKy5cv15VXXqn4+Hh9+eWXVX2YgJBG4AOWeLk1nqEAAAN0SURBVOihh9S8eXM9/PDDys/Pr/J648aNU0JCgv7+97/roosu0uOPP66kpCQ1bdpUs2bNUvfu3TVv3jy9//77brd78cUXFR4ertTUVN18881KS0vTzJkzXfPvv/++/vznP6t58+aaNWuWxowZo9WrV2vUqFFuLwxOnjypF154QY899pgmTJigqKioKvcEhLLq/i4AwNnhdDr11FNPafjw4Xr++ef12GOPVWm9G2+8UaNHj5b061sGgwcPVmRkpCZNmiRJ6t69u1avXq0vvvhC1113net2Xbp00bRp0yRJvXr1Un5+vhYtWqSRI0eqQYMGeu655xQfH6/U1FTXbdq1a6ehQ4dq9erVGjBggGt89OjR6tevX5X6AGzBET5gka5du2rIkCFKT0/Xxo0bq7TWZZdd5vp3RESEJCkuLs41FhYWpvr16+unn35yu91vw1+Srr76ahUWFmrHjh3av3+/Dh8+rH79+qmkpMT1ExMTo4iICG3YsMHttpGRkVXqAbAJgQ9YZsKECWrRooUeeeQR5eXl/eF1wsPDTxmrXbu22+8Oh+OUbRo3buz2e8OGDSVJJ06c0LFjxyRJKSkp6tChg9vP0aNHlZmZ6XbbshcaACrHKX3AMrVr19bTTz+tpKQkPf3006fMl5aWuv1+uhcF5YW5J35/xJ+dnS3p1+CvW7euJOmBBx5Q165dT7lteS8yAHiGI3zAQp07d1ZSUpKWLFmib775xjXudDp15MgRt0/Wb9myxav3vW7dOrffV61apXPOOUexsbFq1aqVGjdurO+//16XXnqp66dFixZ6/vnnT/vlQQAqxxE+YKkHHnhAn376qQ4cOOAa6927t9atW6fHH39cV199tXbt2qXXX3/9Dx/Nl2f9+vWaOnWqevTooc8++0xvvfWWkpOTVa9ePUnS+PHj9eijj0qS+vTpo/z8fM2bN08HDx6s8gcNAZsR+IClatWqpWeeeUZDhw51jQ0cOFCHDh3S0qVLtXz5csXExCgtLU0333yz1+73kUce0QcffKD09HRFRETooYce0p133uma/9Of/iSn06l58+bpnXfeUe3atRUTE6OUlBS1bt3aa3UAtnGY338rBgAACDm8hw8AgAUIfAAALEDgAwBgAQIfAAALEPgAAFiAwAcAwAIEPgAAFiDwAQCwAIEPAIAF/g9fX2VZZqu7vAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure();\n",
    "df['categorie'].value_counts().plot(kind='barh', figsize=(8, 6))\n",
    "plt.xlabel(\"Number\", labelpad=14)\n",
    "plt.ylabel(\"Categorie\", labelpad=14)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train and test split : <a id=\"4\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Comme pour la tâche de prédiction, lors d'une tâche de classification nous devons constituer un dataset d'entrainement et un dataset de validation. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df.drop('categorie', axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "Y=df['categorie']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Random Forest Classification : <a id=\"5\"></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To classify our data we will first use a very classical model in machine learning and especially in classification, the [Random Forest](https://fr.wikipedia.org/wiki/For%C3%AAt_d%27arbres_d%C3%A9cisionnels), which as its name indicates is based on decision trees. To evaluate the efficiency of the model, we will use a metric that we did not use in forecasting, the [accuracy](https://developers.google.com/machine-learning/crash-course/classification/accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model accuracy score : 0.8578\n"
     ]
    }
   ],
   "source": [
    "rfc = RandomForestClassifier(random_state=1, n_estimators=1000, max_depth= 20)\n",
    "rfc.fit(X_train, Y_train)\n",
    "y_pred = rfc.predict(X_test)\n",
    "print('Model accuracy score : {0:0.4f}'. format(accuracy_score(Y_test, y_pred)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Confusion  Matrix : <a id=\"6\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Confusion Matrix is a matrix that shows how our model has classified the classes. If you don't know how this matrix works please refer to this [article](https://towardsdatascience.com/reading-a-confusion-matrix-60c4dd232dd4) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  70,   81,    0],\n",
       "       [  45,  957,  156],\n",
       "       [   0,  150, 1671]], dtype=int64)"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "confusion_matrix(Y_test, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAj4AAAGiCAYAAADnfswJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdf3zN9f//8fsxjtnW2NI2EsbyM1bDJiVlxZuot37YB9Ua3ohN5Mc78qP8qE9+vP2aam9Sq48oJT8qpdI7CZv58aYhP0fEhszMfhy28/3D16njNaMczrzO7drlXC7vPZ/P8zqPl/f73R4ej+fzdSx2u90uAAAAD1DO3QEAAABcLyQ+AADAY5D4AAAAj0HiAwAAPAaJDwAA8BgkPgAAwGOUd+eHB/iFufPjYTJe5cjj4VrZBWfcHQJM5pzt8HX9vLPH97nsWhWq1nHZtdyJ3xQAAMBjuLXiAwAArqHiIndHUOaQ+AAAYFb2YndHUObQ6gIAAB6Dig8AAGZVTMXnYiQ+AACYlJ1WlwGtLgAA4DGo+AAAYFa0ugxIfAAAMCtaXQa0ugAAgMcg8QEAwKyKi1z3+ouSkpLUrVs3p7Hc3FyNHTtWLVu2VLNmzdSvXz/98ssvTmtWrFihjh07qkmTJurcubNWr17tNH/y5EkNGTJEkZGRatGihUaPHq0zZy7/NTMkPgAAmJW92HWvv2D+/PmaNm2aYTwhIUFr167VrFmzNH/+fJ0+fVr9+vVTUdH5BGvdunUaNmyYunXrpiVLlqhNmzbq37+/du3a5bjGwIEDdfDgQb3zzjtKTEzU2rVrNWbMmMvGROIDAABcKjMzU/369dOUKVMUGhrqNJeSkqK1a9dq5syZatGihRo0aKBx48YpNzdX+/fvlyTNmTNH0dHRevrpp1W3bl0NHTpUTZs21bvvvitJ2rRpk1JTU/Xaa6+pcePGioqK0oQJE/T555/r119/LTU2Eh8AAMyquNh1rz8hPT1dvr6+WrZsmcLDw53mfvjhB4WFhalhw4aOsbp16+r7779XWFiYiouLtXnzZkVFRTm9LzIyUmlpaZKktLQ03XzzzQoLC3PMN2vWTBaLxbHmUjjVBQCASbnyAYY5OTnKyckxjPv7+8vf399prG3btmrbtm2J18nIyFDNmjW1aNEivffeezp58qSaNWumkSNHKjg4WDk5OcrLy1NISIjT+4KCgnTkyBFJUlZWlmHearUqICBAR48eLfU+SHwAAMBlJScnKzEx0TAeHx+vhISEK75Obm6u0tPTderUKY0ZM0YWi0VTpkzRM888o6VLl6qgoEDS+UTmj6xWq2w2m+x2u/Lz8w3zF9YUFhaW+vkkPgAAmJULH2AYGxurLl26GMYvrvZcToUKFZSfn6/Zs2crICBAkjRr1iy1bt1a33zzje655x5Jks1mc3qfzWaTj4+PLBaLvL29DfN/XFMaEh8AAMzKha2uklpaf0VISIiCgoIcSY8k3XLLLQoICNChQ4dUpUoV+fj4KCsry+l9WVlZCg4Odlzj4nmbzaaTJ08aWmAXY3MzAAC4bpo3b66jR486JS7Hjh3TyZMnVbNmTVksFkVERCg1NdXpfSkpKYqMjJQktWjRQseOHdO+ffsc8xc2NTdv3rzUzyfxAQDArMrAAwwv1qFDB4WGhmrgwIHatm2btm/frhdeeEG1atVSdHS0JCkuLk5ffvml3n77be3du1dTp05Venq6YmNjJUnh4eGKiIjQkCFDtHXrVqWmpmrMmDF69NFHHVWhSyHxAQDArNz8AMOSWK1WJScnq0aNGoqLi1OPHj1UuXJlvfvuu6pYsaIk6d5779Vrr72mDz/8UH//+9+1Zs0avfXWW6pbt64kyWKxKDExUbfddptiY2OVkJCgVq1a6eWXX77s51vsdrvdZXfzJwX4hV1+EXCFvMqRx8O1sgsu//h74M84Zzt8XT+vcMd3LrtWxYYPuOxa7sTmZgAAzMqFp7rMgsQHAACzcmGLyizoDQAAAI9BxQcAALOi1WVA4gMAgEnZ7a47hm4WtLoAAIDHoOIDAIBZsbnZgMQHAACzYo+PAYkPAABmRcXHgD0+AADAY1DxAQDArFz45aJmQeIDAIBZ0eoyoNUFAAA8BhUfAADMilNdBiQ+AACYFa0uA1pdAADAY1DxAQDArGh1GZD4AABgViQ+BrS6AACAx6DiAwCASdntPMDwYiQ+AACYFa0uA1pdAADAY1DxAQDArHiOjwGJDwAAZkWry4BWFwAA8BhUfAAAMCtaXQYkPgAAmBWtLgNaXQAAwGNQ8QEAwKxodRmQ+AAAYFa0ugxodQEAAI9BxQcAALOi4mNA4gMAgFmxx8eAVlcZcE/rKJ3M3XPJ1223VZckDRn2nLbtWK3DWdu0eNm7ur1eHTdHjrKsXLlySni+t1K3fK2MXzfrq1WL1Pq+liWu7fPcM/ph/WfXOUKYQa+e3bUjfY1On9qjNauXqWVUM3eHhDImKSlJ3bp1u+T8m2++qfr16+vcuXNO4ytWrFDHjh3VpEkTde7cWatXr3aaP3nypIYMGaLIyEi1aNFCo0eP1pkzZy4bD4lPGbB1S7oeeuAJp1enDj104sRvWvXtGh06dETDRyRoyPABmjVzrno/O0j+/jdpyWfvyd/fz93ho4yKf763Xhr7gj54/xM9032AMvYf1IeL56pJ04ZO6zp2ekgvjx/upihxI3vqqSf0xuz/1QcLPlHXmD7Kzj6lLz6fr9q1b3N3aLiguNh1r79g/vz5mjZt2iXnt27dqsTERMP4unXrNGzYMHXr1k1LlixRmzZt1L9/f+3atcuxZuDAgTp48KDeeecdJSYmau3atRozZsxlY7LY7Xb7X7obFwjwC3PXR5d5r77+kp6MeVQtm/9NhQWF2r77R02d9IZmTPu3JKlyFX9t3b5ar786U28kznNztGWDVzny+D9au2GFNm/apgF9zyc15cqV06Ztq/TlilV6ceg4+fn5aug/B6h/Qk+dOpWjo0ey1LplJzdHXbZkF1z+b4+ebM+u9fryq+8UnzBCklS+fHlt/2m1Pv/iGw1+4fK/gDzROdvh6/p5+UsnuexalR698r8gZWZmauzYsUpJSVFISIiqVKmiBQsWOK3Jy8tTly5dFBISovXr1ys9PV3ly5/fgdOzZ0/ddNNNmjFjhmN99+7dVbt2bb366qvatGmTunXrps8//1xhYedziXXr1ikuLk6rVq1S9erVLxkbvynKoPoNwvSPvk9r4rh/6cTx39S8xZ266SY/rfjiW8eaU9k5WrsmVdEP3efGSFGWWa1WnT6d6/i5uLhYOTmnFRBQWZLU45kn9ETXzurXe4i+/GKVu8LEDSosLFS1a9+mzz5b6Rg7d+6cvljxrdq3f8CNkaEsSE9Pl6+vr5YtW6bw8PAS10ycOFH16tXTo48+6jReXFyszZs3Kyoqymk8MjJSaWlpkqS0tDTdfPPNjqRHkpo1ayaLxeJYcymX3dxss9n05ZdfKi0tTUeOHFFhYaF8fHwUEhKiyMhItWvXzpGhwTVGjX1Be/ZkKPmdDyVJYbeHSpL27zvotC4j46A6PvzgdY8PN4Z5c+Zr6D8H6PPlX2vL5m3q1uMxNWh4u14dd77s/OUXq5Q8b6EKCgr1QHRrN0eLG02928/vMdyzN8NpfP/+g6pbp5bKlSunYk4UuZ8L/zvIyclRTk6OYdzf31/+/v5OY23btlXbtm0vea2vv/5a33//vZYvX67vvvvO8Dl5eXkKCQlxGg8KCtKRI0ckSVlZWYZ5q9WqgIAAHT16tNT7KDVjOXjwoHr16qXjx4+rUaNGCgoKUmBgoGw2m3bv3q2lS5dq1qxZmjNnjmrUqFHqB+HK1KxVQx06RmtQwihd6ELedJOfCgoKdfbsWae1ublndNNN7PFByd55+wO1btNSny5PdoxNHPcvfbnifHXnQMYv7goNJnDT/99f+Meq4oWfvby85OvrY5iDG7jwVFdycnKJ+3Hi4+OVkJBwxdfJzMzU6NGjNWnSJAUEBBjmCwoKJJ1PZP7IarXKZrPJbrcrPz/fMH9hTWFhYamfX2ri88orryg0NFSffvqp/PyMv2Bzc3M1ePBgjR8/XklJSaV+EK5M7LMxys4+pY8WLnWMWSwWlbQVy2KxqJijiriEjz6dp/oN6mrY4LHa9fNe3fdAKw17MV6nTp3WvDnz3R0ebnAWi0WSDP9uujBOtcd8YmNj1aVLF8P4xdWe0tjtdr344ovq0KGD7ruv5K0aFStWlHS+4/RHNptNPj4+slgs8vb2Nsz/cU1pSk18Nm7cqI8++qjEpEeS/Pz8NGTIEHXv3r3UD8GV69j5IX2+/Bun/0Jzck6rYkWrypcv73Tcz9fXRzmn+BsVjKJaNtPdrZqr5zMDtWzJl5KkH9ekqrxXeY0dN0wffvCpzpzJc3OUuJHlnDot6XxFOivruGPcz89XRUVF/O+rrHBhAlpSS+vP+vXXX7V27Vpt2rRJS5YskSTH77UWLVqob9++6tu3r3x8fJSVleX03qysLAUHB0uSQkJCDPM2m00nT540tMAuVurmZn9/f2VmZpZ6gcOHD182u8KVqVGjmho0CNNny75yGt+7J0PlypVTrdrO7cTatWtqz+591zNE3CBurXH+//hpG7Y4jaesT5Ovr49uq3mrO8KCiezes1+SVCe0ptN4aGhN/bxrrztCQkncfJz9YsHBwVq5cqWWLVumJUuWaMmSJRo4cKAk6ZNPPtH//M//yGKxKCIiQqmpqU7vTUlJUWRkpKTzSdKxY8e0b9/vvwMvbGpu3rx5qTGUmvg88cQTevHFF7Vw4ULt27dPeXl5OnfunPLy8pSRkaGPPvpIL730kh577LE/f/cwiGh+fud7Wtp/ncZTUzYpP79AD3d6yDFWuYq/Wt0bqe//s+66xogbw949GZLOV37+KKJ5uM6ePatffy198x9wObt379PBg4f1yCN/c4yVL19eHTtEa9WqNW6MDGVZ+fLlVatWLafXzTffLEmqWbOmqlSpIkmKi4vTl19+qbffflt79+7V1KlTlZ6ertjYWElSeHi4IiIiNGTIEG3dulWpqakaM2aMHn30UUdV6JIxlDaZkJAgi8WiSZMmKT8/3zDv6+urHj166Pnnn/9LfwBw1rBRPR0//ptO/pbtNH7mTJ7mvPWeXhozWMXFxdq7J0MvDHtOp0/n6v3kD90ULcqy/25J18ovv9Okf41VlYDK2v3zXt3TOkoDB/1D/37zPUebArgakybP1swZE5SdfUpr125Q/+eeVdWqgZoxc467Q8MF7ntU31W599579dprr2n27NmaPn26wsLC9NZbb6lu3bqSzu8lS0xM1CuvvKLY2FhZrVa1b99eI0eOvOy1r+gBhjabTTt37lRmZqby8/Pl7e2tkJAQNWjQoMRd1VeKBxg6mzLtFd3/wD1qfqfxiLqXl5dGjXlB3Z56TL6+PkpN2awXh43T7l20ui7gAYbOvL0rauTowery+MOqElBZ+/Zm6J25H+jdeQsNa2e9+b+68647eIDhRXiA4eUNHtRXCfG9VLVqoP7733QNGz5O61M2ujusMuu6P8BwwViXXatSt1dcdi134snNMA0SH7gaiQ9cjcTH/XjyIAAAZsVjBQxIfAAAMCue9WZAbwAAAHgMKj4AAJgVrS4DEh8AAMzqBj3Ofi3R6gIAAB6Dig8AAGZFq8uAxAcAALMi8TGg1QUAADwGFR8AAMyK5/gYkPgAAGBS9mJOdV2MVhcAAPAYVHwAADArNjcbkPgAAGBW7PExoNUFAAA8BhUfAADMis3NBiQ+AACYFXt8DEh8AAAwKxIfA/b4AAAAj0HFBwAAs7Kzx+diJD4AAJgVrS4DWl0AAMBjUPEBAMCsOM5uQOIDAIBZ8eRmA1pdAADAY1DxAQDArGh1GZD4AABgUnZOdRnQ6gIAAB6Dig8AAGZFq8uAxAcAALPiVJcBrS4AAOAxqPgAAGBWtLoMSHwAADArTnUZ0OoCAADXTFJSkrp16+Y0lp6erp49eyoyMlKtWrXSoEGDdOTIEac1K1asUMeOHdWkSRN17txZq1evdpo/efKkhgwZosjISLVo0UKjR4/WmTNnLhsPiQ8AAGZVbHfd6y+YP3++pk2b5jR25MgRPfvsswoKCtIHH3ygpKQknThxQj179lRBQYEkad26dRo2bJi6deumJUuWqE2bNurfv7927drluM7AgQN18OBBvfPOO0pMTNTatWs1ZsyYy8ZE4gMAgFnZi133+hMyMzPVr18/TZkyRaGhoU5zy5cvV8WKFTVhwgSFhYWpSZMmmjRpkvbt26eNGzdKkubMmaPo6Gg9/fTTqlu3roYOHaqmTZvq3XfflSRt2rRJqampeu2119S4cWNFRUVpwoQJ+vzzz/Xrr7+WGhuJDwAAcKn09HT5+vpq2bJlCg8Pd5pr166dpk+frvLlf99mbLFYJEmnTp1ScXGxNm/erKioKKf3RUZGKi0tTZKUlpamm2++WWFhYY75Zs2ayWKxONZcCpubAQAwKxee6srJyVFOTo5h3N/fX/7+/k5jbdu2Vdu2bUu8Tu3atVW7dm2nsbfeekve3t5q2bKlcnJylJeXp5CQEKc1QUFBjn1AWVlZhnmr1aqAgAAdPXq01Psg8QEAwKRc+V1dycnJSkxMNIzHx8crISHhL1/3nXfe0YIFCzRq1CgFBgY6Eher1eq0zmq1ymazyW63Kz8/3zB/YU1hYWGpn0fiAwAALis2NlZdunQxjF9c7blSxcXFmjp1qubOnasBAwbo6aefliRVrFhRkmSz2ZzW22w2+fj4yGKxyNvb2zD/xzWlIfEBAMCsXNjqKqml9VcVFhZq2LBh+vrrrzVmzBj16NHDMVelShX5+PgoKyvL6T1ZWVkKDg6WJIWEhBjmbTabTp48aWiBXYzNzQAAmJWbj7OXGFJxsRISEvTdd99p1qxZTkmPdH6jc0REhFJTU53GU1JSFBkZKUlq0aKFjh07pn379jnmL2xqbt68eamfT+IDAACum+TkZH3//fcaOXKkwsPDdezYMcfrwnN84uLi9OWXX+rtt9/W3r17NXXqVKWnpys2NlaSFB4eroiICA0ZMkRbt25VamqqxowZo0cffdRRFboUi91ud9sXeQT4hV1+EXCFvMqRx8O1sgsu/xRY4M84Zzt8XT8vd+ijLruW35Slf+l9L774og4cOKAFCxZIkrp06aLt27eXuHbChAl68sknJUlLly7V7NmzdeTIEYWFhWnYsGFq1aqVY+2JEyf0yiuv6IcffpDValX79u01cuRIeXt7lxoPiQ9Mg8QHrkbiA1e77onPC4+47Fp+/1rmsmu5E78pAACAx+BUFwAAJmV34aZksyDxAQDArEh8DGh1AQAAj0HFBwAAs3LhV1aYBYkPAABmRavLgFYXAADwGFR8AAAwKyo+BiQ+AACYlBufUVxm0eoCAAAeg4oPAABmRavLgMQHAACzIvExcGviYys+586Ph8nsaFDX3SHAZB7IOOnuEAC4GBUfAABMiu/qMiLxAQDArEh8DDjVBQAAPAYVHwAAzIqv6jIg8QEAwKTY42NEqwsAAHgMKj4AAJgVFR8DEh8AAMyKPT4GtLoAAIDHoOIDAIBJsbnZiMQHAACzotVlQKsLAAB4DCo+AACYFK0uIxIfAADMilaXAYkPAAAmZSfxMWCPDwAA8BhUfAAAMCsqPgYkPgAAmBStLiNaXQAAwGNQ8QEAwKyo+BiQ+AAAYFK0uoxodQEAgGsmKSlJ3bp1cxo7dOiQ+vbtq4iICLVq1UqTJ0/WuXPnnNbMnz9f0dHRatq0qWJiYrR169Y/fY2SkPgAAGBS9mLXvf6K+fPna9q0aU5jNptNvXr1ksVi0cKFCzV+/Hh9/PHHmjVrlmPN4sWLNXnyZA0aNEiLFy9WaGioevfurRMnTlzxNS6FxAcAAJNyV+KTmZmpfv36acqUKQoNDXWa++qrr3T48GG9/vrrqlevnqKjozV06FC99957KigokHS+StS9e3d17txZYWFhmjhxovz8/LRw4cIrvsalkPgAAACXSk9Pl6+vr5YtW6bw8HCnubS0NDVs2FCVK1d2jEVFRSkvL0/p6ek6fvy4MjIyFBUV5Zj38vJSs2bNlJaWdkXXKA2bmwEAMCu7xWWXysnJUU5OjmHc399f/v7+TmNt27ZV27ZtS7xOZmamQkJCnMaCgoIkSUePHpW3t7cklbhm27ZtV3SN0pD4AABgUq481ZWcnKzExETDeHx8vBISEq74OgUFBfL19XUas1qtkqTCwkLl5+c7jf1xjc1mu6JrlIbEBwAAXFZsbKy6dOliGL+42nM53t7ejgTmggs/+/j4OCo+Ja3x8fG5omuUhsQHAACTshe7rtVVUkvrrwgJCdGOHTucxrKyshxz1atXd4zVr1/faU1wcPAVXaM0bG4GAMCk3H2cvSQtWrTQjh07nPYLpaSkyNfXV40aNVJgYKBCQ0OVmprqmC8qKtLGjRsVGRl5RdcoDYkPAAC4bh588EEFBwdr8ODB2rlzp1atWqWpU6cqLi7OsU+nZ8+eSk5O1uLFi7Vnzx6NGjVKZ86c0ZNPPnnF17gUWl0AAJiU3YWnulylYsWKmjt3rsaNG6euXbvK399fMTExGjBggGNN165dlZubqxkzZig7O1uNGzfWvHnzFBgYeMXXuBSL3W63X7O7uwxfn9ru+miY0I5Gdd0dAkzmgYyT7g4BJrP3+Kbr+nmHoko+Uv5X1EhZ5bJruROtLgAA4DFodQEAYFKuPNVlFiQ+AACYlPs2s5RdtLoAAIDHoOIDAIBJ0eoyIvEBAMCkSHyMaHUBAACPQcUHAACTYnOzEYkPAAAmRavLiFYXAADwGFR8AAAwqbL4XV3uRuIDAIBJ2YvdHUHZQ6sLAAB4DCo+AACYVDGtLgMSHwAATIo9Pka0ugAAgMeg4gMAgEnxHB8jEh8AAEyKJzcb0eoCAAAeg4oPAAAmRavLiMQHAACT4ji7Ea0uAADgMaj4AABgUjzHx4jEp4yxWq1at/4LpW3Yor59h0qS7opoojVrlhvWzpj+b40c+er1DhFlkKWSt6ok/EM+0W1k8a6owq3pyp75b53dvU+SZG1YTyHvv2l4X877Hyl7RpIq93lGlfvElnjtc4eP6NdHn7qm8aNsi/7bffrXWxMVXru1Y6zJnQ215Jv5hrVzZ7+n18ZOd/z8TO8YxfbpppBqQTqw/xclTp2rL5Z+fV3iBqe6SkLiU8aMfOl5NWgQprQNWxxjTe5ooNzcM+r0sPMvnyNHMq93eCijqk56WRXD79Cpfyfr7O598unwoILnTNfR2P46d+CQKtxeR8V5+crqP8zpfUXHTkiScpd8ofy1G5zmKtSuqcAxQ5W79Ivrdh8oeyJaNNXUNyfIIufKQf1G9XQmN0/PPP6c03jm0WOO/9wnIVYvjOyvaa+9qW2bt+vhLu00Y85r+u3ESa1fk3Zd4gcuRuJThoSHN9Zzzz2rY///l9EFd9zRUNu379KGDZvdFBnKsgoNblelu1vot1enKXfxZ5KkgpSNqvD2TFXuF6cTI8arQlgdnd27X7afdpR4jaKs4yrKOv77QLlyChieoMJNW5Uz74PrcRsoY6zWCnq2b3cNevE55eflq1wF5y2hDRrdrl0792rLxm0lvt/Pz1cJQ/toyoREzZ39viRp7Q+pCg2rpXvvb0nic52wudmIxKeM8PLy0ptvTdL06f/WI4+0d5q7444G+ukSv7CACrVqSJLy1zlXbAr/+5P8HntYkmS9vY6j7XUl/P7eUdZ6dXW0ex/XBYobSpvoe9Tv+Ti9/vJ0VQmsol7POVecGzS+XT9v333J99/7QEtV9Lbqw/c/dRrv/sg/rkm8KBl7fIw41VVGvDCknypUqKApk98wzDVqXF81alTXuvVf6GT2Lm3d9h/16PG4G6JEWVT0/1sL5UOCnMbL31pN5fz8VM7/JlUIC5VXcJBC5ifptnVfqtqn78n34XYlX9BaQZX7PKPcZV/q7L4D1zp8lFFbN6erTUQnJc9ZKHsJG0XqNQxTterBWv7dAu34NUWrUpfqsZhOjvkGjW9XVuZxNWpSX0tXzdfOIyn6NnWJ2ndqez1vAzCg4lMG1KtXV8OHx+vhjj109uxZp7mQakG65ZabVbdubY0dM0nZ2af0ZNdH9O85U2W32/XBB4vdFDXKisLtP+vsgV8U8OLz+u3lSTr7y2H5tntA3vdESpIsPpXkFVBF5WvequzEuSo+nSvf9m118yv/lGTXmc+dN5r6tmurcgFVdPr9j9xwNygr/rhX52JBIVV1c9UA1a5TU5MnzNKp7Bx1fuxvmjx7nOx2uz796HMF3hwgX18fzfj3a5o1ZY727zmgrk//XYnzJqlrx57anLb1Ot6N52JzsxGJj5tZLBa98ebrSk7+UKmpmwzzp7Jz9Ogjz+inn3bo6P//F9F33/2oatWCNWLk8yQ+kM6e1fFhY3XzhJccJ7cK/5uu0+99eP6kVlGxsuL/KdvufSo+8dv5+dRN8qp6s/z/8Ywh8fHr8rAK1m7QuV8OX/dbwY0h51Sunn1ygHZu361jmef3hq1dnargkFuUMKyPPv3oc5WvUF7+lW/S0BGT9OlHn0uS1q3ZoLB6dRQ/pLd6dRvozlvwGOzxMbps4tO9e3dZLFf2Bzd/vvFoI0r33HPPqmbNW/XE4z3l5eXlGLdYLPLy8lJ+foG++Wa14X1fr/xe7drdL19fH505k3c9Q0YZdHbfAR3t3kdewbdIXl4q+vWo/P/xtOxFRSr67TcVHTtueE/Bug0KuCdSlkresucXSJLKBQbI2qShTox9/XrfAm4gBfkF+uG7dYbx1avWqs2D98jHt5LycvMdYxfY7XatW7NBHTpHX7dYgYtdNvFp06aNpk+frjp16qhp06bXIyaP0vmR9rr11mo6/Ktz2bdp00bq8dTjCm/6gNrc30rvv7dINpvNMV+pkrfy8vJJeiBLxYqqFN1ahambVZT5e3vCensdnd2bofK3Vpd3izuVu+wr6Q+tVEtFq4oLChxJjyRVuru5VFys/NVrBVxK7bo1dXfrFvrkg2Wy2X7/36cVKPsAACAASURBVFRF74rKz8tX3pl8Hdj/iySpQoUKTu8tX758iXuGcG2wudnosolP37595efnp6lTpyopKUk1atS4HnF5jIEJI+V3k6/T2Lx507Vnz369+uoM1a59m2bOnKisrONavuwrx5pHHm2vtT9uuPhy8ED2c+cUOGKQTr31rk7P/1iS5FU9RN6tonR6/iJ53VJVgSMGq+jESeX/50fH+yq1ba3Czc5Hka2NG+js/oOyk1CjFCHVgjRhyks6kfWbVn7xnWO8fae22rD+/DPIfvjP+YpQh0cf0jtvne8GeHl5qfUDLbVpA/t7rhdaXUZXtMenR48e+uGHHzR9+nRNmTLlWsfkUXaXcMS4oKBQv53I1uZN21SuXDn9+GOqZs6YoIAq/jp69Jh69uquO+5oqAcffMINEaPMKSrSmSUr5B/XQ0W/Zct+5oyqJPxDxdnZOv3BJyo+nauCzdsUOGKwsv1vUtHx3+T3WCdZw+oos/fzTpeqULe2zh34xU03ghtF6tpN2rBus8ZPGSn/Kv46lnlM3WIfV4NGtyumU09JUsbeg/ro/5Zo6EsDZLFIu3fuU4+4J1S9RjUNiBvu5jvAtWaz2TRr1iwtX75cp06dUsOGDTV06FBFRERIkg4dOqTx48drw4YN8vb2VpcuXTR48GCVL/97WjJ//nzNmzdPx44dU8OGDfXSSy+5pPN0xZubx40bp/T09Kv+QPw5xcXFiun6D738ynCNGv2CAgMDtGXLT+rc6Slt3lTyg8PgebIT58hutyvg+T6yWK0qSNuskzP+reJTOZKk40NGq/KAXqrc91mVq+yvsz/vVtaA4bLt2OV0Ha+AAJ07yKZmlK64uFh9nx6soaPiNejFfgoIqKz0rTv1zBP9tW3L788cGzVkoo4eyVLPfj0UEFhFO9J3K/aJ/qU+/weu5a6m4uzZs/XJJ5/o9ddf12233aa5c+eqd+/eWrFihQICAtSrVy+FhoZq4cKF+uWXXzRy5EiVL19egwcPliQtXrxYkydP1vjx49WwYUOn9998881XFZvF7sZmq69PbXd9NExoR6O67g4BJvNAxkl3hwCT2XvceHr3WlpbzXXPfGt15JMrXtupUyfde++9evHFFyVJubm5atasmaZNm6aioiKNGDFCP/74oypXrixJWrRokV599VWtW7dO3t7eat++vaKjozV8+PnqYFFRkR566CE9/vjjGjBgwFXdBw8wBADApOx2i8tef0ZgYKC+++47HTp0SEVFRVq0aJGsVqsaNWqktLQ0NWzY0JH0SFJUVJTy8vKUnp6u48ePKyMjQ1FRUY55Ly8vNWvWTGlpV/9VJzzHBwAAXFZOTo5ycnIM4/7+/vL393caGz16tAYNGqTo6Gh5eXmpXLlymj59umrXrq3MzEyFhIQ4rQ8KOv/k+aNHj8rb21uSSlyzbdvVb/Eg8QEAwKSKXXit5ORkJSYmGsbj4+OVkJDgNLZ79275+flp9uzZCg4O1qJFizR8+HC9//77KigokK+v82lmq9UqSSosLFR+fr7T2B/X/PGxLn8ViQ8AACZll+uOs8fGxqpLly6G8YurPb/++quGDx+uuXPnqmXLlpKkJk2aaM+ePZo5c6a8vb0NCcyFn318fBwVn5LW+Pj4XPV9kPgAAIDLKqmlVZKtW7fq7NmzatKkidN4eHi4Vq1apbvvvls7duxwmsvKypJ0vr1VvXp1x1j9+vWd1gQHB1/tbbC5GQAAsyq2u+51pS7szfn555+dxnft2qXQ0FC1aNFCO3bscNovlJKSIl9fXzVq1EiBgYEKDQ1VamqqY76oqEgbN25UZGTk1f2BiMQHAADTKpbFZa8r1bRpUzVr1kwjR47U+vXrlZGRoenTp2vt2rXq06ePHnzwQQUHB2vw4MHauXOnVq1apalTpyouLs6xr6dnz55KTk7W4sWLtWfPHo0aNUpnzpzRk08+edV/JrS6AACAy5QrV05vvvmmpk+frhEjRig7O1v169fXO++843hy89y5czVu3Dh17dpV/v7+iomJcXo+T9euXZWbm6sZM2YoOztbjRs31rx58xQYGHjV8fEAQ5gGDzCEq/EAQ7ja9X6A4bfBMS67VnTmhy67ljtR8QEAwKRceZzdLNjjAwAAPAYVHwAATMqVz/ExCxIfAABMilaXEa0uAADgMaj4AABgUlR8jEh8AAAwKfb4GNHqAgAAHoOKDwAAJlVMwceAxAcAAJP6M9+x5SlodQEAAI9BxQcAAJNy25dxlmEkPgAAmBTH2Y1odQEAAI9BxQcAAJMqtrC5+WIkPgAAmBR7fIxodQEAAI9BxQcAAJNic7MRiQ8AACbFk5uNaHUBAACPQcUHAACT4isrjEh8AAAwKU51GdHqAgAAHoOKDwAAJsXmZiMSHwAATIrj7Ea0ugAAgMeg4gMAgEmxudmIxAcAAJNij48RrS4AAOAxqPgAAGBSbG42IvEBAMCkSHyMaHUBAACPQcUHAACTsrO52YDEBwAAk6LVZUSrCwAAuNySJUvUsWNHNWnSRA8//LBWrFjhmDt06JD69u2riIgItWrVSpMnT9a5c+ec3j9//nxFR0eradOmiomJ0datW10SF4kPAAAmVezC15+xdOlSjRw5UjExMfrss8/UqVMnvfDCC9q4caNsNpt69eoli8WihQsXavz48fr44481a9Ysx/sXL16syZMna9CgQVq8eLFCQ0PVu3dvnThx4mr+OCSR+AAAYFp2F76u+DPtds2YMUNPPfWUYmNjVatWLT333HNq1aqV1q9fr6+++kqHDx/W66+/rnr16ik6OlpDhw7Ve++9p4KCAklSUlKSunfvrs6dOyssLEwTJ06Un5+fFi5ceNV/JuzxAQAALrNv3z4dPnxYnTp1chp/++23JUljx45Vw4YNVblyZcdcVFSU8vLylJ6erlq1aikjI0NRUVGOeS8vLzVr1kxpaWlXHR+JDwAAJuXKr6zIyclRTk6OYdzf31/+/v6OnzMyMiRJNptNffr00bZt21SjRg0999xzatu2rTIzMxUSEuJ0jaCgIEnS0aNH5e3tLUklrtm2bdtV3weJDwAAJuXKU13JyclKTEw0jMfHxyshIcHxc25uriRp+PDhGjBggAYPHqyVK1eqf//+evvtt1VQUCBfX1+na1itVklSYWGh8vPzncb+uMZms131fZD4AACAy4qNjVWXLl0M43+s9khShQoVJElxcXF6/PHHJUkNGzbUTz/9pHnz5snb29uQwFz42cfHx1HxKWmNj4/PVd8HiQ8AACblyorPxS2tS7nQoqpXr57T+O23365vv/1Wd999t3bs2OE0l5WV5Xhv9erVHWP169d3WhMcHHxV9yBxqgsAANNyx6muRo0aydfX17AfZ9euXapZs6ZatGihHTt2OO0XSklJka+vrxo1aqTAwECFhoYqNTXVMV9UVKSNGzcqMjLyz/0BlIDEBwAAuIy3t7d69+6tN954Q8uWLdPBgwf15ptvas2aNerZs6cefPBBBQcHa/Dgwdq5c6dWrVqlqVOnKi4uzrGvp2fPnkpOTtbixYu1Z88ejRo1SmfOnNGTTz551fHR6gIAwKRcearrz+jfv798fHw0c+ZMHT16VHXq1NGsWbN09913S5Lmzp2rcePGqWvXrvL391dMTIwGDBjgeH/Xrl2Vm5urGTNmKDs7W40bN9a8efMUGBh41bFZ7Hb7n6lguZSvT213fTRMaEejuu4OASbzQMZJd4cAk9l7fNN1/bz/rfWUy6714oH/c9m13ImKDwAAJuW2ykYZxh4fAADgMdxa8Sk8d9adHw+TuT/jN3eHAJP5eecn7g4BuCrF1HwMaHUBAGBSrnyOj1nQ6gIAAB6Dig8AACZFo8uIxAcAAJOi1WVEqwsAAHgMKj4AAJiUu57cXJaR+AAAYFIcZzei1QUAADwGFR8AAEyKeo8RiQ8AACbFqS4jWl0AAMBjUPEBAMCk2NxsROIDAIBJkfYY0eoCAAAeg4oPAAAmxeZmIxIfAABMij0+RrS6AACAx6DiAwCASVHvMSLxAQDApNjjY0SrCwAAeAwqPgAAmJSdZpcBiQ8AACZFq8uIVhcAAPAYVHwAADApnuNjROIDAIBJkfYY0eoCAAAeg4oPAAAmRavLiMQHAACT4lSXEa0uAADgMaj4AABgUjzA0IiKDwAAJlXswtdftX//ft11111atGiRY+zQoUPq27evIiIi1KpVK02ePFnnzp1zet/8+fMVHR2tpk2bKiYmRlu3br2KKH5H4gMAAK6Js2fPaujQocrLy3OM2Ww29erVSxaLRQsXLtT48eP18ccfa9asWY41ixcv1uTJkzVo0CAtXrxYoaGh6t27t06cOHHVMZH4AABgUnYX/vNXzJo1S76+vk5jX331lQ4fPqzXX39d9erVU3R0tIYOHar33ntPBQUFkqSkpCR1795dnTt3VlhYmCZOnCg/Pz8tXLjwqv9MSHwAADApd7a6NmzYoA8//FCvv/6603haWpoaNmyoypUrO8aioqKUl5en9PR0HT9+XBkZGYqKinLMe3l5qVmzZkpLS/sLkThjczMAALisnJwc5eTkGMb9/f3l7+9vWDt8+HCNGjVK1apVc5rLzMxUSEiI01hQUJAk6ejRo/L29pakEtds27btqu+DxAcAAJMqtrvuVFdycrISExMN4/Hx8UpISHAae/nll3XnnXeqc+fOhvUFBQWG9pfVapUkFRYWKj8/32nsj2tsNttV3YNE4gMAgGm58jB7bGysunTpYhi/uNqzZMkSpaWlafny5SVex9vb25DAXPjZx8fHUfEpaY2Pj89fjv8CEh8AAHBZJbW0SvLJJ5/oxIkTuv/++53Gx40bp3fffVctWrTQjh07nOaysrIknW9vVa9e3TFWv359pzXBwcFXeRckPgAAmJY7vqtrypQpjtNZF7Rr107x8fHq1KmTtmzZosWLFysnJ8eRSKWkpMjX11eNGjWS1WpVaGioUlNT1bp1a0lSUVGRNm7cqJiYmKuOj8QHAACTcseTmy9VlQkMDNStt96qqlWravr06Ro8eLCGDRumX3/9VVOnTlVcXJxjX0/Pnj01YcIEhYaGqmnTpnr77bd15swZPfnkk1cdH4kPAAC4bipWrKi5c+dq3Lhx6tq1q/z9/RUTE6MBAwY41nTt2lW5ubmaMWOGsrOz1bhxY82bN0+BgYFX/fkWu92FW77/pPLWW9310TChmv5B7g4BJvPzzk/cHQJMpkLVOtf182Jq/d1l1/rwwBKXXcudqPgAAGBS7tjjU9bx5GYAAOAxqPgAAGBS7tjcXNaR+AAAYFJ/5Tu2zI5WFwAA8BhUfAAAMCk3Htwus0h8AAAwKU51GdHqAgAAHoOKDwAAJsXmZiMSHwAATIrj7EYkPgAAmBR7fIzY4wMAADwGFR8AAEyK4+xGJD4AAJgUm5uNaHWVYb16dteO9DU6fWqP1qxeppZRzdwdEm4A0X9ro60Za5zGmtzZSPuObza8Rrwy2LHGaq2gUROGKCX9a23NWKPZ8yYrKOSW6x0+yoDvflivyAcfM4x/8c1/1OXp5xTxwCPqGNNL8xctdczNfvv/dMc9HUp8tX/i2RI/5/kR4zVx6hvX6jaAElHxKaOeeuoJvTH7fzVh4jSlpf1XA/rH6YvP5yui+UPKyPjF3eGhjIpoEa5/vTlBFlmcxhs0ul1ncvP09OP9nMazjh5z/OcJU15S9N/a6NUx/1LemXwNG52geQtm6ZHo7iou5u+NnmLztu16cdwkw2mgFd98r3++PEmx//OYXhzUV+vTtui16W/Jz9dHj3Z8SI93bq97L/rL2f4DhzT6tWl6rHN7p3G73a5/vTFP365eq26Pdb7m9+TJONVlROJTRr08ZqjmzJ2v8ROmSZK+/ma1tv+0Ws8P/IcGvzDGzdGhrLFaK+jZvt01+MX+ys/LV7kKzsXcBo1u166de7Rl47YS31+zdg11iemkQX1H6vMlKyVJO9J36Zv1n+qhDvfrq89XXfN7gHvZbDb936KlmjXnPVXy9lbxuXOOuQuJyv906aSh8b0lSVHN7tSvRzK1dsNmPdrxIYUE3aKQoN8rhEVFRZr4rzfV/M4m6vPM/zjGfzl8RK9Nf0spaVvkXbHi9btBD8WpLqPLtrpSUlI0bNgw9enTRx988IGKioqc5k+dOqUePXpcswA9UVhYqGrXvk2ffbbSMXbu3Dl9seJbtW//gBsjQ1nVJvoePfd8T/3vy9P13tyFhvn6jW/Xzu27L/n+u1u3kCStWrnaMZax76B279yr+6JbuT5glDk/rE/T3Pc/0pABvdX9iUec5tJ37taRzCw98WgHp/HXX/6nXh87vMTrfbL8K+3as08vvdBfFsvvFcjJs+bo+Inf9P5bUxUYUNn1NwJcRqmJz6pVqxQXF6djx47p7NmzGj9+vJ566inl5OQ41pw9e1abNm265oF6knq315Ek7dmb4TS+f/9B1a1TS+XKsTULzrZuTtd9EQ8rec6CEk9x1G94u6pVD9Fn3y3Uzl9TtSp1qR6L+b3FEFq3lo5lnVB+XoHT+w4eOKzQurWuefxwvzsa1tOXi97RU08+6pSoSNKuvfslna/iPDtgmO5s01nRXZ7WwsWflXitwkKb3pj3f+rSqZ3C6jj/72dgn1h9+PZMNaofdm1uBE7sdrvLXmZRaqvrjTfe0PPPP6++fftKkrZs2aL4+HjFxcXp/fffl4+Pz3UJ0tPc5O8nSTp9Otdp/PTpXHl5ecnX18cwB8+W+Ye9OhcLCrlFN1cNUO06t2nyhFnKyT6tzo/9TVNmj5PdbtenH30mv5t8dSb3jOG9Z3LPqNqtwdcydJQRwbdUveTcyexT8vIqp4R/vqyYLp30XM8e+vb7tZowdbYq+9+kDg+2cVq/4pvv9dvJU3q22+OGa12cCOHaotVlVGris3fvXk2bNs3x85133qnk5GR1795dAwcOVFJS0jUP0BNd+NvWxRn2hXE2muLPyDl1WrFP9tfO7bt1LPO4JOnH1SkKCrlFA4f10acffSaLxVLi3+gsFovsxfyL09OdO1ekoqJiPfFIB/WJPb9fJ6rZnTp0JFNvvjPfkPgsWrZC97Zsrlq33eqOcIFSldozCQgI0OHDh53G6tatq8TERKWkpOill14yVfmrrMg5dVqSdNNNfk7jfn6+Kioq0pkzee4ICzeogvwC/fDdOkfSc8HqVT+qVuht8vGtpNM5ufL18zW818fXR6dzTl+vUFFG+VTyliTd27K50/jdLe7SgV8O6+zZs46x47+d1Nb0nYZkCO5hd+E/ZlFq4vPggw9q7NixWr16tfLyfv9l26JFC02cOFFLly7V8OElb2zDX7d7z/l+ep3Qmk7joaE19fOuve4ICTew0Lo11f3ZJ2S1VnAa9/b2Vn5evvLO5Ctj30HdEnSzKno7n7KpWetW7dtz4HqGizLothrVJckpwZHOH7qw2yWL5fdfJT+mbJRXuXJ64N6o6xojSlZst7vsZRalJj7PP/+86tWrp759+xo2MD/yyCOaNGmStmzZck0D9ES7d+/TwYOH9cgjf3OMlS9fXh07RGvVqjWlvBMwCq4WpAlTXtL9D97rNN6+U1ttWL9ZkrR2darKly+v6Pb3OeZr16mp2xvU1drVqdc1XpQ9ze+8QxWtVn31nfO/f1av3aA7Gtyu8uW9HGM/bd+l0Nq3yc/XWEEEyoJS9/j4+vpq1qxZOn78uCpVqmSY79y5syIjI7Vy5coS3o2rMWnybM2cMUHZ2ae0du0G9X/uWVWtGqgZM+e4OzTcYFLXbtKGdZs0YcpLqlzFX1mZx9Ut9nE1aHS7unbqKUk6mHFIny9ZqdemjdZN/n7KyT6tYaMTtDN9t1Z+8Z2b7wDu5ufrq388E6M35s2Xn4+Pmt/VRF9+u1ppW7bpzSnjnNbu3p+h0Jo13BQpLmaeOo3rXNEDDKtWvfRu/+DgYD399NMuCwjnvZWUrEqVvJUQ30vPD/yH/vvfdHV8uIf27z/o7tBwgykuLlafpwdr2KgEDXrxOQUEVNZPW3fqmSee07Yt2x3rhg8cq1EThuqfY55XuXLl9OP3KXpl5CQ200OS1C+uu/z8fPXBx8v0zoKPVeu2WzVt4kuGfT+/ncxWrRpsai4rONVlZLG7cXdyeSv/54Dr1PQPcncIMJmfd37i7hBgMhWq1rmun3fPrW1ddq0fD5vjCe58ZQUAACZFxceIxAcAAJPikTNGfPcBAADwGFR8AAAwKVpdRiQ+AACYlJmeuOwqtLoAAIDHoOIDAIBJsbnZiIoPAAAmVSy7y15/Rm5url599VW1bdtWd911lx577DF9++23jvlDhw6pb9++ioiIUKtWrTR58mSdO3fO6Rrz589XdHS0mjZtqpiYGG3dutUlfyYkPgAAwKVGjBih//znP5owYYKWLFmidu3aKT4+XuvWrZPNZlOvXr1ksVi0cOFCjR8/Xh9//LFmzZrleP/ixYs1efJkDRo0SIsXL1ZoaKh69+6tEydOXHVsPLkZpsGTm+FqPLkZrna9n9x8V8g9LrvW5qM/XtG6Y8eO6d5771VSUpLuv/9+x3hsbKyqVq2q+++/XyNGjNCPP/6oypUrS5IWLVqkV199VevWrZO3t7fat2+v6OhoDR8+XJJUVFSkhx56SI8//rgGDBhwVffBHh8AAEzKHcfZK1WqpDlz5igiIsJp3GKx6NSpU0pLS1PDhg0dSY8kRUVFKS8vT+np6apVq5YyMjIUFRXlmPfy8lKzZs2UlpZ21fGR+AAAgMvKyclRTk6OYdzf31/+/v6On/38/HTfffc5rdmyZYvWr1+vUaNGac2aNQoJCXGaDwo6X7E/evSovL29JanENdu2bbvq+yDxAQDApFz5HJ/k5GQlJiYaxuPj45WQkHDJ9+3du1fx8fEKDw9XTEyMvvnmG/n6+jqtsVqtkqTCwkLl5+c7jf1xjc1mu9rbIPEBAMCsil24jTc2NlZdunQxjP+x2nOxDRs2KD4+XtWrV1dSUpIqVKggb29vQwJz4WcfHx9HxaekNT4+Pld7GyQ+AADg8i5uaV3OsmXLNHLkSEVGRmrmzJny8/OTdL6FtWPHDqe1WVlZjrnq1as7xurXr++0Jjg4+Gpvg+PsAACYld2F//wZy5cv1/Dhw9WhQwclJSU5kh5JatGihXbs2OG0XyglJUW+vr5q1KiRAgMDFRoaqtTUVMd8UVGRNm7cqMjIyKv+MyHxAQDApIrtdpe9rtTRo0c1evRoRUVFadiwYcrOztaxY8d07NgxZWdn68EHH1RwcLAGDx6snTt3atWqVZo6dari4uIc+3p69uyp5ORkLV68WHv27NGoUaN05swZPfnkk1f9Z0KrCwAAuMzKlSuVn5+v9evXq3Xr1k5zERERWrBggebOnatx48apa9eu8vf3V0xMjNPzebp27arc3FzNmDFD2dnZaty4sebNm6fAwMCrjo8HGMI0eIAhXI0HGMLVrvcDDBsEtXDZtXZmbXDZtdyJig8AACblylNdZsEeHwAA4DGo+AAAYFKufIChWZD4AABgUrS6jGh1AQAAj0HFBwAAk6LVZUTiAwCASdntxe4Oocyh1QUAADwGFR8AAEyqmFaXAYkPAAAm5cYvZyizaHUBAACPQcUHAACTotVlROIDAIBJ0eoyotUFAAA8BhUfAABMiq+sMCLxAQDApHhysxGtLgAA4DGo+AAAYFJsbjYi8QEAwKQ4zm5E4gMAgElR8TFijw8AAPAYVHwAADApjrMbkfgAAGBStLqMaHUBAACPQcUHAACT4lSXEYkPAAAmRavLiFYXAADwGFR8AAAwKU51GZH4AABgUnxJqRGtLgAA4DGo+AAAYFK0uoxIfAAAMClOdRnR6gIAAB6DxAcAAJOyu/CfP6O4uFgzZ85U69atFR4erp49e+rAgQPX6C7/HBIfAABMym63u+z1Z8yePVsLFizQhAkT9OGHH8rLy0u9evVSYWHhNbrTK0fiAwAAXMZms2nevHmKj49XmzZt1KBBA02bNk3Hjx/XihUr3B0eiQ8AAGbljorPjh07lJeXp5YtWzrG/Pz81KhRI6WlpV2L2/xTONUFAIBJufJMV05OjnJycgzj/v7+8vf3d/ycmZkpSQoODnZaFxQUpCNHjrgwor/GrYnPOdthd348AACm5srfs7NmzVJiYqJhPD4+XgkJCY6f8/PzJUlWq9VpndVqlc1mc1k8fxUVHwAAcFmxsbHq0qWLYfyP1R5J8vb2lnR+r88fkx+bzSYfH59rG+QVIPEBAACXdXFL61KqVasmScrKypKfn59jPCsrS2FhYdcsvivF5mYAAOAyDRo0kJ+fn1JTUx1jubm52r59uyIjI90Y2XlUfAAAgMtYrVY99dRTmjZtmqpWraoaNWpo6tSpCg4OVrt27dwdHokPAABwrYEDB6qoqEhjxoxRfn6+mjVrprlz5xo2PLuDxc43mAEAAA/BHh8AAOAxSHwAAIDHIPEBAAAeg8SnjCouLtbMmTPVunVrhYeHq2fPnjpw4IC7w4JJJCUlqVu3bu4OAze43Nxcvfrqq2rbtq3uuusuPfbYY/r222/dHRZQKhKfMmr27NlasGCBJkyYoA8//FBeXl7q1auXCgsL3R0abnDz58/XtGnT3B0GTGDEiBH6z3/+owkTJmjJkiVq166d4uPjtW7dOneHBlwSiU8ZZLPZNG/ePMXHx6tNmzZq0KCBpk2bpuPHj2vFihXuDg83qMzMTPXr109TpkxRaGiou8PBDe7YsWNauXKlRo4cqVatWqlWrVrq16+fIiMj9fHHH7s7POCSSHzKoB07digvL08tW7Z0jPn5+alRo0ZKS0tzY2S4kaWnp8vX11fLli1TeHi4u8PBDa5SpUqaM2eOmjdv7jRusVh06tQpN0UFXB4PMCyDMjMzJUnBwcFO40FBQTpynIKNaAAAAatJREFU5Ig7QoIJtG3bVm3btnV3GDAJPz8/3XfffU5jW7Zs0fr16zVq1Cg3RQVcHhWfMig/P1+SDE+4tFqtstls7ggJAEq1d+9excfHKzw8XDExMe4OB7gkEp8yyNvbW5IMSY7NZpOPj487QgKAS9qwYYO6d++uW265RUlJSapQoYK7QwIuicSnDKpWrZokKSsry2k8KyvL0P4CAHdatmyZ4uLi1LhxY73//vuqUqWKu0MCSkXiUwY1aNBAfn5+Sk1NdYzl5uZq+/btioyMdGNkAPC75cuXa/jw4erQoYOSkpLk5+fn7pCAy2JzcxlktVr11FNPadq0aapatapq1KihqVOnKjg4WO3atXN3eACgo0ePavTo0YqKitKwYcOUnZ3tmKtQoQKVH5RZJD5l1MCBA1VUVKQxY8YoPz9fzZo109y5cw0bngHAHVauXKn8/HytX79erVu3dpqLiIjQggUL3BQZUDqL3W63uzsIAADw/9q1QwIAAAAGYf1bv8UNWwoEPHh8AIAM4QMAZAgfACBD+AAAGcIHAMgQPgBAhvABADKEDwCQIXwAgIwBcN5iiX9mIe4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x504 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_cm = pd.DataFrame(confusion_matrix(Y_test, y_pred), range(3), range(3))\n",
    "plt.figure(figsize=(10,7))\n",
    "sn.set(font_scale=1.4)\n",
    "sn.heatmap(df_cm, annot=True, annot_kws={\"size\": 16}, fmt='g')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that despite the fact that the classification is not exact, our model does not confuse opposing classes, in fact thanks to the matrix we can see that no entry-level product has been classified as high-end and vice versa, so this means that our model has a certain understanding of the environment and understands the characteristics between the opposing classes, it still has a bit of trouble with the mid-range class, which is more abstract than the other two classes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# XGBOOST : <a id=\"7\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "XGBoost is an optimized distributed gradient boosting library designed to be highly efficient, flexible and portable. It implements machine learning algorithms under the Gradient Boosting framework. XGBoost provides a parallel tree boosting (also known as GBDT, GBM) that solve many data science problems in a fast and accurate way. \n",
    "Here we bring out the heavy artillery!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = XGBClassifier(eval_metric='mlogloss',\n",
    "                      learning_rate =0.1, \n",
    "                      n_estimators=1000, \n",
    "                      max_depth=1, \n",
    "                      use_label_encoder=False, \n",
    "                      min_child_weight=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBClassifier(base_score=None, booster=None, colsample_bylevel=None,\n",
      "              colsample_bynode=None, colsample_bytree=None,\n",
      "              eval_metric='mlogloss', gamma=None, gpu_id=None,\n",
      "              importance_type='gain', interaction_constraints=None,\n",
      "              learning_rate=0.1, max_delta_step=None, max_depth=1,\n",
      "              min_child_weight=1, missing=nan, monotone_constraints=None,\n",
      "              n_estimators=1000, n_jobs=None, num_parallel_tree=None,\n",
      "              objective='binary:logistic', random_state=None, reg_alpha=None,\n",
      "              reg_lambda=None, scale_pos_weight=None, subsample=None,\n",
      "              tree_method=None, use_label_encoder=False,\n",
      "              validate_parameters=None, verbosity=None)\n"
     ]
    }
   ],
   "source": [
    "print(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model accuracy score : 0.8324\n"
     ]
    }
   ],
   "source": [
    "model.fit(X_train,Y_train)\n",
    "y_pred = model.predict(X_test)\n",
    "print('Model accuracy score : {0:0.4f}'. format(accuracy_score(Y_test, y_pred)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  79,   72,    0],\n",
       "       [  43,  989,  126],\n",
       "       [   0,  132, 1689]], dtype=int64)"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "confusion_matrix(Y_test, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAj4AAAGiCAYAAADnfswJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdfXzN9f/H8ccxZrYZlmwpF2O5/EXRtoiSSaUk32JfVGsoYRO5SC6LUUIuNmUR5puQWtKFLqRILoeipVzl2jZkZnZxbDu/P3ydr+PDkNPO+pzn3e38sff7fT6f92ft1l57vd7v97HYbDYbIiIiIm6glKsnICIiIlJcFPiIiIiI21DgIyIiIm5DgY+IiIi4DQU+IiIi4jYU+IiIiIjbKO3Km/uXv9WVtxeT0ckM4mynrTmunoKYTL71cLHe7+zxvU67VpnKtZx2LVdSxkdERETchkszPiIiIvI3Kixw9QxKHAU+IiIiZmUrdPUMShyVukRERMRtKOMjIiJiVoXK+FxMgY+IiIhJ2VTqMlCpS0RERNyGMj4iIiJmpVKXgQIfERERs1Kpy0ClLhEREXEbyviIiIiYlQ4wNFDgIyIiYlYqdRmo1CUiIiJuQxkfERERs9KuLgNlfEREREzKZit02uuvSkhIoEuXLg5tWVlZjB49mrvuuoumTZvy/PPPc/DgQYcxy5cvp127dtx22220b9+e1atXO/SfPHmSgQMHEhoaSkhICCNHjuTMmTNXnI8CHxEREflbLFiwgClTphjaY2JiWLt2LXFxcSxYsIDTp0/z/PPPU1BwbjH2unXrGDx4MF26dGHp0qXce++99OnTh507d9qv0a9fPw4cOMDcuXOJj49n7dq1jBo16opzUuAjIiJiVoWFzntdg7S0NJ5//nkmTZpEUFCQQ9+GDRtYu3Yt06dPJyQkhHr16jFmzBiysrL4448/AJg1axbh4eE89dRT1K5dm0GDBtGoUSPmzZsHwJYtW9i4cSOvvfYaDRs2JCwsjNjYWD7//HOOHDlS5NwU+IiIiJiVrdB5r2uQkpKCj48Py5Yto3Hjxg59P/zwA8HBwdSvX9/eVrt2bVatWkVwcDCFhYVs3bqVsLAwh/eFhoaSnJwMQHJyMjfccAPBwcH2/qZNm2KxWOxjLkeLm0VEROSKMjMzyczMNLT7+fnh5+fn0Na6dWtat259yevs27eP6tWrs2TJEubPn8/Jkydp2rQpw4YNIyAggMzMTLKzswkMDHR4X5UqVTh69CgA6enphn5PT08qVapEampqkc+hwEdERMSsnHiAYWJiIvHx8Yb26OhoYmJirvo6WVlZpKSkcOrUKUaNGoXFYmHSpEk8/fTTfPLJJ+Tm5gLnApkLeXp6YrVasdls5OTkGPrPj8nLyyvy/gp8REREzMqJBxhGRkbSsWNHQ/vF2Z4rKVOmDDk5OcyYMYNKlSoBEBcXR8uWLVmxYgV33303AFar1eF9VqsVb29vLBYLXl5ehv4LxxRFgY+IiIhc0aVKWn9FYGAgVapUsQc9ADfeeCOVKlXi0KFDVKxYEW9vb9LT0x3el56eTkBAgP0aF/dbrVZOnjxpKIFdTIubRUREzMpFu7qKcuedd5KamuoQuBw7doyTJ09SvXp1LBYLTZo0YePGjQ7v27BhA6GhoQCEhIRw7Ngx9u7da+8/v6j5zjvvLPL+CnxERETMykW7uory0EMPERQURL9+/di+fTu//vorL774IjVq1CA8PByAqKgovvzyS95991327NnD5MmTSUlJITIyEoDGjRvTpEkTBg4cyLZt29i4cSOjRo2iQ4cO9qzQ5SjwERERkWLj6elJYmIit9xyC1FRUXTr1o0KFSowb948ypYtC0CLFi147bXXWLx4MY899hhr1qxh5syZ1K5dGwCLxUJ8fDzVqlUjMjKSmJgYmjdvziuvvHLF+1tsNpvt73zAoviXv9VVtxYTcuGPspjUaWuOq6cgJpNvPVys98vb9pXTrlW20QNOu5YraXGziIiISdlsztvObhYqdYmIiIjbUMZHRETErJy4KNksFPiIiIiYlRO3oZuFAh8RERGzUsbHQGt8RERExG0o4yMiImJWTvyQUrNQ4CMiImJWKnUZqNQlIiIibkMZHxEREbPSri4DBT4iIiJmpVKXgUpdIiIi4jaU8RERETErlboMFPiIiIiYlQIfA5W6RERExG0o4yMiImJSNpsOMLyYAh8RERGzUqnLQKUuERERcRvK+IiIiJiVzvExUOAjIiJiVip1GajUJSIiIm5DGR8RERGzUqnLQIGPiIiIWanUZaBSl4iIiLgNZXxERETMSqUuAwU+IiIiZqVSl4FKXSIiIuI2lPERERExK2V8DBT4iIiImJXW+Bio1FUC3N0ilD9P77rs65ZqValc2Z+Zsyax90Ayew8kM+8/cVSrfrOrpy4l1N0twziZtfuyr2rVquLlVZYRo19k88/fcjD1Z1b9uIyOj7dz9dTlH6ZH967sSFnD6VO7WbN6GXeFNXX1lESKpIxPCbDt519p27qTQ1vZsp7Mey+ObT/9SlrqMb774WOqBNzIq6MncujgEZ57/mmWf7OIls3ac/LPDBfNXEqqbT+lcP99Tzi0lfUqS+J7cfz8068cOnSU+Ldf5+FH7mfc2Cns/H0PDz0czpzE6dhssDTpCxfNXP5JnnzyCd6a8Tqx46aQnPwzfftE8cXnC2hy5/3s23fQ1dMTUKnrEiw2m83mqpv7l7/VVbcu8ca/PpwnIh6lWchDNG8eQuKCeJ54LIqV364BwNPTk41bvmLpx8t5ZeQbLp5tyeDCH+V/hPEThtMpogN33fkgALv3bSSmz8u8N3+JfcziD2dxQ2V/2rR63FXTLFFOW3NcPYUSbffO9Xz51XdEx7wMQOnSpfn1l9V8/sUKBrw4ysWzK5nyrYeL9X45nzjv90O5DkOcdi1XUqmrBKpbN5ievZ5k3JgpnDj+J7VvrUl+fj6rvl9nH2O1Wtm6ZTvhbVq6cKbyT1G3XjDP9nqKcWPe5MTxP/H19WHO7AV8999A+rxdu/6gRs1qLpql/JMEBwdRs2Y1Pvvsa3tbfn4+Xyz/lgceuM+FMxMp2hVLXVarlS+//JLk5GSOHj1KXl4e3t7eBAYGEhoaStu2bSldWhUzZxo+egB7du9j/rzFABw+dJTSpUsTeFMVDh86ah9XvcYtVK9+i6umKf8gI0a/yO7d+0ice+5nav++gwzsP9phTKlSpWjT9h527dzjiinKP0ydW2sBsHvPPof2P/44QO1aNShVqhSFKrO4Xgn4b5CQkMD333/PwoULL9n/9ttvM3XqVFJSUhziieXLlxMXF8fBgwepWbMmgwcP5p577rH3nzx5ktjYWH744QdsNhsPPvggQ4cOxcfHp8j5FJnxOXDgAA8//DCjR49mz549+Pr6ctNNN1GuXDl27drF8OHDad++PYcOHbqW74EUoXqNW3ioXTgz4t61l26+XfEDx4//ycxZk6hTtzaV/Cvy0ssx1G9QB2+fci6esZR09p+p6e8WWQ58ecQL1K0bzPQps4pxdvJPVd7PF4DTp7Mc2k+fzsLDwwMfH29XTEsuZit03usvWLBgAVOmTLls/7Zt24iPjze0r1u3jsGDB9OlSxeWLl3KvffeS58+fdi5c6d9TL9+/Thw4ABz584lPj6etWvXMmrUlUusRaZqXn31VYKCgvj444/x9fU19GdlZTFgwADGjh1LQkLCFW8mV/b0M53JyDjFB4uW2dv+PHGSp7r2YeY7E1mf/CUAXy5fyfx5i+nS7V+umqr8Q0Q+E/Hfn6lPLjvmhQHPMWhIX+KmzebL5SuLcXbyT2WxWADj2rrz7cr2uLe0tDRGjx7Nhg0bCAoKuuSY7OxsBg8ezJ133sn69esd+mbNmkV4eDhPPfUUAIMGDWLLli3MmzeP8ePHs2XLFjZu3Mjnn39OcHAwALGxsURFRTFw4ECqVq162bkVmfHZvHkzgwYNumTQA+Dr68vAgQPZtGlTUZeRa/DwI/fzxWcrsFqtDu0b1m3mjttac8dtrWlYpwVdO/fC19eHjJOnXDRT+ado1/5+Pv/U+DN1Xuxrw3hl7BBmv/Meo4a/Xsyzk3+qzFOnAShf3vH3g6+vDwUFBZw5k+2KacnFCgud97oGKSkp+Pj4sGzZMho3bnzJMePGjaNOnTp06NDhoikXsnXrVsLCwhzaQ0NDSU5OBiA5OZkbbrjBHvQANG3aFIvFYh9zOUVmfPz8/EhLS6NOnTqXHXP48GG8vZXSdIabb7mJuvWCGXnRLx//GyrxwIP3sWzpl+y/YItog4Z12b59R3FPU/5BbrnlJurVC2bUsNcMfRaLhbffmUhEl8eYPPEtYl990wUzlH+qXbv/AKBWUHX2XLDOJyioOr9rnVjJ4cTMW2ZmJpmZmYZ2Pz8//Pz8HNpat25N69atL3utb775hlWrVvHpp5/y3XffGe6TnZ1NYGCgQ3uVKlU4evTcOtf09HRDv6enJ5UqVSI1NbXI5ygy4/PEE08wdOhQFi1axN69e8nOziY/P5/s7Gz27dvHBx98wPDhw/nXv1RucYamTc9FxZuTf3Zo9yxThhkzJ3BfeAt7W0jo7dx+x//xlcoSUoQmd577mUq+6GcKzmV6Iro8xvCh4xX0yDXbtWsvBw4c5tFHH7S3lS5dmnYPhbNy5Zoi3in/VImJiYSHhxteiYmJ13SdtLQ0Ro4cyfjx46lUqZKhPzc3FzgXyFzI09MTq9WKzWYjJyfH0H9+TF5eXpH3LzLjExMTg8Vi4Y033iAnx3iehY+PD926deOFF14o8iZydeo3uJXjx/80HEiYmprOF5+vYOy4odhsNjzLlGHchOFs37aDRe8vddFs5Z+gfoM6l/yZatS4Ic/3iWTlt2vYuGELd4bcbu8rKChg65btxT1V+Qd6Y+IMpk+LJSPjFGvXbqJP72eoXNmfadO1QL7EcOL5ZpGRkXTs2NHQfnG2p+jp2Bg6dCgPPfSQww6tC5UtWxbAUJ63Wq14e3tjsVjw8vK6ZPn+/JiiFBn4WCwWYmJi6NWrF7/99htpaWnk5OTg5eVFYGAg9erVu2TEJX9N5Rtv4FSGMY0IEN17KONfH860+HEUFtr4+svvGD3yDc6ePVvMs5R/khsv8zP10MPhlCpVitbhLWh9QSYRICvrDNUCL12TF7nQzIREypXzIia6By/0e5aff06h3cPd+OOPA66empznxFLXpUpa1+rIkSOsXbuWLVu2sHTpuT/c8/PzAQgJCaFXr1706tULb29v0tPTHd6bnp5OQEAAAIGBgYZ+q9XKyZMnDSWwi+nkZjENndwszqaTm8XZiv3k5oWjrzzoKpXr8upfet/QoUPZv38/CxcuJD8/n8OHHb8HX3/9NZMmTWL58uX4+/tTsWJFevToQYUKFXjzzf+V4bt06cKtt97KmDFj+Omnn4iIiGD58uXUqnXuTKm1a9fSo0cPvv/+e3uAdCk6eVBERMSsStixAqVLl6ZGjRoObTfccAMA1atXtx9gGBUVxXPPPUfDhg1p1aoVS5cuJSUlhdjYWAAaN25MkyZNGDhwIK+++iq5ubmMGjWKDh06FBn0gD6yQkRExLxcfIDhX9WiRQtee+01Fi9ezGOPPcaaNWuYOXMmtWvXBs4txYmPj6datWpERkYSExND8+bNeeWVV654bZW6xDRU6hJnU6lLnK3YS13vDXfatco9Oc5p13IllbpERETMqoSVukoCBT4iIiJmpUy4gdb4iIiIiNtQxkdERMSsVOoyUOAjIiJiVgp8DFTqEhEREbehjI+IiIhZFfP5O/8ECnxERERMylaoXV0XU6lLRERE3IYyPiIiImalxc0GCnxERETMSmt8DFTqEhEREbehjI+IiIhZaXGzgQIfERERs9IaHwMFPiIiImalwMdAa3xERETEbSjjIyIiYlY2rfG5mAIfERERs1Kpy0ClLhEREXEbyviIiIiYlbazGyjwERERMSud3GygUpeIiIi4DWV8REREzEqlLgMFPiIiIiZl064uA5W6RERExG0o4yMiImJWKnUZKPARERExK+3qMlCpS0RERNyGMj4iIiJmpVKXgQIfERERs9KuLgOVukRERMRtKOMjIiJiVip1GSjwERERMSvt6jJQqUtERET+NgkJCXTp0sWhLSUlhe7duxMaGkrz5s3p378/R48edRizfPly2rVrx2233Ub79u1ZvXq1Q//JkycZOHAgoaGhhISEMHLkSM6cOXPF+SjwERERMatCm/Nef8GCBQuYMmWKQ9vRo0d55plnqFKlCu+//z4JCQmcOHGC7t27k5ubC8C6desYPHgwXbp0YenSpdx777306dOHnTt32q/Tr18/Dhw4wNy5c4mPj2ft2rWMGjXqinNS4CMiImJStsJCp72uRVpaGs8//zyTJk0iKCjIoe/TTz+lbNmyxMbGEhwczG233cYbb7zB3r172bx5MwCzZs0iPDycp556itq1azNo0CAaNWrEvHnzANiyZQsbN27ktddeo2HDhoSFhREbG8vnn3/OkSNHipybAh8RERFxqpSUFHx8fFi2bBmNGzd26Gvbti1Tp06ldOn/LTO2WCwAnDp1isLCQrZu3UpYWJjD+0JDQ0lOTgYgOTmZG264geDgYHt/06ZNsVgs9jGXo8XNIiIiZuXEXV2ZmZlkZmYa2v38/PDz83Noa926Na1bt77kdWrWrEnNmjUd2mbOnImXlxd33XUXmZmZZGdnExgY6DCmSpUq9nVA6enphn5PT08qVapEampqkc+hwEdERMSsnBj4JCYmEh8fb2iPjo4mJibmL1937ty5LFy4kBEjRuDv728PXDw9PR3GeXp6YrVasdls5OTkGPrPj8nLyyvyfgp8RERE5IoiIyPp2LGjof3ibM/VKiwsZPLkycyePZu+ffvy1FNPAVC2bFkArFarw3ir1Yq3tzcWiwUvLy9D/4VjiqLAR0RExKyceI7PpUpaf1VeXh6DBw/mm2++YdSoUXTr1s3eV7FiRby9vUlPT3d4T3p6OgEBAQAEBgYa+q1WKydPnjSUwC6mxc0iIiJm5eLt7JecUmEhMTExfPfdd8TFxTkEPXBuoXOTJk3YuHGjQ/uGDRsIDQ0FICQkhGPHjrF37157//lFzXfeeWeR91fgIyIiIsUmMTGRVatWMWzYMBo3bsyxY8fsr/Pn+ERFRfHll1/y7rvvsmfPHiZPnkxKSgqRkZEANG7cmCZNmjBw4EC2bdvGxo0bGTVqFB06dLBnhS7HYrPZXPZBHv7lb3XVrcWEXPijLCZ12prj6imIyeRbDxfr/U73b++0a5Wf+ulfet/QoUPZv38/CxcuBKBjx478+uuvlxwbGxtLp06dAPjkk0+YMWMGR48eJTg4mMGDB9O8eXP72BMnTvDqq6/yww8/4OnpyQMPPMCwYcPw8vIqcj4KfMQ0FPiIsynwEWcr9sCn3yNOu1b56Z857VqupFKXiIiIuA3t6hIRETGra/yoCXegwEdERMSsnLgbyyxU6hIRERG3oYyPiIiIWSnjY6DAR0RExKS029VIpS4RERFxG8r4iIiImJVKXQYKfERERMxKgY+BSwOfvIKzrry9mMzOxrVcPQUxmWa/H3P1FETEyZTxERERMSmbMj4GCnxERETMSoGPgXZ1iYiIiNtQxkdERMSs9FFdBgp8RERETEprfIxU6hIRERG3oYyPiIiIWSnjY6DAR0RExKy0xsdApS4RERFxG8r4iIiImJQWNxsp8BERETErlboMVOoSERERt6GMj4iIiEmp1GWkwEdERMSsVOoyUOAjIiJiUjYFPgZa4yMiIiJuQxkfERERs1LGx0CBj4iIiEmp1GWkUpeIiIi4DWV8REREzEoZHwMFPiIiIialUpeRSl0iIiLiNpTxERERMSllfIyU8RERETEpW6HzXn9VQkICXbp0cWg7dOgQvXr1okmTJjRv3pyJEyeSn5/vMGbBggWEh4fTqFEjIiIi2LZt2zVf41IU+IiIiMjfYsGCBUyZMsWhzWq10qNHDywWC4sWLWLs2LF8+OGHxMXF2cckJSUxceJE+vfvT1JSEkFBQfTs2ZMTJ05c9TUuR4GPiIiIWdkszntdg7S0NJ5//nkmTZpEUFCQQ99XX33F4cOHmTBhAnXq1CE8PJxBgwYxf/58cnNzgXNZoq5du9K+fXuCg4MZN24cvr6+LFq06KqvcTkKfEREREzKVaWulJQUfHx8WLZsGY0bN3boS05Opn79+lSoUMHeFhYWRnZ2NikpKRw/fpx9+/YRFhZm7/fw8KBp06YkJydf1TWKosXNIiIickWZmZlkZmYa2v38/PDz83Noa926Na1bt77kddLS0ggMDHRoq1KlCgCpqal4eXkBXHLM9u3br+oaRVHgIyIiYlK2wmsrURUlMTGR+Ph4Q3t0dDQxMTFXfZ3c3Fx8fHwc2jw9PQHIy8sjJyfHoe3CMVar9aquURQFPiIiIiblzO3skZGRdOzY0dB+cbbnSry8vOwBzHnnv/b29rZnfC41xtvb+6quURQFPiIiInJFlypp/RWBgYHs2LHDoS09Pd3eV7VqVXtb3bp1HcYEBARc1TWKosXNIiIiJmWzWZz2cpaQkBB27NjhsF5ow4YN+Pj40KBBA/z9/QkKCmLjxo32/oKCAjZv3kxoaOhVXaMoCnxERERMqiQcYHixNm3aEBAQwIABA/jtt99YuXIlkydPJioqyr5Op3v37iQmJpKUlMTu3bsZMWIEZ86coVOnTld9jctRqUtERESKTdmyZZk9ezZjxoyhc+fO+Pn5ERERQd++fe1jOnfuTFZWFtOmTSMjI4OGDRsyZ84c/P39r/oal2Ox2Wy2v+3prsDHu6arbi0mtLNxLVdPQUym2e/HXD0FMZkDf24v1vsdDAl32rWqbfrWaddyJWV8RERETMp1qY2SS2t8RERExG0o4yMiImJSzjzA0CwU+IiIiJiUAh8jlbpERETEbSjjIyIiYlJa3GykwEdERMSkVOoyUqlLRERE3IYyPiIiIiblzM/YMgsFPiIiIiblzM/YMguVukRERMRtKOMjIiJiUoUqdRko8BERETEprfExUqlLRERE3IYyPiIiIialc3yMFPiIiIiYlE5uNlKpS0RERNyGMj4iIiImpVKXkQIfERERk9J2diOVukRERMRtKOMjIiJiUjrHx0gZnxLG09OTzVtWkJAwyd7m51eeKVPHsmfvRlLTfmHxB7MICqruwllKSWMp50WFgf0I/PxDblrxGTdMmUDp4Fr/G1DWE78+zxKQtJCbvllG5bjJlKkT7HANj1uq4j9hLDet+IzAL5KoOGwQpfz8ivlJpCS6/8FW/Lp/vUNbWa+yDB4ew+rkz9lxYANffP8B7Ts+YHjvM892ZdWmz9h5eBNfr0nikceMY+TvY7M572UWCnxKmGHDX6BePcdfSHPnTaN9+7aMGP4akU9Hc+ONN/DlV4soX97XRbOUksZ//Kt4t3uArAUf8OewVyj8809unDmN0tWrAVDxhT74PP4YWQsW8eeIMdgKC6gcN5lSN1YGoJSfHze+NZUytYPIeGMKGeMnUqZWEJVnTIbSSgy7s6ahjZma8BoWi2PmYPykkTzd89+8O/M9nn3qBTat28KMdyc5BDbPx0QxMnYQi9/7mKgu0WzZ9DPxs9+gWYuQ4n4METv9H60Eady4Ib17P8OxYyfsbfXqBfPgg63p0qUXyz75CoBff93Fb7//yMMPt2HRoqWumq6UEGXq3orXXSGcnPAm2Us/AyBvYzKVq91C+eeiODlyLOUeuJ+sRUs489EnAFi3p3DTFx/jfX9rst7/AO+HH6DUDf6kd+1O/v4D9jEBH72Pz6PtOJO0zGXPJ67h6VmG7r2eZOCwaHKycyhV5n9/J/vfUIlOXTswuN8oFr/3MQBrVq2nelA1nusbyWdLv8K3vA/9hzzPhLHTeCc+EYAfV2+gVnBN7rmvOevWbHLJc7kbLW42UsanhPDw8ODtmW8wdeo7HD2aZm//44+D3HvvY3z15ff2trNnrQB4lvUs7mlKCXQ+q5O33vEXiXXbL3iF3QmlLFjKlMZ2JtveZ8vJxXbWSim/8v+9xi0UpKXbgx6AwlOZ5O8/QNm7QovhKaSkadWmJX0H9GT86MnMm/W+Q5+vrw//mbOY1d+tdWjfu3sf1WrcDMA99zWnrFdZFs1PchjTuX0UE8ZO+3snL3Y2m8VpL7NQ4FNCvDjwecqUKcOkiW85tOfl5ZG86Sfy8vLw8PCgXr1g3p45kbS0Y3z26dcumq2UJAVp6QB4BAY4tJe+KZBSvr5YfHw4s/QzfJ7oSJn6dbGU98Wv73NYypYl57vV/73GMUpVqAAXBtMepfAIqELpmxyvK+7h562/cPftDzL3nfexXbTA48D+QwwfFMvRw//7I61UqVK0atOCPbv+AKBewzqkpx2nwW11+fy7xexJ28KqTZ/xUPs2xfocIhdT4FMC1KlTmyFDounbZyhnz5697LgZb73O5i0rCA9vycgRr/PnnxnFOEspqaw7fufs/oNUHPQCZerVweLrg/djj1C2eRgApby8OP1uIoUnT1JlzttU/XoZvl06cXL8JM7+vguAnJWrsHh44D/qZTwCAyjlX4mKg/qfC5y8yrny8cRF0o6mk5l5+qrHvzi0L7fWqcXM6XMBuOGGSnj7lCN+9kQW/SeJpzv1ZttPKbw9dzJNQhr/XdOWi2hxs5HW+LiYxWLhrbcnkJi4mI0btxQ5dtas93h/QRLt27flnVmT8SjtwfzED4ppplJinT3Lny+Pxv/V4VSZOxOAvO0pZL23GL+ekdhsNm58Jw7KePLnq69RcOw45e5rSaVhg7CdOUPuD2vJP3CQP0fHUvGlFwn8eCG2ggKyl39Dzpq1lKmpHYRStN79utNv0HMkxM9jxVerAChdpjQVKvgxYOgwPlr8KQBrf9jIrXVr029QL56J6OPKKbsNrfExumLg07VrV8Nq/stZsGDBdU/I3fTu/QzVq9/ME493x8PDw95usVjw8PCgoKDA3rY5+WcAVq9eR9WbAxk8uK8CHwEg/499pD/9LB5VbgQPDwqOplK++wiYaroAACAASURBVNPYCgoo2+R2SlevRnr33pzd8TsA1s1bKeXnR4UXY8j94dw6jdxVa0j9YS2lb65K4enTFGaconL8ZAqv4a9+cT8jYwfzbJ+nSZy9iHGjJtvbs/+7puz7b3+0t9lsNtb+sJF2j95f7PMUOe+Kgc+9997L1KlTqVWrFo0aNSqOObmV9o8+wM0338ThI9sc2hs1akC3Jx+nYYOW3HPPXcyfv8Sh/+efU3jwwdbFOVUpoSxly+J13z3kJW+hIP2Yvb1McC3y9+7DI+BGbPkF9qDnPOu2X/C+vzWWcl6UqlCBsnfeQfZnX5J/8NB/L2yhdK0gcr7+tjgfR/4hLBYLU94ax78i2hM3+R0mjotz6N+39yAAZTzLOLSXLl3asGZI/j5mWpTsLFcMfHr16oWvry+TJ08mISGBW265pTjm5Tb6xQzDt7yPQ9ucOVPZvfsPxo+fRoOGdXl75kT27TvE6tXr7GNat25JSspvxT1dKYFs+flUHNKf0+/MJWvRhwB43BSIV/MwTr+/hPwDh7CU9qBMw/qcTdlhf59ng/oU/HkSW04uHsG1qTR8CGd37uHsznPrfsqFt8KjUkVyf1x3yfuKexsZO5h/RbRnzIiJzH5rvqH//I6vhzu05d23/wOc2716z33N2Lzxp2KdqztTqcvoqtb4dOvWjR9++IGpU6cyadKkK79BrtquXXsNbbm5efx5IoOtW7azzeNXNmzYQsI7E3n1lUmcOHGSpyM706xZUzo+9kzxT1hKnoICspd9QflnulFwMgPbmWz8+j5LwckMziz6kMIzZ7Du3IV/7Cgy35lD4fETeN3dDO+H7idj8nQArCk7sP6+i4rDB5H59rt43HgDFfpHk7t2A3mbil57Ju7n/xrVp3uvbqz+bi2bN/7EHXf+rxpQUFDAtq0p/LFnP4veS+KlEf2wWCzs/G03T0VFcHO1qjz/zEAXzl7c3VUvbh4zZgwpKSl/51zkEgoKCnji8e68OmYIY2OHUqlSRbZu3U77R55yyACJezv19iwAKkT3wuLpSd7mrZyKT6AwMxOAEzGD8Yt+jgoxvbGULUv+vv2cGPYKuf/dzk5hIX++NJIKL8bgP3YEtpxcsj/5jMyEOa56JCnB7n+oFaVKleKe+5pzz33NHfrOZGVTv/q5HYUvDxhD6pE0evZ+Cn//ivya8jvd/vUcv/260xXTdksqKhpZbC4stvp413TVrcWEdjaudeVBIteg2e/HrjxI5Boc+HN7sd5v7U2PO+1azY9+dNVjrVYrcXFxfPrpp5w6dYr69eszaNAgmjRpAsChQ4cYO3YsmzZtwsvLi44dOzJgwABKX/AROQsWLGDOnDkcO3aM+vXrM3z4cKesNdY5PiIiIiblqpObZ8yYwUcffcTYsWP5+OOPqVWrFj179iQtLQ2r1UqPHj2wWCwsWrSIsWPH8uGHHxIX978F8klJSUycOJH+/fuTlJREUFAQPXv25MSJE0Xc9eoo8BERERGn+vbbb3n00Udp2bIlNWvWZOjQoZw5c4bNmzfz1VdfcfjwYSZMmECdOnUIDw9n0KBBzJ8/n9zcXAASEhLo2rUr7du3Jzg4mHHjxuHr68uiRYuue24KfEREREyq0Imva+Hv7893333HoUOHKCgoYMmSJXh6etKgQQOSk5OpX78+FSpUsI8PCwsjOzublJQUjh8/zr59+wgLC7P3e3h40LRpU5KTk//S9+FCOrlZRETEpGw4bzt7ZmYmmf/dMHEhPz8//Pz8HNpGjhxJ//79CQ8Px8PDg1KlSjF16lRq1qxJWloagYGBDuOrVKkCQGpqKl5eXgCXHLN9+/WvkVLgIyIiIleUmJhIfHy8oT06OpqYmBiHtl27duHr68uMGTMICAhgyZIlDBkyhP/85z/k5ubi4+N4fp2n57kPSM7LyyMnJ8eh7cIxVqv1up9DgY+IiIhJFTpx33ZkZCQdO3Y0tF+c7Tly5AhDhgxh9uzZ3HXXXQDcdttt7N69m+nTp+Pl5WUIYM5/7e3tbc/4XGqMt7f3dT+HAh8RERGTKnRiqetSJa1L2bZtG2fPnuW2225zaG/cuDErV66kWbNm7Nixw6EvPT0dOFfeqlq1qr2tbt26DmMCAgKu9zG0uFlERESc5/zanN9/d/x8wJ07dxIUFERISAg7duxwWC+0YcMGfHx8aNCgAf7+/gQFBbFx40Z7f0FBAZs3byY0NPS656fAR0RExKRsWJz2ulqNGjWiadOmDBs2jPXr17Nv3z6mTp3K2rVree6552jTpg0BAQEMGDCA3377jZUrVzJ58mSioqLs63q6d+9OYmIiSUlJ7N69mxEjRnDmzBk6dep03d8TlbpERERM6lq3oTtDqVKlePvtt5k6dSovv/wyGRkZ1K1bl7lz59pPbp49ezZjxoyhc+fO+Pn5ERERQd++fe3X6Ny5M1lZWUybNo2MjAwaNmzInDlz8Pf3v+756SMrxDT0kRXibPrICnG24v7Iim8CIpx2rfvTFjvtWq6kjI+IiIhJOfMcH7NQ4CMiImJSrih1lXRa3CwiIiJuQxkfERERk1LGx0iBj4iIiElpjY+RSl0iIiLiNpTxERERMalCJXwMFPiIiIiYlDM/q8ssVOoSERERt6GMj4iIiEm57KMZSjAFPiIiIial7exGKnWJiIiI21DGR0RExKQKLVrcfDEFPiIiIialNT5GKnWJiIiI21DGR0RExKS0uNlIgY+IiIhJ6eRmI5W6RERExG0o4yMiImJS+sgKIwU+IiIiJqVdXUYqdYmIiIjbUMZHRETEpLS42UiBj4iIiElpO7uRSl0iIiLiNpTxERERMSktbjZS4CMiImJSWuNjpFKXiIiIuA1lfERERExKi5uNFPiIiIiYlAIfI5W6RERExG0o4yMiImJSNi1uNlDgIyIiYlIqdRmp1CUiIiJuQ4GPiIiISRU68XWtli5dSrt27bjtttt4+OGHWb58ub3v0KFD9OrViyZNmtC8eXMmTpxIfn6+w/sXLFhAeHg4jRo1IiIigm3btv2FWRgp8BERETEpmxNf1+KTTz5h2LBhRERE8Nlnn/HII4/w4osvsnnzZqxWKz169MBisbBo0SLGjh3Lhx9+SFxcnP39SUlJTJw4kf79+5OUlERQUBA9e/bkxIkT1/PtABT4iIiIiBPZbDamTZvGk08+SWRkJDVq1KB37940b96c9evX89VXX3H48GEmTJhAnTp1CA8PZ9CgQcyfP5/c3FwAEhIS6Nq1K+3btyc4OJhx48bh6+vLokWLrnt+WtwsIiJiUq74yIq9e/dy+PBhHnnkEYf2d999F4DRo0dTv359KlSoYO8LCwsjOzublJQUatSowb59+wgLC7P3e3h40LRpU5KTk697fgp8RERETMqZu7oyMzPJzMw0tPv5+eHn52f/et++fQBYrVaee+45tm/fzi233ELv3r1p3bo1aWlpBAYGOlyjSpUqAKSmpuLl5QVwyTHbt2+/7udQqUtERESuKDExkfDwcMMrMTHRYVxWVhYAQ4YM4YEHHmDOnDm0aNGCPn368OOPP5Kbm4unp6fDe85/nZeXR05OjkPbhWOsVut1P4cyPiIiIiblzIxPZGQkHTt2NLRfmO0BKFOmDABRUVE8/vjjANSvX59ffvmFOXPm4OXlZQhgzn/t7e1tz/hcaoy3t/d1P4cCHxEREZO61t1YRbm4pHU550tUderUcWi/9dZb+fbbb2nWrBk7duxw6EtPT7e/t2rVqva2unXrOowJCAi4rmcAlbpERETEiRo0aICPj49hPc7OnTupXr06ISEh7Nixw2G90IYNG/Dx8aFBgwb4+/sTFBTExo0b7f0FBQVs3ryZ0NDQ656fMj4iIiIm5YpdXV5eXvTs2ZO33nqLKlWqcPvtt/P555+zZs0a5s6dS5MmTZg6dSoDBgxg8ODBHDlyhMmTJxMVFWVf19O9e3diY2MJCgqiUaNGvPvuu5w5c4ZOnTpd9/wU+IiIiJiUqz6rq0+fPnh7ezN9+nRSU1OpVasWcXFxNGvWDIDZs2czZswYOnfujJ+fHxEREfTt29f+/s6dO5OVlcW0adPIyMigYcOGzJkzB39//+uem8VmszmzBHhNfLxruurWYkI7G9dy9RTEZJr9fszVUxCTOfDn9W/Hvhav1XjSadd6ef97TruWK2mNj4iIiLgNl5a68vLPuvL2YjLNdx539RTEZPbs/MTVUxC5LoVO3ddlDlrjIyIiYlKuWuNTkqnUJSIiIm5DGR8RERGTUqHLSIGPiIiISanUZaRSl4iIiLgNZXxERERMyhUnN5d0CnxERERMStvZjVTqEhEREbehjI+IiIhJKd9jpMBHRETEpLSry0ilLhEREXEbyviIiIiYlBY3GynwERERMSmFPUYqdYmIiIjbUMZHRETEpLS42UiBj4iIiElpjY+RSl0iIiLiNpTxERERMSnle4wU+IiIiJiU1vgYqdQlIiIibkMZHxEREZOyqdhloMBHRETEpFTqMlKpS0RERNyGMj4iIiImpXN8jBT4iIiImJTCHiOVukRERMRtKOMjIiJiUip1GSnwERERMSnt6jJSqUtERETchjI+IiIiJqUDDI0U+IiIiJiUSl1GKnWJiIjI3+aPP/7gjjvuYMmSJfa2Q4cO0atXL5o0aULz5s2ZOHEi+fn5Du9bsGAB4eHhNGrUiIiICLZt2+aU+SjwERERMSmbE//9FWfPnmXQoEFkZ2fb26xWKz169MBisbBo0SLGjh3Lhx9+SFxcnH1MUlISEydOpH///iQlJREUFETPnj05ceLEdX9PFPiIiIiYVKETX39FXFwcPj4+Dm1fffUVhw8fZsKECdSpU4fw8HAGDRrE/Pnzyc3NBSAhIYGuXbvSvn17goODGTduHL6+vixatOgvzuR/FPiIiIiI023atInFixczYcIEh/bk5GTq169PhQoV7G1hYWFkZ2eTkpLC8ePH2bdvH2FhYfZ+Dw8PmjZtSnJy8nXPS4ubRURETKrQ5rxdXZmZmWRmZhra/fz88PPzM4wdMmQII0aM4KabbnLoS0tLIzAw0KGtSpUqAKSmpuLl5QVwyTHbt2+/7udQ4CMiImJSztzMnpiYSHx8vKE9OjqamJgYh7ZXXnmF22+/nfbt2xvG5+bmGspfnp6eAOTl5ZGTk+PQduEYq9V6Xc8ACnxERETkKkRGRtKxY0dD+8XZnqVLl5KcnMynn356yet4eXkZApjzX3t7e9szPpca4+3t/Zfnf54CHxEREZNy5md1XaqkdSkfffQRJ06coFWrVg7tY8aMYd68eYSEhLBjxw6HvvT0dOBceatq1ar2trp16zqMCQgIuM6nUOAjIiJiWq44uXnSpEn23VnntW3blujoaB555BF++uknkpKSyMzMtAdSGzZswMfHhwYNGuDp6UlQUBAbN26kZcuWABQUFLB582YiIiKue34KfERERMRpLpeV8ff35+abb6Zy5cpMnTqVAQMGMHjwYI4cOcLkyZOJioqyr+vp3r07sbGxBAUF0ahRI959913OnDlDp06drnt+CnxERERMqiR+ZEXZsmWZPXs2Y8aMoXPnzvj5+REREUHfvn3tYzp37kxWVhbTpk0jIyODhg0bMmfOHPz9/a/7/habzYl73a5Rac+bXXVrMaGby9/g6imIyez+famrpyAmU6ZyrWK9X6caHZx2rSX7P3HatVxJBxiKiIiI21CpS0RExKRcsbi5pFPgIyIiYlIlcY2Pq6nUJSIiIm5DGR8RERGTcuH+pRJLgY+IiIhJOfPkZrNQqUtERETchjI+IiIiJqXFzUYKfERERExK29mNFPiIiIiYlNb4GGmNj4iIiLgNZXxERERMStvZjRT4iIiImJQWNxup1FWC9ejelR0pazh9ajdrVi/jrrCmrp6S/AO0ebAVKfvXObSVL+9L7MThbEr5lpT965j13jSq17zFYUyFin7EThzOjz99yS/71vLxl//h7nvCinPqUkJ898N6Qtv8y9D+xYrv6fhUb5rc9yjtInqwYInjp3WfyjzN6Nencd+j3Wj+YCdiXnqVg4ePOoxJ+W0X3WOG0vS+Dtz3aDcmxs0iNy/vb30ekQsp8CmhnnzyCd6a8TrvL/yIzhHPkZFxii8+X0DNmtVcPTUpwZqGNGbqzPFYsDi0T3/nddq2u4/xr04husdgKlf2Z/GyOfiW97GPmTnvTdo82IopE96i9zMDOXjgCP/5cCZN7mxU3I8hLrR1+68MHfOGYTfQ8hWreOmVN7g7rClvTxrDA61b8trUmXzyxTf2MUNemcB3a9bTv3cU40cOIjX9GD1feJns7BwA9h04RFT0S2SePs2EV4bw0gu9WPXjBvoNHVOsz+hObE78ZxYqdZVQr4waxKzZCxgbOwWAb1as5tdfVvNCv2cZ8OIoF89OShpPzzJE9XqSgS/3JSc7h1Jl/vc3za11a9G67T30ihzAl599C8DO3/aw9uevuP/BVny85HMa3dGQ5i1D6drxWX5cvQGANavWU6debXr0footPQa75Lmk+FitVt5b8glxs+ZTzsuLwvx8e5/NZuPNt+bw746PMCi6JwBhTW/nyNE01m7aSod293PiZAY/btjMmJf70+GhNgDUrHYzj3R5ljUbkml7X0ve/3AZpUpZeGfKOPwrVQSgbnAt2nd9ltVrN3JP89Dif3CT064uoytmfDZs2MDgwYN57rnneP/99ykoKHDoP3XqFN26dfvbJuiOgoODqFmzGp999rW9LT8/ny+Wf8sDD9znwplJSdWqTQv69u/B+FfeZN6shQ59B/YdokPbbnz3zQ/2trNnzwLg6ekJQGFhIQvnf0Tyhq32MTabjf1/HKBajZuL4QnE1X5Yn8zs/3zAwL496frEow59Kb/t4mhaOk90eMihfcIrLzFh9BAArHlWAHx9vO39FSv4AedKYAD7Dh6mXp3a9qAHIKjGLVSq6MeaDZud/1Ail1Bk4LNy5UqioqI4duwYZ8+eZezYsTz55JNkZmbax5w9e5YtW7b87RN1J3VurQXA7j37HNr/+OMAtWvVoFQpVSjF0c9bU2hxx0PMe+d9wy6OvDwrP23eTl6eFQ8PD26tW4uJ08eQnnacr75YCcAvP+9g6IBXyfvvLy8A3/I+hDZryp5dfxTrs4hr/F/9Ony5ZC5PduqAxeJYKt2559zPQEFBAc/0Hczt97YnvONTLEr6zD7mpsAq3Ht3KO8kLmLv/oOcOJnBa1PextfHm5bNQgAIrHIjaenHHX5GM09nkXk6iyNH04rhKd2PzWZz2sssivwN+tZbb/HCCy8wb9485s6dy8KFCzl48CBRUVFkZ2cX1xzdTnk/XwBOn85yaD99OgsPDw98LviLSgQg7Wg6mf/9q7oor08dzYq1S2l5XzNeHzOVjJOnLjt27BvDKO/ny+y3/uPMqUoJFXBjZfzK+16y72TGKTw8ShHz0is0D21KwptjCW/ZjNjJM1i+YpV93NAXnifrTDaPdn2Oex/pwopVa5k6fgSBVW4E4JG293Hw8FHGv/k2x0/8yZHUNIaNnYSHhwc5ubnF8pzuphCb015mUWTgs2fPHtq1a2f/+vbbbycxMZFDhw7Rr18/Q9lLnOP8X1sXR9jn2wsLtUFR/pr35n7Avzv0IHH2It6cEUtEt46XHDf2jWH8q3N7YkdOImX7b8U8Sylp8vMLKCgo5IlHH+K5yH8T1vR2hr3Yh3uah/L23AUApB07TrdeL1LOy4spscN5Z8o4WrUI44WXx/LzLzsACG3amJGDolm6/BtaPdqNh//dk9pB1WlQ91a8vMq68hHFjRQZ+FSqVInDhw87tNWuXZv4+Hg2bNjA8OHDTZX+KikyT537y738RX99+fr6UFBQwJkzyrbJX/Pzll9Yt2YTrw6bwPJPV9B3QE+H/jJlShM/+w2e7vFvXnt1imG9kLgn73JeALS4606H9mYhd7D/4GHOnj3L0s+/IfN0FjPfHMv997WgeWgTJo15meCgGrz51hz7eyI6PszaL5fwyYIEVn26kAG9u3Ps+AkqlC9frM/kLrSry6jIwKdNmzaMHj2a1atXO5S2QkJCGDduHJ988glDhgz52yfpbnbtPldPrxVU3aE9KKg6v+/c44opyT9YtRo307nrY4b2lO07CLipiv3rsl5lmbf4Ldo9ej/DBo5l5vS5xTlNKcGq3VIV+N+i+PPy8/Ox2cBiKUVq+jECq1Qm4MbK9n6LxcLtjRqwZ98BAPb8sZ/lK1ZRpnRpatesjl95XzJPZ5Gafox6t9YuvgdyI4U2m9NeZlFk4PPCCy9Qp04devXqZVjA/Oijj/LGG2/w008//a0TdEe7du3lwIHDPProg/a20qVL0+6hcFauXOPCmck/Ua3aNZkYN4ZmLUIc2lvc24zfd+yyfz094TXCmjcl5tmXWDBvSXFPU0qwO2//P8p6evLVd47//1m9dhP/V+9WSpf2oEa1m0lNO0Zq+jGHMdtTfufmmwIA+G3XXoaOeYMTJzPs/R8s/QKbDfsCaJG/W5Hn+Pj4+BAXF8fx48cpV66cob99+/aEhoby9ddfX+Ldcj3emDiD6dNiycg4xdq1m+jT+xkqV/Zn2vRZrp6a/MOsWbWeLZt+ZlL8WCaNi+PPExlEPNmRO8NuJzKiDwAPPNyaBx9pw4eLlnHkcCp3XHBoYU52Dr/9uutylxc34Ovjw7NPR/DWnAX4entz5x238eW3q0n+aTtvTzp3+GDHh9vynw+W0nvgKJ6P6oqvjzfLlq9g6/Zfmf7aubPHWt0dRmX/Srz0ygS6d+vErr37iJuVSKcODxFU45aipiB/kXnyNM5jsblwkU5pT50PUpQB/XsRE92DypX9+fnnFAYPGcN6nXVxWTeXv8HVUygR+g/pzXN9I2lQ4y57W8VKFXhp5Au0btuSihUrsP3nX5n82gzWrdkEwKT4sXTq0uGS1/t9x27atjB+fIE72P37UldPwSVmvPse8xZ+xKYVHzu0v7fkE97/cBmp6ceoUe1m+vZ4kjb33m3vP5KaxqT42azbtBWbzUbd4Fr07fkUoU3+F0jv2ruP8W++Tcpvu6hU0Y+OD7fl2acj8PDwKLbnc6UylWsV6/3uvrm106714+GVTruWKynwEdNQ4CPO5q6Bj/x9FPi4nj6yQkRExKTMdP6OsyjwERERMSkdOWOkzz4QERERt6GMj4iIiEmp1GWkwEdERMSkzHTisrOo1CUiIiJuQxkfERERk9LiZiNlfEREREyqEJvTXtciKyuL8ePH07p1a+644w7+9a9/8e2339r7Dx06RK9evWjSpAnNmzdn4sSJ5OfnO1xjwYIFhIeH06hRIyIiIti2bZtTvicKfERERMSpXn75Zb7//ntiY2NZunQpbdu2JTo6mnXr1mG1WunRowcWi4VFixYxduxYPvzwQ+Li4uzvT0pKYuLEifTv35+kpCSCgoLo2bMnJ06cuO656eRmMQ2d3CzOppObxdmK++TmOwLvvvKgq7Q19cerGnfs2DFatGhBQkICrVq1srdHRkZSuXJlWrVqxcsvv8yPP/5IhQoVAFiyZAnjx49n3bp1eHl58cADDxAeHs6QIUMAKCgo4P777+fxxx+nb9++1/UcWuMjIiJiUq7Yzl6uXDlmzZpFkyZNHNotFgunTp0iOTmZ+vXr24MegLCwMLKzs0lJSaFGjRrs27ePsLAwe7+HhwdNmzYlOTn5uuenwEdERESuKDMzk8zMTEO7n58ffn5+9q99fX255557HMb89NNPrF+/nhEjRrBmzRoCAwMd+qtUqQJAamoqXl5eAJccs3379ut+DgU+IiIiJuXMc3wSExOJj483tEdHRxMTE3PZ9+3Zs4fo6GgaN25MREQEK1aswMfHx2GMp6cnAHl5eeTk5Di0XTjGarVe72Mo8BERETGrQicu442MjKRjx46G9guzPRfbtGkT0dHRVK1alYSEBMqUKYOXl5chgDn/tbe3tz3jc6kx3t7e1/sYCnxERETkyi4uaV3JsmXLGDZsGKGhoUyfPh1fX1/gXAlrx44dDmPT09PtfVWrVrW31a1b12FMQEDA9T6GtrOLiIiYlc2J/67Fp59+ypAhQ3jooYdISEiwBz0AISEh7Nixw2G90IYNG/Dx8aFBgwb4+/sTFBTExo0b7f0FBQVs3ryZ0NDQ6/6eKPARERExqUKbzWmvq5WamsrIkSMJCwtj8ODBZGRkcOzYMY4dO0ZGRgZt2rQhICCAAQMG8Ntvv7Fy5UomT55MVFSUfV1P9+7dSUxMJCkpid27dzNixAjOnDlDp06drvt7olKXiIiIOM3XX39NTk4O69evp2XLlg59TZo0YeHChcyePZsxY8bQuXNn/Pz8iIiIcDifp3PnzmRlZTFt2jQyMjJo2LAhc+bMwd/f/7rnpwMMxTR0gKE4mw4wFGcr7gMM61UJcdq1fkvf5LRruZIyPiIiIiblzF1dZqE1PiIiIuI2lPERERExKWceYGgWCnxERERMSqUuI5W6RERExG0o4yMiImJSKnUZKfARERExKZut0NVTKHFU6hIRERG3oYyPiIiISRWq1GWgwEdERMSkXPjhDCWWSl0iIiLiNpTxERERMSmVuowU+IiIiJiUSl1GKnWJiIiI21DGR0RExKT0kRVGCnxERERMSic3G6nUJSIiIm5DGR8RERGT0uJmIwU+IiIiJqXt7EYKfERERExKGR8jrfERERERt6GMj4iIiElpO7uRAh8RERGTUqnLSKUuERERcRvK+IiIiJiUdnUZKfARERExKZW6jFTqEhEREbehjI+IiIhJaVeXkQIfERERk9KHlBqp1CUiIiJuQxkfERERk1Kpy0iBj4iIiElpV5eRSl0iIiLiNhT4iIiImJTNif+uRWFhIdOnT6dly5Y0btyY7t27s3///r/pKa+NAh8RERGTstlsTntdixkzfw8yuwAABJNJREFUZrBw4UJiY2NZvHgxHh4e9OjRg7y8vL/pSa+eAh8RERFxGqvVypw5c4iOjubee++lXr16TJkyhePHj7N8+XJXT0+Bj4iIiFm5IuOzY8cOsrOzueuuu+xtvr6+NGjQgOTk5L/jMa+JdnWJiIiYlDP3dGVmZpKZmWlo9/Pzw8/Pz/51WloaAAEBAQ7jqlSpwtGjR504o7/GpYFPvvWwK28vIiJias78PRsXF0d8fLyhPTo6mpiYGPvXOTk5AHh6ejqM8/T0xGq1Om0+f5UyPiIiInJFkZGRdOzY0dB+YbYHwMvLCzi31ufC4MdqteLt7f33TvIqKPARERGRK7q4pHU5N910E/D/7d0/KPx/AMfxl3SXP5/BcFxKSRkuhovT+aaOuuHKahFZsNxw3UYRpstEV8pwuSySlEEMSimbC4PFmQwm7sNAXS5Xx2/4lV8SfoPf7318no/x/Vme46vPvfucZNu2LMt6PbdtW62trf9Z37/F5WYAAPBtfD6fLMvS8fHx61k+n1c2m1UwGDRY9jfe+AAAgG/jdrs1MjKiZDIpj8ejpqYmLS4uyuv1KhKJmM5j+AAAgO8Vj8dVKpU0NzenQqGgQCCgdDr97sKzCRUv/IMZAABwCO74AAAAx2D4AAAAx2D4AAAAx2D4lKnn52ctLS0pFArJ7/drbGxMV1dXprPwS6RSKQ0NDZnOwA+Xz+c1Pz+vcDisjo4ODQwM6ODgwHQW8CmGT5laXl7WxsaGEomENjc3VVlZqfHxcT09PZlOww+3vr6uZDJpOgO/wNTUlA4PD5VIJLS9va1IJKJYLKajoyPTacCHGD5lqFgsanV1VbFYTH19ffL5fEomk7q7u9Pe3p7pPPxQuVxO0WhUCwsLamlpMZ2DH+729lb7+/uanp5WT0+PmpubFY1GFQwGtbW1ZToP+BDDpwxdXFzo8fFRf/78eT2zLEttbW06PT01WIaf7Pz8XLW1tdrZ2ZHf7zedgx+uurpaKysr6urqenNeUVGhh4cHQ1XA1/iAYRnK5XKSJK/X++a8oaFB19fXJpLwC4TDYYXDYdMZ+CUsy1Jvb++bs7OzM2UyGc3MzBiqAr7GG58yVCgUJOndFy7dbreKxaKJJAD41OXlpWKxmPx+vwYHB03nAB9i+JShqqoqSXo3corFompqakwkAcCHTk5ONDw8rPr6eqVSKblcLtNJwIcYPmWosbFRkmTb9ptz27bf/fwFACbt7OxodHRU7e3tWltbU11dnekk4FMMnzLk8/lkWZaOj49fz/L5vLLZrILBoMEyAPjH7u6uJicn1d/fr1QqJcuyTCcBX+Jycxlyu90aGRlRMpmUx+NRU1OTFhcX5fV6FYlETOcBgG5ubjQ7O6vu7m5NTEzo/v7+9ZnL5eLND8oWw6dMxeNxlUolzc3NqVAoKBAIKJ1Ov7vwDAAm7O/vq1AoKJPJKBQKvXnW2dmpjY0NQ2XA5ypeXl5eTEcAAAD8H7jjAwAAHIPhAwAAHIPhAwAAHIPhAwAAHIPhAwAAHIPhAwAAHIPhAwAAHIPhAwAAHIPhAwAAHOMv3INxHTj3ehsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x504 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_cm = pd.DataFrame(confusion_matrix(Y_test, y_pred), range(3), range(3))\n",
    "plt.figure(figsize=(10,7))\n",
    "sn.set(font_scale=1.4)\n",
    "sn.heatmap(df_cm, annot=True, annot_kws={\"size\": 16}, fmt='g')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Grid Search : <a id=\"8\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We notice that the XGBOOST model is composed of a lot of hyperparameters, the question is how can we be sure to have the right ones? Well, we are going to perform what we call a grid search, the goal is to test all the combinations of hyperparameters to find the most optimal ones, a bit like testing all the combinations of a code box to open it. This process can be very long for some models (from several hours to several months for some models with billions of hyperparameters like the [GPT3](https://fr.wikipedia.org/wiki/GPT-3))\n",
    "\n",
    "![image.png](img/lock.gif)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The parameters combination that would give best accuracy is : \n",
      "{'max_depth': 3, 'min_child_weight': 2}\n",
      "The best accuracy achieved after parameter tuning via grid search is :  0.8584214696485624\n"
     ]
    }
   ],
   "source": [
    "param_test1 = {\n",
    " 'max_depth':range(3,25,1),\n",
    " 'min_child_weight':range(1,11,1)\n",
    "}\n",
    "gsearch1 = GridSearchCV(estimator = XGBClassifier(learning_rate =0.1, n_estimators=1000, eval_metric='mlogloss'), \n",
    "param_grid = param_test1,n_jobs=4,iid=False, cv=5)\n",
    "gsearch1.fit(X_train, Y_train)\n",
    "print('The parameters combination that would give best accuracy is : ')\n",
    "print(gsearch1.best_params_)\n",
    "print('The best accuracy achieved after parameter tuning via grid search is : ', gsearch1.best_score_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Class Balancing : <a id=\"9\"></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that despite a powerful model and optimal hyperparameters, our accuracy does not increase enormously, perhaps the solution is elsewhere, for example in the balancing of our classes.\n",
    "Indeed we had seen previously that our classes were not balanced, to remedy this, there are 2 solutions, \n",
    "The undersampling which consists in rebalancing the dataset by decreasing the number of instance of the majority class and the oversampling which has the opposite consists in artificially increasing the number of instance of the minority class.\n",
    "\n",
    "![image.png](img/undersampling_oversampling.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we will use undersampling on our classes 1 and 2 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [],
   "source": [
    "count_class_0 = df[df['categorie']==0].count()['categorie']\n",
    "df_class_0 = df[df['categorie'] == 0]\n",
    "df_class_1 = df[df['categorie'] == 1]\n",
    "df_class_2 = df[df['categorie'] == 2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_class_2_under = df_class_2.sample(count_class_0)\n",
    "df_class_1_under = df_class_1.sample(count_class_0)\n",
    "df_test_under = pd.concat([df_class_2_under, df_class_1_under, df_class_0], axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Categorie')"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfwAAAGJCAYAAABvvYFhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deVTU9f7H8dfgGowroGm2uIGhMWCJC7nkVpndylu5nLCU1Lph1yw1W65WZOaKZBqp11tZZG6FLeZVs27uml5v5tUyLe0qCKFCbCLf3x8d5xeJMMnAMPN5Ps7hnPx+Z77z/vjtzJNZtVmWZQkAAPg0P08PAAAAKh7BBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADVPf0ABUpM/MXFRWZ8TUDgYF2ZWRke3qMSmPaeiXz1sx6fRvrdT8/P5saNAi46H6fDn5RkWVM8CUZtVbJvPVK5q2Z9fo21lu5eEofAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAADbLsixPDwEAgGny8guVdSbXbcfz87MpMNB+0f3V3XZLVVBs/FqlZbrvLxMAAHdZPfMOZVXi7fGUPgAABiD4AAAYgOADAGAAgg8AgAEIPgAABiD4AAAYgOADAGAAgg8AgAEIPgAABiD4AAAYgOADAGAAgg8AgAEIPgAABiD4AAAYgOADAGAAgg8AgAEIPgAABiD4AAAYgOADAGAAgg8AgAGqbPCLioqUmJiorl27yuFwaPjw4frhhx88PRYAAF6pygb/1VdfVXJysuLj47V06VJVq1ZNsbGxys/P9/RoAAB4nSoZ/IKCAv39739XXFycunfvrjZt2mj27NlKT0/XJ5984unxAADwOlUy+Pv371dOTo46derk3Ga32xUWFqadO3d6cDIAALxTlQx+amqqJKlx48bFtjdq1EjHjx/3xEgAAHi1Khn83NxcSVLNmjWLba9Zs6YKCgo8MRIAAF6tuqcHKEnt2rUl/fpa/m+jX1BQIH9/f0+NBQCAWwUH16m026qSwW/SpIkkKS0tTXa73bk9LS1NrVq18tRYAAC41cmTWW47lp+fTYGB9ovvd9stuVGbNm1kt9u1fft257bs7Gx98803ioqK8uBkAAB4pyr5CL9mzZq67777NHv2bAUFBalZs2aaOXOmGjdurL59+3p6PAAAvE6VDL4kPfroozp37pz+9re/KTc3V9dff70WLlx4wRv5AABA2aps8KtVq6YnnnhCTzzxhKdHAQDA61XJ1/ABAIB7EXwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAAD2CzLsjw9BAAApsnLL1TWmVy3Hc/Pz6bAQPtF91d32y1VQRkZ2SoqMuP3meDgOjp5MsvTY1Qa09Yrmbdm1uvbWG/l4yl9AAAMQPABADAAwQcAwAAEHwAAAxB8AAAMQPABADAAwQcAwAAEHwAAAxB8AAAMQPABADAAwQcAwAAEHwAAAxB8AAAMQPABADAAwQcAwAAEHwAAAxB8AAAMQPABADAAwQcAwAAEHwAAAxB8AAAMQPABADAAwQcAwAAEHwAAAxB8AAAMQPABADAAwQcAwAAEHwAAAxB8AAAMQPABADAAwQcAwAAEHwAAAxB8AAAMQPABADAAwQcAwAAEHwAAAxB8AAAMQPABADAAwQcAwAAEHwAAAxB8AAAMQPABADAAwQcAwAAEHwAAAxB8AAAMQPABADBA9T9y4dTUVM2bN0+bNm1SWlqakpOT9eGHH6pt27bq379/Rc0IAADKyeVH+IcPH9Ydd9yhtWvXyuFw6OzZs5KkjIwMjRs3TmvXrq2wIQEAQPm4/Aj/5ZdfVpMmTfTWW2+pdu3a+uijjyRJ06ZNU15enhYuXKi+fftW2KAAAODSufwIf9u2bRoxYoTsdrtsNluxfXfffbe+++47tw8HAADcw+Xg+/n5XRD683Jzc+Xnx/v/AACoqlyudIcOHZSUlKSsrCznNpvNpnPnzuntt9/WDTfcUCEDAgCA8rNZlmW5csFDhw5p0KBBql69uqKiorR27Vrdcsst+u677/TTTz/pnXfeUZs2bSp6XgAAfEJefqGyzuS67Xh+fjYFBtovut/l4EvSjz/+qMTERG3dulWnTp1SnTp1FBUVpbi4OLVu3dotA7tTbPxapWW67y8TAAB3WT3zDp08mVX2BV1UVvD/0Ofwr7rqKs2YMaPcQwEAgMpVavC3bNmi6667Tna7XVu2bCnzYJ07d3bbYAAAwH1KDf6wYcP0zjvvqH379ho2bJhsNpt+/wrA+W02m0379++v0GEBAMClKTX4b775pkJCQpz/DQAAvFOpwY+KinL+97vvvqtBgwYV2wYAALyDy5/D37BhgwoKCipyFgAAUEFcDv4NN9yg9evXq6ioqCLnAQAAFcDlj+W1bt1aS5Ys0YYNG9SyZUsFBQUV22+z2fTyyy+7fUAAAFB+Lgf/008/VXBwsCTpyJEjOnLkSLH9F/uefQAA4HkuB3/Dhg0VOQcAAKhAf+ib9iQpMzNTe/bs0ZkzZ9SwYUM5HA7VrVu3ImYDAABu8oeCP2/ePCUlJSk/P///D1C9ukaOHKlHH33U7cMBAAD3cDn4y5YtU2Jiou666y7dddddCgoKUlpamj744APNnz9fTZs21d13312RswIAgEvkcvDfeOMN3XvvvXr++eed21q0aKFOnTqpRo0aWrJkCcEHAKCKcvlz+D/++KN69+5d4r5evXrp8OHDbhsKAAC4l8vBb9y4sY4ePVrivmPHjqlOnTpuGwoAALiXy8Hv1auX5s6dq507dxbbvnPnTs2dO1c9e/Z0+3AAAMA9XH4NPy4uTps2bVJMTIwaN26soKAgpaenKzU1VS1bttTYsWMrck4AAFAOLgffbrdr+fLlWrFihXbs2KHTp0/ryiuvVIcOHTRgwADVrl27IucEAADl8Ic+h1+rVi0NGTJEQ4YMqah5AABABXA5+BMnTrzoPj8/P/n7+6t58+bq16+f6tev75bhzktKStLGjRuVnJzs1uMCAGAKl4N/4sQJ7d69W3l5ebriiisUFBSkjIwMHTt2TNWqVVNgYKB+/vlnzZ8/X8nJyWrWrJlbBnz77bc1e/ZsRUZGuuV4AACYyOV36ffu3VsBAQFKTk7W+vXrtXTpUq1bt04rVqxQo0aNNHr0aG3atElXXHGFZs2aVe7BUlNT9dBDD2nGjBlq3rx5uY8HAIDJXA7+okWLNGbMmAseabdt21ajR4/W/PnzVa9ePd1///3aunVruQfbt2+fAgIClJKSIofDUe7jAQBgMpef0s/MzFSDBg1K3Fe3bl1lZGRIkho0aKCcnJxyD9azZ08+2w8AgJu4HPywsDAtXrxYXbt2Va1atZzb8/PztXjxYl177bWSpG+++UZNmjRx/6QAAPiY4ODK+5Zal4P/xBNPaNiwYerVq5e6du2qhg0bKiMjQ//617+Uk5OjhQsXavv27UpISNAjjzxSkTMDAOATTp7Mctux/PxsCgy0X3S/y8GPjIzUypUr9dprr2nz5s36+eefdfnll6t79+566KGHdNVVV2nTpk0aM2aMhg8f7pbhAQCAe/yhL95p0aKFpk2bdtH90dHRio6OLvdQAADAvf5Q8IuKirRmzRp9+eWXSktL0zPPPKM9e/YoPDxcLVq0qKgZAQBAObkc/OzsbI0YMUK7d+9WcHCw0tPT9csvv2jVqlWKj4/XkiVL1KZNm4qcFQAAXCKXP4c/Y8YMff/990pOTtZnn30my7IkSQkJCbryyiuVkJBQYUNOnTqVr9UFAKAcXA7+2rVr9dhjjykyMlI2m825vUGDBhoxYoT27NlTIQMCAIDyczn4v/zyiy6//PIS9wUEBCg3N9dtQwEAAPdyOfghISF6//33S9y3fv16hYaGum0oAADgXi6/ae/hhx/WI488omHDhql3796y2WzasmWLli5dquXLl2vOnDkVOScAACgHm3X+3Xcu+PjjjzV9+nQdP37cuS0oKEhjx47VgAEDKmTA8oiNX6u0TF5qAABUPatn3lE1v2lPkvr166d+/frp8OHDyszMVN26ddWiRQv5+bn8ygAAAPAAl0s9dOhQffvtt5Kk5s2bq3379mrVqpX8/Py0b98+3X777RU2JAAAKJ9SH+G///77KiwslCRt375dH374oa688soLLrdt2zYdO3asYiYEAADlVmrwDxw4oMWLF0uSbDabkpKSLnrZkSNHuncyAADgNqUG/7HHHtN9990ny7LUu3dvzZkzR+3atSt2mWrVqslut8tuv/gbBQAAgGeVGvyaNWvqiiuukCS9+eabCgsLI+wAAHghl9+lHxUVpdzcXH311VcqKChwfpe+ZVnKycnRrl27NGHChAobFAAAXDqXg79lyxaNGTNGZ86cKXG/3W4n+AAAVFEuB3/OnDmqX7++XnjhBX344Yey2WwaMGCAPv/8c7377rt6/fXXK3JOAABQDi4H/8CBA3ruuefUt29f5eTkKDk5Wd27d1f37t2Vl5enefPmacGCBRU5KwAAuEQuf/FOUVGRGjVqJOnXL945ePCgc9/NN9+s/fv3u386AADgFi4H/6qrrtKBAwckSddcc41yc3N1+PBhSVJhYaF++eWXipkQAACUm8vBv/322zV79mwtXrxY9erVU0REhJ577jn985//1Lx589SqVauKnBMAAJSDy6/hP/jggzp16pS+/vprSdKzzz6r2NhYjR49WnXq1NG8efMqbEgAAFA+LgXfsixlZWVp/Pjxzm1t27bViy++qODgYLVo0YIv5AEAoAqzWee/Qecidu/erfHjx6t///7661//6tyenp6uG2+8UU2bNtXcuXMVFhZW4cMCAOAr8vILlXUm123H8/OzKTDw4g++S32Ef/jwYcXGxio4OFgOh6PYvrp162rq1Kl69dVXFRMTow8++EDNmjVzz9RukpGRraKiUn+f8RnBwXV08mSWp8eoNKatVzJvzazXt7Heylfqm/Zef/11NWnSRCtWrFCPHj2K7atZs6buvPNOLVu2TPXq1Sv1X9IDAACeVWrwt2/frgceeKDU1+fr16+v+++/X9u3b3f7cAAAwD1KDX56erpLT9O3bt1aJ06ccNtQAADAvUoNfmBgoFJTU8s8SHp6uho0aOC2oQAAgHuVGvyOHTtq5cqVZR7kgw8+0LXXXuu2oQAAgHuVGvz77rtPX331leLj45Wfn3/B/oKCAk2ZMkWbN2/WfffdV2FDAgCA8in1Y3lt27bVM888oxdeeEEfffSROnfurGbNmuncuXP66aeftG3bNp06dUpjxoxRdHR0Zc0MAAD+oDK/aW/QoEFq06aNFi5cqA0bNigvL0+SFBAQoBtvvFHDhw+/4DP6AACganHpq3UjIiI0d+5cSdLPP/+s6tWrq27duhU6GAAAcB+X//Gc8xo2bFgRcwAAgArk8j+PCwAAvBfBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAAD2CzLsjw9BAAApsnLL1TWmVy3Hc/Pz6bAQPtF91d32y1VQbHxa5WW6b6/TAAA3GX1zDuUVYm3x1P6AAAYgOADAGAAgg8AgAEIPgAABiD4AAAYgOADAGAAgg8AgAEIPgAABiD4AAAYgOADAGAAgg8AgAEIPgAABiD4AAAYgOADAGAAgg8AgAEIPgAABiD4AAAYgOADAGAAgg8AgAGqbPCzs7M1ZcoU9ezZU5GRkRowYIDWr1/v6bEAAPBKVTb4EydO1MaNGxUfH6/3339fffv2VVxcnLZs2eLp0QAA8DpVMvgnT57U2rVr9dRTT6lLly66+uqr9dBDDykqKkrLly/39HgAAHid6p4eoCSXXXaZFixYoPbt2xfbbrPZdPr0aQ9NBQCA96qSj/Dtdru6desmu93u3LZnzx5t3bpVPXr08NxgAAB4qSoZ/N87dOiQ4uLi5HA4NHDgQE+PAwCA16mST+n/1o4dOxQXF6emTZsqKSlJNWrU8PRIAAC4RXBwnUq7rSod/JSUFD311FOKiopSYmJisaf4AQDwdidPZrntWH5+NgUGXryTVfYp/dWrV2v8+PG69dZblZSUROwBACiHKvkI/8SJE3r22WfVsWNHjRs3TqdOnXLuq1GjhurXr+/B6QAA8D5VMvhr165Vbm6utm7dqq5duxbb1759eyUnJ3toMgAAvFOVDP7QoUM1dOhQT48BAIDPqLKv4QMAAPch+AAAGIDgAwBgAIIPAIABCD4AAAYg+AAAGIDgAwBgAIIPAIABCD4AAAYg+AAAGIDgAwBgAIIPAIABCD4AAAYg+AAAGIDgAwBgAIIPAIABCD4AAAYg+AAAGIDgAwBgAIIPAIABCD4AAAawWZZleXoIAABMk5dfqKwzuW47np+fTYGB9ovur+62W6qCMjKyVVRkxu8zwcF1dPJklqfHqDSmrVcyb82s17ex3srHU/oAABiA4AMAYACCDwCAAQg+AAAGIPgAABiA4AMAYACCDwCAAQg+AAAGIPgAABiA4AMAYACCDwCAAQg+AAAGIPgAABiA4AMAYACCDwCAAQg+AAAGIPgAABiA4AMAYACCDwCAAQg+AAAGIPgAABiA4AMAYACCDwCAAQg+AAAGqO7pASqSn5/N0yNUKtbr+0xbM+v1bay3co9vsyzLqtAJAACAx/GUPgAABiD4AAAYgOADAGAAgg8AgAEIPgAABiD4AAAYgOADAGAAgg8AgAEIPgAABvC54BcVFSkxMVFdu3aVw+HQ8OHD9cMPP3h6LLf5/vvvFRoaesHPsmXLJEn79+9XTEyMIiIi1KNHDy1atMjDE1+apKQkDR48uNi2Y8eOadSoUWrfvr26dOmi6dOnq7CwsNhl3n77bfXq1Uvh4eEaOHCg9u7dW5ljX7KS1jtz5swSz/Vv1+xN683OztaUKVPUs2dPRUZGasCAAVq/fr1zv6+d37LW62vnV5JSU1M1duxYdezYUZGRkRo5cqS+/fZb5/6y7p+87f67rPWOHTv2gvPbrVs35/5KX6/lYxITE61OnTpZGzdutPbv3289+OCDVq9evay8vDxPj+YWH3/8sdW+fXsrLS2t2E9ubq6VkZFhRUVFWU8//bT13XffWStXrrTCw8OtpUuXenrsP2TJkiVWaGioNWjQIOe2/Px8q2/fvtaoUaOsAwcOWOvWrbOioqKsWbNmOS+zYsUKy+FwWCkpKda3335rTZgwwerQoYOVnp7uiWW4rKT1WpZljRgxwnrmmWcuONfnedt64+LirD59+libNm2yjhw5Ys2fP99q06aNtXnzZp88v6Wt17J87/wWFRVZt99+uzV48GDrP//5j/Xdd99Zo0ePtrp06WJlZ2e7dP/kTfffZa3XsiyrX79+1ty5c4ud34yMDOcxKnu9PhX8/Px8KyIiwlqyZIlzW1ZWluVwOKxVq1Z5cDL3SUhIsO65554S982fP9+Kjo62zp4969w2e/Zsq1evXpU1XrmcOHHCGjVqlBUREWHdcsstxQKYkpJitW3b1jp16pRz23vvvWdFRERYubm5lmVZVt++fa2XX37Zub+wsNC66aabrLlz51beIv6A0tZrWZbVvXt365133rno9b1pvWlpaVZISIj12WefFds+dOhQa+zYsT53fstar2X51vm1rF/XPGbMGOv77793btu/f78VEhJiffXVV2XeP3nb/XdZ683Pz7fCwsKszz//vMTre2K9PvWU/v79+5WTk6NOnTo5t9ntdoWFhWnnzp0enMx9Dhw4oJYtW5a4b+fOnbrhhhtUvfr//yOIHTt21NGjR5WamlpZI16yffv2KSAgQCkpKXI4HMX27dy5U9dee63q1avn3NaxY0fl5ORo3759Sk9P15EjR9SxY0fn/mrVqun666+vsue+tPWeOXNGx48fV6tWrUq8rret97LLLtOCBQt0ww03FNtus9l0+vRpnzu/Za3X186vJAUHB2v27Nlq3ry5pF/XsGjRIjVq1EghISFl3j952/13Wes9dOiQCgsLL3qOPbFenwr++ag1bty42PZGjRrp+PHjnhjJ7Q4ePKi0tDQNGjRIXbp00ZAhQ/Tll19K+nX9l19+ebHLN2rUSJK8Yv09e/bUzJkzdeWVV16wr7S1nThxwnnuS7pMVV17aes9ePCgJGn16tXq27evbrrpJk2YMEFpaWmS5HXrtdvt6tatm+x2u3Pbnj17tHXrVvXo0cPnzm9Z6/W18/t7Tz75pKKjo7VmzRq99NJLCggIKPP+yZvvv0ta74EDB1S9enUlJSWpZ8+e6tOnj+Lj45WVlSXJM73yqeDn5uZKkmrWrFlse82aNVVQUOCJkdwqJydHx44dU1ZWlh577DG9/vrrateunR588EFt3rxZeXl5Ja5dkvLz8z0xstuUtTZfO/fng2C325WYmKjnn39e3377rWJiYpSbm+v16z106JDi4uLkcDg0cOBAnz+/v1+vr5/f2NhYLV++XP3799cjjzyir7/+2qfPcUnrPf/mvWbNmum1117T+PHjtXHjRj388MMqKiryyHqrl30R71G7dm1JUkFBQbG/xIKCAvn7+3tqLLfx9/fXrl27VKNGDef62rVrp0OHDmnhwoWqXbv2Bf+jnP+zt6+/rLX99tz//jLeuPbBgwfrtttucz7F3aZNG4WEhKh79+5at26d82lEb1zvjh07FBcXp6ZNmyopKUk1atTw6fNb0np9+fxKUuvWrSVJL774ov7973/rrbfe+kPn2Nvuv0ta70svvaRRo0apbt26kqSQkBAFBQVp0KBB2rNnj0fW61OP8Js0aSJJzqfFzktLS7vgaRNvFRAQcMFvhCEhIfrf//6nyy+/vMS1Sxc+Nehtylpb06ZNi2377WW88dzbbLZir2dLvz71V79+fR0/ftxr15uSkqJhw4apbdu2euutt1S/fn1Jvnt+L7ZeXzy/aWlpWr16tSzLcm7z8/NTq1atnE/nl3aOve3+u6z1+vn5OWN/XmhoqKRfX8LwxHp9Kvht2rSR3W7X9u3bnduys7P1zTffKCoqyoOTucfu3bsVGRl5wWdxv/76a7Vu3VodOnTQrl27in2Od+vWrbrmmmsUHBxc2eO6VYcOHbR//36dOXPGuW3btm0KCAhQWFiYGjZsqObNmxc79+fOndOuXbu88tzHx8frzjvvLLbt6NGjyszMVKtWrbxyvatXr9b48eN16623Kikpqdjr2754fktbry+e3+PHj+uJJ57Qrl27nNvOnj2rb775Ri1btizz/snb7r/LWu8jjzyihx9+uNh1zt93t2rVyiPrrTZ58uTJFXJkD6hWrZpycnK0cOFCNW/eXAUFBZo0aZLOnTunZ599VtWqVfP0iOUSFBSkTz75RBs3blRISIhyc3M1b948rVu3TtOmTVOHDh30j3/8Q4cOHVLz5s31xRdfKCEhQY8//riuvfZaT4//h6xbt06nT5/W3XffLUm6+uqrlZKSoq1btyo0NFR79+5VfHy8hg4dqi5dukj69bWvV155RY0bN1aNGjU0c+ZMHTx4UFOmTNFll13myeWU6ffr9ff316JFi5SZmamrrrpKBw8e1MSJE9W0aVONGzdONpvNq9Z74sQJxcbG6vrrr9fkyZOVn5+vnJwc5eTk6OzZswoNDfWp81vWehs2bOhT51f69c1mO3fu1EcffaTQ0FBlZ2dr6tSpOnDggKZOnSqHw1Hq/ZO33X+Xtd66devqtddeU61atRQcHKzdu3frb3/7m7p06aKYmBjPrLdCPuznQYWFhdb06dOtzp07WxEREVZsbKz1448/enostzlx4oQ1btw4q0uXLla7du2sgQMHWtu2bXPu37t3rzVw4ECrXbt21k033WS98cYbHpz20k2YMOGCz6UfOXLEGj58uHXddddZ0dHR1qxZs6xz584Vu8yiRYusbt26WeHh4dbgwYOtr7/+ujLHvmQlrfeLL76w7r33XisiIsKKioqyJk6caP3888/FLuMt633jjTeskJCQEn/Or9uXzq8r6/Wl83veqVOnrGeeecaKjo62wsPDreHDh1v//e9/nfvLun/ytvvvstabkpJi/elPf7LCw8Ot6Ohoa8qUKc7vlbCsyl+vzbJ+8wIEAADwST71Gj4AACgZwQcAwAAEHwAAAxB8AAAMQPABADAAwQcAwAAEHwAAAxB8AAAMQPABADAAwQcAwAAEHwAAAxB8AAAMQPABADAAwQcAwAAEHwAAAxB8AAAMQPABHxcTE6OwsDDt2bOnxP2DBw9WTExMhc9x7NgxhYaGatmyZRV+WwAuRPABA5w7d04TJkxQXl6ep0cB4CEEHzBAnTp1dOTIEc2aNcvTowDwEIIPGKB169a655579Oabb2rHjh0XvVxMTIwGDx5cbNsPP/yg0NBQrVy5UpK0bds2hYaGasuWLbr//vvlcDjUo0cPLV26VBkZGRo7dqwiIyN14403asaMGbIsq9jxTp48qYcffljh4eHq1q2bEhISVFhYWOwyGzZs0N13363w8HB17txZkyZNUlZWlnP/ypUrFRYWplWrVunGG29UVFSU9u7dW96/JsCnEXzAEE8++aSaNm2qiRMnKicnp9zHe+yxxxQdHa1XXnlFV199tSZPnqyYmBg1bvu2lMkAAANFSURBVNxYCQkJ6tq1qxYsWKCPP/642PVeffVVBQQEKDExUXfddZeSkpI0e/Zs5/6PP/5Yf/nLX9S0aVMlJCRo9OjRWrNmjUaMGFHsF4Nz585pzpw5mjRpksaNG6ewsLByrwnwZdU9PQCAymG32/Xiiy9q2LBhmj59uiZNmlSu491xxx0aOXKkpF9fMhg0aJBCQkI0YcIESVLXrl21Zs0affXVV7rtttuc1+vYsaNmzJghSerRo4dycnK0ZMkSxcbGqkGDBpo2bZqioqKUmJjovE5oaKiGDBmiNWvWqH///s7tI0eOVJ8+fcq1DsAUPMIHDNK5c2cNHjxYycnJ2rJlS7mOdf311zv/Ozg4WJIUGRnp3Obn56f69evr9OnTxa732/hL0s0336y8vDzt3r1bhw8f1vHjx9WnTx8VFhY6fxwOh4KDg7Vp06Zi1w0JCSnXGgCTEHzAMOPGjVOzZs301FNPKTs7+5KPExAQcME2f3//Yn+22WwXXCYoKKjYnxs2bChJOnPmjDIzMyVJ8fHxatu2bbGfkydPKjU1tdh1z/+iAaBsPKUPGMbf318vvfSSYmJi9NJLL12wv6ioqNifL/ZLQUkxd8XvH/Gnp6dL+jX8devWlSQ9/vjj6ty58wXXLemXDACu4RE+YKAOHTooJiZGy5cv18GDB53b7Xa7Tpw4Ueyd9du3b3frbW/YsKHYnz/88EPVqlVLERERatGihYKCgnT06FFdd911zp9mzZpp+vTpF/3yIABl4xE+YKjHH39cX3zxhY4cOeLc1rNnT23YsEGTJ0/WzTffrH379untt9++5EfzJdm4caOmTJmibt266csvv9R7772nuLg41atXT5I0duxYPf3005KkXr16KScnRwsWLNAPP/xQ7jcaAiYj+IChateuralTp2rIkCHObQMGDNCxY8e0YsUKrVq1Sg6HQ0lJSbrrrrvcdrtPPfWUPv30UyUnJys4OFhPPvmkHnjgAef+P//5z7Lb7VqwYIHef/99+fv7y+FwKD4+Xi1btnTbHIBpbNbvvxUDAAD4HF7DBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAAQfAAADEHwAAAxA8AEAMADBBwDAAP8HoCKhGI7NVuQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure();\n",
    "df_test_under['categorie'].value_counts().plot(kind='barh', figsize=(8, 6))\n",
    "plt.xlabel(\"Number\", labelpad=14)\n",
    "plt.ylabel(\"Categorie\", labelpad=14)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df_test_under.drop('categorie', axis=1)\n",
    "Y=df_test_under['categorie']\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Random Forest <a id=\"9.1\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model accuracy score : 0.8559\n"
     ]
    }
   ],
   "source": [
    "rfc = RandomForestClassifier(random_state=1, n_estimators=100, max_depth= 20)\n",
    "rfc.fit(X_train, Y_train)\n",
    "y_pred = rfc.predict(X_test)\n",
    "print('Model accuracy score : {0:0.4f}'. format(accuracy_score(Y_test, y_pred)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjYAAAGiCAYAAAD0qYz9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de3zO9f/H8edlB7PN5RDbKqchOXxRwyYlNaovRa0DOZRGmcMW+pqQU6i+QuQQ+zmuvpKUHColKRKWJaWZQ6ScttE3LrPNZdv1+8O31dVk5LJre1+Pu9t1u7X3+3N93q/PjVteXq/35/OxOBwOhwAAAAxQxt0BAAAAuAqJDQAAMAaJDQAAMAaJDQAAMAaJDQAAMAaJDQAAMIa3OxfPXjjUncvDMO1HbXd3CDDMpoxUd4cAw+TajxTreudOHHDZuXyq1HbZua4mKjYAAMAYbq3YAACAqyg/z90RFDsSGwAATOXId3cExY5WFAAAMAYVGwAATJXveRUbEhsAAAzloBUFAABQelGxAQDAVLSiAACAMWhFAQAAlF5UbAAAMBUP6AMAAMagFQUAAOA6CQkJ6tq1q9NYZmamxowZo5YtW6pZs2bq27evDh065HTMmjVr1KFDBzVu3FgdO3bUxo0bL2k9EhsAAEyVn++6z9+wePFiTZ06tdB4XFycNm/erBkzZmjx4sU6ffq0+vbtq7y8862zLVu2KD4+Xl27dtWKFSvUpk0b9e/fX3v37i1yTRIbAAAM5XDku+xzOdLT09W3b19NnjxZoaGhTnNJSUnavHmzpk+frhYtWqh+/foaN26cMjMz9eOPP0qS5s6dq7Zt2+qxxx5TnTp1NGTIEDVp0kSLFi0qcm0SGwAA4FIpKSkKCAjQqlWr1LRpU6e5L774QnXr1lWDBg0KxurUqaMNGzaobt26ys/P1zfffKOIiAin74WHhys5ObnItdk8DACAqVz4gD6bzSabzVZo3Gq1ymq1Oo1FRkYqMjLyguc5ePCgatSooWXLlun111/Xr7/+qmbNmmnEiBEKDg6WzWZTVlaWQkJCnL4XFBSkY8eOFRkniQ0AAKZy4V1RiYmJmjlzZqHx2NhYxcXFXfJ5MjMzlZKSolOnTmn06NGyWCyaPHmyHn/8ca1cuVI5OTmSJF9fX6fv+fr6ym63y+FwyGKx/OX5SWwAAECRevbsqaioqELjf67WFMXHx0fZ2dmaNWuWKlWqJEmaMWOGWrdurXXr1unWW2+VJNntdqfv2e12+fv7XzSpkUhsAAAwlwsf0HehltPfERISoqCgoIKkRpKqVq2qSpUq6fDhw6pYsaL8/f2VkZHh9L2MjAwFBwcXeX42DwMAYCpHvus+LtK8eXOlpaU5JS7Hjx/Xr7/+qho1ashisSgsLExfffWV0/eSkpIUHh5e5PlJbAAAQLFp3769QkND9fTTT2vnzp3atWuXnnnmGdWsWVNt27aVJEVHR+ujjz7S/PnztX//fk2ZMkUpKSnq2bNnkecnsQEAwFRufkDfhfj6+ioxMVHVqlVTdHS0unfvrgoVKmjRokUqW7asJOm2227TSy+9pKVLl+qBBx7Qpk2bNGfOHNWpU6fI81scDofDZdFepuyFQ921NAzUftR2d4cAw2zKSHV3CDBMrv1Isa539vtPXHausv+4y2Xnupqo2AAAAGNwVxQAAKZyYQuptCCxAQDAUA6H6273Li1oRQEAAGNQsQEAwFQufP5MaUFiAwCAqdhjAwAAjOGBFRv22AAAAGNQsQEAwFQufAlmaUFiAwCAqWhFAQAAlF5UbAAAMBV3RQEAAGPQigIAACi9qNgAAGAqWlEAAMAYHpjY0IoCAADGoGIDAIChHA4e0AcAAExBKwoAAKD0omIDAICpPPA5NiQ2AACYilYUAABA6UXFBgAAU9GKAgAAxqAVBQAAUHpRsQEAwFS0ogAAgDFoRQEAAJReVGwAADCVB1ZsSGwAADAVe2xQ3D7fd0wjVn+tzc/cVzB2Ouecpm/Ypc/2HlPWuVyF16iqf7X9h6pXCig45pPdRxW/Yluh8w27q7EebVa7WGJHyVamTBk91DtK93XroKDrg5R+OEMrX1+l9xatlCT5+vnqiWceV2SnOxVoDdTenXv12rgE/ZDyg5sjR2nSu1c3DflXf1WrFqJvv92lIfHPa2vS1+4OCx6MxMaNdhz+r55bvV0Oh/P48NVfa0/6SQ26s5EqlPPV3C/36Mk3N+ndJyMVWNZHkrQv45SqVwrQC/eFOX33+or+xRU+SrjHB/VQt/6P6vVX/6Nd21PVJKKxYsf2V9lyZfXW7LcVO6af7nqwnRJenKsjB4+qS8wjmrp0knrd9ZSOHzvh7vBRCvTo8bBem/VvTXhhqpKTv9WA/tH68IPFCmt+lw4ePOTu8CB5ZCuKzcNuYM/N08Kt+/TUki/lVcbiNLf/hE2b9qdr2F1NdN8/qqt1nWD9+/7mSj+dow370gqO23vcpoYhFdXk+spOn2sC/Ir7clACWSwWPfLUQ3prztv6z4w3tf3Lb7Tolde16j/vq0vMI7JYLLrrwXZ6e+47WpG4Sts2JGvUU2NV1q+sIjvd6e7wUUqMHT1Ec+ct1vgJU7Xmo/V64MFonTjxXw18+il3h4bfOPJd9/mbEhIS1LVr17+cnz17tm688Ubl5uY6ja9Zs0YdOnRQ48aN1bFjR23cuPGS1iOxcYNNBzK0YMs+Db6zYaG2UbWKAXrj8da6rU5wwZiP1/nfJnve73+w9mXYdENVa/EEjFInwBqgte9+oo1rNjmNHzpwSJWqVJJ/oL+8fbyVdTqrYC4nK0d2+zmVr1i+uMNFKVS3bqhq1aqu999fWzCWm5urD9d8qnvuITnGeYsXL9bUqVP/cv67777TzJkzC41v2bJF8fHx6tq1q1asWKE2bdqof//+2rt3b5FrFtmKstvt+uijj5ScnKxjx47p7Nmz8vf3V0hIiMLDw3X33XfL25uO1uVodG1FfdDvLln9fDT7i91Oc2W9vdT4usqSpNz8fP3030y9sj5F1wSU1Z31rpUkZdlzdfRUlnann1KnhHU6eipLodeU19N3NFTrPyRE8FyZpzL16sjC/7No1e4WZRzN0JnTZ7R68QeKin5AO7Z+pyMHj6hHXDeV9fPVxg+/cEPEKG3q3XD+H2U/7D/oNP7jjz+rTu2aKlOmjPI9sA1S4rjp9yA9PV1jxoxRUlKSQkNDL3hMVlaW4uPj1bx5c23dutVpbu7cuWrbtq0ee+wxSdKQIUO0fft2LVq0SC+++OJF175oxebnn3/WvffeqzFjxmj//v0KDAzUtddeq3Llymnfvn167rnn1LFjRx0+fPhyrtfjBZcvJ6ufT5HHjVuzQw/N+0xbfszQwDsaqmI5X0nS3gybHJKOnjqjIW3/oVcfjtB1Ffw18J2t2vbT8ascPUqre7u2V/Pbm2nJ7LclSYmvvK6TJ04q4YNZej9lhTr3eVgvx0/R3p373BwpSoPy1kBJ0unTmU7jp09nysvLSwEB7PcrEdzUikpJSVFAQIBWrVqlpk2bXvCYF154QfXq1dP999/vNJ6fn69vvvlGERERTuPh4eFKTk4ucu2Lllqef/55hYaG6r333lNgYGCh+czMTA0ePFjjx49XQkJCkYvh8jxyc6g6/qO61u9N0+gPvlFevkNRTWuqdpXymvFIS91crXLBZuKWtYLUecFnmrt5r1rUrOrmyFHStIuK1DMvDdLn72/QewtXqKxfWc1cOV0+vj56YeC/deLYCd3eobWGTvqXsk5n6cu1m90dMko4i+X8/kDHn+5++G2cao15bDabbDZboXGr1Sqr1XlrRGRkpCIjI//yXJ988ok2bNig1atX67PPPiu0TlZWlkJCQpzGg4KCdOzYsSLjvGhi8/XXX+vtt9++YFIjSYGBgfrXv/6lbt26FbkQLl/j6ypJklrUrKqM09mav2WvoprWlNXPp1DLyauMRS1rVdUHKVTP4OzhJx9S/9Ex2rx2iybEvSRJur3Dbapeu5pi7h2gPd/ukSR9s3mHKlSy6unxsSQ2KJLt1GlJUvnygcrI+P0uusDAAOXl5enMmay/+iqKkwsTzMTExAvuh4mNjVVcXNwlnyc9PV2jRo3Syy+/rEqVKhWaz8nJkST5+vo6jfv6+sput8vhcBQk0Bdy0cTGarUqPT1d9erV+8tjjhw5In9/So6ucvjkGSX/dEL3N6nh9BtXP7iCvtifLknanXZSu9JP6cGmNZ2+ezY3v6BdBUjSk8/2Uo+4bvpo2VpNGjJZef/bgF712iDl5eYVJDW/2bnte0Xef6fK+fspOyvHHSGjlNj3w4+SpNqhNbT/D/tsQkNraM/e/W6KCoW4MLHp2bOnoqKiCo3/uVpzMQ6HQ8OGDVP79u11++23X/CYsmXLSjq/x/eP7Ha7/P39L5rUSEUkNg8//LCGDRumuLg4hYeHKyQkpCBjysjI0FdffaVXXnlFnTt3vuSLwsX99N9MjV2zQ9dX9C9oKTkcDm09eFx1/3cX1O6MUxq3ZocaBldQ/ZCKkqScc3nadCBdt9YOclvsKFke6h2lHnHd9M68dzVz7GynucM/HpaXt5cahjXQru2pBeMNbq6vX0/8SlKDIu3bd0A//3xEnTr9U5+sO38brre3tzq0b6sP13zq5uhwNVyo5XS5jh49qs2bN2v79u1asWKFJBXc5t2iRQvFxMQoJiZG/v7+ysjIcPpuRkaGgoOLvkHmoolNXFycLBaLXn75ZWVnZxeaDwgIUPfu3TVw4MBLvihcXEStqmpyXSWN/uAbxd7eQBX9ffXetz9rx+H/ambnlpKku+tfrwVb9il+RbJi2zRQWe8ySkz6QVn2XD3V6kY3XwFKgspBlRUz/CntTz2g9as+V8OwBk7zX67drH3f/6Axs0dp/ssL9Uv6L7rlrpa6+6G79OrIGW6KGqXNy5NmafqrE3Ty5Clt3rxN/fs9oSpVKuvV6XPdHRp+8+cnwLpZcHCw1q5d6zS2du1aTZ48We+++64qV64si8WisLAwffXVV3r00UcLjktKSlJ4eHiRa1w0sbFYLIqLi1NMTIx2796t9PR0ZWdny8/PTyEhIapfv36hHhiujHeZMpr+SEtN/3yXpn2+S7YcuxqEVNScR28pqOD4+3rr/7reqmmfpWjiJzuVdS5XN1errAXdb1OItZybrwAlQXib5vL181WdBrX12qrCicr9jR/Uvx4dqpiRT6n/6BiV9Surn/b9rDExz2vDB9zujUszJyFR5cr5KS62twY+/ZS+/TZFHe7trh9//NndoeE3JWwTt7e3t2rWdN5Gcc0110iSatSoUfD4mOjoaPXp00eNGjXSHXfcoRUrViglJUUTJkwocg2L489b2otR9sKh7loaBmo/aru7Q4BhNmWkFn0QcBly7UeKdb3sJWNcdq5yXZ//W98bNmyYfvrpJy1ZsuSC88uXL9fw4cOVkpLi9Fy8lStXatasWTp27Jjq1q2r+Ph4tWrVqsj1SGxgDBIbuBqJDVyt2BObxaNcdq5y3ce77FxXE48MBgDAVFfwjqfSindFAQAAY1CxAQDAVCVs83BxILEBAMBUJex27+JAKwoAABiDig0AAKaiFQUAAIzhgYkNrSgAAGAMKjYAAJjKA59jQ2IDAIChHPncFQUAAFBqUbEBAMBUHrh5mMQGAABTeeAeG1pRAADAGFRsAAAwlQduHiaxAQDAVOyxAQAAxvDAxIY9NgAAwBhUbAAAMJWDPTYAAMAUtKIAAABKLyo2AACYitu9AQCAMXjyMAAAQOlFxQYAAFPRigIAAKZwcFcUAABA6UXFBgAAU9GKAgAAxuCuKAAAgNKLig0AAKaiFQUAAIzBXVEAAAClF4kNAACmyne47vM3JSQkqGvXrk5jKSkp6tWrl8LDw9WqVSsNGjRIx44dczpmzZo16tChgxo3bqyOHTtq48aNl7QeiQ0AAKZy5Lvu8zcsXrxYU6dOdRo7duyYnnjiCQUFBenNN99UQkKCfvnlF/Xq1Us5OTmSpC1btig+Pl5du3bVihUr1KZNG/Xv31979+4tck0SGwAA4FLp6enq27evJk+erNDQUKe51atXq2zZspowYYLq1q2rxo0b6+WXX9aBAwf09ddfS5Lmzp2rtm3b6rHHHlOdOnU0ZMgQNWnSRIsWLSpybRIbAABM5aZWVEpKigICArRq1So1bdrUae7uu+/WtGnT5O39+/1LFotFknTq1Cnl5+frm2++UUREhNP3wsPDlZycXOTa3BUFAICh3PWuqMjISEVGRl5wrlatWqpVq5bT2Jw5c+Tn56eWLVvKZrMpKytLISEhTscEBQUV2odzISQ2AACgSDabTTabrdC41WqV1Wr92+dduHChlixZopEjR6py5cpKS0uTJPn6+jod5+vrK7vdLofDUVDhuRASGwAATOXCB/QlJiZq5syZhcZjY2MVFxd32efLz8/XlClTNG/ePA0YMECPPfaYJKls2bKSJLvd7nS83W6Xv7//RZMaicQGAABzuTCx6dmzp6KiogqN/51qzdmzZxUfH69PPvlEo0ePVvfu3QvmKlasKH9/f2VkZDh9JyMjQ8HBwUWem8QGAAAU6UpbTr/Jz89XXFyctmzZohkzZqhdu3ZO8xaLRWFhYfrqq6/06KOPFownJSUpPDy8yPOT2AAAYKoS+HbvxMREbdiwQWPHjlXTpk11/Pjxgrny5cvLz89P0dHR6tOnjxo1aqQ77rhDK1asUEpKiiZMmFDk+bndGwAAU5WAJw//2apVqyRJY8eO1W233eb0Wb16tSTptttu00svvaSlS5fqgQce0KZNmzRnzhzVqVOnyPNbHA6H2179mb1wqLuWhoHaj9ru7hBgmE0Zqe4OAYbJtR8p1vUyn+nksnMFvrLKZee6mmhFAQBgKIcLKy2lBYkNAACm8sDEhj02AADAGFRsAAAwlZteqeBOJDYAAJiKVhQAAEDpRcUGAABTeWDFhsQGAABDufFRdW5DKwoAABiDig0AAKaiFQUAAIxBYlO84p//yZ3LwzDv3ZLr7hBgmIjPr3V3CAAuExUbAAAMxbuiAACAOTwwseGuKAAAYAwqNgAAmMrzXhVFYgMAgKk8cY8NrSgAAGAMKjYAAJjKAys2JDYAAJjKA/fY0IoCAADGoGIDAIChPHHzMIkNAACmohUFAABQelGxAQDAULSiAACAOTywFUViAwCAoRwemNiwxwYAABiDig0AAKbywIoNiQ0AAIaiFQUAAFCKUbEBAMBUHlixIbEBAMBQtKIAAABKMRIbAAAM5ch33efvSkhIUNeuXZ3GDh8+rJiYGIWFhalVq1aaNGmScnNznY5ZvHix2rZtqyZNmqhLly767rvvLmk9EhsAAAzl7sRm8eLFmjp1qtOY3W5X7969ZbFY9NZbb2n8+PF65513NGPGjIJjli9frkmTJmnQoEFavny5QkND9eSTT+qXX34pck0SGwAA4FLp6enq27evJk+erNDQUKe5jz/+WEeOHNHEiRNVr149tW3bVkOGDNHrr7+unJwcSeerPN26dVPHjh1Vt25dvfDCCwoMDNRbb71V5NokNgAAmMphcd3nMqSkpCggIECrVq1S06ZNneaSk5PVoEEDVahQoWAsIiJCWVlZSklJ0YkTJ3Tw4EFFREQUzHt5ealZs2ZKTk4ucm3uigIAwFCuvCvKZrPJZrMVGrdarbJarU5jkZGRioyMvOB50tPTFRIS4jQWFBQkSUpLS5Ofn58kXfCYnTt3FhkniQ0AAChSYmKiZs6cWWg8NjZWcXFxl3yenJwcBQQEOI35+vpKks6ePavs7GynsT8eY7fbizw/iQ0AAIZy5F9eC+lievbsqaioqELjf67WFMXPz69QgvLbz/7+/gUVmwsd4+/vX+T5SWwAADCUK1tRF2o5/R0hISFKTU11GsvIyCiYu+666wrGbrzxRqdjgoODizw/m4cBAECxadGihVJTU5326yQlJSkgIEANGzZU5cqVFRoaqq+++qpgPi8vT19//bXCw8OLPD+JDQAAhnI4LC77uEq7du0UHByswYMHa/fu3Vq/fr2mTJmi6Ojogn01vXr1UmJiopYvX64ffvhBI0eO1JkzZ/TII48UeX5aUQAAGKokviuqbNmymjdvnsaNG6fOnTvLarWqS5cuGjBgQMExnTt3VmZmpl599VWdPHlSjRo10oIFC1S5cuUiz29xOByOq3kBFxNbq4u7loaBxrdId3cIMEzE55nuDgGG2Xu86OewuNLhiAvfcv13VEta77JzXU1UbAAAMJQr74oqLUhsAAAwlPt6Mu7D5mEAAGAMKjYAABiKVhQAADCGJyY2tKIAAIAxqNgAAGAoT9w8TGIDAIChaEUBAACUYlRsAAAwlCvf8VRakNgAAGCokviuqKuNVhQAADAGFRsAAAyVTysKAACYwhP32NCKAgAAxqBiAwCAoTzxOTYkNgAAGMoTnzxMKwoAABiDig0AAIaiFQUAAIzhibd704oCAADGoGIDAIChPPE5NiQ2JYy3r7eGffiyDu7Yp/8MmV0w3qxjK90zIEpVQ0P069FftGHRR9qQ+JEbI0VJ5dO8lQKeHqmTj3dwGvd7sIfK3tVRlvIVlLvne2XNn678oz8XzFsCAlWuR1/5hLWUfH2Vm/qdshfNUn7GseK+BJQCkffcrslzxisstI0kKerR+zRxxti/PL5e1ebFFBn+yBPviiKxKWHaD3xYIXWv18Ed+wrGwu67RT1fjdP6ue9r1/M7dOOt/9Ajz0crJzNLSe9udGO0KGm86jVSQNxzksX5X2l+D/eUX1Q3Zf8nQfnH0+T30GMqP2aKTg1+Qso6I0kKGDRaXrXqKvuNOXJknpZfl2gFjnlFtn/1knKy3XA1KKlubtFEk2ePk+UPf84+/2STHvnnE07HVb6mkqbP/7dWLvuwmCOEJyOxKUGqNaqlO55or9O/2JzGHxjWXV+8sVYrXlosSdq7JUWVqwXpxtuakNjgPG8fle3wkMo92kuOszmylPnD9jm/cvLr1EXZby/S2TXLJUm5qd+pwmtLVTayg86+v0wWa0X53BSuM69NlH3jWklS3rFDqjD9P/K5KVzntm5wx1WhhPHx9VHPPl01aFhfZWVly8f39z9nv/5yUr/+ctLp+NcSJ+vIoWMaP2JycYeK//HEzcMkNiVEGa8y6v5yX637v9Vqek+LgvEajWurcrWq+nLJp07HJw6aUdwhogTzuTlCflHdlf3GHFnKW+XXsUvBnHe9hrKU89e55C8LxhxnMnVu17fyuSlcZ99fJvn6nh/Pzvr9mNPnE2xLYPliugqUdG3atlLMwCc08flXValSBUX37/GXx952Z0u163CHenWO1dmcs8UYJf7IE/fYcFdUCXFX3/vl7eOtta+95zR+Xf0akqQyXl4auHSMpu1drPGbZ6l1j7vdESZKqNz9u2Ub0PV8ReZPPXWva6tLkvLTjzqN52ccVZn/zTlOZMievFl+D/ZQmetqyGKtqHK94uTIytS57UnFcg0o+XZ+s0uRzTrpjblLi9y7MWRUnL74bIs2fba1eIID/oeKTQkQXOc63RMbpRndxyvvXJ7TXOA1VuXl5ilmXry+eGOt1kx7R03vCVeXCb115uRpbX9/i5uiRkni+O+Jv5yzlPOXw26XcnOdv5OdLYu/f8HP2QtnKHDUZFV49fXz8/azynxpuBz/PX51gkapk552aX8Wwls1U8PGN6rng/2uckQoCpuHUewsFou6TeyrLUs/04/b9xWa9/L2lpe3l75c8qnWvrZC0vk9NtdUr6r2Ax8msUHRLBYVKuNIkkVS/vlxS+UqKv/CLDlsJ5U5ebQc2WdUtu19Cowfr9MT4pW3b1exhozSrcvjUdqz6wdt+WKbu0PxeOyxuYBu3bo57Xy/mMWLF19xQJ6mzRP/VOXrq2hOr4kq4/V7Z9BisaiMVxmdzcqRJKVu2OH0vd2bdirquZvk5eNVqMoD/JEj64zk7SN5eUl5v/9ZsfiVkyMrU5JU9o72sgSWl+3ZPgXVn9zvvlb5F2apXI8YZY4Z6JbYUfp4e3upTbtbNf+1/7g7FHioIhObNm3aaNq0aapdu7aaNGlSHDF5lKb3tFCla6/RpO8WOI1Xa1hLEQ+10dujz497+Tj/Vnl5e8lisciR74F1RlyWvGOHZSlTRmWCrlX+scMF42WCrlP+0UPn/7tKVeWfyCjU0srd/b187/hnscaL0u2mFk1krVBea99f7+5QIM/cPFxkYhMTE6PAwEBNmTJFCQkJqlatWnHE5TGWjJgrv4ByTmM9p8Up48djWvPqO8r48ZgeGNFDN9/bUgeS9xQc0ygyTD99t1/5efnFHTJKmdy9KXLYz8qnxW06u+otSecfxufTsKmylyVKkvKOHpbvnR1kqVzVaU+N1w0NeEAfLkuTmxvptC1T+/f+6O5QIFpRf6l79+764osvNG3aNE2ezPMIXCnjQOG/NM6dtevMydP6eecBSdLaWe+pw6BHlHM6Wz8kpSrsvltUN6KBZkf/u7jDRWmUk62za5arXNfeksOh/GOH5PdgDzmys3R2/QeSJPtnH8rv3odV/rmJyl6WKEd2lnzb3C3vG/+hM5NGuvkCUJrUa1BHB/f/5O4w4MEuefPwuHHjlJKScjVjwV/4aMZyZZ/OUpue/1S7mI7KOHBM8/q9otQN37o7NJQS2W/OkxwO+XXqIotfOeXu+V6nZ/674KnDjjOZOj0qTuUe7yf/vvGyWKTcg/uV+fwzyt21o4izA7+7pkpl2U5lujsM/I+7NivY7XbNmDFDq1ev1qlTp9SgQQMNGTJEYWFhkqTDhw9r/Pjx2rZtm/z8/BQVFaXBgwfL2/vK72myOBzuuxkstlaXog8CLtH4FunuDgGGificv6DhWnuPJxfrepuvfchl52p17N1LPnbq1KlatmyZJk6cqOrVq2vevHn68MMPtWbNGlWqVEkdO3ZUaGionnnmGR06dEgjRozQo48+qsGDB19xnDygDwAAQzkcFpd9Lsenn36qTp06qXXr1qpVq5aGDRumM2fO6Ouvv9bHH3+sI0eOaOLEiapXr57atm2rIUOG6PXXX1dOTrce0xYAABm1SURBVM4VXzOJDQAAcKnKlSvrs88+0+HDh5WXl6dly5bJ19dXDRs2VHJysho0aKAKFSoUHB8REaGsrCyXbHnhAX0AABjKlffN2mw22Wy2QuNWq1VWq9VpbNSoURo0aJDatm0rLy8vlSlTRtOmTVOtWrWUnp6ukJAQp+ODgoIkSWlpaVccJ4kNAACGcsh1t3snJiZq5syZhcZjY2MVFxfnNLZv3z4FBgZq1qxZCg4O1rJlyzR06FC98cYbysnJUUBAgNPxvv97Ee/Zs1f+wlQSGwAAUKSePXsqKiqq0PifqzVHjx7V0KFDNW/ePLVs2VKS1LhxY/3www+aPn26/Pz8ZLfbnb7z28/+f3h/3d9FYgMAgKFc+XD6C7WcLuS7777TuXPn1LhxY6fxpk2bav369brllluUmprqNJeRkSFJhVpUfwebhwEAMFS+LC77XKrfkpM9e/Y4je/du1ehoaFq0aKFUlNTnfbrJCUlKSAgQA0bNrziayaxAQAALtOkSRM1a9ZMI0aM0NatW3Xw4EFNmzZNmzdvVp8+fdSuXTsFBwdr8ODB2r17t9avX68pU6YoOjq6YK/NlaAVBQCAoVy5efhSlSlTRrNnz9a0adM0fPhwnTx5UjfeeKMWLlxY8OThefPmady4cercubOsVqu6dOmiAQMGuGR9njwMY/DkYbgaTx6GqxX3k4c/CXbd37N3pS912bmuJlpRAADAGLSiAAAwlDtaUe5GYgMAgKFc+eTh0oJWFAAAMAYVGwAADOWJFRsSGwAADOWJe2xoRQEAAGNQsQEAwFD5nlewIbEBAMBUl/OOJ1PQigIAAMagYgMAgKHc9s4kNyKxAQDAUJ54uzetKAAAYAwqNgAAGCrf4nmbh0lsAAAwlCfusaEVBQAAjEHFBgAAQ3ni5mESGwAADOWJTx6mFQUAAIxBxQYAAEN54isVSGwAADAUd0UBAACUYlRsAAAwlCduHiaxAQDAUJ54uzetKAAAYAwqNgAAGMoTNw+T2AAAYChP3GNDKwoAABiDig0AAIbyxM3DJDYAABjKExMbWlEAAMAYVGwAADCUwwM3D5PYAABgKFpRAAAALrBixQp16NBBjRs31r333qs1a9YUzB0+fFgxMTEKCwtTq1atNGnSJOXm5rpkXSo2AAAYyl0Vm5UrV2rEiBF69tlndccdd+jDDz/UM888o6CgIDVu3Fi9e/dWaGio3nrrLR06dEgjRoyQt7e3Bg8efMVrk9gAAGAodzx52OFw6NVXX1WPHj3Us2dPSVK/fv2UnJysrVu36ujRozpy5IjefvttVahQQfXq1dOQIUP04osvql+/fvLz87ui9WlFAQAAlzlw4ICOHDmi++67z2l8/vz5GjBggJKTk9WgQQNVqFChYC4iIkJZWVlKSUm54vWp2AAAYChXvlLBZrPJZrMVGrdarbJarQU/Hzx4UJJkt9vVp08f7dy5U9WqVVO/fv0UGRmp9PR0hYSEOJ0jKChIkpSWlnbFcZLYAABgKFfusUlMTNTMmTMLjcfGxiouLq7g58zMTEnS0KFDNWDAAA0ePFhr165V//79NX/+fOXk5CggIMDpHL6+vpKks2fPXnGcJDYAAKBIPXv2VFRUVKHxP1ZrJMnHx0eSFB0drYceekiS1KBBA33//fdasGCB/Pz8ZLfbnb7z28/+/v5XHCeJDQAAhnJlxebPLae/8lubqV69ek7jN9xwgz799FPdcsstSk1NdZrLyMhw+u6VYPMwAACGcrjwc6kaNmyogIAA7dy502l87969qlGjhlq0aKHU1FSn/TpJSUkKCAhQw4YN/9Z1/hGJDQAAcBk/Pz89+eSTeu2117Rq1Sr9/PPPmj17tjZt2qRevXqpXbt2Cg4O1uDBg7V7926tX79eU6ZMUXR0dMFemytBKwoAAEO58q6oy9G/f3/5+/tr+vTpSktLU+3atTVjxgzdcsstkqR58+Zp3Lhx6ty5s6xWq7p06aIBAwa4ZG0SGwAADOXOd0U98cQTeuKJJy44V7NmTc2fP/+qrEtiAwCAodzx5GF3Y48NAAAwhlsrNnOObnLn8jBM6pZG7g4BhklJfdvdIQBXJN8Daza0ogAAMJQ799i4C60oAABgDCo2AAAYyvMaUSQ2AAAYi1YUAABAKUbFBgAAQ7nrycPuRGIDAIChPPF2b1pRAADAGFRsAAAwlOfVa0hsAAAwFndFAQAAlGJUbAAAMJQnbh4msQEAwFCel9bQigIAAAahYgMAgKE8cfMwiQ0AAIbyxD02tKIAAIAxqNgAAGAoz6vXkNgAAGAsT9xjQysKAAAYg4oNAACGcnhgM4rEBgAAQ9GKAgAAKMWo2AAAYChPfI4NiQ0AAIbyvLSGVhQAADAIFRsAAAxFKwoAABiDu6IAAABKMSo2AAAYigf0AQAAY9CKAgAAcKEff/xRN998s5YtW1YwdvjwYcXExCgsLEytWrXSpEmTlJub65L1qNgAAGAod7eizp07pyFDhigrK6tgzG63q3fv3goNDdVbb72lQ4cOacSIEfL29tbgwYOveE0qNgAAGCrfhZ+/Y8aMGQoICHAa+/jjj3XkyBFNnDhR9erVU9u2bTVkyBC9/vrrysnJ+Zsr/Y7EBgAAuNy2bdu0dOlSTZw40Wk8OTlZDRo0UIUKFQrGIiIilJWVpZSUlCtel1YUAACGyne4rhVls9lks9kKjVutVlmt1kLHDh06VCNHjtS1117rNJeenq6QkBCnsaCgIElSWlraFcdJYgMAgKFcucMmMTFRM2fOLDQeGxuruLg4p7GxY8fqpptuUseOHQsdn5OTU6g95evrK0k6e/bsFcdJYgMAAIrUs2dPRUVFFRr/c7VmxYoVSk5O1urVqy94Hj8/P9ntdqex33729/e/4jhJbAAAMJQr3xV1oZbThbz77rv65ZdfdMcddziNjxs3TosWLVKLFi2UmprqNJeRkSFJhVpUfweJDQAAhnLH7d6TJ08udHfT3XffrdjYWN13333asWOHli9fLpvNVpAoJSUlKSAgQA0bNrzi9UlsAACAywQHB19wvHLlyrr++utVpUoVTZs2TYMHD1Z8fLyOHj2qKVOmKDo6umCvzZXgdm8AAAzl7ufYXEjZsmU1b948SVLnzp01evRodenSRQMGDHDJ+anYAABgKFfusbkSe/bscfq5Zs2amj9//lVZi4oNAAAwBhUbAAAM5e53RbkDiQ0AAIZy5d6Y0oJWFAAAMAYVGwAADOVw4buiSgsSGwAADFVS7ooqTrSiAACAMajYAABgKE/cPExiAwCAobjdGwAAGIM9NgAAAKUYFRsAAAzF7d4AAMAYnrh5mFZUCda7VzelpmzS6VM/aNPGVWoZ0czdIaEUKVOmjB5+6iEt+Gye3t+7SgvWz9X9T3QqmK94TUUNn/6sVnz/rlZ8/67GJIxScLVgN0aMkuqzL7YqvN2DhcY/XPe5oh7rp7A7O6lDl95avGyl0/wp22mN+ferurNTd7X65yOKe/Z5HTpyrLjChocisSmhevR4WK/N+rfeXPKuOnfpo5MnT+nDDxarVq3q7g4NpUSPQd3V+9lorVv+qUZFj9bn72/UgLH91KXfI/L28dakpRPVvE0zzX1pvl6IfUlly5XV9BVTZa1Y3t2howT5ZucuDRv3cqG7a9as26Bnx76sWyOaafbkcbonsrVemjZHKz/8pOCYoWMn6rNNWzWoX7ReHDVEaRnH9eTA4crKyi7uy/BYDhf+Ki1oRZVQY0cP0dx5izV+wlRJ0ifrNmrX9xs18OmnNPiZ0W6ODiWdxWLRw089qKVzlunNGUskSd98uUMVK1fQIzGP6OhPx1S7fqie7T5cyRu+liTt2PytFm2Yr0cHdNH/vTDPneGjBLDb7frPspWaMfd1lfPzU35ubsGcw+HQK68t0KNR92lI7JOSpIhmN+nosXRt3vaN7u9wl3759aS+TPpa44YP0v3t20mSalW/Xvd1fUqbkpJ1952t3XJdnoa7oi4gKSlJ8fHx6tOnj958803l5eU5zZ86dUrdu3e/agF6orp1Q1WrVnW9//7agrHc3Fx9uOZT3XPPnW6MDKVFgDVAn7y7TpvWbHIaP3TgsCpVqajqdaorLzdP2zd9UzB3zn5Oe77dqxZ3NC/ucFECfbE1WfPeeFv/GvCkuj3cyWkuZfc+HUvP0MP3t3canzj2WU0cM1SSZD9rlyQFBvgXzFesYJV0vkUFXC0XTWzWr1+v6OhoHT9+XOfOndP48ePVo0cP2Wy2gmPOnTun7du3X/VAPUm9G2pLkn7Yf9Bp/Mcff1ad2jVVpgwdRFxc5qlMzRg5Sz+k7Hcav6VdS2UcPa6MIxny8vZSleBrnOZDqoewzwaSpH80qKePli1Uj0ful8VicZrbu/9HSVJeXp6eGBCvm9p0VNuox/TW8vcLjrk2JEhtbg3X/yW+pQM/HdIvv57US1NnKzDAX61vaVGs1+LJHA6Hyz6lxUX/hnzttdc0cOBALVq0SAsXLtSSJUt06NAhRUdHKysrq7hi9DjlrYGSpNOnM53GT5/OlJeXlwL+8C8g4FJ16PpPNbs9TEtnv62vPtumk7+c1LDpz6pG3eqyViyvx595TKE31pKfv5+7Q0UJEFy1iqzlAy849+vJU/LyKqO4Z8eqVXgzJbwyXm1b36IJU2ZpzboNBccNG9hXmWey1KlbH7W5r6vWbdisaS+OVEhQ1eK6DI+XL4fLPqXFRROb/fv3q0OHDgU/33TTTUpMTNThw4f19NNPF2pLwTV++9fRnzPk38bz8z3xBj5cibZRkRr00kBteH+jVixcKduvNo158nkFXRekhZ/P13vfv6t6jW/QB0s+1Nnss+4OFyVcbm6e8vLy9XCn9urT81FFNLtJI57pr9tbhWv2wsWSpPTjJ9Q95hmV8/PT1AnP6f+mvqA7bovQwOHj9e33qW6+ApjsoolNpUqVdOTIEaexOnXqaObMmUpKStJzzz1XqspTpYXt1Pn+c/k//WspMDBAeXl5OnOGahku3UNPPqhhrw7V1nVJejHu3wXj329LUY9Wj6tHq8fVpXlXjYwerXL+5XT6JPsfcHH+5c5X9W5r6bwf65YWN+unQ0d07tw5rfjgE9lOZ2rOK+N11523qVV4mCaPG666oTX1ymsL3BG2R/LEu6Iumti0a9dOY8aM0caNG51aTy1atNALL7yglStXaujQoVc9SE+z74fz/evaoTWcxkNDa2jP3v0X+gpwQb2fjVb/sX217t1PNTZmnHLPnb+zxVrJqnseuUt+/n469nOaTqT9Ikmq3aB2oX05wJ9Vr3adpPN7LP8oNzdXDodksZRRWsZxhQRVUXDVKgXzFotFNzVpqP0Hfy7WeD1ZvsPhsk9pcdHEZuDAgapXr55iYmIKbRDu1KmTXn75Ze3YseOqBuiJ9u07oJ9/PqJOnf5ZMObt7a0O7dtq/fpNF/km8LsHez+gbnFd9e685Zo4eJLy835vYfr4emvo1Hg1b/P7Qx8bhjVQvSY3aOu6re4IF6VI85v+obK+vvr4M+f/H23cvE3/qH+DvL29VLP69UpLP660jONOx+xM2aPrr2WDOq6eiz7HJiAgQDNmzNCJEydUrly5QvMdO3ZUeHi41q5de4Fv40q8PGmWpr86QSdPntLmzdvUv98TqlKlsl6dPtfdoaEUqBxUWU8Nf1IHUg/os1Wfq0FYfaf5Pd/u1Zcfb1bfUTFyOCQfH2/1H9tXP6Ts19p31rkpapQWgQEBeurxLnptwWIF+vur+c2N9dGnG5W8Y6dmTx4nSYq692698fYK9fvXaPWN7qbAAH+tWrNO3+zcpekv8Syu4lJ66iyuY3G4cZOMt+/17lq6VBg8KEZxsb1VpUplffttiuKHjtPWpK/dHVaJ1SaokbtDKDHueeQuDZ0a/5fzUY0flsPhUP+xfdWybYTy8/O1dV2S/u+FeTr131PFGGnJ9tGOOe4OoUSYNf8/WrTkXW1b957T+H+WrdSb76xSWsZx1ax+vQb07qF2bW4tmD+alq7JM+dpy7Zv5HA4dGPd2hrw5GMKD2tS3JdQYvhUqV2s6916faTLzvXlkfUuO9fVRGIDY5DYwNVIbOBqJDZXH69UAADAUKXp+TOuQmIDAIChPPGRLDybHwAAGIOKDQAAhqIVBQAAjFGanhjsKrSiAACAMajYAABgKDYPAwAAY+TL4bLP5cjMzNSLL76oyMhI3XzzzXrwwQf16aefFswfPnxYMTExCgsLU6tWrTRp0iTl5ua65Jqp2AAAAJcaPny49uzZowkTJuj666/XmjVrFBsbqwULFqhZs2bq3bu3QkND9dZbb+nQoUMaMWKEvL29NXjw4Ctem8QGAABDuaMVdfz4ca1du1YJCQlq1aqVJKlv377asmWL3nnnHZ04cUJHjhzR22+/rQoVKqhevXoaMmSIXnzxRfXr109+fn5XtD6tKAAADOWOVlS5cuU0d+5cNW/e3GncYrHo1KlTSk5OVoMGDVShQoWCuYiICGVlZSklJeWKr5mKDQAAKJLNZpPNZis0brVaZbVaC34ODAzU7bff7nTMjh07tHXrVo0cOVKbNm1SSEiI03xQUJAkKS0t7YrjJLEBAMBQrnyOTWJiombOnFloPDY2VnFxcX/5vf379ys2NlZNmzZVly5dtG7dOgUEBDgd4+vrK0k6e/bsFcdJYgMAgKHyXbjHpmfPnoqKiio0/sdqzZ9t27ZNsbGxuu6665SQkCAfHx/5+fnJbrc7Hffbz/7+/lccJ4kNAAAo0p9bTkVZtWqVRowYofDwcE2fPl2BgYGSpJCQEKWmpjodm5GRUTB3pdg8DACAoRwu/HU5Vq9eraFDh6p9+/ZKSEgoSGokqUWLFkpNTXXar5OUlKSAgAA1bNjwiq+ZxAYAAEPlOxwu+1yqtLQ0jRo1ShEREYqPj9fJkyd1/PhxHT9+XCdPnlS7du0UHByswYMHa/fu3Vq/fr2mTJmi6Ojogr02V4JWFAAAcJm1a9cqOztbW7duVevWrZ3mwsLCtGTJEs2bN0/jxo1T586dZbVa1aVLFw0YMMAl61scbnyRhLfv9e5aGgZqE9TI3SHAMB/tmOPuEGAYnyq1i3W9+kEtXHau3RnbXHauq4mKDQAAhnLlXVGlBXtsAACAMajYAABgKFc+oK+0ILEBAMBQtKIAAABKMSo2AAAYilYUAAAwhsOR7+4Qih2tKAAAYAwqNgAAGCqfVhQAADCFG18u4Da0ogAAgDGo2AAAYChaUQAAwBi0ogAAAEoxKjYAABjKE1+pQGIDAIChPPHJw7SiAACAMajYAABgKE/cPExiAwCAobjdGwAAGMMTKzbssQEAAMagYgMAgKG43RsAABiDVhQAAEApRsUGAABDcVcUAAAwBq0oAACAUoyKDQAAhuKuKAAAYAxeggkAAFCKUbEBAMBQtKIAAIAxuCsKAACgFCOxAQDAUA4X/roc+fn5mj59ulq3bq2mTZuqV69e+umnn67SVTojsQEAwFAOh8Nln8sxa9YsLVmyRBMmTNDSpUvl5eWl3r176+zZs1fpSn9HYgMAAFzGbrdrwYIFio2NVZs2bVS/fn1NnTpVJ06c0Jo1a676+iQ2AAAYyh0Vm9TUVGVlZally5YFY4GBgWrYsKGSk5OvxmU64a4oAAAM5cp7omw2m2w2W6Fxq9Uqq9Va8HN6erokKTg42Om4oKAgHTt2zIURXZhbE5tc+xF3Lg8AgNFc+ffsjBkzNHPmzELjsbGxiouLK/g5OztbkuTr6+t0nK+vr+x2u8vi+StUbAAAQJF69uypqKioQuN/rNZIkp+fn6Tze23+mNzY7Xb5+/tf3SBFYgMAAC7Bn1tOf+Xaa6+VJGVkZCgwMLBgPCMjQ3Xr1r1q8f2GzcMAAMBl6tevr8DAQH311VcFY5mZmdq1a5fCw8Ov+vpUbAAAgMv4+vqqR48emjp1qqpUqaJq1appypQpCg4O1t13333V1yexAQAALvX0008rLy9Po0ePVnZ2tpo1a6Z58+YV2lB8NVgcnviGLAAAYCT22AAAAGOQ2AAAAGOQ2AAAAGOQ2JRQ7nzlO8yXkJCgrl27ujsMlHKZmZl68cUXFRkZqZtvvlkPPvigPv30U3eHBQ9HYlNCufOV7zDb4sWLNXXqVHeHAQMMHz5cn3/+uSZMmKAVK1bo7rvvVmxsrLZs2eLu0ODBSGxKIHe/8h1mSk9PV9++fTV58mSFhoa6OxyUcsePH9fatWs1YsQItWrVSjVr1lTfvn0VHh6ud955x93hwYOR2JRA7n7lO8yUkpKigIAArVq1Sk2bNnV3OCjlypUrp7lz56p58+ZO4xaLRadOnXJTVAAP6CuR3P3Kd5gpMjJSkZGR7g4DhggMDNTtt9/uNLZjxw5t3bpVI0eOdFNUABWbEsndr3wHgMu1f/9+xcbGqmnTpurSpYu7w4EHI7Epgf74yvc/Kq5XvgPA5di2bZu6deumqlWrKiEhQT4+Pu4OCR6MxKYE+uMr3/8oIyOjUHsKANxp1apVio6OVqNGjfTGG2+oYsWK7g4JHo7EpgRy9yvfAeBSrF69WkOHDlX79u2VkJCgwMBAd4cEsHm4JHL3K98BoChpaWkaNWqUIiIiFB8fr5MnTxbM+fj4ULmB25DYlFDufOU7ABRl7dq1ys7O1tatW9W6dWunubCwMC1ZssRNkcHTWRwOh8PdQQAAALgCe2wAAIAxSGwAAIAxSGwAAIAxSGwAAIAxSGwAAIAxSGwAAIAxSGwAAIAxSGwAAIAxSGwAAIAx/h8mMU1fP9D/IwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x504 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_cm = pd.DataFrame(confusion_matrix(Y_test, y_pred), range(3), range(3))\n",
    "plt.figure(figsize=(10,7))\n",
    "sn.set(font_scale=1.4) \n",
    "sn.heatmap(df_cm, annot=True, annot_kws={\"size\": 16}, fmt='g')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### XGBOOST  <a id=\"9.2\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model accuracy score : 0.8324\n"
     ]
    }
   ],
   "source": [
    "model.fit(X_train,Y_train)\n",
    "y_pred = model.predict(X_test)\n",
    "print('Model accuracy score : {0:0.4f}'. format(accuracy_score(Y_test, y_pred)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjYAAAGiCAYAAAD0qYz9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de3zO9f/H8edlB7PN5RDbKqchOXxRwyYlNaovRa0DOZRGmcMW+pqQU6i+QuQQ+zmuvpKUHColKRKWJaWZQ6ScttE3LrPNZdv1+8O31dVk5LJre1+Pu9t1u7X3+3N93q/PjVteXq/35/OxOBwOhwAAAAxQxt0BAAAAuAqJDQAAMAaJDQAAMAaJDQAAMAaJDQAAMAaJDQAAMIa3OxfPXjjUncvDMO1HbXd3CDDMpoxUd4cAw+TajxTreudOHHDZuXyq1HbZua4mKjYAAMAYbq3YAACAqyg/z90RFDsSGwAATOXId3cExY5WFAAAMAYVGwAATJXveRUbEhsAAAzloBUFAABQelGxAQDAVLSiAACAMWhFAQAAlF5UbAAAMBUP6AMAAMagFQUAAOA6CQkJ6tq1q9NYZmamxowZo5YtW6pZs2bq27evDh065HTMmjVr1KFDBzVu3FgdO3bUxo0bL2k9EhsAAEyVn++6z9+wePFiTZ06tdB4XFycNm/erBkzZmjx4sU6ffq0+vbtq7y8862zLVu2KD4+Xl27dtWKFSvUpk0b9e/fX3v37i1yTRIbAAAM5XDku+xzOdLT09W3b19NnjxZoaGhTnNJSUnavHmzpk+frhYtWqh+/foaN26cMjMz9eOPP0qS5s6dq7Zt2+qxxx5TnTp1NGTIEDVp0kSLFi0qcm0SGwAA4FIpKSkKCAjQqlWr1LRpU6e5L774QnXr1lWDBg0KxurUqaMNGzaobt26ys/P1zfffKOIiAin74WHhys5ObnItdk8DACAqVz4gD6bzSabzVZo3Gq1ymq1Oo1FRkYqMjLyguc5ePCgatSooWXLlun111/Xr7/+qmbNmmnEiBEKDg6WzWZTVlaWQkJCnL4XFBSkY8eOFRkniQ0AAKZy4V1RiYmJmjlzZqHx2NhYxcXFXfJ5MjMzlZKSolOnTmn06NGyWCyaPHmyHn/8ca1cuVI5OTmSJF9fX6fv+fr6ym63y+FwyGKx/OX5SWwAAECRevbsqaioqELjf67WFMXHx0fZ2dmaNWuWKlWqJEmaMWOGWrdurXXr1unWW2+VJNntdqfv2e12+fv7XzSpkUhsAAAwlwsf0HehltPfERISoqCgoIKkRpKqVq2qSpUq6fDhw6pYsaL8/f2VkZHh9L2MjAwFBwcXeX42DwMAYCpHvus+LtK8eXOlpaU5JS7Hjx/Xr7/+qho1ashisSgsLExfffWV0/eSkpIUHh5e5PlJbAAAQLFp3769QkND9fTTT2vnzp3atWuXnnnmGdWsWVNt27aVJEVHR+ujjz7S/PnztX//fk2ZMkUpKSnq2bNnkecnsQEAwFRufkDfhfj6+ioxMVHVqlVTdHS0unfvrgoVKmjRokUqW7asJOm2227TSy+9pKVLl+qBBx7Qpk2bNGfOHNWpU6fI81scDofDZdFepuyFQ921NAzUftR2d4cAw2zKSHV3CDBMrv1Isa539vtPXHausv+4y2Xnupqo2AAAAGNwVxQAAKZyYQuptCCxAQDAUA6H6273Li1oRQEAAGNQsQEAwFQufP5MaUFiAwCAqdhjAwAAjOGBFRv22AAAAGNQsQEAwFQufAlmaUFiAwCAqWhFAQAAlF5UbAAAMBV3RQEAAGPQigIAACi9qNgAAGAqWlEAAMAYHpjY0IoCAADGoGIDAIChHA4e0AcAAExBKwoAAKD0omIDAICpPPA5NiQ2AACYilYUAABA6UXFBgAAU9GKAgAAxqAVBQAAUHpRsQEAwFS0ogAAgDFoRQEAAJReVGwAADCVB1ZsSGwAADAVe2xQ3D7fd0wjVn+tzc/cVzB2Ouecpm/Ypc/2HlPWuVyF16iqf7X9h6pXCig45pPdRxW/Yluh8w27q7EebVa7WGJHyVamTBk91DtK93XroKDrg5R+OEMrX1+l9xatlCT5+vnqiWceV2SnOxVoDdTenXv12rgE/ZDyg5sjR2nSu1c3DflXf1WrFqJvv92lIfHPa2vS1+4OCx6MxMaNdhz+r55bvV0Oh/P48NVfa0/6SQ26s5EqlPPV3C/36Mk3N+ndJyMVWNZHkrQv45SqVwrQC/eFOX33+or+xRU+SrjHB/VQt/6P6vVX/6Nd21PVJKKxYsf2V9lyZfXW7LcVO6af7nqwnRJenKsjB4+qS8wjmrp0knrd9ZSOHzvh7vBRCvTo8bBem/VvTXhhqpKTv9WA/tH68IPFCmt+lw4ePOTu8CB5ZCuKzcNuYM/N08Kt+/TUki/lVcbiNLf/hE2b9qdr2F1NdN8/qqt1nWD9+/7mSj+dow370gqO23vcpoYhFdXk+spOn2sC/Ir7clACWSwWPfLUQ3prztv6z4w3tf3Lb7Tolde16j/vq0vMI7JYLLrrwXZ6e+47WpG4Sts2JGvUU2NV1q+sIjvd6e7wUUqMHT1Ec+ct1vgJU7Xmo/V64MFonTjxXw18+il3h4bfOPJd9/mbEhIS1LVr17+cnz17tm688Ubl5uY6ja9Zs0YdOnRQ48aN1bFjR23cuPGS1iOxcYNNBzK0YMs+Db6zYaG2UbWKAXrj8da6rU5wwZiP1/nfJnve73+w9mXYdENVa/EEjFInwBqgte9+oo1rNjmNHzpwSJWqVJJ/oL+8fbyVdTqrYC4nK0d2+zmVr1i+uMNFKVS3bqhq1aqu999fWzCWm5urD9d8qnvuITnGeYsXL9bUqVP/cv67777TzJkzC41v2bJF8fHx6tq1q1asWKE2bdqof//+2rt3b5FrFtmKstvt+uijj5ScnKxjx47p7Nmz8vf3V0hIiMLDw3X33XfL25uO1uVodG1FfdDvLln9fDT7i91Oc2W9vdT4usqSpNz8fP3030y9sj5F1wSU1Z31rpUkZdlzdfRUlnann1KnhHU6eipLodeU19N3NFTrPyRE8FyZpzL16sjC/7No1e4WZRzN0JnTZ7R68QeKin5AO7Z+pyMHj6hHXDeV9fPVxg+/cEPEKG3q3XD+H2U/7D/oNP7jjz+rTu2aKlOmjPI9sA1S4rjp9yA9PV1jxoxRUlKSQkNDL3hMVlaW4uPj1bx5c23dutVpbu7cuWrbtq0ee+wxSdKQIUO0fft2LVq0SC+++OJF175oxebnn3/WvffeqzFjxmj//v0KDAzUtddeq3Llymnfvn167rnn1LFjRx0+fPhyrtfjBZcvJ6ufT5HHjVuzQw/N+0xbfszQwDsaqmI5X0nS3gybHJKOnjqjIW3/oVcfjtB1Ffw18J2t2vbT8ascPUqre7u2V/Pbm2nJ7LclSYmvvK6TJ04q4YNZej9lhTr3eVgvx0/R3p373BwpSoPy1kBJ0unTmU7jp09nysvLSwEB7PcrEdzUikpJSVFAQIBWrVqlpk2bXvCYF154QfXq1dP999/vNJ6fn69vvvlGERERTuPh4eFKTk4ucu2Lllqef/55hYaG6r333lNgYGCh+czMTA0ePFjjx49XQkJCkYvh8jxyc6g6/qO61u9N0+gPvlFevkNRTWuqdpXymvFIS91crXLBZuKWtYLUecFnmrt5r1rUrOrmyFHStIuK1DMvDdLn72/QewtXqKxfWc1cOV0+vj56YeC/deLYCd3eobWGTvqXsk5n6cu1m90dMko4i+X8/kDHn+5++G2cao15bDabbDZboXGr1Sqr1XlrRGRkpCIjI//yXJ988ok2bNig1atX67PPPiu0TlZWlkJCQpzGg4KCdOzYsSLjvGhi8/XXX+vtt9++YFIjSYGBgfrXv/6lbt26FbkQLl/j6ypJklrUrKqM09mav2WvoprWlNXPp1DLyauMRS1rVdUHKVTP4OzhJx9S/9Ex2rx2iybEvSRJur3Dbapeu5pi7h2gPd/ukSR9s3mHKlSy6unxsSQ2KJLt1GlJUvnygcrI+P0uusDAAOXl5enMmay/+iqKkwsTzMTExAvuh4mNjVVcXNwlnyc9PV2jRo3Syy+/rEqVKhWaz8nJkST5+vo6jfv6+sput8vhcBQk0Bdy0cTGarUqPT1d9erV+8tjjhw5In9/So6ucvjkGSX/dEL3N6nh9BtXP7iCvtifLknanXZSu9JP6cGmNZ2+ezY3v6BdBUjSk8/2Uo+4bvpo2VpNGjJZef/bgF712iDl5eYVJDW/2bnte0Xef6fK+fspOyvHHSGjlNj3w4+SpNqhNbT/D/tsQkNraM/e/W6KCoW4MLHp2bOnoqKiCo3/uVpzMQ6HQ8OGDVP79u11++23X/CYsmXLSjq/x/eP7Ha7/P39L5rUSEUkNg8//LCGDRumuLg4hYeHKyQkpCBjysjI0FdffaVXXnlFnTt3vuSLwsX99N9MjV2zQ9dX9C9oKTkcDm09eFx1/3cX1O6MUxq3ZocaBldQ/ZCKkqScc3nadCBdt9YOclvsKFke6h2lHnHd9M68dzVz7GynucM/HpaXt5cahjXQru2pBeMNbq6vX0/8SlKDIu3bd0A//3xEnTr9U5+sO38brre3tzq0b6sP13zq5uhwNVyo5XS5jh49qs2bN2v79u1asWKFJBXc5t2iRQvFxMQoJiZG/v7+ysjIcPpuRkaGgoOLvkHmoolNXFycLBaLXn75ZWVnZxeaDwgIUPfu3TVw4MBLvihcXEStqmpyXSWN/uAbxd7eQBX9ffXetz9rx+H/ambnlpKku+tfrwVb9il+RbJi2zRQWe8ySkz6QVn2XD3V6kY3XwFKgspBlRUz/CntTz2g9as+V8OwBk7zX67drH3f/6Axs0dp/ssL9Uv6L7rlrpa6+6G79OrIGW6KGqXNy5NmafqrE3Ty5Clt3rxN/fs9oSpVKuvV6XPdHRp+8+cnwLpZcHCw1q5d6zS2du1aTZ48We+++64qV64si8WisLAwffXVV3r00UcLjktKSlJ4eHiRa1w0sbFYLIqLi1NMTIx2796t9PR0ZWdny8/PTyEhIapfv36hHhiujHeZMpr+SEtN/3yXpn2+S7YcuxqEVNScR28pqOD4+3rr/7reqmmfpWjiJzuVdS5XN1errAXdb1OItZybrwAlQXib5vL181WdBrX12qrCicr9jR/Uvx4dqpiRT6n/6BiV9Surn/b9rDExz2vDB9zujUszJyFR5cr5KS62twY+/ZS+/TZFHe7trh9//NndoeE3JWwTt7e3t2rWdN5Gcc0110iSatSoUfD4mOjoaPXp00eNGjXSHXfcoRUrViglJUUTJkwocg2L489b2otR9sKh7loaBmo/aru7Q4BhNmWkFn0QcBly7UeKdb3sJWNcdq5yXZ//W98bNmyYfvrpJy1ZsuSC88uXL9fw4cOVkpLi9Fy8lStXatasWTp27Jjq1q2r+Ph4tWrVqsj1SGxgDBIbuBqJDVyt2BObxaNcdq5y3ce77FxXE48MBgDAVFfwjqfSindFAQAAY1CxAQDAVCVs83BxILEBAMBUJex27+JAKwoAABiDig0AAKaiFQUAAIzhgYkNrSgAAGAMKjYAAJjKA59jQ2IDAIChHPncFQUAAFBqUbEBAMBUHrh5mMQGAABTeeAeG1pRAADAGFRsAAAwlQduHiaxAQDAVOyxAQAAxvDAxIY9NgAAwBhUbAAAMJWDPTYAAMAUtKIAAABKLyo2AACYitu9AQCAMXjyMAAAQOlFxQYAAFPRigIAAKZwcFcUAABA6UXFBgAAU9GKAgAAxuCuKAAAgNKLig0AAKaiFQUAAIzBXVEAAAClF4kNAACmyne47vM3JSQkqGvXrk5jKSkp6tWrl8LDw9WqVSsNGjRIx44dczpmzZo16tChgxo3bqyOHTtq48aNl7QeiQ0AAKZy5Lvu8zcsXrxYU6dOdRo7duyYnnjiCQUFBenNN99UQkKCfvnlF/Xq1Us5OTmSpC1btig+Pl5du3bVihUr1KZNG/Xv31979+4tck0SGwAA4FLp6enq27evJk+erNDQUKe51atXq2zZspowYYLq1q2rxo0b6+WXX9aBAwf09ddfS5Lmzp2rtm3b6rHHHlOdOnU0ZMgQNWnSRIsWLSpybRIbAABM5aZWVEpKigICArRq1So1bdrUae7uu+/WtGnT5O39+/1LFotFknTq1Cnl5+frm2++UUREhNP3wsPDlZycXOTa3BUFAICh3PWuqMjISEVGRl5wrlatWqpVq5bT2Jw5c+Tn56eWLVvKZrMpKytLISEhTscEBQUV2odzISQ2AACgSDabTTabrdC41WqV1Wr92+dduHChlixZopEjR6py5cpKS0uTJPn6+jod5+vrK7vdLofDUVDhuRASGwAATOXCB/QlJiZq5syZhcZjY2MVFxd32efLz8/XlClTNG/ePA0YMECPPfaYJKls2bKSJLvd7nS83W6Xv7//RZMaicQGAABzuTCx6dmzp6KiogqN/51qzdmzZxUfH69PPvlEo0ePVvfu3QvmKlasKH9/f2VkZDh9JyMjQ8HBwUWem8QGAAAU6UpbTr/Jz89XXFyctmzZohkzZqhdu3ZO8xaLRWFhYfrqq6/06KOPFownJSUpPDy8yPOT2AAAYKoS+HbvxMREbdiwQWPHjlXTpk11/Pjxgrny5cvLz89P0dHR6tOnjxo1aqQ77rhDK1asUEpKiiZMmFDk+bndGwAAU5WAJw//2apVqyRJY8eO1W233eb0Wb16tSTptttu00svvaSlS5fqgQce0KZNmzRnzhzVqVOnyPNbHA6H2179mb1wqLuWhoHaj9ru7hBgmE0Zqe4OAYbJtR8p1vUyn+nksnMFvrLKZee6mmhFAQBgKIcLKy2lBYkNAACm8sDEhj02AADAGFRsAAAwlZteqeBOJDYAAJiKVhQAAEDpRcUGAABTeWDFhsQGAABDufFRdW5DKwoAABiDig0AAKaiFQUAAIxBYlO84p//yZ3LwzDv3ZLr7hBgmIjPr3V3CAAuExUbAAAMxbuiAACAOTwwseGuKAAAYAwqNgAAmMrzXhVFYgMAgKk8cY8NrSgAAGAMKjYAAJjKAys2JDYAAJjKA/fY0IoCAADGoGIDAIChPHHzMIkNAACmohUFAABQelGxAQDAULSiAACAOTywFUViAwCAoRwemNiwxwYAABiDig0AAKbywIoNiQ0AAIaiFQUAAFCKUbEBAMBUHlixIbEBAMBQtKIAAABKMRIbAAAM5ch33efvSkhIUNeuXZ3GDh8+rJiYGIWFhalVq1aaNGmScnNznY5ZvHix2rZtqyZNmqhLly767rvvLmk9EhsAAAzl7sRm8eLFmjp1qtOY3W5X7969ZbFY9NZbb2n8+PF65513NGPGjIJjli9frkmTJmnQoEFavny5QkND9eSTT+qXX34pck0SGwAA4FLp6enq27evJk+erNDQUKe5jz/+WEeOHNHEiRNVr149tW3bVkOGDNHrr7+unJwcSeerPN26dVPHjh1Vt25dvfDCCwoMDNRbb71V5NokNgAAmMphcd3nMqSkpCggIECrVq1S06ZNneaSk5PVoEEDVahQoWAsIiJCWVlZSklJ0YkTJ3Tw4EFFREQUzHt5ealZs2ZKTk4ucm3uigIAwFCuvCvKZrPJZrMVGrdarbJarU5jkZGRioyMvOB50tPTFRIS4jQWFBQkSUpLS5Ofn58kXfCYnTt3FhkniQ0AAChSYmKiZs6cWWg8NjZWcXFxl3yenJwcBQQEOI35+vpKks6ePavs7GynsT8eY7fbizw/iQ0AAIZy5F9eC+lievbsqaioqELjf67WFMXPz69QgvLbz/7+/gUVmwsd4+/vX+T5SWwAADCUK1tRF2o5/R0hISFKTU11GsvIyCiYu+666wrGbrzxRqdjgoODizw/m4cBAECxadGihVJTU5326yQlJSkgIEANGzZU5cqVFRoaqq+++qpgPi8vT19//bXCw8OLPD+JDQAAhnI4LC77uEq7du0UHByswYMHa/fu3Vq/fr2mTJmi6Ojogn01vXr1UmJiopYvX64ffvhBI0eO1JkzZ/TII48UeX5aUQAAGKokviuqbNmymjdvnsaNG6fOnTvLarWqS5cuGjBgQMExnTt3VmZmpl599VWdPHlSjRo10oIFC1S5cuUiz29xOByOq3kBFxNbq4u7loaBxrdId3cIMEzE55nuDgGG2Xu86OewuNLhiAvfcv13VEta77JzXU1UbAAAMJQr74oqLUhsAAAwlPt6Mu7D5mEAAGAMKjYAABiKVhQAADCGJyY2tKIAAIAxqNgAAGAoT9w8TGIDAIChaEUBAACUYlRsAAAwlCvf8VRakNgAAGCokviuqKuNVhQAADAGFRsAAAyVTysKAACYwhP32NCKAgAAxqBiAwCAoTzxOTYkNgAAGMoTnzxMKwoAABiDig0AAIaiFQUAAIzhibd704oCAADGoGIDAIChPPE5NiQ2JYy3r7eGffiyDu7Yp/8MmV0w3qxjK90zIEpVQ0P069FftGHRR9qQ+JEbI0VJ5dO8lQKeHqmTj3dwGvd7sIfK3tVRlvIVlLvne2XNn678oz8XzFsCAlWuR1/5hLWUfH2Vm/qdshfNUn7GseK+BJQCkffcrslzxisstI0kKerR+zRxxti/PL5e1ebFFBn+yBPviiKxKWHaD3xYIXWv18Ed+wrGwu67RT1fjdP6ue9r1/M7dOOt/9Ajz0crJzNLSe9udGO0KGm86jVSQNxzksX5X2l+D/eUX1Q3Zf8nQfnH0+T30GMqP2aKTg1+Qso6I0kKGDRaXrXqKvuNOXJknpZfl2gFjnlFtn/1knKy3XA1KKlubtFEk2ePk+UPf84+/2STHvnnE07HVb6mkqbP/7dWLvuwmCOEJyOxKUGqNaqlO55or9O/2JzGHxjWXV+8sVYrXlosSdq7JUWVqwXpxtuakNjgPG8fle3wkMo92kuOszmylPnD9jm/cvLr1EXZby/S2TXLJUm5qd+pwmtLVTayg86+v0wWa0X53BSuM69NlH3jWklS3rFDqjD9P/K5KVzntm5wx1WhhPHx9VHPPl01aFhfZWVly8f39z9nv/5yUr/+ctLp+NcSJ+vIoWMaP2JycYeK//HEzcMkNiVEGa8y6v5yX637v9Vqek+LgvEajWurcrWq+nLJp07HJw6aUdwhogTzuTlCflHdlf3GHFnKW+XXsUvBnHe9hrKU89e55C8LxhxnMnVu17fyuSlcZ99fJvn6nh/Pzvr9mNPnE2xLYPliugqUdG3atlLMwCc08flXValSBUX37/GXx952Z0u163CHenWO1dmcs8UYJf7IE/fYcFdUCXFX3/vl7eOtta+95zR+Xf0akqQyXl4auHSMpu1drPGbZ6l1j7vdESZKqNz9u2Ub0PV8ReZPPXWva6tLkvLTjzqN52ccVZn/zTlOZMievFl+D/ZQmetqyGKtqHK94uTIytS57UnFcg0o+XZ+s0uRzTrpjblLi9y7MWRUnL74bIs2fba1eIID/oeKTQkQXOc63RMbpRndxyvvXJ7TXOA1VuXl5ilmXry+eGOt1kx7R03vCVeXCb115uRpbX9/i5uiRkni+O+Jv5yzlPOXw26XcnOdv5OdLYu/f8HP2QtnKHDUZFV49fXz8/azynxpuBz/PX51gkapk552aX8Wwls1U8PGN6rng/2uckQoCpuHUewsFou6TeyrLUs/04/b9xWa9/L2lpe3l75c8qnWvrZC0vk9NtdUr6r2Ax8msUHRLBYVKuNIkkVS/vlxS+UqKv/CLDlsJ5U5ebQc2WdUtu19Cowfr9MT4pW3b1exhozSrcvjUdqz6wdt+WKbu0PxeOyxuYBu3bo57Xy/mMWLF19xQJ6mzRP/VOXrq2hOr4kq4/V7Z9BisaiMVxmdzcqRJKVu2OH0vd2bdirquZvk5eNVqMoD/JEj64zk7SN5eUl5v/9ZsfiVkyMrU5JU9o72sgSWl+3ZPgXVn9zvvlb5F2apXI8YZY4Z6JbYUfp4e3upTbtbNf+1/7g7FHioIhObNm3aaNq0aapdu7aaNGlSHDF5lKb3tFCla6/RpO8WOI1Xa1hLEQ+10dujz497+Tj/Vnl5e8lisciR74F1RlyWvGOHZSlTRmWCrlX+scMF42WCrlP+0UPn/7tKVeWfyCjU0srd/b187/hnscaL0u2mFk1krVBea99f7+5QIM/cPFxkYhMTE6PAwEBNmTJFCQkJqlatWnHE5TGWjJgrv4ByTmM9p8Up48djWvPqO8r48ZgeGNFDN9/bUgeS9xQc0ygyTD99t1/5efnFHTJKmdy9KXLYz8qnxW06u+otSecfxufTsKmylyVKkvKOHpbvnR1kqVzVaU+N1w0NeEAfLkuTmxvptC1T+/f+6O5QIFpRf6l79+764osvNG3aNE2ezPMIXCnjQOG/NM6dtevMydP6eecBSdLaWe+pw6BHlHM6Wz8kpSrsvltUN6KBZkf/u7jDRWmUk62za5arXNfeksOh/GOH5PdgDzmys3R2/QeSJPtnH8rv3odV/rmJyl6WKEd2lnzb3C3vG/+hM5NGuvkCUJrUa1BHB/f/5O4w4MEuefPwuHHjlJKScjVjwV/4aMZyZZ/OUpue/1S7mI7KOHBM8/q9otQN37o7NJQS2W/OkxwO+XXqIotfOeXu+V6nZ/674KnDjjOZOj0qTuUe7yf/vvGyWKTcg/uV+fwzyt21o4izA7+7pkpl2U5lujsM/I+7NivY7XbNmDFDq1ev1qlTp9SgQQMNGTJEYWFhkqTDhw9r/Pjx2rZtm/z8/BQVFaXBgwfL2/vK72myOBzuuxkstlaXog8CLtH4FunuDgGGificv6DhWnuPJxfrepuvfchl52p17N1LPnbq1KlatmyZJk6cqOrVq2vevHn68MMPtWbNGlWqVEkdO3ZUaGionnnmGR06dEgjRozQo48+qsGDB19xnDygDwAAQzkcFpd9Lsenn36qTp06qXXr1qpVq5aGDRumM2fO6Ouvv9bHH3+sI0eOaOLEiapXr57atm2rIUOG6PXXX1dOTrce0xYAABm1SURBVM4VXzOJDQAAcKnKlSvrs88+0+HDh5WXl6dly5bJ19dXDRs2VHJysho0aKAKFSoUHB8REaGsrCyXbHnhAX0AABjKlffN2mw22Wy2QuNWq1VWq9VpbNSoURo0aJDatm0rLy8vlSlTRtOmTVOtWrWUnp6ukJAQp+ODgoIkSWlpaVccJ4kNAACGcsh1t3snJiZq5syZhcZjY2MVFxfnNLZv3z4FBgZq1qxZCg4O1rJlyzR06FC98cYbysnJUUBAgNPxvv97Ee/Zs1f+wlQSGwAAUKSePXsqKiqq0PifqzVHjx7V0KFDNW/ePLVs2VKS1LhxY/3www+aPn26/Pz8ZLfbnb7z28/+f3h/3d9FYgMAgKFc+XD6C7WcLuS7777TuXPn1LhxY6fxpk2bav369brllluUmprqNJeRkSFJhVpUfwebhwEAMFS+LC77XKrfkpM9e/Y4je/du1ehoaFq0aKFUlNTnfbrJCUlKSAgQA0bNrziayaxAQAALtOkSRM1a9ZMI0aM0NatW3Xw4EFNmzZNmzdvVp8+fdSuXTsFBwdr8ODB2r17t9avX68pU6YoOjq6YK/NlaAVBQCAoVy5efhSlSlTRrNnz9a0adM0fPhwnTx5UjfeeKMWLlxY8OThefPmady4cercubOsVqu6dOmiAQMGuGR9njwMY/DkYbgaTx6GqxX3k4c/CXbd37N3pS912bmuJlpRAADAGLSiAAAwlDtaUe5GYgMAgKFc+eTh0oJWFAAAMAYVGwAADOWJFRsSGwAADOWJe2xoRQEAAGNQsQEAwFD5nlewIbEBAMBUl/OOJ1PQigIAAMagYgMAgKHc9s4kNyKxAQDAUJ54uzetKAAAYAwqNgAAGCrf4nmbh0lsAAAwlCfusaEVBQAAjEHFBgAAQ3ni5mESGwAADOWJTx6mFQUAAIxBxQYAAEN54isVSGwAADAUd0UBAACUYlRsAAAwlCduHiaxAQDAUJ54uzetKAAAYAwqNgAAGMoTNw+T2AAAYChP3GNDKwoAABiDig0AAIbyxM3DJDYAABjKExMbWlEAAMAYVGwAADCUwwM3D5PYAABgKFpRAAAALrBixQp16NBBjRs31r333qs1a9YUzB0+fFgxMTEKCwtTq1atNGnSJOXm5rpkXSo2AAAYyl0Vm5UrV2rEiBF69tlndccdd+jDDz/UM888o6CgIDVu3Fi9e/dWaGio3nrrLR06dEgjRoyQt7e3Bg8efMVrk9gAAGAodzx52OFw6NVXX1WPHj3Us2dPSVK/fv2UnJysrVu36ujRozpy5IjefvttVahQQfXq1dOQIUP04osvql+/fvLz87ui9WlFAQAAlzlw4ICOHDmi++67z2l8/vz5GjBggJKTk9WgQQNVqFChYC4iIkJZWVlKSUm54vWp2AAAYChXvlLBZrPJZrMVGrdarbJarQU/Hzx4UJJkt9vVp08f7dy5U9WqVVO/fv0UGRmp9PR0hYSEOJ0jKChIkpSWlnbFcZLYAABgKFfusUlMTNTMmTMLjcfGxiouLq7g58zMTEnS0KFDNWDAAA0ePFhr165V//79NX/+fOXk5CggIMDpHL6+vpKks2fPXnGcJDYAAKBIPXv2VFRUVKHxP1ZrJMnHx0eSFB0drYceekiS1KBBA33//fdasGCB/Pz8ZLfbnb7z28/+/v5XHCeJDQAAhnJlxebPLae/8lubqV69ek7jN9xwgz799FPdcsstSk1NdZrLyMhw+u6VYPMwAACGcrjwc6kaNmyogIAA7dy502l87969qlGjhlq0aKHU1FSn/TpJSUkKCAhQw4YN/9Z1/hGJDQAAcBk/Pz89+eSTeu2117Rq1Sr9/PPPmj17tjZt2qRevXqpXbt2Cg4O1uDBg7V7926tX79eU6ZMUXR0dMFemytBKwoAAEO58q6oy9G/f3/5+/tr+vTpSktLU+3atTVjxgzdcsstkqR58+Zp3Lhx6ty5s6xWq7p06aIBAwa4ZG0SGwAADOXOd0U98cQTeuKJJy44V7NmTc2fP/+qrEtiAwCAodzx5GF3Y48NAAAwhlsrNnOObnLn8jBM6pZG7g4BhklJfdvdIQBXJN8Daza0ogAAMJQ799i4C60oAABgDCo2AAAYyvMaUSQ2AAAYi1YUAABAKUbFBgAAQ7nrycPuRGIDAIChPPF2b1pRAADAGFRsAAAwlOfVa0hsAAAwFndFAQAAlGJUbAAAMJQnbh4msQEAwFCel9bQigIAAAahYgMAgKE8cfMwiQ0AAIbyxD02tKIAAIAxqNgAAGAoz6vXkNgAAGAsT9xjQysKAAAYg4oNAACGcnhgM4rEBgAAQ9GKAgAAKMWo2AAAYChPfI4NiQ0AAIbyvLSGVhQAADAIFRsAAAxFKwoAABiDu6IAAABKMSo2AAAYigf0AQAAY9CKAgAAcKEff/xRN998s5YtW1YwdvjwYcXExCgsLEytWrXSpEmTlJub65L1qNgAAGAod7eizp07pyFDhigrK6tgzG63q3fv3goNDdVbb72lQ4cOacSIEfL29tbgwYOveE0qNgAAGCrfhZ+/Y8aMGQoICHAa+/jjj3XkyBFNnDhR9erVU9u2bTVkyBC9/vrrysnJ+Zsr/Y7EBgAAuNy2bdu0dOlSTZw40Wk8OTlZDRo0UIUKFQrGIiIilJWVpZSUlCtel1YUAACGyne4rhVls9lks9kKjVutVlmt1kLHDh06VCNHjtS1117rNJeenq6QkBCnsaCgIElSWlraFcdJYgMAgKFcucMmMTFRM2fOLDQeGxuruLg4p7GxY8fqpptuUseOHQsdn5OTU6g95evrK0k6e/bsFcdJYgMAAIrUs2dPRUVFFRr/c7VmxYoVSk5O1urVqy94Hj8/P9ntdqex33729/e/4jhJbAAAMJQr3xV1oZbThbz77rv65ZdfdMcddziNjxs3TosWLVKLFi2UmprqNJeRkSFJhVpUfweJDQAAhnLH7d6TJ08udHfT3XffrdjYWN13333asWOHli9fLpvNVpAoJSUlKSAgQA0bNrzi9UlsAACAywQHB19wvHLlyrr++utVpUoVTZs2TYMHD1Z8fLyOHj2qKVOmKDo6umCvzZXgdm8AAAzl7ufYXEjZsmU1b948SVLnzp01evRodenSRQMGDHDJ+anYAABgKFfusbkSe/bscfq5Zs2amj9//lVZi4oNAAAwBhUbAAAM5e53RbkDiQ0AAIZy5d6Y0oJWFAAAMAYVGwAADOVw4buiSgsSGwAADFVS7ooqTrSiAACAMajYAABgKE/cPExiAwCAobjdGwAAGIM9NgAAAKUYFRsAAAzF7d4AAMAYnrh5mFZUCda7VzelpmzS6VM/aNPGVWoZ0czdIaEUKVOmjB5+6iEt+Gye3t+7SgvWz9X9T3QqmK94TUUNn/6sVnz/rlZ8/67GJIxScLVgN0aMkuqzL7YqvN2DhcY/XPe5oh7rp7A7O6lDl95avGyl0/wp22mN+ferurNTd7X65yOKe/Z5HTpyrLjChocisSmhevR4WK/N+rfeXPKuOnfpo5MnT+nDDxarVq3q7g4NpUSPQd3V+9lorVv+qUZFj9bn72/UgLH91KXfI/L28dakpRPVvE0zzX1pvl6IfUlly5XV9BVTZa1Y3t2howT5ZucuDRv3cqG7a9as26Bnx76sWyOaafbkcbonsrVemjZHKz/8pOCYoWMn6rNNWzWoX7ReHDVEaRnH9eTA4crKyi7uy/BYDhf+Ki1oRZVQY0cP0dx5izV+wlRJ0ifrNmrX9xs18OmnNPiZ0W6ODiWdxWLRw089qKVzlunNGUskSd98uUMVK1fQIzGP6OhPx1S7fqie7T5cyRu+liTt2PytFm2Yr0cHdNH/vTDPneGjBLDb7frPspWaMfd1lfPzU35ubsGcw+HQK68t0KNR92lI7JOSpIhmN+nosXRt3vaN7u9wl3759aS+TPpa44YP0v3t20mSalW/Xvd1fUqbkpJ1952t3XJdnoa7oi4gKSlJ8fHx6tOnj958803l5eU5zZ86dUrdu3e/agF6orp1Q1WrVnW9//7agrHc3Fx9uOZT3XPPnW6MDKVFgDVAn7y7TpvWbHIaP3TgsCpVqajqdaorLzdP2zd9UzB3zn5Oe77dqxZ3NC/ucFECfbE1WfPeeFv/GvCkuj3cyWkuZfc+HUvP0MP3t3canzj2WU0cM1SSZD9rlyQFBvgXzFesYJV0vkUFXC0XTWzWr1+v6OhoHT9+XOfOndP48ePVo0cP2Wy2gmPOnTun7du3X/VAPUm9G2pLkn7Yf9Bp/Mcff1ad2jVVpgwdRFxc5qlMzRg5Sz+k7Hcav6VdS2UcPa6MIxny8vZSleBrnOZDqoewzwaSpH80qKePli1Uj0ful8VicZrbu/9HSVJeXp6eGBCvm9p0VNuox/TW8vcLjrk2JEhtbg3X/yW+pQM/HdIvv57US1NnKzDAX61vaVGs1+LJHA6Hyz6lxUX/hnzttdc0cOBALVq0SAsXLtSSJUt06NAhRUdHKysrq7hi9DjlrYGSpNOnM53GT5/OlJeXlwL+8C8g4FJ16PpPNbs9TEtnv62vPtumk7+c1LDpz6pG3eqyViyvx595TKE31pKfv5+7Q0UJEFy1iqzlAy849+vJU/LyKqO4Z8eqVXgzJbwyXm1b36IJU2ZpzboNBccNG9hXmWey1KlbH7W5r6vWbdisaS+OVEhQ1eK6DI+XL4fLPqXFRROb/fv3q0OHDgU/33TTTUpMTNThw4f19NNPF2pLwTV++9fRnzPk38bz8z3xBj5cibZRkRr00kBteH+jVixcKduvNo158nkFXRekhZ/P13vfv6t6jW/QB0s+1Nnss+4OFyVcbm6e8vLy9XCn9urT81FFNLtJI57pr9tbhWv2wsWSpPTjJ9Q95hmV8/PT1AnP6f+mvqA7bovQwOHj9e33qW6+ApjsoolNpUqVdOTIEaexOnXqaObMmUpKStJzzz1XqspTpYXt1Pn+c/k//WspMDBAeXl5OnOGahku3UNPPqhhrw7V1nVJejHu3wXj329LUY9Wj6tHq8fVpXlXjYwerXL+5XT6JPsfcHH+5c5X9W5r6bwf65YWN+unQ0d07tw5rfjgE9lOZ2rOK+N11523qVV4mCaPG666oTX1ymsL3BG2R/LEu6Iumti0a9dOY8aM0caNG51aTy1atNALL7yglStXaujQoVc9SE+z74fz/evaoTWcxkNDa2jP3v0X+gpwQb2fjVb/sX217t1PNTZmnHLPnb+zxVrJqnseuUt+/n469nOaTqT9Ikmq3aB2oX05wJ9Vr3adpPN7LP8oNzdXDodksZRRWsZxhQRVUXDVKgXzFotFNzVpqP0Hfy7WeD1ZvsPhsk9pcdHEZuDAgapXr55iYmIKbRDu1KmTXn75Ze3YseOqBuiJ9u07oJ9/PqJOnf5ZMObt7a0O7dtq/fpNF/km8LsHez+gbnFd9e685Zo4eJLy835vYfr4emvo1Hg1b/P7Qx8bhjVQvSY3aOu6re4IF6VI85v+obK+vvr4M+f/H23cvE3/qH+DvL29VLP69UpLP660jONOx+xM2aPrr2WDOq6eiz7HJiAgQDNmzNCJEydUrly5QvMdO3ZUeHi41q5de4Fv40q8PGmWpr86QSdPntLmzdvUv98TqlKlsl6dPtfdoaEUqBxUWU8Nf1IHUg/os1Wfq0FYfaf5Pd/u1Zcfb1bfUTFyOCQfH2/1H9tXP6Ts19p31rkpapQWgQEBeurxLnptwWIF+vur+c2N9dGnG5W8Y6dmTx4nSYq692698fYK9fvXaPWN7qbAAH+tWrNO3+zcpekv8Syu4lJ66iyuY3G4cZOMt+/17lq6VBg8KEZxsb1VpUplffttiuKHjtPWpK/dHVaJ1SaokbtDKDHueeQuDZ0a/5fzUY0flsPhUP+xfdWybYTy8/O1dV2S/u+FeTr131PFGGnJ9tGOOe4OoUSYNf8/WrTkXW1b957T+H+WrdSb76xSWsZx1ax+vQb07qF2bW4tmD+alq7JM+dpy7Zv5HA4dGPd2hrw5GMKD2tS3JdQYvhUqV2s6916faTLzvXlkfUuO9fVRGIDY5DYwNVIbOBqJDZXH69UAADAUKXp+TOuQmIDAIChPPGRLDybHwAAGIOKDQAAhqIVBQAAjFGanhjsKrSiAACAMajYAABgKDYPAwAAY+TL4bLP5cjMzNSLL76oyMhI3XzzzXrwwQf16aefFswfPnxYMTExCgsLU6tWrTRp0iTl5ua65Jqp2AAAAJcaPny49uzZowkTJuj666/XmjVrFBsbqwULFqhZs2bq3bu3QkND9dZbb+nQoUMaMWKEvL29NXjw4Ctem8QGAABDuaMVdfz4ca1du1YJCQlq1aqVJKlv377asmWL3nnnHZ04cUJHjhzR22+/rQoVKqhevXoaMmSIXnzxRfXr109+fn5XtD6tKAAADOWOVlS5cuU0d+5cNW/e3GncYrHo1KlTSk5OVoMGDVShQoWCuYiICGVlZSklJeWKr5mKDQAAKJLNZpPNZis0brVaZbVaC34ODAzU7bff7nTMjh07tHXrVo0cOVKbNm1SSEiI03xQUJAkKS0t7YrjJLEBAMBQrnyOTWJiombOnFloPDY2VnFxcX/5vf379ys2NlZNmzZVly5dtG7dOgUEBDgd4+vrK0k6e/bsFcdJYgMAgKHyXbjHpmfPnoqKiio0/sdqzZ9t27ZNsbGxuu6665SQkCAfHx/5+fnJbrc7Hffbz/7+/lccJ4kNAAAo0p9bTkVZtWqVRowYofDwcE2fPl2BgYGSpJCQEKWmpjodm5GRUTB3pdg8DACAoRwu/HU5Vq9eraFDh6p9+/ZKSEgoSGokqUWLFkpNTXXar5OUlKSAgAA1bNjwiq+ZxAYAAEPlOxwu+1yqtLQ0jRo1ShEREYqPj9fJkyd1/PhxHT9+XCdPnlS7du0UHByswYMHa/fu3Vq/fr2mTJmi6Ojogr02V4JWFAAAcJm1a9cqOztbW7duVevWrZ3mwsLCtGTJEs2bN0/jxo1T586dZbVa1aVLFw0YMMAl61scbnyRhLfv9e5aGgZqE9TI3SHAMB/tmOPuEGAYnyq1i3W9+kEtXHau3RnbXHauq4mKDQAAhnLlXVGlBXtsAACAMajYAABgKFc+oK+0ILEBAMBQtKIAAABKMSo2AAAYilYUAAAwhsOR7+4Qih2tKAAAYAwqNgAAGCqfVhQAADCFG18u4Da0ogAAgDGo2AAAYChaUQAAwBi0ogAAAEoxKjYAABjKE1+pQGIDAIChPPHJw7SiAACAMajYAABgKE/cPExiAwCAobjdGwAAGMMTKzbssQEAAMagYgMAgKG43RsAABiDVhQAAEApRsUGAABDcVcUAAAwBq0oAACAUoyKDQAAhuKuKAAAYAxeggkAAFCKUbEBAMBQtKIAAIAxuCsKAACgFCOxAQDAUA4X/roc+fn5mj59ulq3bq2mTZuqV69e+umnn67SVTojsQEAwFAOh8Nln8sxa9YsLVmyRBMmTNDSpUvl5eWl3r176+zZs1fpSn9HYgMAAFzGbrdrwYIFio2NVZs2bVS/fn1NnTpVJ06c0Jo1a676+iQ2AAAYyh0Vm9TUVGVlZally5YFY4GBgWrYsKGSk5OvxmU64a4oAAAM5cp7omw2m2w2W6Fxq9Uqq9Va8HN6erokKTg42Om4oKAgHTt2zIURXZhbE5tc+xF3Lg8AgNFc+ffsjBkzNHPmzELjsbGxiouLK/g5OztbkuTr6+t0nK+vr+x2u8vi+StUbAAAQJF69uypqKioQuN/rNZIkp+fn6Tze23+mNzY7Xb5+/tf3SBFYgMAAC7Bn1tOf+Xaa6+VJGVkZCgwMLBgPCMjQ3Xr1r1q8f2GzcMAAMBl6tevr8DAQH311VcFY5mZmdq1a5fCw8Ov+vpUbAAAgMv4+vqqR48emjp1qqpUqaJq1appypQpCg4O1t13333V1yexAQAALvX0008rLy9Po0ePVnZ2tpo1a6Z58+YV2lB8NVgcnviGLAAAYCT22AAAAGOQ2AAAAGOQ2AAAAGOQ2JRQ7nzlO8yXkJCgrl27ujsMlHKZmZl68cUXFRkZqZtvvlkPPvigPv30U3eHBQ9HYlNCufOV7zDb4sWLNXXqVHeHAQMMHz5cn3/+uSZMmKAVK1bo7rvvVmxsrLZs2eLu0ODBSGxKIHe/8h1mSk9PV9++fTV58mSFhoa6OxyUcsePH9fatWs1YsQItWrVSjVr1lTfvn0VHh6ud955x93hwYOR2JRA7n7lO8yUkpKigIAArVq1Sk2bNnV3OCjlypUrp7lz56p58+ZO4xaLRadOnXJTVAAP6CuR3P3Kd5gpMjJSkZGR7g4DhggMDNTtt9/uNLZjxw5t3bpVI0eOdFNUABWbEsndr3wHgMu1f/9+xcbGqmnTpurSpYu7w4EHI7Epgf74yvc/Kq5XvgPA5di2bZu6deumqlWrKiEhQT4+Pu4OCR6MxKYE+uMr3/8oIyOjUHsKANxp1apVio6OVqNGjfTGG2+oYsWK7g4JHo7EpgRy9yvfAeBSrF69WkOHDlX79u2VkJCgwMBAd4cEsHm4JHL3K98BoChpaWkaNWqUIiIiFB8fr5MnTxbM+fj4ULmB25DYlFDufOU7ABRl7dq1ys7O1tatW9W6dWunubCwMC1ZssRNkcHTWRwOh8PdQQAAALgCe2wAAIAxSGwAAIAxSGwAAIAxSGwAAIAxSGwAAIAxSGwAAIAxSGwAAIAxSGwAAIAxSGwAAIAx/h8mMU1fP9D/IwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x504 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_cm = pd.DataFrame(confusion_matrix(Y_test, y_pred), range(3), range(3))\n",
    "plt.figure(figsize=(10,7))\n",
    "sn.set(font_scale=1.4) \n",
    "sn.heatmap(df_cm, annot=True, annot_kws={\"size\": 16}, fmt='g') \n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that here, balancing class 0 finally results in an equavalent score than with an unbalanced data set.\n",
    " **Unbalanced classes are not a problem, and oversampling will not solve a non-problem !**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}