780 lines
64 KiB
Text
780 lines
64 KiB
Text
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# 朴素贝叶斯模型实验 （选作）\n",
|
||
"\n",
|
||
"> 本实验目标是用朴素贝叶斯模型对Yelp网站的评论文本进行分类"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 第一步：读入数据\n",
|
||
"\n",
|
||
"把`yelp.csv`读入一个DataFrame中。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 66,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"<div>\n",
|
||
"<style scoped>\n",
|
||
" .dataframe tbody tr th:only-of-type {\n",
|
||
" vertical-align: middle;\n",
|
||
" }\n",
|
||
"\n",
|
||
" .dataframe tbody tr th {\n",
|
||
" vertical-align: top;\n",
|
||
" }\n",
|
||
"\n",
|
||
" .dataframe thead th {\n",
|
||
" text-align: right;\n",
|
||
" }\n",
|
||
"</style>\n",
|
||
"<table border=\"1\" class=\"dataframe\">\n",
|
||
" <thead>\n",
|
||
" <tr style=\"text-align: right;\">\n",
|
||
" <th></th>\n",
|
||
" <th>business_id</th>\n",
|
||
" <th>date</th>\n",
|
||
" <th>review_id</th>\n",
|
||
" <th>stars</th>\n",
|
||
" <th>text</th>\n",
|
||
" <th>type</th>\n",
|
||
" <th>user_id</th>\n",
|
||
" <th>cool</th>\n",
|
||
" <th>useful</th>\n",
|
||
" <th>funny</th>\n",
|
||
" </tr>\n",
|
||
" </thead>\n",
|
||
" <tbody>\n",
|
||
" <tr>\n",
|
||
" <th>0</th>\n",
|
||
" <td>9yKzy9PApeiPPOUJEtnvkg</td>\n",
|
||
" <td>2011-01-26</td>\n",
|
||
" <td>fWKvX83p0-ka4JS3dc6E5A</td>\n",
|
||
" <td>5</td>\n",
|
||
" <td>My wife took me here on my birthday for breakf...</td>\n",
|
||
" <td>review</td>\n",
|
||
" <td>rLtl8ZkDX5vH5nAx9C3q5Q</td>\n",
|
||
" <td>2</td>\n",
|
||
" <td>5</td>\n",
|
||
" <td>0</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <th>1</th>\n",
|
||
" <td>ZRJwVLyzEJq1VAihDhYiow</td>\n",
|
||
" <td>2011-07-27</td>\n",
|
||
" <td>IjZ33sJrzXqU-0X6U8NwyA</td>\n",
|
||
" <td>5</td>\n",
|
||
" <td>I have no idea why some people give bad review...</td>\n",
|
||
" <td>review</td>\n",
|
||
" <td>0a2KyEL0d3Yb1V6aivbIuQ</td>\n",
|
||
" <td>0</td>\n",
|
||
" <td>0</td>\n",
|
||
" <td>0</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <th>2</th>\n",
|
||
" <td>6oRAC4uyJCsJl1X0WZpVSA</td>\n",
|
||
" <td>2012-06-14</td>\n",
|
||
" <td>IESLBzqUCLdSzSqm0eCSxQ</td>\n",
|
||
" <td>4</td>\n",
|
||
" <td>love the gyro plate. Rice is so good and I als...</td>\n",
|
||
" <td>review</td>\n",
|
||
" <td>0hT2KtfLiobPvh6cDC8JQg</td>\n",
|
||
" <td>0</td>\n",
|
||
" <td>1</td>\n",
|
||
" <td>0</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <th>3</th>\n",
|
||
" <td>_1QQZuf4zZOyFCvXc0o6Vg</td>\n",
|
||
" <td>2010-05-27</td>\n",
|
||
" <td>G-WvGaISbqqaMHlNnByodA</td>\n",
|
||
" <td>5</td>\n",
|
||
" <td>Rosie, Dakota, and I LOVE Chaparral Dog Park!!...</td>\n",
|
||
" <td>review</td>\n",
|
||
" <td>uZetl9T0NcROGOyFfughhg</td>\n",
|
||
" <td>1</td>\n",
|
||
" <td>2</td>\n",
|
||
" <td>0</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <th>4</th>\n",
|
||
" <td>6ozycU1RpktNG2-1BroVtw</td>\n",
|
||
" <td>2012-01-05</td>\n",
|
||
" <td>1uJFq2r5QfJG_6ExMRCaGw</td>\n",
|
||
" <td>5</td>\n",
|
||
" <td>General Manager Scott Petello is a good egg!!!...</td>\n",
|
||
" <td>review</td>\n",
|
||
" <td>vYmM4KTsC8ZfQBg-j5MWkw</td>\n",
|
||
" <td>0</td>\n",
|
||
" <td>0</td>\n",
|
||
" <td>0</td>\n",
|
||
" </tr>\n",
|
||
" </tbody>\n",
|
||
"</table>\n",
|
||
"</div>"
|
||
],
|
||
"text/plain": [
|
||
" business_id date review_id stars \\\n",
|
||
"0 9yKzy9PApeiPPOUJEtnvkg 2011-01-26 fWKvX83p0-ka4JS3dc6E5A 5 \n",
|
||
"1 ZRJwVLyzEJq1VAihDhYiow 2011-07-27 IjZ33sJrzXqU-0X6U8NwyA 5 \n",
|
||
"2 6oRAC4uyJCsJl1X0WZpVSA 2012-06-14 IESLBzqUCLdSzSqm0eCSxQ 4 \n",
|
||
"3 _1QQZuf4zZOyFCvXc0o6Vg 2010-05-27 G-WvGaISbqqaMHlNnByodA 5 \n",
|
||
"4 6ozycU1RpktNG2-1BroVtw 2012-01-05 1uJFq2r5QfJG_6ExMRCaGw 5 \n",
|
||
"\n",
|
||
" text type \\\n",
|
||
"0 My wife took me here on my birthday for breakf... review \n",
|
||
"1 I have no idea why some people give bad review... review \n",
|
||
"2 love the gyro plate. Rice is so good and I als... review \n",
|
||
"3 Rosie, Dakota, and I LOVE Chaparral Dog Park!!... review \n",
|
||
"4 General Manager Scott Petello is a good egg!!!... review \n",
|
||
"\n",
|
||
" user_id cool useful funny \n",
|
||
"0 rLtl8ZkDX5vH5nAx9C3q5Q 2 5 0 \n",
|
||
"1 0a2KyEL0d3Yb1V6aivbIuQ 0 0 0 \n",
|
||
"2 0hT2KtfLiobPvh6cDC8JQg 0 1 0 \n",
|
||
"3 uZetl9T0NcROGOyFfughhg 1 2 0 \n",
|
||
"4 vYmM4KTsC8ZfQBg-j5MWkw 0 0 0 "
|
||
]
|
||
},
|
||
"execution_count": 66,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# read csv\n",
|
||
"import pandas as pd\n",
|
||
"\n",
|
||
"url = \"e2.4_yelp.csv\"\n",
|
||
"yelp = pd.read_csv(url)\n",
|
||
"yelp.head()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"创建一个新的DataFrame，只包含5颗星和1颗星评分的数据。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 67,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# filter data\n",
|
||
"yelp_best_worst = yelp[(yelp.stars==5) | (yelp.stars==1)]\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 第二步：生成X和y\n",
|
||
"\n",
|
||
"使用评论文本作为唯一的分类特征，评分星数作为预测目标，并将数据集划分为训练集和测试集。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 68,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# define X and y \n",
|
||
"X = yelp_best_worst.text\n",
|
||
"y = yelp_best_worst.stars\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 69,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# split into training and testing sets\n",
|
||
"from sklearn.model_selection import train_test_split\n",
|
||
"X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)\n",
|
||
"\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 第三步：转换数据\n",
|
||
"\n",
|
||
"使用CountVectorizer将X_train和X_test转换为document-term矩阵。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 70,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# import and instantiate the vectorizer\n",
|
||
"from sklearn.feature_extraction.text import CountVectorizer\n",
|
||
"vect = CountVectorizer()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 71,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# fit and transform X_train, but only transform X_test\n",
|
||
"X_train_dtm = vect.fit_transform(X_train)\n",
|
||
"X_test_dtm = vect.transform(X_test)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 第四步：训练、预测、评价\n",
|
||
"\n",
|
||
"使用朴素贝叶斯预测测试集中评论的星级评分，并计算预测精度。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 72,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"<style>#sk-container-id-3 {color: black;background-color: white;}#sk-container-id-3 pre{padding: 0;}#sk-container-id-3 div.sk-toggleable {background-color: white;}#sk-container-id-3 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-3 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-3 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-3 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-3 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-3 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-3 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-3 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-3 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-3 div.sk-item {position: relative;z-index: 1;}#sk-container-id-3 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-3 div.sk-item::before, #sk-container-id-3 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-3 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-3 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-3 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-3 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-3 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-3 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-3 div.sk-label-container {text-align: center;}#sk-container-id-3 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-3 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-3\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>MultinomialNB()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" checked><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">MultinomialNB</label><div class=\"sk-toggleable__content\"><pre>MultinomialNB()</pre></div></div></div></div></div>"
|
||
],
|
||
"text/plain": [
|
||
"MultinomialNB()"
|
||
]
|
||
},
|
||
"execution_count": 72,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# import/instantiate/fit\n",
|
||
"from sklearn.naive_bayes import MultinomialNB\n",
|
||
"nb = MultinomialNB()\n",
|
||
"nb.fit(X_train_dtm, y_train)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 73,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# make class predictions\n",
|
||
"y_pred_class = nb.predict(X_test_dtm)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 74,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"0.9187866927592955"
|
||
]
|
||
},
|
||
"execution_count": 74,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# calculate accuracy\n",
|
||
"from sklearn import metrics\n",
|
||
"metrics.accuracy_score(y_test, y_pred_class)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"计算AUC。注意：y_test中的取值是1和5，需要先把它转换为取值为0和1的二值数组y_test_binary。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 75,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# create y_test_binary from y_test, which contains ones and zeros instead\n",
|
||
"# of ones and fives\n",
|
||
"y_test_binary = y_test.map({5:1, 1:0})\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 76,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# predict class probabilities\n",
|
||
"y_pred_prob = nb.predict_proba(X_test_dtm)[:, 1]"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 77,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"0.9391635104285566"
|
||
]
|
||
},
|
||
"execution_count": 77,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# calculate the AUC using y_test_binary and y_pred_prob\n",
|
||
"metrics.roc_auc_score(y_test_binary, y_pred_prob)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"绘制ROC曲线。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 78,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"%matplotlib inline\n",
|
||
"import matplotlib.pyplot as plt"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 79,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAHHCAYAAABTMjf2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy89olMNAAAACXBIWXMAAA9hAAAPYQGoP6dpAABhp0lEQVR4nO3deVxUVf8H8M8MMjPsi8gqguK+JC5puIaiqOWSpbikqKllbkmWu7jkUpbZYpmaov4st7RMDVdQNLdU3FBcwDACFBARUBhmzu8PHyZHQBmcYYD5vF8vXo/33HPv/c49Q3yfc889RyKEECAiIiIyQVJjB0BERERkLEyEiIiIyGQxESIiIiKTxUSIiIiITBYTISIiIjJZTISIiIjIZDERIiIiIpPFRIiIiIhMFhMhIiIiMllMhIjIaFJSUvDWW2+hatWqkEgkWLZsmbFDKhFvb28MGzasTK8ZFhYGiUSCW7dulel1C9y6dQsSiQRhYWFa5eHh4fD19YVCoYBEIkFGRgaGDRsGb29vo8RJpCsmQkT4749MwU+VKlXg4eGBYcOGITExschjhBDYsGEDOnToAHt7e1haWqJJkyaYN28esrOzi73Wjh070L17dzg5OUEmk8Hd3R39+/fHoUOHDPXxyq1JkyZh7969mDZtGjZs2IBu3boZ5DqbNm2CRCLBDz/8UOT+MWPGwNzcHOfPnzfI9SurtLQ09O/fHxYWFli+fDk2bNgAKysrY4dFpJMqxg6AqDyZN28eatasiUePHuHEiRMICwvD0aNHcenSJSgUCk09lUqFQYMGYcuWLWjfvj3mzJkDS0tLREVFYe7cudi6dSsOHDgAFxcXzTFCCIwYMQJhYWFo1qwZQkJC4OrqiqSkJOzYsQOdO3fGsWPH0KZNG2N8dKM4dOgQevfujcmTJxv0OgMGDMC6deswdepU9OnTR6tdTp06hZUrV+LDDz9E06ZNDRpHRebl5YWHDx/C3NxcU3b69Gk8ePAA8+fPR0BAgKZ81apVUKvVxgiTSHeCiMTatWsFAHH69Gmt8ilTpggAYvPmzVrlCxcuFADE5MmTC51r586dQiqVim7dummVL1myRAAQH3zwgVCr1YWOW79+vTh58qQePk3pZWVllen1JBKJGDt2rN7O9/DhQ6FSqYrcFx8fLywtLcXAgQM1Zfn5+cLX11d4e3uL7OzsEl/Hy8tLBAcHv2i4Oin4jsbHx5fpdZ9l3bp1Rf7e6JNarRY5OTkGOz8REyEiUXwitGvXLgFALFy4UFOWk5MjHBwcRN26dYVSqSzyfMOHDxcAxPHjxzXHODo6ivr164v8/PxSx6lSqcSyZctE48aNhVwuF05OTiIwMFATd3x8vAAg1q5dW+hYACI0NFSzHRoaKgCIy5cvi4EDBwp7e3vh6+urSdhu3bpV6BxTp04V5ubmIj09XVN24sQJERgYKGxtbYWFhYXo0KGDOHr06DM/R8H9fvqnwM2bN8Vbb70lHBwchIWFhWjdurXYtWuX1jkiIiIEAPHzzz+LGTNmCHd3dyGRSMS9e/eKve5nn30mAIh9+/YJIYRYunSpACD++OMPIYQQjx49ErNnzxY+Pj5CJpOJ6tWri48++kg8evRI6zxPJ0IFn+fw4cNi9OjRwtHRUdjY2IghQ4Zo3atnuXLliujXr59wcnISCoVC1K1bV0yfPr3QNZ5MhH799VfRo0cP4ebmJmQymahVq5aYN29eoe/YtWvXRN++fYWLi4uQy+XCw8NDBAUFiYyMDE2dffv2ibZt2wo7OzthZWUl6tatK6ZNm6bZ//R3q2PHjoXar+CeBAcHCy8vL60YVCqV+PLLL0XDhg2FXC4Xzs7OYvTo0YXuj5eXl3jttddEeHi4aNGihZDL5eLLL78s0T0kKg0+GiN6hoKBqQ4ODpqyo0eP4t69e5g4cSKqVCn6V2jo0KFYu3Ytdu3ahVdeeQVHjx5Feno6PvjgA5iZmZU6nnfeeQdhYWHo3r07Ro4cifz8fERFReHEiRNo2bJlqc7Zr18/1KlTBwsXLoQQAq+//jo+/vhjbNmyBR999JFW3S1btqBr166a+3Ho0CF0794dLVq0QGhoKKRSKdauXYtOnTohKioKrVq1KvKaHTp0wIYNGzBkyBB06dIFQ4cO1exLSUlBmzZtkJOTgwkTJqBq1apYt24devXqhW3btuGNN97QOtf8+fMhk8kwefJk5ObmQiaTFftZJ02ahI0bN2LMmDEIDw/H7NmzMWDAAHTr1g1qtRq9evXC0aNHMXr0aDRo0AAXL17El19+iWvXruHXX3997r0cN24c7O3tMWfOHMTGxuL777/H33//jcjISEgkkmKPu3DhAtq3bw9zc3OMHj0a3t7euHnzJn7//XcsWLCg2OPCwsJgbW2NkJAQWFtb49ChQ5g9ezYyMzOxZMkSAEBeXh4CAwORm5uL8ePHw9XVFYmJidi1axcyMjJgZ2eHy5cv4/XXX8dLL72EefPmQS6X48aNGzh27Fix154xYwbq1auHlStXah4p+/j4FFv/3XffRVhYGIYPH44JEyYgPj4e3377Lc6dO4djx45pPXKLjY3FwIED8e6772LUqFGoV6/es2470YsxdiZGVB4U/L/tAwcOiLt374rbt2+Lbdu2iWrVqgm5XC5u376tqbts2TIBQOzYsaPY86WnpwsAom/fvkIIIb766qvnHvM8hw4dEgDEhAkTCu0reNRWmh6hJx8VFfDz8xMtWrTQKjt16pQAINavX6+5Zp06dURgYKDWo76cnBxRs2ZN0aVLl+d+JgCFHo198MEHAoCIiorSlD148EDUrFlTeHt7ax59FfQI1apVS6dHJydPnhRSqVQ4OjoKe3t7kZycLIQQYsOGDUIqlWpdVwghVqxYIQCIY8eOacqK6xFq0aKFyMvL05QX9ED99ttvz4ypQ4cOwsbGRvz9999a5U/e16J6hIr63O+++66wtLTU9GKdO3dOABBbt24t9vpffvmlACDu3r1bbJ2ivlvF9aQ+3SMUFRUlAIiNGzdq1QsPDy9U7uXlJQCI8PDwYmMh0ie+NUb0hICAAFSrVg2enp546623YGVlhZ07d6J69eqaOg8ePAAA2NjYFHuegn2ZmZla//usY57nl19+gUQiQWhoaKF9z+pteJ733nuvUFlQUBDOnDmDmzdvaso2b94MuVyO3r17AwCio6Nx/fp1DBo0CGlpaUhNTUVqaiqys7PRuXNnHDlypFQDZvfs2YNWrVqhXbt2mjJra2uMHj0at27dQkxMjFb94OBgWFhYlPj8rVq1wnvvvYf09HQsWrRIM3B669ataNCgAerXr6/5LKmpqejUqRMAICIi4rnnHj16tFbPxpgxY1ClShXs2bOn2GPu3r2LI0eOYMSIEahRo4bWvue165Of+8GDB0hNTUX79u2Rk5ODq1evAgDs7OwAAHv37kVOTk6R57G3twcA/PbbbwYZ5Lx161bY2dmhS5cuWve2RYsWsLa2LnRva9asicDAQL3HQVQUJkJET1i+fDn279+Pbdu2oUePHkhNTYVcLteqU5DMFCRERXk6WbK1tX3uMc9z8+ZNuLu7w9HRsdTnKErNmjULlfXr1w9SqRSbN28G8PiNt61bt6J79+6az3L9+nUAjxORatWqaf2sXr0aubm5uH//vs7x/P3330U+CmnQoIFm//Pif56XX34ZALQeJ16/fh2XL18u9Fnq1q0LALhz585zz1unTh2tbWtra7i5uT1z7p+4uDgAQOPGjXX9GLh8+TLeeOMN2NnZwdbWFtWqVcPbb78NAJp7X7NmTYSEhGD16tVwcnJCYGAgli9frtU2QUFBaNu2LUaOHAkXFxcMGDAAW7Zs0VtSdP36ddy/fx/Ozs6F7m9WVlahe1uaNiUqLY4RInpCq1atNH8c+/Tpg3bt2mHQoEGIjY2FtbU1gP/+IF+4cAF9+vQp8jwXLlwAADRs2BAAUL9+fQDAxYsXiz1GH4rrQVCpVMUeU1Rviru7O9q3b48tW7Zg+vTpOHHiBBISEvDpp59q6hT8kVyyZAl8fX2LPHfBPTMkXXqDnkWtVqNJkyZYunRpkfs9PT31ch19ycjIQMeOHWFra4t58+bBx8cHCoUCZ8+exZQpU7SSmC+++ALDhg3Db7/9hn379mHChAlYtGgRTpw4gerVq8PCwgJHjhxBREQEdu/ejfDwcGzevBmdOnXCvn37XmhcG/D43jo7O2Pjxo1F7q9WrZrWtr7alKgkmAgRFcPMzAyLFi2Cv78/vv32W0ydOhUA0K5dO9jb2+Onn37CjBkzivwjsX79egDA66+/rjnGwcEBP//8M6ZPn16qPyw+Pj7Yu3cv0tPTi+0VKhjEnJGRoVX+dC9KSQQFBeH9999HbGwsNm/eDEtLS/Ts2VMrHuBxb9eTc8i8KC8vL8TGxhYqL3jU4+XlpbdrPcnHxwfnz59H586dS/2o8fr16/D399dsZ2VlISkpCT169Cj2mFq1agEALl26pNO1IiMjkZaWhu3bt6NDhw6a8vj4+CLrN2nSBE2aNMHMmTPx559/om3btlixYgU++eQTAIBUKkXnzp3RuXNnLF26FAsXLsSMGTMQERHxwu3r4+ODAwcOoG3btkxyqNzhozGiZ3j11VfRqlUrLFu2DI8ePQIAWFpaYvLkyYiNjcWMGTMKHbN7926EhYUhMDAQr7zyiuaYKVOm4MqVK5gyZQqEEIWO+7//+z+cOnWq2FjefPNNCCEwd+7cQvsKzmdrawsnJyccOXJEa/93331X8g/9xPXMzMzw888/Y+vWrXj99de1Zg1u0aIFfHx88PnnnyMrK6vQ8Xfv3tX5mgDQo0cPnDp1CsePH9eUZWdnY+XKlfD29tb0sulb//79kZiYiFWrVhXa9/Dhw2fOFl5g5cqVUCqVmu3vv/8e+fn56N69e7HHVKtWDR06dMCaNWuQkJCgta+o70mBgmT6yTp5eXmF2jozMxP5+flaZU2aNIFUKkVubi4AID09vdD5C3r5Cuq8iP79+0OlUmH+/PmF9uXn5xdK3InKEnuEiJ7jo48+Qr9+/RAWFqYZWDx16lScO3cOn376KY4fP44333wTFhYWOHr0KP7v//4PDRo0wLp16wqd5/Lly/jiiy8QERGBt956C66urkhOTsavv/6KU6dO4c8//yw2Dn9/fwwZMgRff/01rl+/rnnlOyoqCv7+/hg3bhwAYOTIkVi8eDFGjhyJli1b4siRI7h27ZrOn9vZ2Rn+/v5YunQpHjx4gKCgIK39UqkUq1evRvfu3dGoUSMMHz4cHh4eSExMREREBGxtbfH777/rfN2pU6fi559/Rvfu3TFhwgQ4Ojpi3bp1iI+Pxy+//AKp1DD//23IkCHYsmUL3nvvPURERKBt27ZQqVS4evUqtmzZgr179z53ioK8vDx07twZ/fv3R2xsLL777ju0a9cOvXr1euZxX3/9Ndq1a4fmzZtj9OjRqFmzJm7duoXdu3cjOjq6yGPatGkDBwcHBAcHY8KECZBIJNiwYUOh5OnQoUMYN24c+vXrh7p16yI/Px8bNmyAmZkZ3nzzTQCPZ1Q/cuQIXnvtNXh5eeHOnTv47rvvUL16da1B66XVsWNHvPvuu1i0aBGio6PRtWtXmJub4/r169i6dSu++uorvPXWWy98HaJSMeIba0TlRnGvAQvxeCI4Hx8f4ePjozVRnUqlEmvXrhVt27YVtra2QqFQiEaNGom5c+c+c4bmbdu2ia5duwpHR0dRpUoV4ebmJoKCgkRkZORz48zPzxdLliwR9evXFzKZTFSrVk10795dnDlzRlMnJydHvPPOO8LOzk7Y2NiI/v37izt37hT7+vyzXpletWqVACBsbGzEw4cPi6xz7tw50bdvX1G1alUhl8uFl5eX6N+/vzh48OBzPw+KeH1eiP8mVLS3txcKhUK0atWq2AkVn/VaeHGKa++8vDzx6aefikaNGgm5XC4cHBxEixYtxNy5c8X9+/c19Z43oaKDg4OwtrYWgwcPFmlpaSWK6dKlS+KNN97QfOZ69eqJWbNmFbrGk6/PHzt2TLzyyivCwsJCuLu7i48//ljs3btXABARERFCCCHi4uLEiBEjhI+Pj1AoFMLR0VH4+/uLAwcOaM5z8OBB0bt3b+Hu7i5kMplwd3cXAwcOFNeuXdPUeZHX5wusXLlStGjRQlhYWAgbGxvRpEkT8fHHH4t///1X696+9tprJbpnRPogEeIZfa9ERPRcBRMFnj59utQTWxKRcXCMEBEREZksJkJERERkspgIERERkckyaiJ05MgR9OzZE+7u7pBIJCVa1DAyMhLNmzeHXC5H7dq1ERYWZvA4iYieZdiwYRBCcHwQUQVk1EQoOzsbTZs2xfLly0tUPz4+Hq+99hr8/f0RHR2NDz74ACNHjsTevXsNHCkRERFVRuXmrTGJRIIdO3Y8c/mBKVOmYPfu3VozsA4YMAAZGRkIDw8vgyiJiIioMqlQEyoeP3680FTvgYGB+OCDD4o9Jjc3V2tmVLVajfT0dFStWvWFVuwmIiKisiOEwIMHD+Du7q7XiVUrVCKUnJwMFxcXrTIXFxdkZmbi4cOHRa5hs2jRoiKXJCAiIqKK5/bt26hevbrezlehEqHSmDZtGkJCQjTb9+/fR40aNXDt2rViF66ksqFUKhEREQF/f3+Ym5sbOxyTx/YoveTMR4hLff5aZA9zVdhw8jas5M9edFeoBdLT0+DoWBUSqXF6ru/nKHE56YFRrk2Vl5lUgg3DW0JWRffv9f2MDHT184WNjY1eY6pQiZCrqytSUlK0ylJSUmBra1vsisZyuRxyubxQuaOjI6pWrWqQOKlklEolLC0tUbVqVf7hLQfYHrrZ8tdt/BgVj6zcfCRmPNTxaOXzq8Aa+PfFFzx9EVK5JQDAzuLFvw95+WpUtZahf0vPYut0qFsNdZytX/ha+qRUKrFv3z7N+mj0YizMzSAtZXKflvb4u6HvYS0VKhHy8/PDnj17tMr2798PPz8/I0VERJVNTl4+lPna75D8fuFf3Ml8pFX29aEbhY5t6Gb73PPnqdSo6WSFbo1ci62jUqlw/sJ5NH2pqWaVeWOQSAA/n6pwsyv6/2iaAqVUQG4GWMmrwNy8Qv3JpBIyaqtmZWXhxo3//mMSHx+P6OhoODo6okaNGpg2bRoSExOxfv16AMB7772Hb7/9Fh9//DFGjBiBQ4cOYcuWLdi9e7exPgIRGUhaVi6Sn0o+DO236H+x8kicTsd82KUuWno7opGHLWwV+ukxUCqVkCdFo0czd/ZCEBmYUROhv/76C/7+/prtgrE8wcHBCAsLQ1JSEhISEjT7a9asid27d2PSpEn46quvUL16daxevRqBgYFlHjsRPfbgkRKn4tOh1sNEHFv+uo3Mh0qkZD7CrbScFz+hngX7eWltezhY4J12tWBmpHE8RPTijJoIvfrqq3jWNEZFzRr96quv4ty5cwaMioieFHc3C0dvpOLJX9WYfzOx+2ISHK1kSEg3bMLibFN4jJ8hWcur4MsgXzRy137MVcWMKxIRVUZ84ElUgdx/qESuUlWqY3Pz1djy123k5au1yn84EgczqQQSAGphhg9P7tfan/+Mrp6s3HzNv+0tzeFd1apUsT19zgmd68BMIoGfT1U4Wsle+JxERMVhIkRUjqnUAl/si0VCeg7CLyU/Myl50es8JoG6mF7ajnWrwVrx338ycpUqvPaSG7yqWsHNTmHSA2qJqOJiIkRkBAlpObiQmKFVdvbvDBy7kao1x0z07Ywix96UdkyKSi1gb2le6BVmdzsFAuo74dChQ+jUqVOhAbp2FuZQmBvv7SUiIkNhIkSkJ2HH4vHDkbhie1QKqAVw94Hu88PM6dkQVvIqeO0lN1jK9P+rq1QqYScDXGwVfFOJiEwGEyGi/zl0NQVHrqWW+viwP2/pfExLLwet3p3MR/kY8LIn3OwUmjKpRIKXazrqZVI7IiLSxkSITNqFfzLww+E45KnU2B+T8vwDSmB+n8Zo5mn/3HreTlawlvNXkIjImPhfYarU9lxMwg+HbxY7yPjyv5mFyoa39S51guJhb4Gglz31PgU8EREZBhMhqnRy8vLxy5l/kJGjxBf7r5XomMBGLni1njMautmiaQl6c4iIqHJgIkSVxro/b+FqciZ+PnW70L4RbWuiY71qRR5X1UqGRu627MUhIjJBTISowom7m4Ule2ORnfffxIKJ93Jw8252oboDW3nCxVaB91+tDVkVzgxMRETamAhRhXHsRioS0nPwWfhV3MtRFlvvwy51oTA3wxvNPeBkXbbLMxARUcXCRIgqhBt3HmDw6pNaZR72Fviwa12tsta1qsLDnjMcExFRyTARonItN1+F6ylZuJR4HwBgKTND29pOsJZXwaSAuqhR1dLIERIRUUXGRIjKndx8FU7EpSNXqcLoDWe09rnbW2DV0JZGioyIiCobJkJU7nx14Dq+i7xZqNzNToGgp9bIIiIiehFMhMioLt+T4EJ4LMzM/lvQ89DVOwAejwFysZWjkbsd5vVuxNfbiYhI75gIUZm5lZqN9Jw8nL+dgT9vpkECgX1XzQD8XWT9sf61Mah1jbINkoiITAoTISoTR67dxdA1p4rdP6yNN+RPzPNjZ2mOXr7uZREaERGZMCZCZFBCCKjUAjfvZgEALMzNUM1GjuTMRxjRxgt3E25gYGAbtKzpZORIiYjIFDERIr0TQuCfew+hVKnRb8VxpGXnafb516+G7wa3AAAolUrs2XMdTavbGStUIiIycUyESO+m/nIRm/8qvN6XRAK08nY0QkRERERFYyJEenfxf5MfyqtIYSaVoEOdavj0zZdQxUwCKzm/ckREVH7wrxIZzMqhLdGxbtErvhMREZUHTITohWQ+UiLl/iOtskf5qmJqExERlS9MhKjU7mXnod2nh5Cdx8SHiIgqJiZCVGr/3HuI7DwVJBLAwVKmtc/D3gK+nvbGCYyIiKiEmAiRzhbsjsHGkwnIVwsAgJutAn9O62zkqIiIiHTHRIh0tvP8v8h54nFYE84DREREFRQTISoRpUqN4zfT8FCpwiOlGgCwdtjLqONiDXc7CyNHR0REVDpMhKhYQgjsi0nBvxkPsTziBlKz8rT2u9krUN3B0kjRERERvTgmQqSRlpULpUrgyPW7uJb8ABf+uY9Tt9IL1Wvh5YA6ztao62xjhCiJiIj0h4mQCVr35y2citdOcHZfTHrmMT2busNWUQUTO9eBs63CkOERERGVGSZCJiYrNx9zfr8MIYqvY24mgVIlMLpDLVSRStDL1x31XW3LLkgiIqIywkTIxKhUQpMEhfZsCDOpRLPP0UqGwEauMDeTGik6IiKissVEyIQNecULVZj0EBGRCeNfQSIiIjJZ7BEyEenZeTh87Q4ePMo3dihERETlBhMhEzF9+0WEX07WbJtJJZBIJM84goiIqPJjImQiUrNyAQCN3G1RzUaOjnWraQ2UJiIiMkVMhEzM+E610a2xm7HDICIiKhc4WJqIiIhMFnuEKrGL/9zHyPWncf+hErn5amOHQ0REVO4wEarEjt1MRUpmrmZbXkWKOi5cH4yIiKgAEyET0L2xK2a81gB2FuawUZgbOxwiIqJyg4lQJaNSC7y74QyuJGXiwSMlAMBSVgXVHSyNHBkREVH5w0SokrmVlo0DV1K0ympVszJSNEREROUbE6FKpmBBVWt5FWwc2RoKczPUdbE2blBERETlFBOhSmLLX7ex8WQCHuWpADyeObqpp71xgyIiIirnmAhVcPkqNdQCWHH4JuLuZmvK3ewURoyKiIioYmAiVIEdvJKCsT+dxSPlf3METelWH3VdrNG8hoMRIyMiIqoYmAhVYCfi0rSSoGo2cgxs5Ql7S5kRoyIiIqo4mAhVMI+UKkz4+RwS0nOQkvkIADC8rTcmdakLS3MzVDHjqilEREQlxUSogrmYeB/7Yp5+Pd4atpwokYiISGdMhCqIAzEp2BeTjLsPHi+Z4W6nwGdvNYWV3Ay+fDuMiIioVJgIVRDTdlzUJEEA4GqnQLs6TkaMiIiIqOJjIlRBPFI+nh9oVPuaqGotR5eGLkaOiIiIqOJjIlTBDGrthZpOXDKDiIhIH/iKEREREZksJkJERERksvhorBzLyMnDrgtJyM1XIy9f/fwDiIiISCdMhMqxrw5ex9pjt7TK5FXYiUdERKQvTITKsfs5SgBAI3db1Ha2RiN3W7jbWxg5KiIiosqDiVAF0MfXA6M61DJ2GERERJWO0Z+zLF++HN7e3lAoFGjdujVOnTr1zPrLli1DvXr1YGFhAU9PT0yaNAmPHj0qo2iJiIioMjFqIrR582aEhIQgNDQUZ8+eRdOmTREYGIg7d+4UWf+nn37C1KlTERoaiitXruDHH3/E5s2bMX369DKOnIiIiCoDoyZCS5cuxahRozB8+HA0bNgQK1asgKWlJdasWVNk/T///BNt27bFoEGD4O3tja5du2LgwIHP7UUiIiIiKorRxgjl5eXhzJkzmDZtmqZMKpUiICAAx48fL/KYNm3a4P/+7/9w6tQptGrVCnFxcdizZw+GDBlS7HVyc3ORm/vfGl2ZmZkAAKVSCaVSqadPYxhq9eNX5lVqVbmPtTQKPlNl/GwVEduj/GBblB9si/LDUG1gtEQoNTUVKpUKLi7aa2a5uLjg6tWrRR4zaNAgpKamol27dhBCID8/H++9994zH40tWrQIc+fOLVQeEREBS0vLF/sQBpaYKAUgxZUrV7DnfoyxwzGY/fv3GzsEegLbo/xgW5QfbAvjy8nJMch5K9RbY5GRkVi4cCG+++47tG7dGjdu3MDEiRMxf/58zJo1q8hjpk2bhpCQEM12ZmYmPD094e/vj6pVq5ZV6KUSse0iTqcmoUGDBujR1tvY4eidUqnE/v370aVLF5ibmxs7HJPH9ig/2BblB9ui/EhLSzPIeY2WCDk5OcHMzAwpKSla5SkpKXB1dS3ymFmzZmHIkCEYOXIkAKBJkybIzs7G6NGjMWPGDEilhYc8yeVyyOXyQuXm5ubl/ktd8HnMpGblPtYXURHawpSwPcoPtkX5wbYwPkPdf6MNlpbJZGjRogUOHjyoKVOr1Th48CD8/PyKPCYnJ6dQsmNmZgYAEEIYLlgiIiKqlIz6aCwkJATBwcFo2bIlWrVqhWXLliE7OxvDhw8HAAwdOhQeHh5YtGgRAKBnz55YunQpmjVrpnk0NmvWLPTs2VOTEBERERGVlFEToaCgINy9exezZ89GcnIyfH19ER4erhlAnZCQoNUDNHPmTEgkEsycOROJiYmoVq0aevbsiQULFhjrIxAREVEFZvTB0uPGjcO4ceOK3BcZGam1XaVKFYSGhiI0NLQMIiMiIqLK7oUSodzc3CIHIlPpqdUCuy8mISXzEa7fyTJ2OERERJWaTonQH3/8gU2bNiEqKgq3b9+GWq2GlZUVmjVrhq5du2L48OFwd3c3VKwm4URcGsb/fE6rTG5u9CXhiIiIKqUSJUI7duzAlClT8ODBA/To0QNTpkyBu7s7LCwskJ6ejkuXLuHAgQOYP38+hg0bhvnz56NatWqGjr1Synj4eObMqlYytK/jBHtLGV5/icklERGRIZQoEfrss8/w5Zdfonv37kXO1dO/f38AQGJiIr755hv83//9HyZNmqTfSE2Mj7M1lg1oZuwwiIiIKrUSJULFrf31NA8PDyxevPiFAiIiIiIqKzoPPomIiDBEHCYvN1+Fm3ezkHz/kbFDISIiMhk6vzXWrVs3VK9eHcOHD0dwcDA8PT0NEZdJyMtX48zf96BUqTF0zSljh0NERGRydE6EEhMTsWHDBqxbtw5z585Fp06d8M4776BPnz6QyWSGiLFS+edeDj7cch73cvJwLaXw6/FmUgnsLMzRqykHSBMRERmazo/GnJycMGnSJERHR+PkyZOoW7cu3n//fbi7u2PChAk4f/68IeKsNCJi7+JkfHqhJKi+qw1GtK2Jmwt74OysLnj7FS8jRUhERGQ6XmhCxebNm8PV1RVVq1bF4sWLsWbNGnz33Xfw8/PDihUr0KhRI33FWeE9eKTEP/ceIinjIQCgVU1HfNC5DqrZyFHHxcbI0REREZmmUs3Up1QqsW3bNvTo0QNeXl7Yu3cvvv32W6SkpODGjRvw8vJCv3799B1rhfVIqcKrSyLR/asofBd5EwBQzVqONrWdmAQREREZkc49QuPHj8fPP/8MIQSGDBmCzz77DI0bN9bst7Kywueff84Zpp+Qnp2HtOw8AEA1GzlkZlL0bOpm5KiIiIhI50QoJiYG33zzDfr27VvsOmNOTk58zb4IMjMpTs8IMHYYRERE9D86PxoLDQ1Fv379CiVB+fn5OHLkCIDHq8R37NhRPxESERERGYjOiZC/vz/S09MLld+/fx/+/v56CYqIiIioLOj8aEwIAYlEUqg8LS0NVlZWegmqslh//BZ+OByHPJXa2KEQERFREUqcCPXt2xcAIJFIMGzYMK1HYyqVChcuXECbNm30H2EFtvn0bST+73V54PFCqkRERFR+lDgRsrOzA/C4R8jGxgYWFhaafTKZDK+88gpGjRql/wgrMCEe/+/cXo3QwssBtZkIERERlSslToTWrl0LAPD29sbkyZP5GEwH3k5WaOxhZ+wwiIiI6Ck6jxEKDQ01RBxEREREZa5EiVDz5s1x8OBBODg4oFmzZkUOli5w9uxZvQVHREREZEglSoR69+6tGRzdu3fvZyZCRERERBVFiRKhJx+HzZkzx1CxVAqxyQ8wbfsFPHiUj7/Tc4wdDhERET2DzhMqjhw5EpGRkQYIpXL441ISziZk4PqdLOTlP54/qLqDxXOOIiIiImPQebD03bt30a1bN1SrVg0DBgzA22+/jaZNmxoitgolIycP/2Y8QkpmLgCgS0MXjGhbE+72CnhV5Rt2RERE5ZHOidBvv/2Ge/fuYevWrfjpp5+wdOlS1K9fH4MHD8agQYPg7e1tgDDLt/TsPLRdfAgPlSpNmZudAn4+VY0YFRERET2Pzo/GAMDBwQGjR49GZGQk/v77bwwbNgwbNmxA7dq19R1fhZB47yEeKlWQSgBnGzm8q1qiW2NXY4dFREREz6Fzj9CTlEol/vrrL5w8eRK3bt2Ci4uLvuKqkFxtFfhzWmdjh0FEREQlVKoeoYiICIwaNQouLi4YNmwYbG1tsWvXLvzzzz/6jo+IiIjIYHTuEfLw8EB6ejq6deuGlStXomfPnloLsBIRERFVFDonQnPmzEG/fv1gb29vgHCIiIiIyo7OiRBXmP/Pibg0hF9KRmpWrrFDISIiolIoUSLUt29fhIWFwdbWFn379n1m3e3bt+slsIpg2vaLiE/N1mzbKMyNGA0RERHpqkSJkJ2dnWZ9MVtbW6419j85efkAgKCWnnC2laNrQ74yT0REVJGUKBFau3at5t9hYWGGiqXCGtrGC43c7YwdBhEREelI59fnO3XqhIyMjELlmZmZ6NSpkz5iIiIiIioTOidCkZGRyMvLK1T+6NEjREVF6SUoIiIiorJQ4rfGLly4oPl3TEwMkpOTNdsqlQrh4eHw8PDQb3REREREBlTiRMjX1xcSiQQSiaTIR2AWFhb45ptv9BocERERkSGVOBGKj4+HEAK1atXCqVOnUK1aNc0+mUwGZ2dnmJmZGSRIIiIiIkMocSLk5eUFAFCr1QYLhoiIiKgslSgR2rlzJ7p37w5zc3Ps3LnzmXV79eqll8CIiIiIDK1EiVCfPn2QnJwMZ2dn9OnTp9h6EokEKpVKX7ERERERGVSJEqEnH4fx0RgRERFVFjrPI1SUoiZYJCIiIirvdE6EPv30U2zevFmz3a9fPzg6OsLDwwPnz5/Xa3BEREREhqRzIrRixQp4enoCAPbv348DBw4gPDwc3bt3x0cffaT3AMujO5mPkJCWA5VaGDsUIiIiegElfn2+QHJysiYR2rVrF/r374+uXbvC29sbrVu31nuA5c3PpxIwbftFY4dBREREeqBzj5CDgwNu374NAAgPD0dAQAAAQAhhEm+MXUq8DwAwN5PASmaGpp72qO1sbeSoiIiIqDR07hHq27cvBg0ahDp16iAtLQ3du3cHAJw7dw61a9fWe4Dl1Tj/OpgYUMfYYRAREdEL0DkR+vLLL+Ht7Y3bt2/js88+g7X1496QpKQkvP/++3oPkIiIiMhQdE6EzM3NMXny5ELlkyZN0ktARERERGVF50QIAK5fv46IiAjcuXOn0ASLs2fP1ktgRERERIamcyK0atUqjBkzBk5OTnB1dYVEItHsk0gkTISIiIiowtA5Efrkk0+wYMECTJkyxRDxEBEREZUZnV+fv3fvHvr162eIWIiIiIjKlM49Qv369cO+ffvw3nvvGSKecinzkRIj1/2FfzMe4l52nrHDISIiIj3RORGqXbs2Zs2ahRMnTqBJkyYwNzfX2j9hwgS9BVdenP37Hk7Fp2uV1axmZaRoiIiISF90ToRWrlwJa2trHD58GIcPH9baJ5FIKmUiVLCimE81K3zR3xc2iirwqcbZpImIiCo6nROh+Ph4Q8RRIVjKqsDX097YYRAREZGe6DxYukBeXh5iY2ORn5+vz3iIiIiIyozOiVBOTg7eeecdWFpaolGjRkhISAAAjB8/HosXL9Z7gERERESGonMiNG3aNJw/fx6RkZFQKBSa8oCAAGzevFmvwREREREZks6J0K+//opvv/0W7dq105pVulGjRrh586bOASxfvhze3t5QKBRo3bo1Tp069cz6GRkZGDt2LNzc3CCXy1G3bl3s2bNH5+sSERER6TxY+u7du3B2di5Unp2drZUYlcTmzZsREhKCFStWoHXr1li2bBkCAwMRGxtb5DXy8vLQpUsXODs7Y9u2bfDw8MDff/8Ne3t7XT8GERERke49Qi1btsTu3bs12wXJz+rVq+Hn56fTuZYuXYpRo0Zh+PDhaNiwIVasWAFLS0usWbOmyPpr1qxBeno6fv31V7Rt2xbe3t7o2LEjmjZtquvHICIiItK9R2jhwoXo3r07YmJikJ+fj6+++goxMTH4888/C80r9Cx5eXk4c+YMpk2bpimTSqUICAjA8ePHizxm586d8PPzw9ixY/Hbb7+hWrVqGDRoEKZMmQIzM7Mij8nNzUVubq5mOzMzEwCgVCqhVCpLFKvqf2/GCSFKfAw9X8G95D0tH9ge5QfbovxgW5QfhmoDnROhdu3aITo6GosXL0aTJk2wb98+NG/eHMePH0eTJk1KfJ7U1FSoVCq4uLholbu4uODq1atFHhMXF4dDhw5h8ODB2LNnD27cuIH3338fSqUSoaGhRR6zaNEizJ07t1B5REQELC0tSxRrzD0JADPcv3+f45EMYP/+/cYOgZ7A9ig/2BblB9vC+HJycgxyXp0TIQDw8fHBqlWr9B3Lc6nVajg7O2PlypUwMzNDixYtkJiYiCVLlhSbCE2bNg0hISGa7czMTHh6esLf3x9Vq1Yt0XWtrt3FD1fPwc7ODj16vKKXz0KPs/v9+/ejS5cuhZZqobLH9ig/2BblB9ui/EhLSzPIeUucCOXn50OlUkEul2vKUlJSsGLFCmRnZ6NXr15o165diS/s5OQEMzMzpKSkaJWnpKTA1dW1yGPc3Nxgbm6u9RisQYMGSE5ORl5eHmQyWaFj5HK5VswFzM3NS/ylNqvy+DZJJBL+IhiALm1Bhsf2KD/YFuUH28L4DHX/SzxYetSoUVrriD148AAvv/wyli9fjr1798Lf31+nx0YymQwtWrTAwYMHNWVqtRoHDx4sdtB127ZtcePGDajVak3ZtWvX4ObmVmQSRERERPQsJU6Ejh07hjfffFOzvX79eqhUKly/fh3nz59HSEgIlixZotPFQ0JCsGrVKqxbtw5XrlzBmDFjkJ2djeHDhwMAhg4dqjWYesyYMUhPT8fEiRNx7do17N69GwsXLsTYsWN1ui4RERERoMOjscTERNSpU0ezffDgQbz55puws7MDAAQHB2Pt2rU6XTwoKAh3797F7NmzkZycDF9fX4SHh2sGUCckJEAq/S9X8/T0xN69ezFp0iS89NJL8PDwwMSJEzFlyhSdrktEREQE6JAIKRQKPHz4ULN94sQJrR4ghUKBrKwsnQMYN24cxo0bV+S+yMjIQmV+fn44ceKEztchIiIielqJH435+vpiw4YNAICoqCikpKSgU6dOmv03b96Eu7u7/iMkIiIiMpAS9wjNnj0b3bt3x5YtW5CUlIRhw4bBzc1Ns3/Hjh1o27atQYIkIiIiMoQSJ0IdO3bEmTNnsG/fPri6uqJfv35a+319fdGqVSu9B0hERERkKDpNqNigQQM0aNCgyH2jR4/WS0BEREREZaVEY4R0GZyck5ODy5cvlzqg8iTubha2/HUbUddSjR0KERERGUCJEqEhQ4YgMDAQW7duRXZ2dpF1YmJiMH36dPj4+ODMmTN6DdJYBq06iY+3XcCaY/EAAHMziZEjIiIiIn0q0aOxmJgYfP/995g5cyYGDRqEunXrwt3dHQqFAvfu3cPVq1eRlZWFN954A/v27dNp8dXyLDXr8ar1bXyqwlJmhrdf8TJyRERERKRPJUqEzM3NMWHCBEyYMAF//fUXjh49ir///hsPHz5E06ZNMWnSJPj7+8PR0dHQ8RrFl0G+cLFVGDsMIiIi0jOdV59v2bIlWrZsaYhYiIiIiMpUiSdUJCIiIqpsmAgRERGRyWIiRERERCaLiRARERGZrBdKhB49eqSvOIiIiIjKnM6JkFqtxvz58+Hh4QFra2vExcUBAGbNmoUff/xR7wESERERGYrOidAnn3yCsLAwfPbZZ5DJZJryxo0bY/Xq1XoNjoiIiMiQdE6E1q9fj5UrV2Lw4MEwMzPTlDdt2hRXr17Va3BEREREhqRzIpSYmIjatWsXKler1VAqlXoJioiIiKgs6JwINWzYEFFRUYXKt23bhmbNmuklKCIiIqKyoPMSG7Nnz0ZwcDASExOhVquxfft2xMbGYv369di1a5chYiQiIiIyCJ17hHr37o3ff/8dBw4cgJWVFWbPno0rV67g999/R5cuXQwRIxEREZFB6NwjBADt27fH/v379R0LERERUZnSuUeoVq1aSEtLK1SekZGBWrVq6SUoIiIiorKgcyJ069YtqFSqQuW5ublITEzUS1BEREREZaHEj8Z27typ+ffevXthZ2en2VapVDh48CC8vb31GhwRERGRIZU4EerTpw8AQCKRIDg4WGufubk5vL298cUXX+g1OCIiIiJDKnEipFarAQA1a9bE6dOn4eTkZLCgiIiIiMqCzm+NxcfHGyIOIiIiojJXqtfns7OzcfjwYSQkJCAvL09r34QJE/QSGBEREZGh6ZwInTt3Dj169EBOTg6ys7Ph6OiI1NRUWFpawtnZmYkQERERVRg6vz4/adIk9OzZE/fu3YOFhQVOnDiBv//+Gy1atMDnn39uiBiJiIiIDELnRCg6OhoffvghpFIpzMzMkJubC09PT3z22WeYPn26IWIkIiIiMgidEyFzc3NIpY8Pc3Z2RkJCAgDAzs4Ot2/f1m90RERERAak8xihZs2a4fTp06hTpw46duyI2bNnIzU1FRs2bEDjxo0NESMRERGRQejcI7Rw4UK4ubkBABYsWAAHBweMGTMGd+/exQ8//KD3AImIiIgMReceoZYtW2r+7ezsjPDwcL0GRERERFRWdO4RKs7Zs2fx+uuv6+t0RERERAanUyK0d+9eTJ48GdOnT0dcXBwA4OrVq+jTpw9efvllzTIcRERERBVBiR+N/fjjjxg1ahQcHR1x7949rF69GkuXLsX48eMRFBSES5cuoUGDBoaMlYiIiEivStwj9NVXX+HTTz9FamoqtmzZgtTUVHz33Xe4ePEiVqxYwSSIiIiIKpwSJ0I3b95Ev379AAB9+/ZFlSpVsGTJElSvXt1gwREREREZUokToYcPH8LS0hIAIJFIIJfLNa/RExEREVVEOr0+v3r1alhbWwMA8vPzERYWBicnJ606XHSViIiIKooSJ0I1atTAqlWrNNuurq7YsGGDVh2JRMJEiIiIiCqMEidCt27dMmAYRERERGVPbxMqEhEREVU0TISIiIjIZDERIiIiIpPFRIiIiIhMFhMhIiIiMlmlSoRu3ryJmTNnYuDAgbhz5w4A4I8//sDly5f1GhwRERGRIemcCB0+fBhNmjTByZMnsX37dmRlZQEAzp8/j9DQUL0HSERERGQoOidCU6dOxSeffIL9+/dDJpNpyjt16oQTJ07oNTgiIiIiQ9I5Ebp48SLeeOONQuXOzs5ITU3VS1BEREREZUHnRMje3h5JSUmFys+dOwcPDw+9BEVERERUFnROhAYMGIApU6YgOTkZEokEarUax44dw+TJkzF06FBDxEhERERkEDonQgsXLkT9+vXh6emJrKwsNGzYEB06dECbNm0wc+ZMQ8RIREREZBAlXnS1gEwmw6pVqzBr1ixcunQJWVlZaNasGerUqWOI+IiIiIgMRudE6OjRo2jXrh1q1KiBGjVqGCImIiIiojKh86OxTp06oWbNmpg+fTpiYmIMERMRERFRmdA5Efr333/x4Ycf4vDhw2jcuDF8fX2xZMkS/PPPP4aIj4iIiMhgdE6EnJycMG7cOBw7dgw3b95Ev379sG7dOnh7e6NTp06GiJGIiIjIIF5o0dWaNWti6tSpWLx4MZo0aYLDhw/rKy4iIiIigyt1InTs2DG8//77cHNzw6BBg9C4cWPs3r27VOdavnw5vL29oVAo0Lp1a5w6dapEx23atAkSiQR9+vQp1XWJiIjItOmcCE2bNg01a9ZEp06dkJCQgK+++grJycnYsGEDunXrpnMAmzdvRkhICEJDQ3H27Fk0bdoUgYGBmlXti3Pr1i1MnjwZ7du31/maREREREApEqEjR47go48+QmJiInbt2oWBAwfC0tKy1AEsXboUo0aNwvDhw9GwYUOsWLEClpaWWLNmTbHHqFQqDB48GHPnzkWtWrVKfW0iIiIybTrPI3Ts2DG9XTwvLw9nzpzBtGnTNGVSqRQBAQE4fvx4scfNmzcPzs7OeOeddxAVFfXMa+Tm5iI3N1eznZmZCQBQKpVQKpUlivNxXbMS1aWSK7j/JW0HMiy2R/nBtig/2Bblh6HaoESJ0M6dO9G9e3eYm5tj586dz6zbq1evEl88NTUVKpUKLi4uWuUuLi64evVqkcccPXoUP/74I6Kjo0t0jUWLFmHu3LmFyiMiIp7bk6UWZgAkOHToEOxkJboclcL+/fuNHQI9ge1RfrAtyg+2hfHl5OQY5LwlSoT69OmD5ORkODs7P3NgskQigUql0ldshTx48ABDhgzBqlWr4OTkVKJjpk2bhpCQEM12ZmYmPD094e/vj6pVqz7z2A9P7odaCHTq1AkutooXip0KUyqV2L9/P7p06QJzc3Njh2Py2B7lB9ui/GBblB9paWkGOW+JEiG1Wl3kv1+Uk5MTzMzMkJKSolWekpICV1fXQvVv3ryJW7duoWfPnoXiqVKlCmJjY+Hj46N1jFwuh1wuL3Quc3PzEn+pdalLuuP9LV/YHuUH26L8YFsYn6Huv86DpdevX6815qZAXl4e1q9fr9O5ZDIZWrRogYMHD2rK1Go1Dh48CD8/v0L169evj4sXLyI6Olrz06tXL/j7+yM6Ohqenp66fhwiIiIyYTonQsOHD8f9+/cLlT948ADDhw/XOYCQkBCsWrUK69atw5UrVzBmzBhkZ2drzjV06FDNYGqFQoHGjRtr/djb28PGxgaNGzeGTMaBPERERFRyOr81JoSARCIpVP7PP//Azs5O5wCCgoJw9+5dzJ49G8nJyfD19UV4eLhmAHVCQgKk0heaAJuIiIioSCVOhJo1awaJRAKJRILOnTujSpX/DlWpVIiPjy/VhIoAMG7cOIwbN67IfZGRkc88NiwsrFTXJCIiIipxIlTwtlh0dDQCAwNhbW2t2SeTyeDt7Y0333xT7wESERERGUqJE6HQ0FAAgLe3N4KCgqBQ8HVyIiIiqth0HiMUHBxsiDiIiIiIylyJEiFHR0dcu3YNTk5OcHBwKHKwdIH09HS9BUdERERkSCVKhL788kvY2Nho/v2sRIiIiIiooihRIvTk47Bhw4YZKhYiIiKiMqXzBD1nz57FxYsXNdu//fYb+vTpg+nTpyMvL0+vwREREREZks6J0Lvvvotr164BAOLi4hAUFARLS0ts3boVH3/8sd4DJCIiIjIUnROha9euwdfXFwCwdetWdOzYET/99BPCwsLwyy+/6Ds+IiIiIoPRORESQmhWfD9w4AB69OgBAPD09ERqaqp+oyMiIiIyIJ0ToZYtW+KTTz7Bhg0bcPjwYbz22msAgPj4eM36YEREREQVgc6J0LJly3D27FmMGzcOM2bMQO3atQEA27ZtQ5s2bfQeIBEREZGh6Dyz9EsvvaT11liBJUuWwMzMTC9BEREREZUFnROhAmfOnMGVK1cAAA0bNkTz5s31FhQRERFRWdA5Ebpz5w6CgoJw+PBh2NvbAwAyMjLg7++PTZs2oVq1avqOkYiIiMggdB4jNH78eGRlZeHy5ctIT09Heno6Ll26hMzMTEyYMMEQMRIREREZhM49QuHh4Thw4AAaNGigKWvYsCGWL1+Orl276jU4IiIiIkPSuUdIrVbD3Ny8ULm5ublmfiEiIiKiikDnRKhTp06YOHEi/v33X01ZYmIiJk2ahM6dO+s1OCIiIiJD0jkR+vbbb5GZmQlvb2/4+PjAx8cHNWvWRGZmJr755htDxEhERERkEDqPEfL09MTZs2dx8OBBzevzDRo0QEBAgN6DIyIiIjIknRKhzZs3Y+fOncjLy0Pnzp0xfvx4Q8VFREREZHAlToS+//57jB07FnXq1IGFhQW2b9+OmzdvYsmSJYaMz2iUKjWEsYMgIiIigyrxGKFvv/0WoaGhiI2NRXR0NNatW4fvvvvOkLEZzeV/76P3t8egUgtIJIDCnEuHEBERVUYlToTi4uIQHBys2R40aBDy8/ORlJRkkMCM5bfoRPT+9hhikjJhb2mObwY2g51F4ekCiIiIqOIr8aOx3NxcWFlZabalUilkMhkePnxokMCM5Y+LychXC7Sv44Sl/X1RzUZu7JCIiIjIQHQaLD1r1ixYWlpqtvPy8rBgwQLY2dlpypYuXaq/6IwosJErkyAiIqJKrsSJUIcOHRAbG6tV1qZNG8TFxWm2JRKJ/iIjIiIiMrASJ0KRkZEGDIOIiIio7Ok8szQRERFRZcFEiIiIiEwWEyEiIiIyWUyEiIiIyGQxESIiIiKTVapEKCoqCm+//Tb8/PyQmJgIANiwYQOOHj2q1+CIiIiIDEnnROiXX35BYGAgLCwscO7cOeTm5gIA7t+/j4ULF+o9QCIiIiJD0TkR+uSTT7BixQqsWrUK5ub/rcHVtm1bnD17Vq/BERERERmSzolQbGwsOnToUKjczs4OGRkZ+oiJiIiIqEzonAi5urrixo0bhcqPHj2KWrVq6SUoIiIiorKgcyI0atQoTJw4ESdPnoREIsG///6LjRs3YvLkyRgzZowhYiQiIiIyCJ1WnweAqVOnQq1Wo3PnzsjJyUGHDh0gl8sxefJkjB8/3hAxEhERERmEzomQRCLBjBkz8NFHH+HGjRvIyspCw4YNYW1tbYj4iIiIiAxG50SogEwmQ8OGDfUZCxEREVGZ0jkR8vf3h0QiKXb/oUOHXiggIiIiorKicyLk6+urta1UKhEdHY1Lly4hODhYX3ERERERGZzOidCXX35ZZPmcOXOQlZX1wgEZW75aAACkz+j1IiIiospBb4uuvv3221izZo2+Tmc093LyAACOVubPqUlEREQVnd4SoePHj0OhUOjrdEaTlvV47bSq1nIjR0JERESGpvOjsb59+2ptCyGQlJSEv/76C7NmzdJbYMaSlvW4R6iqlczIkRAREZGh6ZwI2dnZaW1LpVLUq1cP8+bNQ9euXfUWmDHk5qvwIDcfAFDVij1CRERElZ1OiZBKpcLw4cPRpEkTODg4GComo0nPftwbVEUqga1FqadYIiIiogpCpzFCZmZm6Nq1a6VdZV7zWMxa9sy5koiIiKhy0HmwdOPGjREXF2eIWIwu9X8DpR35WIyIiMgk6JwIffLJJ5g8eTJ27dqFpKQkZGZmav1UZAWPxpysOVCaiIjIFJR4IMy8efPw4YcfokePHgCAXr16aT0+EkJAIpFApVLpP8oywjfGiIiITEuJE6G5c+fivffeQ0REhCHjMarUbM4hREREZEpKnAgJ8XjpiY4dOxosGGMr6BFyZI8QERGRSdBpjFBlf5OKY4SIiIhMi06T5dStW/e5yVB6evoLBWRMmuU1+NYYERGRSdApEZo7d26hmaUrk9Qn5hEiIiKiyk+nRGjAgAFwdnY2VCxGl5bNHiEiIiJTUuIxQpV9fFBOXj4eKdUA2CNERERkKkqcCBW8NVZZFbwxpjCXwlJmZuRoiIiIqCyUOBFSq9UGeyy2fPlyeHt7Q6FQoHXr1jh16lSxdVetWoX27dvDwcEBDg4OCAgIeGb9kkp9YqB0Ze/9IiIiosd0XmJD3zZv3oyQkBCEhobi7NmzaNq0KQIDA3Hnzp0i60dGRmLgwIGIiIjA8ePH4enpia5duyIxMfGF4kjjQGkiIiKTY/REaOnSpRg1ahSGDx+Ohg0bYsWKFbC0tMSaNWuKrL9x40a8//778PX1Rf369bF69Wqo1WocPHjwheIomEOIy2sQERGZDp3eGtO3vLw8nDlzBtOmTdOUSaVSBAQE4Pjx4yU6R05ODpRKJRwdHYvcn5ubi9zcXM12wcKwSqUSSqVSU34n8yEAwMHSXKucDKfgPvN+lw9sj/KDbVF+sC3KD0O1gVETodTUVKhUKri4uGiVu7i44OrVqyU6x5QpU+Du7o6AgIAi9y9atAhz584tVB4REQFLS0vN9l+3pACkyEj5B3v2JJT8Q9AL279/v7FDoCewPcoPtkX5wbYwvpycHIOc16iJ0ItavHgxNm3ahMjISCgUiiLrTJs2DSEhIZrtzMxMeHp6wt/fH1WrVtWUH9p2EUhKwstN6qNHO29Dh054nN3v378fXbp0gbm5ubHDMXlsj/KDbVF+sC3Kj7S0NIOc16iJkJOTE8zMzJCSkqJVnpKSAldX12ce+/nnn2Px4sU4cOAAXnrppWLryeVyyOWFJ0g0NzfX+lKn5zzucnO2teCXvYw93RZkXGyP8oNtUX6wLYzPUPffqIOlZTIZWrRooTXQuWDgs5+fX7HHffbZZ5g/fz7Cw8PRsmVLvcTCt8aIiIhMj9EfjYWEhCA4OBgtW7ZEq1atsGzZMmRnZ2P48OEAgKFDh8LDwwOLFi0CAHz66aeYPXs2fvrpJ3h7eyM5ORkAYG1tDWtr61LHweU1iIiITI/RE6GgoCDcvXsXs2fPRnJyMnx9fREeHq4ZQJ2QkACp9L+Oq++//x55eXl46623tM4TGhqKOXPmlCoGIcR/r8+zR4iIiMhkGD0RAoBx48Zh3LhxRe6LjIzU2r5165ber5/5KB9K1eMlRBw5jxAREZHJMPqEiuVB2v+W17CWV4HCnOuMERERmQomQgDS+FiMiIjIJDERwhNvjPGxGBERkUlhIoQn3hiz5htjREREpoSJENgjREREZKqYCOG/wdIcI0RERGRamAjhicHSnEyRiIjIpDARApfXICIiMlVMhMDlNYiIiEwVEyGAy2sQERGZKJNPhFRqrjNGRERkqkw+EcrIyYP68TJjcLBkIkRERGRKTD4RKnhjzN7SHOZmJn87iIiITIrJ/+XnZIpERESmi4kQl9cgIiIyWUyE2CNERERkspgIcXkNIiIik8VEiMtrEBERmSwmQv97NObEHiEiIiKTw0Tof4OlHdkjREREZHKYCHHBVSIiIpPFRCibj8aIiIhMlUknQnn5atx/qATAR2NERESmyKQToXs5j3uDpBLA3sLcyNEQERFRWTPpRKhgfJCjlRxSqcTI0RAREVFZM+1E6H9vjHF8EBERkWky7URI0yPERIiIiMgUmXQilJrFBVeJiIhMmUknQunZXHCViIjIlJl0IsTlNYiIiEybaSdCXF6DiIjIpJl0IpTK5TWIiIhMmkknQulcXoOIiMikmXQilJbFR2NERESmzGQToYd5KmTnqQDw0RgREZGpMtlE6N7Dx4/FZGZS2MirGDkaIiIiMgbTTYSyH686X9VaBomE64wRERGZIpNNhNJzuLwGERGRqTPdRKhgVmkur0FERGSyTDgRevxozIk9QkRERCbLZBOhezmcTJGIiMjUmW4ilF0wRoiPxoiIiEyVySZC6Tn/vTVGREREpslkE6F7/0uEuLwGERGR6TLZRCidj8aIiIhMnukmQgWPxvjWGBERkcky2URIma8GwDFCREREpsxkEyEAsJSZwVLGdcaIiIhMlUknQlxeg4iIyLSZdCLE5TWIiIhMm0knQlxeg4iIyLSZdCLEgdJERESmzaQTIc4hREREZNpMOhHirNJERESmzaQTIT4aIyIiMm0mnQjx0RgREZFpM+lEiMtrEBERmTaTToScOI8QERGRSTPpRIgzSxMREZk2k02ErBVmkFUx2Y9PREREMOFEyNGSvUFERESmjokQERERmaxykQgtX74c3t7eUCgUaN26NU6dOvXM+lu3bkX9+vWhUCjQpEkT7NmzR+drOliZlzZcIiIiqiSMnght3rwZISEhCA0NxdmzZ9G0aVMEBgbizp07Rdb/888/MXDgQLzzzjs4d+4c+vTpgz59+uDSpUs6XdeBPUJEREQmz+iJ0NKlSzFq1CgMHz4cDRs2xIoVK2BpaYk1a9YUWf+rr75Ct27d8NFHH6FBgwaYP38+mjdvjm+//Van6/KNMSIiIjJqIpSXl4czZ84gICBAUyaVShEQEIDjx48Xeczx48e16gNAYGBgsfWL48hHY0RERCavijEvnpqaCpVKBRcXF61yFxcXXL16tchjkpOTi6yfnJxcZP3c3Fzk5uZqtu/fvw8AMFfmIC0t7UXCpxekVCqRk/O4HczNmZgaG9uj/GBblB9si/IjPT0dACCE0Ot5jZoIlYVFixZh7ty5hcrf69UW7xkhHiIiIiq9tLQ02NnZ6e18Rk2EnJycYGZmhpSUFK3ylJQUuLq6FnmMq6urTvWnTZuGkJAQzXZGRga8vLyQkJCg1xtJusvMzISnpydu374NW1tbY4dj8tge5QfbovxgW5Qf9+/fR40aNeDo6KjX8xo1EZLJZGjRogUOHjyIPn36AADUajUOHjyIcePGFXmMn58fDh48iA8++EBTtn//fvj5+RVZXy6XQy4vvKaYnZ0dv9TlhK2tLduiHGF7lB9si/KDbVF+SKX6Hd5s9EdjISEhCA4ORsuWLdGqVSssW7YM2dnZGD58OABg6NCh8PDwwKJFiwAAEydORMeOHfHFF1/gtddew6ZNm/DXX39h5cqVxvwYREREVAEZPREKCgrC3bt3MXv2bCQnJ8PX1xfh4eGaAdEJCQla2V+bNm3w008/YebMmZg+fTrq1KmDX3/9FY0bNzbWRyAiIqIKyuiJEACMGzeu2EdhkZGRhcr69euHfv36lepacrkcoaGhRT4uo7LFtihf2B7lB9ui/GBblB+GaguJ0Pd7aEREREQVhNFnliYiIiIyFiZCREREZLKYCBEREZHJYiJEREREJqtSJkLLly+Ht7c3FAoFWrdujVOnTj2z/tatW1G/fn0oFAo0adIEe/bsKaNIKz9d2mLVqlVo3749HBwc4ODggICAgOe2HelG19+NAps2bYJEItFMfEovTte2yMjIwNixY+Hm5ga5XI66devyv1V6omtbLFu2DPXq1YOFhQU8PT0xadIkPHr0qIyirbyOHDmCnj17wt3dHRKJBL/++utzj4mMjETz5s0hl8tRu3ZthIWF6X5hUcls2rRJyGQysWbNGnH58mUxatQoYW9vL1JSUoqsf+zYMWFmZiY+++wzERMTI2bOnCnMzc3FxYsXyzjyykfXthg0aJBYvny5OHfunLhy5YoYNmyYsLOzE//8808ZR1456doeBeLj44WHh4do37696N27d9kEW8np2ha5ubmiZcuWokePHuLo0aMiPj5eREZGiujo6DKOvPLRtS02btwo5HK52Lhxo4iPjxd79+4Vbm5uYtKkSWUceeWzZ88eMWPGDLF9+3YBQOzYseOZ9ePi4oSlpaUICQkRMTEx4ptvvhFmZmYiPDxcp+tWukSoVatWYuzYsZptlUol3N3dxaJFi4qs379/f/Haa69plbVu3Vq8++67Bo3TFOjaFk/Lz88XNjY2Yt26dYYK0aSUpj3y8/NFmzZtxOrVq0VwcDATIT3RtS2+//57UatWLZGXl1dWIZoMXdti7NixolOnTlplISEhom3btgaN09SUJBH6+OOPRaNGjbTKgoKCRGBgoE7XqlSPxvLy8nDmzBkEBARoyqRSKQICAnD8+PEijzl+/LhWfQAIDAwstj6VTGna4mk5OTlQKpV6X2DPFJW2PebNmwdnZ2e88847ZRGmSShNW+zcuRN+fn4YO3YsXFxc0LhxYyxcuBAqlaqswq6UStMWbdq0wZkzZzSPz+Li4rBnzx706NGjTGKm/+jr73e5mFlaX1JTU6FSqTTLcxRwcXHB1atXizwmOTm5yPrJyckGi9MUlKYtnjZlyhS4u7sX+qKT7krTHkePHsWPP/6I6OjoMojQdJSmLeLi4nDo0CEMHjwYe/bswY0bN/D+++9DqVQiNDS0LMKulErTFoMGDUJqairatWsHIQTy8/Px3nvvYfr06WURMj2huL/fmZmZePjwISwsLEp0nkrVI0SVx+LFi7Fp0ybs2LEDCoXC2OGYnAcPHmDIkCFYtWoVnJycjB2OyVOr1XB2dsbKlSvRokULBAUFYcaMGVixYoWxQzM5kZGRWLhwIb777jucPXsW27dvx+7duzF//nxjh0alVKl6hJycnGBmZoaUlBSt8pSUFLi6uhZ5jKurq071qWRK0xYFPv/8cyxevBgHDhzASy+9ZMgwTYau7XHz5k3cunULPXv21JSp1WoAQJUqVRAbGwsfHx/DBl1JleZ3w83NDebm5jAzM9OUNWjQAMnJycjLy4NMJjNozJVVadpi1qxZGDJkCEaOHAkAaNKkCbKzszF69GjMmDFDa5FwMqzi/n7b2tqWuDcIqGQ9QjKZDC1atMDBgwc1ZWq1GgcPHoSfn1+Rx/j5+WnVB4D9+/cXW59KpjRtAQCfffYZ5s+fj/DwcLRs2bIsQjUJurZH/fr1cfHiRURHR2t+evXqBX9/f0RHR8PT07Msw69USvO70bZtW9y4cUOTjALAtWvX4ObmxiToBZSmLXJycgolOwUJquDSnWVKb3+/dRvHXf5t2rRJyOVyERYWJmJiYsTo0aOFvb29SE5OFkIIMWTIEDF16lRN/WPHjokqVaqIzz//XFy5ckWEhoby9Xk90bUtFi9eLGQymdi2bZtISkrS/Dx48MBYH6FS0bU9nsa3xvRH17ZISEgQNjY2Yty4cSI2Nlbs2rVLODs7i08++cRYH6HS0LUtQkNDhY2Njfj5559FXFyc2Ldvn/Dx8RH9+/c31keoNB48eCDOnTsnzp07JwCIpUuXinPnzom///5bCCHE1KlTxZAhQzT1C16f/+ijj8SVK1fE8uXL+fp8gW+++UbUqFFDyGQy0apVK3HixAnNvo4dO4rg4GCt+lu2bBF169YVMplMNGrUSOzevbuMI668dGkLLy8vAaDQT2hoaNkHXknp+rvxJCZC+qVrW/z555+idevWQi6Xi1q1aokFCxaI/Pz8Mo66ctKlLZRKpZgzZ47w8fERCoVCeHp6ivfff1/cu3ev7AOvZCIiIor8G1Bw/4ODg0XHjh0LHePr6ytkMpmoVauWWLt2rc7XlQjBvjwiIiIyTZVqjBARERGRLpgIERERkcliIkREREQmi4kQERERmSwmQkRERGSymAgRERGRyWIiRERERCaLiRBRKYSFhcHe3t7YYZSaRCLBr7/++sw6w4YNQ58+fcoknvJm1qxZGD16tLHDKDNz5syBr69voTIXFxfNd0WX78OtW7cgkUgQHR39QnGtWLFCa707IoN40ZkgiSqq4ODgImcxvX79+nOPXbt2rbCzszNYbGvXrtXEI5FIhIeHhxg2bJhISUnRy/mTkpLEo0ePhBBCxMfHCwDi3LlzWnUyMjIMPltuaGio5nNKpVJRvXp1MWrUKJGWlqbTefQ563VSUpKwsbERt27d0pQdPnxYvP7668LNzU0AEDt27NDLtYQQYvv27aJ169bC1tZWWFtbi4YNG4qJEyfq7fwl8eDBA5GamqrZjomJ0XzOgu+KLt+H/Px8kZSUJJRKpRDivxmDdf0+5ebmCnd3d3HkyBGdjiPSRaVafZ5IV926dcPatWu1yqpVq2akaLTZ2toiNjYWarUa58+fx/Dhw/Hvv/9i7969L3zu4lbWfpKdnd0LX6ckGjVqhAMHDkClUuHKlSsYMWIE7t+/j82bN5fJ9Z+2evVqtGnTBl5eXpqy7OxsNG3aFCNGjEDfvn31dq2DBw8iKCgICxYsQK9evSCRSBATE4P9+/fr7RolYW1tDWtra832zZs3AQC9e/eGRCIBAMjl8hKfz8zMrETfseeRyWQYNGgQvv76a7Rv3/6Fz0dUJGNnYkTG8qxehC+++EI0btxYWFpaiurVq4sxY8ZoLf76dI9QdHS0ePXVV4W1tbWwsbERzZs3F6dPn9bsj4qKEu3atRMKhUJUr15djB8/XmRlZRUbW1E9TgsWLBBSqVTk5OQIlUol5s6dKzw8PIRMJhNNmzYVf/zxh6Zubm6uGDt2rHB1dRVyuVzUqFFDLFy4ULMfT/Rq4KkesYK1fJ68Pz/88INwc3MTKpVKK6ZevXqJ4cOHa7Z//fVX0axZMyGXy0XNmjXFnDlzNL0CRQkNDRVNmzbVKgsJCREODg6a7fz8fDFixAjh7e0tFAqFqFu3rli2bJnWOZ7+DBEREUKIx4uV9uvXT9jZ2QkHBwfRq1cvER8fX2w8QgjRqFEj8e233xa7H3rsEZo4caJ49dVXn1mn4B6tWLFCVK9eXVhYWIh+/fqJjIwMrXqrVq0S9evXF3K5XNSrV08sX75ca//t27fFgAEDhIODg7C0tBQtWrTQrKn1ZDsUdT+FKPz7olKpxKeffip8fHyETCYTnp6emkVgn+xlLPj3kz/BwcFi3bp1wtHRUdMzWaB3797i7bff1mwfPnxYyGQykZOTU/IbS6QDjhEiKoJUKsXXX3+Ny5cvY926dTh06BA+/vjjYusPHjwY1atXx+nTp3HmzBlMnToV5ubmAB7/v+tu3brhzTffxIULF7B582YcPXoU48aN0ykmCwsLqNVq5Ofn46uvvsIXX3yBzz//HBcuXEBgYCB69eqF69evAwC+/vpr7Ny5E1u2bEFsbCw2btwIb2/vIs976tQpAMCBAweQlJSE7du3F6rTr18/pKWlISIiQlOWnp6O8PBwDB48GAAQFRWFoUOHYuLEiYiJicEPP/yAsLAwLFiwoMSf8datW9i7dy9kMpmmTK1Wo3r16ti6dStiYmIwe/ZsTJ8+HVu2bAEATJ48Gf3790e3bt2QlJSEpKQktGnTBkqlEoGBgbCxsUFUVBSOHTsGa2trdOvWDXl5eUVePz09HTExMWjZsmWJY34Rrq6uuHz5Mi5duvTMejdu3MCWLVvw+++/Izw8HOfOncP777+v2b9x40bMnj0bCxYswJUrV7Bw4ULMmjUL69atAwBkZWWhY8eOSExMxM6dO3H+/Hl8/PHHUKvVha41efJkTS9pwf0syrRp07B48WLMmjULMTEx+Omnn+Di4lKonqenJ3755RcAQGxsLJKSkvDVV1+hX79+UKlU2Llzp6bunTt3sHv3bowYMUJT1rJlS+Tn5+PkyZPPvEdEpWbsTIzIWIKDg4WZmZmwsrLS/Lz11ltF1t26dauoWrWqZvvpHhsbGxsRFhZW5LHvvPOOGD16tFZZVFSUkEql4uHDh0Ue8/T5r127JurWrStatmwphBDC3d1dLFiwQOuYl19+Wbz//vtCCCHGjx8vOnXqJNRqdZHnxxO9GsWNEXq6B6B3795ixIgRmu0ffvhBuLu7a3qJOnfurNXrJIQQGzZsEG5ubkXGIMTj3gepVCqsrKyEQqHQ9BgsXbq02GOEEGLs2LHizTffLDbWgmvXq1dP6x7k5uYKCwsLsXfv3iLPe+7cOQFAJCQkFHtt6LFHKCsrS/To0UMAEF5eXiIoKEj8+OOPWr0koaGhwszMTPzzzz+asj/++ENIpVKRlJQkhBDCx8dH/PTTT1rnnj9/vvDz8xNCPG4rGxubYsdePd0zt2PHDvH0n4cn73FmZqaQy+Vi1apVRZ7v6e9UcWOExowZI7p3767Z/uKLL0StWrUKfW8dHByK/f0ielHsESKT5u/vj+joaM3P119/DeBx70jnzp3h4eEBGxsbDBkyBGlpacjJySnyPCEhIRg5ciQCAgKwePFizRgLADh//jzCwsI04zCsra0RGBgItVqN+Pj4YmO7f/8+rK2tYWlpiXr16sHFxQUbN25EZmYm/v33X7Rt21arftu2bXHlyhUAj9/4io6ORr169TBhwgTs27fvRW8VBg8ejF9++QW5ubkAHvdCDBgwAFKpVPM5582bp/U5R40ahaSkpGLvGwDUq1cP0dHROH36NKZMmYLAwECMHz9eq87y5cvRokULVKtWDdbW1li5ciUSEhKeGe/58+dx48YN2NjYaOJxdHTEo0ePtNrnSQ8fPgQAKBSKEt+XoiQkJGjdh4ULFxZZz8rKCrt378aNGzcwc+ZMWFtb48MPP0SrVq207lmNGjXg4eGh2fbz84NarUZsbCyys7Nx8+ZNvPPOO1rX/OSTTzSfMzo6Gs2aNYOjo+MLfa4CV65cQW5uLjp37vxC5xk1ahT27duHxMREAI/fxhw2bJhmXFIBCwuLZ36HiF4EB0uTSbOyskLt2rW1ym7duoXXX38dY8aMwYIFC+Do6IijR4/inXfeQV5eHiwtLQudZ86cORg0aBB2796NP/74A6Ghodi0aRPeeOMNZGVl4d1338WECRMKHVejRo1iY7OxscHZs2chlUrh5uYGCwsLAEBmZuZzP1fz5s0RHx+PP/74AwcOHED//v0REBCAbdu2PffY4vTs2RNCCOzevRsvv/wyoqKi8OWXX2r2Z2VlYe7cuUUOJn5WYiGTyTRtsHjxYrz22muYO3cu5s+fDwDYtGkTJk+ejC+++AJ+fn6wsbHBkiVLnvuoJCsrCy1atMDGjRsL7StuQLyTkxMA4N69ey80aN7d3V3r1fHnJSA+Pj7w8fHByJEjMWPGDNStWxebN2/G8OHDn3utrKwsAMCqVavQunVrrX1mZmYAoPnu6Iu+ztesWTM0bdoU69evR9euXXH58mXs3r27UL309PRy8xIDVT5MhIiecubMGajVanzxxRea3o6C8SjPUrduXdStWxeTJk3CwIEDsXbtWrzxxhto3rw5YmJiCiVczyOVSos8xtbWFu7u7jh27Bg6duyoKT927BhatWqlVS8oKAhBQUF466230K1bN6Snpxf6o1wwHkelUj0zHoVCgb59+2Ljxo24ceMG6tWrh+bNm2v2N2/eHLGxsTp/zqfNnDkTnTp1wpgxYzSfs02bNlpjYp7u0ZHJZIXib968OTZv3gxnZ2fY2tqW6No+Pj6wtbVFTEwM6tatW+rPUKVKlVLfB29vb1haWiI7O1tTlpCQgH///Rfu7u4AgBMnTkAqlWp6Ct3d3REXF6cZr/W0l156CatXry6y/UujTp06sLCwwMGDBzFy5Mjn1n/Wd2zkyJFYtmwZEhMTERAQAE9PT639N2/exKNHj9CsWbMXjpuoKHw0RvSU2rVrQ6lU4ptvvkFcXBw2bNiAFStWFFv/4cOHGDduHCIjI/H333/j2LFjOH36NBo0aAAAmDJlCv7880+MGzcO0dHRuH79On777TedB0s/6aOPPsKnn36KzZs3IzY2FlOnTkV0dDQmTpwIAFi6dCl+/vlnXL16FdeuXcPWrVvh6upa5CSQzs7OsLCwQHh4OFJSUnD//v1irzt48GDs3r0ba9asKfRHd/bs2Vi/fj3mzp2Ly5cv48qVK9i0aRNmzpyp02fz8/PDSy+9pHmcVKdOHfz111/Yu3cvrl27hlmzZuH06dNax3h7e+PChQuIjY1FamoqlEolBg8eDCcnJ/Tu3RtRUVGIj49HZGQkJkyYgH/++afIa0ulUgQEBODo0aNa5VlZWZrHpwAQHx+P6Ojo5z6ee545c+bg448/RmRkJOLj43Hu3DmMGDECSqUSXbp00dRTKBQIDg7G+fPnERUVhQkTJqB///6aV9Tnzp2LRYsW4euvv8a1a9dw8eJFrF27FkuXLgUADBw4EK6urujTpw+OHTuGuLg4/PLLLzh+/Hip4lYoFJgyZQo+/vhjrF+/Hjdv3sSJEyfw448/Flnfy8sLEokEu3btwt27dzW9WAAwaNAg/PPPP1i1apXWIOkCUVFRqFWrFnx8fEoVK9FzGXuQEpGxPOv1+aVLlwo3NzdhYWEhAgMDxfr167UGez45mDk3N1cMGDBAeHp6CplMJtzd3cW4ceO0BkKfOnVKdOnSRVhbWwsrKyvx0ksvFRrs/KTnTdioUqnEnDlzhIeHhzA3Ny/0+vzKlSuFr6+vsLKyEra2tqJz587i7Nmzmv14asDvqlWrhKenp5BKpUW+Pv/kdQsmFbx582ahuMLDw0WbNm2EhYWFsLW1Fa1atRIrV64s9nMU9fq8EEL8/PPPQi6Xi4SEBPHo0SMxbNgwYWdnJ+zt7cWYMWPE1KlTtY67c+eO5v7iidfnk5KSxNChQ4WTk5OQy+WiVq1aYtSoUeL+/fvFxrRnzx7h4eGhNVVAwWDfp3+Cg4OLPU9JHDp0SLz55pua746Li4vo1q2biIqKKnSPvvvuO+Hu7i4UCoV46623RHp6uta5Nm7cKHx9fYVMJhMODg6iQ4cOYvv27Zr9t27dEm+++aawtbUVlpaWomXLluLkyZNa1yjwvMHSQjz+LnzyySfCy8tLmJuba03RUNQA/Hnz5glXV1chkUgK3bchQ4YU+Sq9EEJ07dpVLFq0qET3k6g0JEIIYaQcjIio3BFCoHXr1ppHnMY2Z84c/Prrry+8XEV51rlzZzRq1EjzskKBy5cvo1OnTrh27VqZTfBJpoePxoiIniCRSLBy5Urk5+cbO5RK7969e9ixYwciIyMxduzYQvuTkpKwfv16JkFkUBwsTUT0FF9f30KLkJL+NWvWDPfu3cOnn36KevXqFdofEBBghKjI1PDRGBEREZksPhojIiIik8VEiIiIiEwWEyEiIiIyWUyEiIiIyGQxESIiIiKTxUSIiIiITBYTISIiIjJZTISIiIjIZDERIiIiIpP1/2H9z4Xs7esUAAAAAElFTkSuQmCC",
|
||
"text/plain": [
|
||
"<Figure size 640x480 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"# plot ROC curve using y_test_binary and y_pred_prob\n",
|
||
"fpr, tpr, thresholds = metrics.roc_curve(y_test_binary, y_pred_prob)\n",
|
||
"plt.plot(fpr, tpr)\n",
|
||
"plt.xlim([0.0, 1.0])\n",
|
||
"plt.ylim([0.0, 1.0])\n",
|
||
"plt.title('ROC curve for Yelp classifier')\n",
|
||
"plt.xlabel('False Positive Rate (1 - Specificity)')\n",
|
||
"plt.ylabel('True Positive Rate (Sensitivity)')\n",
|
||
"plt.grid(True)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"显示混淆矩阵，并计算敏感度和特异性，评论结果。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 80,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"array([[126, 58],\n",
|
||
" [ 25, 813]])"
|
||
]
|
||
},
|
||
"execution_count": 80,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# print the confusion matrix\n",
|
||
"metrics.confusion_matrix(y_test, y_pred_class)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 81,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"0.9701670644391408"
|
||
]
|
||
},
|
||
"execution_count": 81,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# calculate sensitivity\n",
|
||
"tn, fp, fn, tp = metrics.confusion_matrix(y_test, y_pred_class).ravel()\n",
|
||
"sensitivity = tp / float(tp + fn)\n",
|
||
"sensitivity"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 82,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"0.6847826086956522"
|
||
]
|
||
},
|
||
"execution_count": 82,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# calculate specificity\n",
|
||
"specificity = tn / float(tn + fp)\n",
|
||
"specificity\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"对模型的敏感度和特异性做出评论："
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 第五步：错误分析\n",
|
||
"\n",
|
||
"查看测试集中一些被预测错误的评论文本，即false positives和false negatives。试着回答为什么这些评论会被预测错。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 83,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"2175 This has to be the worst restaurant in terms o...\n",
|
||
"1781 If you like the stuck up Scottsdale vibe this ...\n",
|
||
"2674 I'm sorry to be what seems to be the lone one ...\n",
|
||
"9984 Went last night to Whore Foods to get basics t...\n",
|
||
"3392 I found Lisa G's while driving through phoenix...\n",
|
||
"8283 Don't know where I should start. Grand opening...\n",
|
||
"2765 Went last week, and ordered a dozen variety. I...\n",
|
||
"2839 Never Again,\\nI brought my Mountain Bike in (w...\n",
|
||
"321 My wife and I live around the corner, hadn't e...\n",
|
||
"1919 D-scust-ing.\n",
|
||
"2490 Lazy Q CLOSED in 2010. New Owners cleaned up ...\n",
|
||
"9125 La Grande Orange Grocery has a problem. It can...\n",
|
||
"9185 For frozen yogurt quality, I give this place a...\n",
|
||
"436 this another place that i would give no stars ...\n",
|
||
"2051 Sadly with new owners comes changes on menu. ...\n",
|
||
"1721 This is the closest to a New York hipster styl...\n",
|
||
"3447 If you want a school that cares more about you...\n",
|
||
"842 Boy is the name a temptation.Seriously :) I'l...\n",
|
||
"6159 Really, if I could, I would give this place ze...\n",
|
||
"943 Don't waste your time...Arrowhead mall on the ...\n",
|
||
"5977 You want good food? You'd be better off smuggl...\n",
|
||
"8833 The owner has changed hands & this place isn't...\n",
|
||
"6584 Jimmy Johns is cheaper and better ... The Capr...\n",
|
||
"1899 Buca Di Beppo is literally, italian restaurant...\n",
|
||
"9953 \"Hipster,Trendy\" ????-I think NOT !!!! Very di...\n",
|
||
"2060 This place is closed. Good riddance.\n",
|
||
"3082 Currently having a liquidation sale, but it's ...\n",
|
||
"8220 Maybe I ate at a different restaurant than the...\n",
|
||
"3634 Seriously?! With grocery stores like Fresh & E...\n",
|
||
"3266 Absolutely awful... these guys have NO idea wh...\n",
|
||
"7397 This place sucks!! I moved to the valley and h...\n",
|
||
"4473 It is what you would expect from any themed pl...\n",
|
||
"5502 Angry Bro Bar ! Please go here if you wear si...\n",
|
||
"2615 Great in its day, now leaves a lot to be desir...\n",
|
||
"3413 I purchased the Enotria groupon when it was re...\n",
|
||
"2999 I can't even believe I actually went to this r...\n",
|
||
"1372 No offense to everyone who gave this place 5 s...\n",
|
||
"1291 Every time I come here the staff is so rude! I...\n",
|
||
"6222 My mother always told me, if I didn't have any...\n",
|
||
"9296 My boyfriend and I tried this place last year ...\n",
|
||
"7975 What are you all talking about?! This place is...\n",
|
||
"4630 I used to always go here for tires until my me...\n",
|
||
"7130 I was not impressed. The food was bad & expens...\n",
|
||
"5818 Most horrible buffet I have ever been to.\\n\\nM...\n",
|
||
"3704 Staff is nice, and that's it.\\nThey use very c...\n",
|
||
"8741 They served us stale rice. Average main dishe...\n",
|
||
"3938 We were so disappointed! We were on vacation ...\n",
|
||
"7631 this is a business located in the fry's grocer...\n",
|
||
"8681 As I promised myself, I'd go back again to try...\n",
|
||
"1532 Cold, under done chips. If a Mexican food rest...\n",
|
||
"113 Unless you are a regular or look like your wal...\n",
|
||
"4165 OMG! what is the rave about? this place is dis...\n",
|
||
"9299 The salad plates were not chilled... As they u...\n",
|
||
"4311 Donuts are really good, if they have any when ...\n",
|
||
"7035 Totally excited to try this place out, my gran...\n",
|
||
"8000 Still a place that is unacceptable in my book-...\n",
|
||
"3755 Have been going to LGO since 2003 and have alw...\n",
|
||
"507 HELLISH HELLISH SUMMER WEATHER (March thru Oct...\n",
|
||
"Name: text, dtype: object"
|
||
]
|
||
},
|
||
"execution_count": 83,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# first 10 false positives (meaning they were incorrectly classified as 5-star reviews)\n",
|
||
"X_test[y_pred_class > y_test]\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 84,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"7148 I now consider myself an Arizonian. If you dri...\n",
|
||
"4963 This is by far my favourite department store, ...\n",
|
||
"6318 Since I have ranted recently on poor customer ...\n",
|
||
"380 This is a must try for any Mani Pedi fan. I us...\n",
|
||
"5565 I`ve had work done by this shop a few times th...\n",
|
||
"3448 I was there last week with my sisters and whil...\n",
|
||
"6050 I went to sears today to check on a layaway th...\n",
|
||
"2504 I've passed by prestige nails in walmart 100s ...\n",
|
||
"2475 This place is so great! I am a nanny and had t...\n",
|
||
"241 I was sad to come back to lai lai's and they n...\n",
|
||
"3149 I was told to see Greg after a local shop diag...\n",
|
||
"423 These guys helped me out with my rear windshie...\n",
|
||
"763 Here's the deal. I said I was done with OT, bu...\n",
|
||
"8956 I took my computer to RedSeven recently when m...\n",
|
||
"750 This store has the most pleasant employees of ...\n",
|
||
"9765 You can't give anything less than 5 stars to a...\n",
|
||
"6334 I came here today for a manicure and pedicure....\n",
|
||
"1282 Loved my haircut. Walked in and waited for jus...\n",
|
||
"1266 I've never been to this location before. My hu...\n",
|
||
"402 Once again Wildflower proves why it's my favor...\n",
|
||
"4034 \"Fine dining\" is not just a setting. it isn't...\n",
|
||
"2444 EXCELLENT CUSTOMER SERVICE! \\n\\nEven with Happ...\n",
|
||
"2494 What a great surprise stumbling across this ba...\n",
|
||
"5736 Thank goodness for Sue at Mill Avenue Travel. ...\n",
|
||
"7903 First, I'm sorry this review is lengthy, but i...\n",
|
||
"Name: text, dtype: object"
|
||
]
|
||
},
|
||
"execution_count": 84,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# first 10 false negatives (meaning they were incorrectly classified as 1-star reviews)\n",
|
||
"X_test[y_pred_class < y_test]"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 第六步：多分类预测\n",
|
||
"\n",
|
||
"使用所有的评论做预测，而不仅仅是评分1星和5星的评论。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 85,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# define X and y using the original DataFrame\n",
|
||
"X = yelp.text\n",
|
||
"y = yelp.stars\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 86,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# split into training and testing sets\n",
|
||
"X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)\n",
|
||
"\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 87,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# create document-term matrices\n",
|
||
"X_train_dtm = vect.fit_transform(X_train)\n",
|
||
"X_test_dtm = vect.transform(X_test)\n",
|
||
"\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 88,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"<style>#sk-container-id-4 {color: black;background-color: white;}#sk-container-id-4 pre{padding: 0;}#sk-container-id-4 div.sk-toggleable {background-color: white;}#sk-container-id-4 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-4 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-4 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-4 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-4 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-4 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-4 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-4 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-4 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-4 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-4 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-4 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-4 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-4 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-4 div.sk-item {position: relative;z-index: 1;}#sk-container-id-4 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-4 div.sk-item::before, #sk-container-id-4 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-4 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-4 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-4 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-4 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-4 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-4 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-4 div.sk-label-container {text-align: center;}#sk-container-id-4 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-4 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-4\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>MultinomialNB()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-4\" type=\"checkbox\" checked><label for=\"sk-estimator-id-4\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">MultinomialNB</label><div class=\"sk-toggleable__content\"><pre>MultinomialNB()</pre></div></div></div></div></div>"
|
||
],
|
||
"text/plain": [
|
||
"MultinomialNB()"
|
||
]
|
||
},
|
||
"execution_count": 88,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# fit a Naive Bayes model\n",
|
||
"nb.fit(X_train_dtm, y_train)\n",
|
||
"\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 89,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# make class predictions\n",
|
||
"y_pred_class = nb.predict(X_test_dtm)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 90,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"0.4712"
|
||
]
|
||
},
|
||
"execution_count": 90,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# calculate the testing accuary\n",
|
||
"metrics.accuracy_score(y_test, y_pred_class)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 91,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"array([[ 55, 14, 24, 65, 27],\n",
|
||
" [ 28, 16, 41, 122, 27],\n",
|
||
" [ 5, 7, 35, 281, 37],\n",
|
||
" [ 7, 0, 16, 629, 232],\n",
|
||
" [ 6, 4, 6, 373, 443]])"
|
||
]
|
||
},
|
||
"execution_count": 91,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# print the confusion matrix\n",
|
||
"metrics.confusion_matrix(y_test, y_pred_class)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"有何评论："
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3.10.8 ('.venv': venv)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.10.8"
|
||
},
|
||
"vscode": {
|
||
"interpreter": {
|
||
"hash": "1f0d395e06aa83586067b19165efc9b683889967164248deef4bbf1fa27cfb00"
|
||
}
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 1
|
||
}
|