{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "hi2CmTEDvGij" }, "source": [ "# Purpose of Notebook\n", "\n", "The purpose of this notebook is to offer an example answer to the guided project for the Sequential Models for Deep Learning course. Since the choice of model predictors is up to the student, results can differ. Use this solution as a guide for how to structure your own answer." ] }, { "cell_type": "markdown", "metadata": { "id": "e65739c8" }, "source": [ "# Time-Series Forecasting on the S&P 500" ] }, { "cell_type": "markdown", "metadata": { "id": "18150cb2" }, "source": [ "**Context**: We are working as traders on the S&P 500 futures desk. We have been tasked with building a model to better forecast how this index will move based on its behavior over the past several years. The better our forecast performs, the more effectively and lucratively our desk will be able to trade these futures." ] }, { "cell_type": "markdown", "metadata": { "id": "2e64de5f" }, "source": [ "## Introduction" ] }, { "cell_type": "markdown", "metadata": { "id": "9091a3ff" }, "source": [ "The dataset we will be working with is from [Yahoo Finance via Kaggle](https://www.kaggle.com/datasets/arashnic/time-series-forecasting-with-yahoo-stock-price), and it contains S&P 500 Index prices from 2015 through 2020.\n", "\n", "Before we get into the data, let's set some random seed values to improve the reproducibility of the models we will build later on." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "id": "e101a4be" }, "outputs": [], "source": [ "# Imports\n", "import tensorflow as tf\n", "import numpy as np\n", "import random\n", "\n", "# Seed code\n", "np.random.seed(1)\n", "random.seed(1)\n", "tf.random.set_seed(1)" ] }, { "cell_type": "markdown", "metadata": { "id": "240f8d2d" }, "source": [ "## Data Wrangling and Exploration" ] }, { "cell_type": "markdown", "metadata": { "id": "395dc3c2" }, "source": [ "First, we will load in the data and inspect it to determine what steps will be required for cleaning and preprocessing." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 206 }, "id": "f55e074c", "outputId": "f2091de4-28f5-4d49-c8c8-8420aee93d51" }, "outputs": [ { "data": { "text/html": [ "\n", "
\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
DateHighLowOpenCloseVolumeAdj Close
02015-11-232095.6101072081.3898932089.4099122086.5900883.587980e+092086.590088
12015-11-242094.1201172070.2900392084.4199222089.1398933.884930e+092089.139893
22015-11-252093.0000002086.3000492089.3000492088.8701172.852940e+092088.870117
32015-11-262093.0000002086.3000492089.3000492088.8701172.852940e+092088.870117
42015-11-272093.2900392084.1298832088.8200682090.1101071.466840e+092090.110107
\n", "
\n", " \n", " \n", " \n", "\n", " \n", "
\n", "
\n", " " ], "text/plain": [ " Date High Low Open Close \\\n", "0 2015-11-23 2095.610107 2081.389893 2089.409912 2086.590088 \n", "1 2015-11-24 2094.120117 2070.290039 2084.419922 2089.139893 \n", "2 2015-11-25 2093.000000 2086.300049 2089.300049 2088.870117 \n", "3 2015-11-26 2093.000000 2086.300049 2089.300049 2088.870117 \n", "4 2015-11-27 2093.290039 2084.129883 2088.820068 2090.110107 \n", "\n", " Volume Adj Close \n", "0 3.587980e+09 2086.590088 \n", "1 3.884930e+09 2089.139893 \n", "2 2.852940e+09 2088.870117 \n", "3 2.852940e+09 2088.870117 \n", "4 1.466840e+09 2090.110107 " ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Import\n", "import pandas as pd\n", "\n", "# Load and inspect the data\n", "stock_data = pd.read_csv(\"yahoo_stock.csv\")\n", "stock_data.head()" ] }, { "cell_type": "markdown", "metadata": { "id": "192e4514" }, "source": [ "We can see that the data contains seven columns: `Date`, `High`, `Low`, `Open`, `Close`, `Volume`, and `Adj Close`.\n", "\n", "We will want to set the index of the DataFrame to the `Date` column to prepare for time series forecasting, and decide what other column(s) to use for the forecast itself. For now, we are going to use only the `Adj Close` column, which is the closing price of the S&P 500 index, [adjusted for dividends](https://www.investopedia.com/articles/investing/091015/how-dividends-affect-stock-prices.asp). Based on this decision, we modify the DataFrame to drop the other columns.\n", "\n", "We should also ensure that the data is sorted by its `Date` column." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 238 }, "id": "d4e04bd1", "outputId": "d9068a8e-5a59-4934-ba1b-f036db02772e" }, "outputs": [ { "data": { "text/html": [ "\n", "
\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Adj Close
Date
2015-11-232086.590088
2015-11-242089.139893
2015-11-252088.870117
2015-11-262088.870117
2015-11-272090.110107
\n", "
\n", " \n", " \n", " \n", "\n", " \n", "
\n", "
\n", " " ], "text/plain": [ " Adj Close\n", "Date \n", "2015-11-23 2086.590088\n", "2015-11-24 2089.139893\n", "2015-11-25 2088.870117\n", "2015-11-26 2088.870117\n", "2015-11-27 2090.110107" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Select relevant columns, sort data, and set index\n", "stock_data = stock_data[[\"Date\", \"Adj Close\"]]\n", "stock_data = stock_data.sort_values(\"Date\")\n", "stock_data = stock_data.set_index(\"Date\")\n", "\n", "# Inspect the data\n", "stock_data.head()" ] }, { "cell_type": "markdown", "metadata": { "id": "c0d02adc" }, "source": [ "We should also double-check that we don't have any missing or erroneous values in our dataset, and consider forward-filling or interpolating if necessary." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "ff57c079", "outputId": "199f8938-e72b-4d2a-f8fc-ddeab6256faa" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Info: \n", "\n", "Index: 1825 entries, 2015-11-23 to 2020-11-20\n", "Data columns (total 1 columns):\n", " # Column Non-Null Count Dtype \n", "--- ------ -------------- ----- \n", " 0 Adj Close 1825 non-null float64\n", "dtypes: float64(1)\n", "memory usage: 28.5+ KB\n", "\n", "Describe: \n", " Adj Close\n", "count 1825.000000\n", "mean 2647.856284\n", "std 407.301177\n", "min 1829.079956\n", "25% 2328.949951\n", "50% 2683.340088\n", "75% 2917.520020\n", "max 3626.909912\n", "\n", "Skew: \n", " Adj Close 0.081869\n", "dtype: float64\n" ] } ], "source": [ "# Check for missing or erroneous values\n", "print(\"Info: \")\n", "stock_data.info()\n", "print(\"\\nDescribe: \\n\", stock_data.describe())\n", "print(\"\\nSkew: \\n\", stock_data.skew())" ] }, { "cell_type": "markdown", "metadata": { "id": "0f794226" }, "source": [ "Great! No missing values, and everything seems to be within a reasonable range. The low skew value for `Adj Close` indicates we don't have any outliers to be concerned about.\n", "\n", "Before we begin preparing the data for modeling by scaling the variable to be forecasted (`Adj Close`) and splitting the dataset for training, validation, and testing, let's quickly visualize the data." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 274 }, "id": "062d8991", "outputId": "e5244674-9c69-49b3-9064-e6687ec13ba7" }, "outputs": [ { "data": { "text/plain": [ "Text(0, 0.5, 'Adjusted Close')" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAADwCAYAAADxckg9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3zU9f3A8dc7exIChD0CyBBRQEFxoKAyrLZi66x1j7pa62odP+uobe1ytlqte9Q9iltUxIkyZO9NmAECCdm5vH9/fL93+V5yl1zgLiHJ+/l43MO7z3fc59Jy7/us90dUFWOMMaY+cc1dAWOMMfs/CxbGGGMaZMHCGGNMgyxYGGOMaZAFC2OMMQ2yYGGMMaZBFiyMaaVEZKyI5DV3PUzrYMHCtAoicoyIfCMiu0Vkp4h8LSKjPMf7ish0ESkSkXUicn6Ie6iIFIvIHhHZKCL3iUh8mPdbKyKl7rl7ROTjWsevE5EtIlIoIk+JSLLnWK6ITBOREhFZKiIn1vO5nhGRe/bur2JM9FiwMC2eiLQD3gUeBjoAPYC7gHLPaX8C1rrHjwAWh7ndMFXNAE4Afg5cVs9b/1hVM9zHBE99JgI3u/foA/Rz6+P3EvAD0BG4DXhdRHIi+rDGNBMLFqY1GAigqi+pqk9VS1X1Y1Wd7zmnEshT1UpV3aKqs+q7oaouBb4Ehu5FfS4AnlTVRapaAPwBuBBARAYChwJ3uPV8A1gA/Kyhm7otEhWRC0RkvYhsF5HbPMdT3ZZIgYgsBkbVur67iLwhIvkiskZEfu2WdxCRPBH5sfs6Q0RWhmp9mbbLgoVpDZYDPhF5VkROEpHsEOfMBG4UkUmR3FBEhgBjcFoA4bzofvF+LCLDPOUHAfM8r+cBXUSko3tstaoW1Tp+UCT1ch0DDMJpufxeRA50y+8A+ruPiThBy/954oB33Pfq4V77GxGZqKo7gYuB/4hIZ+B+YK6qPteIOplWzoKFafFUtRDnC1SB/wD5IjJFRLoAiMjRwPXABOAJf8AQkQPcX+fiud0cESnA+WJ9Ang6zNueC+TidDNNAz4SkfbusQxgt+dc//PMEMf8xzMb8ZHvclsl83C+/P2B6kzgj6q6U1U3AA95rhkF5Kjq3apaoaqrcf5WZwOo6sfAa8CnwI+AXzaiPqYNsGBhWgVVXaKqF6pqT5yuo+7AA+7ha4DHVXU6cBrwvBswjgamaXA2zUNVNVtV+6vq/6lqdZj3+9r9wi5R1T8Du3BaIgB7gHae0/3Pi0Ic8x8vInJbPM9LcAIQOJ95g+fYOs/zPkB3EdnlfwC3Al085zyO87d7RlV3NKI+pg2wYGFaHXe84RlqxhsSgET32EzgLOBl4E7gb9F6W8DfQllEza993Odb3S/gRUA/EcmsdXxRFOqwGejled3b83wDsEZV23semar6IwB31tfjwHPAVSJyQBTqY1oRCxamxRORwSJyg4j0dF/3As4BZrinvAb8WkSOdfvuN+PMjOoKVO3F+/UWkaNFJElEUkTkJqAT8LV7ynPAJSIyxO2a+j+c4IWqLgfmAne4154GHAK8sTefvZZXgVtEJNv9W/zKc+x7oEhEfucOhMeLyFDP9OJbcQLexTgB9Llw04ZN22TBwrQGRTjTYb8TkWKcILEQuAFAVV/Fmcr6uHvuW8BjwE3AuyLSO9RN65EJPAoUABuBScBJ/q4bVf0Q+CvOWMZ6nO6gOzzXnw2MdK+/FzhdVfMbWYdQ7nLfaw3wMfC8/4Cq+oBTgOHu8e04YzJZInIYzpjO+e55f8EJHDdHoU6mlRDb/MgYY0xDrGVhjDGmQRYsjDHGNMiChTHGmAZZsDDGGNMgCxbGGGMalNDcFYiVTp06aW5ubnNXwxhjWozZs2dvV9WQGZBbbbDIzc1l1qx6E4saY4zxEJF14Y5ZN5QxxpgGWbAwxhjTIAsWxhhjGmTBwhhjTIMsWBhjjGmQBQtjjDENsmBhjDGtQF5BCWP/No28gpKY3N+ChTHGtAKvz85j7Y4SXv5+Q8Mn7wULFsYY0wpkJDtrrPeUN3rzx4hYsDDGmFYgTpwt4GO1oZ0FC2OM2Q9t3l3KtsKyiM+fungrAKvyi2NSn1abG8oYY1qyI//8GQBr7z05ovO/Xb0DgK9Wbo9JfaxlYYwxLdyKrUUxfw8LFsYY08LtKK6I+XtYsDDGmBbukc9XBZ7ffsqQmLyHBQtjjGlBqquVe95dzIadNYvvBnfNDDy/5Ji+MXnfmAULEUkRke9FZJ6ILBKRu9zyZ0RkjYjMdR/D3XIRkYdEZKWIzBeRQz33ukBEVriPC2JVZ2OM2d8t31bEE1+t4er/zgmUJcU7X+Vf3DQuZu8by9lQ5cDxqrpHRBKBr0TkA/fYTar6eq3zTwIGuI8jgEeBI0SkA3AHMBJQYLaITFHVghjW3Rhjmo2vOvxaiYQ4JzDsKqkMlH21cjspiXH07pgWszrFrGWhjj3uy0T3Ud9qkVOB59zrZgDtRaQbMBGYqqo73QAxFZgUq3obY0xzu+2tBWGPVVVXA1BS4QNgZ3EFczfsoqyyOqZ1iumYhYjEi8hcYBvOF/537qE/ul1N94tIslvWA/AmNclzy8KVh3q/y0VklojMys/Pj+pnMcaYprJg4+6wx8rdoFBa4aT1KHbTe1w/fmBM6xTTYKGqPlUdDvQEDheRocAtwGBgFNAB+F0U3+9xVR2pqiNzcnKidVtjjGlS/jxP/Tql1zlW4XOCRXGFj9X5e6h0X/eJYRcUNNFsKFXdBUwDJqnqZrerqRx4GjjcPW0j0MtzWU+3LFy5Mca0SsvcRXa+EHmevFllv165PRA8EuNj+3Uey9lQOSLS3n2eCowHlrrjEIiIAJOBhe4lU4Dz3VlRo4HdqroZ+AiYICLZIpINTHDLjDGm1fFVa2DwOtRAtz84ANz+v0W8NisPiH2wiOVsqG7AsyISjxOUXlXVd0XkMxHJAQSYC1zhnv8+8CNgJVACXASgqjtF5A/ATPe8u1V1ZwzrbYwxzWZPWU2K8VDBYk9ZJQO7ZLB8qzN/6MXv1gGQGC8xrVfMgoWqzgdGhCg/Psz5Clwd5thTwFNRraAxxuyHCsucVkV6UnxgxpPXnvIqstOSAq8zkhMoq6wIrLWIFVvBbYwx+5HdpU6w6JGdyu7Syjr7U5RU+AID4A6nRZGUYMHCGGPaDH/Lok9HZyaUN+8TQGmlj5Sk+MBr/9TZFjvAbYwxpvEKS50v/4uPdnI81d4AaXV+MamJ8bx99dHkZCZTWul0VVmwMMaYNiR/TzkAPbNTSYwXPlmyjWlLtwEwZd4mAGas3sHwXu3pnpUSuC4pIbYD3BYsjDFmP1Fdrdz+trOaoH1aIpU+ZeOuUi56xpkMushd2Z1XUApAu9TEwLXWsjDGmCa0ZnsxuTe/R+7N7wVWRzeFSl81P/7nV4HXmSmJQcfLKn1MW+a0MBLinFZEj/apgeMWLIwxpgmN+/vngedbdpeFPzHK3vphI4s2FQIwZkCnOscLSysDaytSEp0Bbm+AsGBhjDFNKDut5hd9U2xX6rdpV2ng+YNn11mixvY9NXVJdqfJxsfVjFPYOgtjjGlCmSmJdEh3Fr2VlFc1cHZ0VFRV88AnKwD4/Maxgff3+m7NjsDzm08aDAQHi+RECxbGGNMkLn9uFut3lnBo7/YAbG6ibqhvVm0PPPd3MdV21zuLA89POaQ7UDN2ATWtjVixYGGMadM+WbyVDxduAeDjxVsBOKCzs6f1G3PymqQO+UXlgefe1kIoN04YSKq7KC/OPTc5IQ4nN2vsxDKRoDHG7NeqfNVc+twsoKZrB+DQ3u1pn5YY9ld+tO10x0auHtefThk1XVDXnjCAdTuKeXvupkDZVWMPCDz3tyyaop4WLIwxbdbOkppB48+XbaNTRhI5mSmceGAXcjumU1XPXtjRtH1POckJcdw4YVBQC+G68QMpKqsMBAuRmtYEgD9tVKwzzoJ1Qxlj2rCdntlO3bJSqapWDs/NJi5OSIgTfNVNs85ix54KOmUkh+xK8q63qL0X0vMznPTk3plSsWItC2NMm7VsS1Hg+Vs/bCQpPi6QpC8+TqjyNU3LIn9POZ0ykxt9nT9DbVOI5U55KSLyvYjME5FFInKXW/6iiCwTkYUi8pSIJLrlY0Vkt4jMdR+/99xrknvNShG5OVZ1Nsa0Lde+PBeAQ3pmAc4udKlu/39CvITcfCiaKqqqqfRVs31PBTkZdafL1jZ+SJeg1zEe0w4Sy5ZFOXC8qu5xA8JXIvIB8CLwC/ec/wKXAo+6r79U1VO8N3F32vsXzrasecBMEZmiqosxxpi9NHNtzYabR/XvxPw8J++SP1jEx8VRVV1386FoOuSuj0hNjKegpJJhbsAK5fDcDny/dieZycFf2fEiVIXYpzsWYtayUMce92Wi+1BVfd89psD3QM8GbnU4sFJVV6tqBfAycGqs6m2Maf2Ky6v4ywdLAejSLpndpTV9/v5pqc6YRWy/iMsqqylw99vul5Me9rzTRzpfk3G1ptU+deEoAM4b3SdGNawR0wFuEYkXkbnANmCqqn7nOZYInAd86LnkSLfb6gMROcgt6wFs8JyT55YZY8xeuerFOcxaVwDA85ccETQ2kZLoGbOIYbAoqwxutZw0tFvYc6vdesTX6nc6dmAOa+89mT9MHhr9CtYS02Chqj5VHY7TejhcRLyf6BHgC1X90n09B+ijqsOAh4G3G/t+InK5iMwSkVn5+fn7Wn1jTCs0e10B05fXfD8M7JIZNOMozdOyqIph1tmNnlxQ4KQkD6e8yqlHQhNMkQ2nSabOquouYBowCUBE7gBygOs95xT6u61U9X0gUUQ6ARuBXp7b9XTLQr3P46o6UlVH5uTkxOSzGGNatnveqzvceePEgYHn2WnOQHN8FLqhyip9XPLMTJZvLapzbNW2PUGva6ck9+rdIQ2A4wd33qf67ItYzobKEZH27vNUnAHqpSJyKTAROEdVqz3ndxV3krGIHO7WbQcwExggIn1FJAk4G5gSq3obY1qvKfM28cP6XYHXn95wHABpSQkc0bcDADnuFNaEKHRDzduwi0+XbuOWNxfUOXb587Mjvs+4wZ35+ubjOeHALg2fHCOxnA3VDXjWnc0UB7yqqu+KSBWwDvjWjQ1vqurdwOnAle7xUuBsdxC8SkSuAT4C4oGnVHVRDOttjGmlXp/t5Hoa0DmDeyYPpX9ORuDYg2ePYPHm3Qzo7JQlxMftc8si2R3/KK0IHp/Ym/t6NzpqDjELFqo6H6iTlF1VQ76nqv4T+GeYY+8D70e1gsaYNmfzrlImHtSFx84bWedY16wUunr2tHZaFvs2ZlHuDmKXVwUHi1K3/OxRvRg/pAtHH1B3s6P9jaX7MMa0CZt2lbJi2x46pEe2UjoaYxavzHQmclbUGigvcNOMDO2RxQkHdmmyhIX7woKFMaZN+GaVs3nQcQMj+xUfjTELn7tgLrdjzRqKjbtKGfPXaUDTJACMFgsWxpg24bVZzq/8I/tHFizi4+Lw7WNuKP/6DW+CwIufnhl4XtFEuaeiwRIJGmNatepqpVqV79Y46T1qp8wIJyF+31sW/u4n9aTkWOaZRlvdRCnQo8FaFsaYVuuVmesZeudHfLp0GwB3/nhInZQZ4cRHYYDbv6gv3NhH745p+3T/pmTBwhjTav3nyzWUVPj4pbumob6Fb7VFY8zCf321p2WR7q4Q//Gw7owb1HyL7BrLgoUxptWqnUupQ3rDacAD18YJqnD3O4vZVlS2V+9f4abpKKus5rVZG/jLh0spdtdcPHxOnZUF+zUbszDGtDqz1xVw+9sLg8YHDuicwbEDI08DlJ7kfD0+9fUa1u8s4YkL6q7NaIh/nGTuhl3M3VCzcvzQ3u0bfa/mZi0LY0yr8/GiLSzeXBhUdtvJBxIf4XgFQHfPiuna6yQiofXsM3HbyUMafb/mZsHCGNPq7Ciuuyf14K6ZjbpHUkLN12N9X/zh+DPFhtIupeV16liwMMa0Kne9s4jXZ+eRlBDHC5ccwUHd2wGNG68AiPd8O+7NSu7a+1V4pUU4fXd/0mCNRSQNuAHoraqXicgAYJCqvhvz2hljTCPsLq3k6a/XAnD/mcM5ZkAnnu12OIs3FZKc0LiUGvFxNdGiuKLx26uW1HONf0ZUSxJJy+JpnP20j3RfbwTuiVmNjDFmL936lpMK/MKjcjn5EGfnuU4ZyY0a2PbzzqTy7cV6iz3lVWGPpSW1vJZFJMGiv6r+FagEUNUSoOUkNDHGtBkLN+4GnMHsfeVpWASmwDbG7lJnb+20EK0I73hISxFJjSvczYsUQET647Q0jDFmv6GqrNtRwuTh3UmM3/cvY2/LYm+CxZrtxUDLbEWEEslf9A7gQ6CXiLwIfAr8Nqa1MsaYRnrk81UAvD13U1Tu551muzfB4revzwdg+x7nt/VhfbKjUq/m0mCwUNWpwE+BC4GXgJGq+nlD14lIioh8LyLzRGSRiNzllvcVke9EZKWIvOJulYqIJLuvV7rHcz33usUtXyYiE/fmgxpj9m+qytTFW9m0q3Svrp+x2klBftPEQVGpjzeHVGPXWazbUVyn7JpxB+xznZpTg8FCRI4GylT1PaA9cKuI9Ing3uXA8ao6DBgOTBKR0cBfgPtV9QCgALjEPf8SoMAtv989DxEZgrPv9kHAJOARd6tWY0wr4KtWlm4p5Nlv1nLZc7O45NlZjb5Hla+aL1dsp0/HNK6O0pdygidY1LdmIhT/TKgLj8oNlPXq0Lzbou6rSLqhHgVKRGQYcD2wCniuoYvUscd9meg+FDgeeN0tfxaY7D4/1X2Ne/wEcZLAnwq8rKrlqroGWAkcHkG9jTEtwKOfr2TSA19y5zuLAVhSa+W1X3F5FVc8P5uLn5lZJ1fTx4u3AnDC4C5Rq1fcPoxZ+IPF2EE1s7CiMY7SnCKpfZU6yxdPBf6lqv8CIloKKSLxIjIX2AZMxQk0u1TVP6csD+jhPu8BbABwj+8GOnrLQ1xjjGnBvlqxnY8Wba1THmpB24KNu/lw0RY+W7qNJ75cEyhfsrmQq16cA8DZh/eKWt28YxadMiLbitXPX/9Uz3apXbNSGDcohz9MHhqdCjaxSIbpi0TkFuA8YIyIxOG0Ehqkqj5guIi0B94CBu91TSMgIpcDlwP07t07lm9ljNlHJRVV/OLJ70IeKyyrrLMv9ZbdNa2Jx79YzUVH59ItK5WTHvwyUN6vUzrR4g0Wo3IbNzjtb1mkJSUw7cax5BeVk5wQz9MXtdxOkUhaFmfhjD9crKpbgJ7A3xrzJqq6C5iGs7CvvYj4g1RPnEV+uP/tBeAezwJ2eMtDXFP7fR5X1ZGqOjInp/GLcIwxTWdbYfgZ+IWlwQvadpdWcve7TjfVGYf1BODIP38W1AI5/8g+JESxq8fbDdXYFdwlFU79U5Pi6NspncP7dohavZpLJLOhtgAvAlkicgrOYHeDYxYikuO2KHDXaYwHluAEjdPd0y4A/uc+n+K+xj3+mdv9NQU4250t1RcYAHwf4eczxuyn/FNKAc4c6QSAU9xV14VllUHnfr5sGzuLK/jxsO4cM6BmD+1rX/4h8PyunxwU1fqlehbTTV28NeL8UD+sL+Dal+e692gdaywgstxQZ+K0JD7HWbn9sIjcpKqv13shdAOedWcuxQGvquq7IrIYeFlE7gF+AJ50z38SeF5EVgI7cWZAoaqLRORVYDFQBVztdm8ZY1qoldv2sK3ICRbv/uoYhvbI4t6fHsIPG3bx7vzNFJYGB4vCMueX+u2nHEiVr+ZL+/Nl+QBMOqgrItFNLJGVGtzbXlVdTXxcwxMx//vd+sDztMTWM3EzkrB3GzBKVbeB02IAPqFmRlNIqjofqLMVlKquJsRsJlUtA84Ic68/An+MoK7GmGa2dEshr83K44Ijc0PuMb10SyGTHqgZZ+ialQI46xqyUp2vJH9w8Ctx8yylJyWQnpzAcxcfzvlPfR+Y0vq3Mw6J+ueonewv0izlr83Oq7lHC8wuG04kHXxx/kDh2hHhdcaYNmb68nwmPfAlT361hmtf+SHkOduLgveayE6rSR3ezt0j+553F7PCs8udf8zAP7toeK2d5jJi8KUsInz1u3FcNqYvELyPdm3//GwFz327NqjsP+ePbJE5oMKJ5C/8oYh8hLN6G5wB7/djVyVjTKT+/P4SPl26jcuP7UeHtCROHBK9dQZ744KnaoYTf1i/K+Q5RZ7xiN4d0oJmHbVzu362FZUz/v4v+M/5I+nVIZWHPl0B1KyqTq3VvRPtLii/ntlpdM50Wj71DVn8/ePlALzuaVWMbOHpPWprMFio6k0i8jPgaLfocVV9K7bVMsaEUlFVzfqdJRzQOQOAV2ZtYFdJZSAP0dp7T27O6kVks2cK7NEHdAo6lpIYT3JCXKB76bLnQq/mbsoFbv44VF/Lwm9+3u7A89pTf1u6iP7iqvqGql7vPixQGNNM/vT+Ek68bzqbdpWyaNNudpUEDwTvTcK7aPhk8VYOvvMjAE4Y3BmALu1CL2TLK6jJ/eRd4ew3rFf7OmXNyd9q0Ub8aVMS44JmU7UGYVsWIlKEm5a89iGcbB7tYlYrY0xIHy/aAsCPHvqyTqAAKCipoEu7lCat04adJVzqaQFccFQunTKS+Xz5tpDn7y6tpFtWCq/+8kh6ZtfNl1Qdpr/nyH4do1PhRvL3kmnIr8PQDuzW+r4ew7YsVDVTVduFeGRaoDCmeWS7+0h7A8VNEwfx718cCsA3q7Y3eZ3G/HVa4PnLl4/m2IE5pCTGUVRWVeeL/7evz+ONOXlU+qrp1SEt5FiDL0R3T7+cdF66fHTI989Mie2MI//ivMZsw50Y13oGtv3CfiIRGSUiJ4UoP0lEDotttYzZP6gqy7cWNVv3jleVr5pFmwoZ4vnVOrBLBucf2YcBXZx0bQ99urJJ6+T9uxzWJ5vR7q//lKR4Sip89Lv1/UCajipfNa/OcgaALzmmX9h7erup/PwbCXldPa4/N4wfyFe/O36fPkND8gpKAHjp+/V1ji3dUsjny+q2oBLiW99movWFv7/gLISrbTGNTPdhTEs1Y/VOJtz/BX/9cGlzV4WpbmZV77TRKdccQ2ZKIv1znKCxY0/TbmL57nxno6H0pHj+/Yua35CDu9bkGh3950+prtZAwsCe2alcObZ/2HvuDtG9Fmps+aaJg/nVCQPqLJ6LttX5TqD6YOHmOscm/+trLnx6Zp1y7wyv1qK+9lumqq6rXaiq60SkU6gLjGlt/KmwF2zc3cCZsVFRVc2Nr83jZ4f15Eo3s+rlY/oxebiTeNk74yY5IY6qxvSV7IOySh/frtoRmP3z/W0nBi1AO3VYD657ZV7g9SOfrwyshXjzyqPqvXeojYZ+PKx7NKq9V/xdZdUhGpdllaFbnI3d/6IlqC9Y1DdJuO6yTGNaoT3uyuGm+hL2K6/y8fy36xjWqz1T5m1iyryarUJ7ZKeSGyK7amJ8HJWN3NFtb93+9kJem53HsJ5ZdM9KqbNSOS5O6N0hjfU7nS6c79bs5PBcJ5lee88ivEideGDnfa/0XooLM3U2VBp1P++mR61Ffd1Qn4jIH8UzAiWOu4HPYl81Y5rfG+4iq6YOFq/OyuOe95Zwxr+/DSr/9pbjw64xSIiPo9KnaKR5KfbBNLeffl7e7rApLYrLa1J2+KrVTTse1+Cq5l8eFzyeMaxnFof0bL7ptP5vwNp/1vunLg97Tff2LXtXvFDqa1ncADwBrHQ3MAIYBswCLo11xYxpbqrKHHcV8rwNoVcjx/K9Q+mWFf5LKMkdVK30KUkJsesz91Ur2/dUBL0OpcgTLL5ZtYOUxHhyMhveROiWkw7klpMO5JPFWxnRuz0dG7nxULQJ7jqLWlNnd5fWHVvxy06L7ThKc6hv6myxqp6Dk1r8GfcxQVXP9myXakyrVVJrD4NQM3JiIb+onJe+r9kc8qj+ka0v8Lc4qjyd6+t3lLBuR3TrPWVe8HYy4fa8rj2D7LOl2+jQiC6oE4d0afZAAeCfBVs7ftee9vuHUw9iYBdnZX2sB92bQyTpPlYDq5ugLsbsV/xbdx47MIcvluezY085faO4E1s45z4xg+Vba36PHdYnm0N7ZzPA/SIKxx8sDv3DVN7/9Rj65WRw7N+cNRDRSAOyfkcJa3YUc/2r84LKO2aEDgAj+2Qza10Bb1x5FD979BuAFrmqOTDAXStaeCc8JcXHcd6RuUwc2pWZawr2alxmf9d68ucaE2X3f+L0SZ83ug9fLM9vkrUWL8xYFxQooGZFdEMS3W6osspqFmzcTb+c+oNLY/ywvoDTHvkm5LHhYdJzPH3RKPKLyunhWaWd1gI3A/IvyqvdsvDupOefwdU5M4WT3Q2cWpuW97+cMU3AuxLa/8u5vNZMo53FFfzxvSWcNarXXm2buWxLEfd+sIT/nD8ysB3o/729EIBLjunLhUflEh8nEQUKCE6uV/tXcKWvmsT4OF6dtYHfvj6feXdMiLirpMpXXSdQ/PTQHuR2TKdPx7Swv6IzUxLJTAl+j9rZYlsCf0jw/k0Lyyp5fkadlQWtWn0ruDvU92joxiLSS0SmichiEVkkIte65a+IyFz3sdY/eC4iuSJS6jn2b8+9DhORBSKyUkQektqdhcZE2WPTa3pek9wv4XJ3Tn1BcQW5N7/H3z9exhtz8rjxtXkh71GfXSUVTHzgC6Yty+eA2z6oc/z2U4bQq0Nao2bVeIOFdzc5gLd+cMYZHpu+CoBthWVE6me1ZmQBDOicya9PGMCp7nqPSLXEbqiaqbM1ZbX/fv5pwa1ZfXPYZuPMfJoN5APLgRXu89kR3LsKuEFVhwCjgatFZIiqnqWqw1V1OPAG8KbnmlX+Y6p6haf8UeAynP23BwCTIvt4xuwd/7TPZy4aRUqi888k310dvXhzIVCzfeamXXXTUzTk8S+ChwG/WbU9bAK9SHlXDT/99Vqg5pf8rhJn9pJ/3Uikm/IUFFcEZoI9dM6IQOBsl9q4TolJB3UFIK0lBou4umMWtRfdtYWfr/XNhuqrqv1wtlD9sap2UtWOwCnAxw3dWFU3q+oc93kRsAQI/AxxWwdnUrOpUkgi0g1opyQ82vcAACAASURBVKoz1JlP+BwwucFPZsw+WLaliFMO6cbYQZ0DG/JMmev8Oq/dd11VrSFTVNTHv3jukXOdBIA//893vLvASSfx+1OG7FWdC0pqprMu3lxIla+a9GTny9nfKtpa6AS8cNNda9tR7Nzz72cM4yfDupPlTglt7Je+f8psS9zjITB11vMnq/TVHuxu/dEikp8Xo1U1sDOeqn4A1L9evxYRycXZj/s7T/EYYKuqrvCU9RWRH0RkuoiMcct6AHmec/LwBB1jou3jRVsoKq9iq9vV0DkzhY7pSYHB2ekhUm//JswWouFsKSynQ3oSJw3tGij7YX0BAD8ZvnepLWp3PW3eXRaY/lvhqw5quUS6yPD3/3PGUPx/C/84R2PHHtLcoPXd6h2Num5/4m1Z+Cc7+FtMrTDJbB2RfMRNIvJ/7phCrojcBmxq8CqXiGTgdDf9RlULPYfOIbhVsRnoraojgOuB/4pIo1Khi8jlIjJLRGbl5+c35lJjAh50t/C8bvzAQFlup3R81cqmXaW8v2BLnWu2FDacwK/KV82rMzewIG8378zbxIhe7RGRQN6jjQWlZCYnRDygXVulu74iwe02GfPXaYFg8fBnK7nznUU150aQFuSZr9fwzSrny/20Ec7vM//GRvUtDgzlqrHOWowLj85t1HX7gxsnOv8/mHhQTWD3Bwv/dOa20LKIpOPxHOAO4C2czZC+cMsaJCKJOIHiRVV901OeAPwUCKSpVNVyoNx9PltEVgEDgY1AT89te7pldajq48DjACNHjmza/Aym1dhaWMbJB3fjqP41+TLj44Sq6mqOurdupptxg3IC4xnh3PjavKD9mYFAWvEJQ7rwzrxNfOxmld1bKQnOr/exg3L4ZEnd1s/LM2sW+tVuhYRy5ztO0unuWSmBgfbfTRrMpWP6RbQS2ysrNbFFbPkaSresVJLig3e+8wfbMQNyWLejhBsnDGqu6jWZSBbl7QSuFZF0VY14Kag7JvEksERV76t1+ERgqarmec7PAXaqqk9E+uEMZK9W1Z0iUigio3G6sc4HHo60HsY0Rlmlj53FFfTvHLxGISFOKA2ROC4zOYGlW4rYvLuMKl91YApsbbUDBcDphzm/gbY2YmZSfX4xug9lVT5G5XYIGSx8Qd1Qka8Z8Y4zxMVJowNFa1BZXR1Iigg1A9zpyfE8dM6I5qpWk2qwG0pEjhKRxTgD1IjIMBF5JIJ7Hw2cBxzvmQ77I/fY2dQd2D4WmO9OpX0duMINVABX4eapAlYBdecaGhMFG3eVUq2Q2zE4sXJ8nFAYIhfQpWP6Bb5Ml24pCpQ/9OkKTndXLYdbzOcfJPZOPx0XYk/qSCUlxHHV2AOCxhMmDOkSeO4NFrUHaGtbkOdJyd76e1gapArvzd/MBjdg+BfhJUc4q6w1iOST3g9MBHYAqOo8nC/2eqnqV6oqqnqIZzrs++6xC1X137XOf0NVD3LPO1RV3/Ecm6WqQ1W1v6peo02RVtO0KZW+aqp81exwE+TV/vW8aVcpq/LrNqwzUxK4+9SDgOBcUvdNXc6sdQX4qjWQcO7Co3KZc/v4wDn+9/D/NzM5gScuGLXPn8W73qJHdiqH9q67wnr68vrH9N6Y47SEslIT+ePkg/e5Tq2F/3/LSvcHQLgMwK1RRJOlVXVDrXVw4RO5G9MCDb3jo6C58x3Tg4OFN1DcNHEQf/toGQCdMpNJdscKQrUgthbWzEga0bs9HdJrVjt7v2hev+JIenVIi8oOa94tPTOSE3jx0tGc8vCXQZ9hpydrbCiFZZX0aJ/K1zfHdsvSlsb/M9Xfsoh0vUprEEmw2CAiRwHqDlhfi9slZUxrsK2wrM4iq05hkuNB8JahPxralYWbnEl+Fb66v6HmbdhFitvd5F+v8ekNxwVmLPmNjOIKYO+905MTSE2Kp2d2WiBYZKYkhNyNDpzFiFsKy9hTVhXY2c7U8E+f9Q9wJ1nLIsgVwIM4axs24izIuyqWlTKmKf3jYydh4KjcbGauLaBru5SgFkBt6ckJvH310azfWUJCfFzgC6OiqpoVW4uCUpn7t0IF6OUm1OsfxQR/oXg7af1f+P5V6O1SEsjJTA47jnLZc7MC02UHNpDlti3y/2kXuz8QEq1lEWSQqp7rLRCRo4GvY1MlY5pOSUUVr8xyppQ+cu5hdMpIQrUmxUMoHdKTGNglM5Bt1d8VccULc8JeA3BA58x6j0eLd8Gdv1vLPwifmZJIUkI8GwpK2FVSUScJ4Ox1BYHntbPfmppNqfzTkG2AO1ioaao2ddW0eHe/s5ghv/8o8Lp9WiIiUm+gAOrsaRHJF8a5R/Teu0ruhX6e+vlTgPxvrrOOduOuUpIS4piftzuwx4SXzRypnz8Oj3VnrfnHq9qC+rLOHikiNwA5InK953En0Hb+QqbVeurrNYHnC++aWO/MlvvOHBZ4Xvu8SAY5yypjvxeGX1yc8PSFzqyqId2cJAjDemYFjm8vchYQhprd5XXEXqRdb+2qVdlTXkV6cgL9c2K/Edb+pL5uqCQgwz3H234uBE6PZaWMibWSipr9ofvnpDc4mFvfDnmhBjlvPmkwHdOTePDTFeQVlNK7Q1qIK2Nn3ODOzLjlBLpmpQDwwqVHcPCdH3PigV3IyUzmpe/X13v92aN6cdPE1r8qubHu+N+iQNbhAZ3b1phO2H8hqjodmC4iz6jqOgARiQMyauV4MqbF8Q9Cj8rN5m+nD2vgbEioJ1NcqJZFenICZ4zsxRkjezFnfUHgF35T8gcKcMYqlv5hEonxccSJk6rcn3rcr6zSR0VVNTeMH8ivThjQ1NVtEfyBAojKNOeWJJIxiz+LSDsRSQcWAotF5KYY18uYmPrMTYdx/fhB5Eawr7Y/VoTKFxcqWLT37EJ3aO/s/SI1d0piPPFxgoiQnhRPXkFJ0HH/CvTaqU5MaG1tD7ZIgsUQtyUxGSfNRl+cNB7GtCjTlm5zVlav3ck/pjrTZXtmR5Y91Z8e4+AeWXWO1V4zAXBEv/2/v79aCbQuKn3VTP6XM8Gxm6dFYsJrQ0ssgMimzia6i/EmA/9U1UoRsUkTpsUoLq/iya/WcJ8bIB5yU5D/44xh9IpwLMG/LiHU+ETtX5g3nzSYzpn79xfuyYd04+WZG9hR7Ax2b95Vk8xwf2gFtQRtIS25VyTB4jFgLTAP+EJE+uAMchuz33tv/mau/m/o9Q+N2WTI/2t7kmezIq8nzh/J7PUFPPr5Ko45oFPIc/Yn/nxU/h30tngy3zZm3++2zIJFLar6EPCQp2idiIyLXZWMiZ5b31oQeH79+IEs3Libjxdv5cKjchuVBK5XhzTm/n58YKe42k4c0oUTh3ThpgmDGlynsT/wrw/wpznxB4uPrzs27Gc0wVrA/8xR1WCwEJHfhzl0d5TrYkxUqCovz9zALW8uCCr/9QkD2LirFIAb92JaaO3VzqG0hEABNQsJy6ucfFbb3GDRpd3+3X22P2lrs6Ei6YbyrtxJAU7BEgma/diyrUVBgeJfPz+UrllOt0uP9qk8fv7I5qrafsMfLL5euYOzRvVmy+4yUhLjaJdiyQMj1dZmQ0XSDfUP72sR+TvwUZjTjWl2y9wpoL06pDL9xnEt5td+U/InSvRP+31vwWa6tktpc1+A+yK+jf2t9mbyVxrBe2KHJCK9RGSaiCwWkUUicq1bfqeIbAyxex4icouIrBSRZSIy0VM+yS1bKSI370WdTRvy5pyNtE9LZNoNYy1QhCEi9M9Jp7TSR2FZJZt3R2dr17YkWtvhthSRjFksoCa/WDyQQ2TjFVXADao6R0QygdkiMtU9dr+q/r3W+wzB2W71IKA78ImIDHQP/wsYD+QBM0VkiqoujqAOpg1RVVZvL2b68nyO7Ncx7H7YxpGaFE9ZhY8yd3Omi4/p28w1allWb68/t1ZrE0kH5Sme51XAVlWtCneyn6puBja7z4tEZAnOnhjhnAq8rKrlwBoRWQkc7h5bqaqrAUTkZfdcCxYmoKC4guP/8TkFJc62l9eeaOkqGpKaGE9ppY/x938BQHG5bYBpwqsv66w/mU2R51EKtBORbBGJeOWOiOQCI4Dv3KJrRGS+iDwlItluWQ9gg+eyPLcsXHmo97lcRGaJyKz8/Pr3GDatR3W1MuIPUwOBAmB0v47NWKOWISUxnuIKX2Bf6bJKCxYmvPra6f91/zsbmOX+1/+YA2wRkT819AYikgG8AfzGTRvyKNAfGI7T8vhHPZc3iqo+rqojVXVkTk5OtG5r9gNllT5UlRmrd1BQHLx/dF6BMx12YJcMHjn3UJ6w2U4RSU2MD0omaGkZGqdHG1u8WF/W2VPc/4bsyHRbFguBW8Pdw00T8gbwoqq+6d5vq+f4f4B33ZcbgV6ey3u6ZdRTblq5skofv3x+NtOX17QUD8/twLMXH05qUjxXvjCbDxZuAeCWHx3IuEGdm6uqLU5qUnDngKqFi8Z4+qJRzV2FJlVfN9Sh9T1U1aeqB9ZzvQBPAktU9T5PeTfPaafhBByAKcDZIpIsIn2BAcD3wExggIj0FZEknEHwKXv7gU3L8u78zUGBAuD7tTs58Pcf4qvWQKAAaJdiK48bI7VWDqhqCxZh/feyI+qU5Xa0zY/8/N1DKcBInNxQAhyC0y11ZAP3PhonO+0CEZnrlt0KnCMiw3FavWuBXwKo6iIReRVn4LoKuFpVfQAicg3O2o544ClVXdSIz2haiIqqav7y4VKuGtufjhnJlFf5uNVdXPfONcdwQOcM7n53cWDjnj+8GzzHoVeHttUtsK9qJwystlgRljcP1Mg+2fz3stER7ZDYmtTXDTUOQETeBA5V1QXu66HAnQ3dWFW/wgkutb1fzzV/BP4Yovz9+q4zLduq/D1kpyXxt4+W8tL3G3h15gZeuPQI4uOECl811504kIPdbUG966Ce+WYtADdOGMhlx/ZrU/shR0Ptbqg+TbybX0vibXRlpiS0uUABkU2dHeQPFACqulBEwnY/GdMY5VU+TvjH9KCyovIqTv3X1wzu6uzme9qImslvybX+kfbPSefKsQe0uTw90eDvhuqUkcw/fz7C9tyuh3c8pzEJKFuTSILFfBF5AnjBfX0uMD92VTJtyaZd4VfB+ndu69wuOVB2w4RB5GQm89cPlwHwyfXHWYqKveQPFhVVPptq3ABvF11bS03uF0mwuAi4ErjWfT0dZ/qrMftk4cbdnPLwVwA8ePZwRvfrSJd2KeTe/F7gnPZpiUF96xnJCVx6TD9Kyn2MG5xjgWIfpCQ6v5ALyxpcY9vmqWdicT3bsbdqkSQSLAPudx+IyBjgPuDq2FbNtEbbCsuYMm8TeQWl7HDXS3TLSmHswM5kpTmzmebfOYHXZ+Vx97uLef2Ko+rcIykhbq9SjJtgJxzYhdv/Z3NFIuFtWbTVHygR5SMWkRHAOcCZwBrgzVhWyrQsT3y5mgc/XcGCOyeGPUdV+XTJNi59blZQeW7HND6/KXgvrXYpiVx8TF/OPrwXaUmWMjtWOmUkN3ySAYKnFVs3VC1uEr9z3Md24BVA/LOkjPG75z1ne5PHpq/i9dl5/ObEgZx8SLegcz5Zso3LagUKgEN7Z9cp87NAEVuJ8W3zS29vZCbX/H+xrc6lqK/3bSlwPHCKqh6jqg8DljzGBPHOEvnzB0tZsW0PV/93DtsKy9hVUpOWY90OJ0Pn4X07sOTuSYHyI/rZDJzm0la7U/bGyNwOgY2h2to+Fn71BYuf4uRumiYi/xGREwi9bsK0Yac98k3I8sP/9CnD757Khwu3UOWrZkdxBYnxwiuXjw6a33/aiAa3RjFmvzDZncLdVoNsfYvy3gbeFpF0nJTgvwE6i8ijwFuq+nET1dHsp6bM28RcNxHdeaP7sHZHMRVV1Xy3ZmfgnCtemE2/nHRW5zstC/8/tKSEOCqqqtvk4ibTMvnX8rTVbqhIZkMV42Sg/a+bTvwM4HeABYs27tcv/QDAf84fyfghXQCYsXoHZz8+A4BOGUls31MRCBReX9w0jlJLid3sjhuYQ89sS5MSiYRAsGib0aJRI4iqWgA87j5MG1ZU5uyBcO4RvQOBAmBE7/Z0y0qh0qe8fPloPl2ylT9/sBSA9359TOC8rlkpTVthE9KzFx/e8EkGgHh3gYWtszCmEfythTEDOgWVJyfE8+0tJwRer99Zc95B3bOaroLGRJm/ZWFjFsY0wl8/cloLB3TOqPe84wd34eubj6dLps3pNy2bf8yibYYKCxamEXzVyqJNu+naLoWZawvo2i6FAzpnNnhdW9tRzLRO/rGKOet3NXBm62TBwkTs3fmbuPbluYHXk0eE3ArdmFbpyP4duf8T2LCzpLmr0ixiNlQjIr1EZJqILBaRRSJyrVv+NxFZKiLzReQtEWnvlueKSKmIzHUf//bc6zARWSAiK0XkIWmrnYbNoLzKx4qtRagqa7cH/yM5/8g+zVQrY5pen47Ofh+VvupmrknziGXLogq4QVXniEgmMFtEpgJTgVtUtUpE/gLcgjMVF2CVqg4Pca9HgcuA73A2QZoEfBDDurdZczfs4oMFm2mflsSAzhnc+Po8dpVUBp0THyc8feEoulv3kmlDktx9LHxtdEvBmAULVd2MswIcVS0SkSVAj1qL+WYAp9d3H3fP7naqOsN9/RwwGQsWUVflq2byv75u8LxFd02ssyWnMa1doruAtKqNBosmmTEsIrnACJyWgdfFBH/p9xWRH0RkupsKHaAHkOc5J88tM1E2a11ByPJfHtePL24axwmDO/P21UdboDBtUlIb3SHPL+afXkQygDeA36hqoaf8Npyuqhfdos1Ab1UdAVyPs2K8XSPf63IRmSUis/Lz86PzAdqQFdv2AHBgt3bMuX18oLx/Tga9O6bx5IWjGN6rfXNVz5hm1daz9MY0WIhIIk6geFFV3/SUXwicApyrbtpSVS1X1R3u89nAKmAgsBHwZpvr6ZbVoaqPq+pIVR2Zk5MTg0/Uuq3YWkRmSgLv//oYOqQncerw7gCBvbCNacv882omHdS1mWvSPGI2ZuHOWHoSWKKq93nKJwG/BY5T1RJPeQ6wU1V9ItIPGACsVtWdIlIoIqNxurHOBx6OVb3bsqKyKrJSEwP/KO4/czh/Ou1g0pNthrUxAD/cPr7N/nuI5ac+GjgPWCAi/sn5twIPAcnAVPdLaYaqXgEcC9wtIpVANXCFqvrTl14FPAOk4oxx2OB2FL02awMPf7aSxHghMyUxUB4XJ232H4YxoWSnJzV3FZpNLGdDfUXolfHvhzn/DZwuq1DHZgFDo1c741fpq+am1+cHXh8/uHMz1sYYs79q28P7rcD05fnc/c5iFuTtRlXZvLuUwrLKhi90TZm7Kej1gC7153oyxrRN1sfQwqgqu0srKa30ceSfPwuUP/X1msDzQV0y+ei6Y5m3YReH9MwKmyWzrNLHDa/NA+B3kwbz7eod/GRY99h+AGNMi2TBogVRVfreErIXL8iyrUV8umQrlzw7C4AHzx7OqcPrLk154JMVgeeXjunLlWP7R6+yxphWxbqhWpDD//RpnbJ//+JQVv/pR4Fprn5frtgeeH7ty3PJvfk9lm0pCpSt21HMv6evAmDajWNJbOMLjowx9bNviBYir6CE/KJyAF645Ag6ZTj7Q0wa2o24OOHBs0ew9t6Tue7EgQA8883aOveYungL9328jI8WbWHdDmfW8jmH96Zvp/Sm+RDGmBbLuqFaiIUbdwPwxPkjOWZAJ7763Tg0RIqalMTw8f/vHy8PPH/wbCdf4yXH5Ea1nsaY1slaFi3ECzPWAzAyNxuAlMR4UpPq5mjyjmUffUBHvrv1BJbcPanOedOWbgOgczvbC9sY0zALFvuht37IC7Qk/OZu2EVyQhzt0+pfFNS3U83U1xcvHU2XdimkJsVzx4+HADB2kJMG5e25m8hKTaSdZxGeMcaEY91Q+5m124u57hVnOusDZw3nN6/MZeFdEympqOKqsQc0eP34IV2Yc/v4Ohu0XHR0X34yrDvt05K48Onv+XLFdlIte6wxJkLWsgjh/qnLOeuxb2Ny722FZfzowS/5cOEWyqt8Qcde/G4dY//+eeD1P6YuA2DoHR9RrTUbxjekQ3oSXUJ0L3XMSCY+Tjj6gE4AVFW3zR2/jDGNZ8Gilo8WbeHBT1fw3Zqdgaml0fT5snwWby7kihdm87vX51Ppq6bSV82OPeXc9tbCoHM37CwNep2eHJ2WQJ8OzvaQ2/dUROV+xpjWz4KFx+6SSq5/ZW7g9b0fLA08/3DhZtbtKN6n+6/ZXsxv36jJw/T23E0MuO0DjvnLZ8zdsAuAP512MB9cOybk9eceEZ09rw/qngXAyD7ZUbmfMab1szELj6y0RN799RjGebqCcm9+j/vPGsZ1r8wjLSmexSFmFjVk5bYiTrzvC/y9SBOGdGHO+l1s3+Osm9haWM68PGdAe8yATuRkJte5x/SbxkYtA2zvjmm8+6tjAms1jDGmIdayqKVvp3ROP6xnUJl/wLmkwhfqkgad9dgMAPxb994zeShjBnQKOuehT53UG9npSaQkxnPjhIFBx6OdKnxojyy6Ztm0WWNMZCxYhBCt1BdfrsjnpAe/ZEdx8NhAdnoS90wOnXE93V07cc3xA3j6wlGB8gzbV8IY04wsWITwmxMHkJQQ+k8zdfHWiO/zwox1LNlcWKc8MT6O9OQE/nHGMCYM6RJ0zJsh1hu0ksPUxxhjmkLMvoFEpJeITBORxSKySESudcs7iMhUEVnh/jfbLRcReUhEVorIfBE51HOvC9zzV4jIBbGqs1+XdinM/r8Tg8quGeesccgrKAl1SUhrttcMiP/yuH6MGdCJB84aHij72WE9efz8kXx+41h+Mqw7c38/Puj6Htmpgefh0owbY0xTiGXfRhVwg6rOEZFMYLaITAUuBD5V1XtF5GbgZuB3wEk4+24PAI4AHgWOEJEOwB3ASEDd+0xR1YIY1p20pOA/zZH9O/LPaSuZ585aasgtb85n+dY9nDaiBzdNHES3rJSwX/i5ndJ56JwRdcotwZ8xZn8Rs5aFqm5W1Tnu8yJgCdADOBV41j3tWWCy+/xU4Dl1zADai0g3YCIwVVV3ugFiKtD4KUmNVHsBXI/2zq/8t2vtLBfO1MVO7qXjBubQvX2qtQyMMS1ak3SEi0guMAL4DuiiqpvdQ1sAf6d9D2CD57I8tyxcecw9em6gJ4yuWSnEx0lE002vffkHtu8p57iBOXX2mTDGmJYo5lNsRCQDeAP4jaoWen9hq6qKSIhE23v9XpcDlwP07t17n+930sHdWHz3RIrLfaQkxnPywd2Yl1d/N9T2PeX8z219xMm+jzU8dt5hIVORG2NMU4ppy0JEEnECxYuq+qZbvNXtXsL97za3fCPQy3N5T7csXHkdqvq4qo5U1ZE5OTlR+QxpSQmBRXKZKQnsKauq9/xZa2uGUqKxNmLiQV2ZNLTrPt/HGGP2RSxnQwnwJLBEVe/zHJoC+Gc0XQD8z1N+vjsrajSw2+2u+giYICLZ7sypCW5Zk0tPTqC4ov5gsX6nMwNq3KAc/jj54KaoljHGxFwsu6GOBs4DFoiIP+HSrcC9wKsicgmwDjjTPfY+8CNgJVACXASgqjtF5A/ATPe8u1V1ZwzrHVZ6UgJlldVU+apJCLNwr7TCyeT6xAWjIs4Sa4wx+7uYBQtV/QoI9215QojzFbg6zL2eAp6KXu32jj/ra3GFj6zUusFi2tJt3P+Js3WpBQpjTGtiy4IbwT8G8cjnKwF44JPlnPnYtxS46TwuemZm2GuNMaYls2DRCP5g8dj01QA88MkKvl+zk2Vbi1i5rag5q2aMMTFl2ekaoV1KzZ9rxdaa4LBw427++uGywOs3rzqqSetljDGxZi2LRjiyf8fA8+terdkk6Z73lpDqZos9c2RPDu1tmwoZY1oXa1k0QnJCzbamCzcGZ5Md2qMdZZXV/PX0YU1dLWOMiTlrWTRSqFTh/XPS2VNWZXtOGGNaLQsWjfTipUfUKVuVX8zq/GIyUixYGGNaJwsWjXRYn2wyPUHhpomDACgqryLTWhbGmFbKgkUjiQhXHNcfgJtPGswJB3YOHLM9rY0xrZX9FN4LFxyVS1FZFRcelUtphS9Q/qvjBzRjrYwxJnYsWOyFjOQEbj5pMAApiTUzpCzFhzGmtbJuKGOMMQ2ylkUUvHHlUazK39Pc1TDGmJixYBEFh/XJ5rA+tmrbGNN6WTeUMcaYBlmwMMYY06BYbqv6lIhsE5GFnrJXRGSu+1jr30FPRHJFpNRz7N+eaw4TkQUislJEHnK3azXGGNOEYjlm8QzwT+A5f4GqnuV/LiL/AHZ7zl+lqsND3OdR4DLgO5ytVycBH8SgvsYYY8KIWctCVb8AQu6V7bYOzgRequ8eItINaKeqM9xtV58DJke7rsYYY+rXXGMWY4CtqrrCU9ZXRH4QkekiMsYt6wHkec7Jc8uMMcY0oeaaOnsOwa2KzUBvVd0hIocBb4vIQY29qYhcDlwO0Lt376hU1BhjTDMECxFJAH4KHOYvU9VyoNx9PltEVgEDgY1AT8/lPd2ykFT1ceBx933yRWRd1D+AMca0Xn3CHWiOlsWJwFJVDXQviUgOsFNVfSLSDxgArFbVnSJSKCKjcQa4zwcejuRNVDUnBnU3xpg2KZZTZ18CvgUGiUieiFziHjqbugPbxwLz3am0rwNXqKp/cPwq4AlgJbAKmwlljDFNTpxJRsYYY0x4toLbmCgQEZ+7oHSRiMwTkRtEpN5/X+5i1J83VR2N2RcWLIyJjlJVHa6qBwHjgZOAOxq4JhewYGFaBOuGMiYKRGSPqmZ4XvcDZgKdcGaYPA+ku4evUdVvRGQGcCCwBngWeAi4FxgLJAP/UtXHmuxDGFMPCxbGREHtYOGW7QIGAUVAtaqWicgA4CVVHSkiY4EbVfUU9/zLgc6qeo+IJANfEkdUCgAAAPJJREFUA2eo6pom/TDGhGD7WRgTe4nAP0VkOODDWUMUygTgEBE53X2dhTON3IKFaXYWLIyJAbcbygdswxm72AoMwxknLAt3GfArVf2oSSppTCPYALcxUeYuMv038E83AWYWsFlVq4HzgHj31CIg03PpR8CVIpLo3megiKRjzH7AWhbGREequ6g0EajCGdC+zz32CPCGiJwPfAgUu+XzAZ+IzMNJ6f8gzgypOW5m5nwsy7LZT9gAtzHGmAZZN5QxxpgGWbAwxhjTIAsWxhhjGmTBwhhjTIMsWBhjjGmQBQtjjDENsmBhjDGmQRYsjDHGNOj/AREA0LiCFiQlAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Imports\n", "import matplotlib.pyplot as plt\n", "import matplotlib.dates as mdates\n", "\n", "# Plot the data\n", "plt.plot(stock_data)\n", "\n", "# Add title and axis labels\n", "plt.title('S&P 500 Index')\n", "plt.xlabel('Date')\n", "plt.xticks(rotation=45)\n", "plt.gca().xaxis.set_major_locator(mdates.YearLocator())\n", "plt.gcf().autofmt_xdate()\n", "plt.ylabel('Adjusted Close')" ] }, { "cell_type": "markdown", "metadata": { "id": "bdbb420b" }, "source": [ "The plot is looking great! We can see that we have data from about five years to work with. There are no gaps, and there are some visible dips and spikes.\n", "\n", "Clearly there is a pattern here (it's not just random noise), and we want to build a model that can predict that pattern. Before we can do that, we need to preprocess the data so we can build and train an RNN model. Let's move on to that data preprocessing." ] }, { "cell_type": "markdown", "metadata": { "id": "743cc550" }, "source": [ "## Data Preprocessing" ] }, { "cell_type": "markdown", "metadata": { "id": "8454185d" }, "source": [ "Before we can build and train an RNN model to make forecasts based on this data, we need to complete two preprocessing steps:\n", "- Split the data into `train`, `validation`, and `test` sets, using 50% of the data for training and 25% each for validation and testing.\n", "- Scale the data to between `0` and `1`, fitting the scaler to the training data and using the fitted scaler to scale all three datasets." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "id": "007b1366" }, "outputs": [], "source": [ "# Import\n", "from sklearn.preprocessing import MinMaxScaler\n", "\n", "# Split into train, validation, and test sets\n", "train_size = int(len(stock_data) * 0.5)\n", "validation_size = int(len(stock_data) * 0.25)\n", "train_df = stock_data.iloc[0:train_size, :]\n", "validation_df = stock_data.iloc[train_size:train_size + validation_size, :]\n", "test_df = stock_data.iloc[train_size + validation_size:len(stock_data), :]\n", "\n", "# Fit scaler\n", "scaler = MinMaxScaler()\n", "scaler.fit(train_df)\n", "\n", "# Scale data\n", "train = pd.DataFrame(scaler.transform(train_df), columns=['Adj Close'], index=train_df.index)\n", "validation = pd.DataFrame(scaler.transform(validation_df), columns=['Adj Close'], index=validation_df.index)\n", "test = pd.DataFrame(scaler.transform(test_df), columns=['Adj Close'], index=test_df.index)" ] }, { "cell_type": "markdown", "metadata": { "id": "0dbb236a" }, "source": [ "We will also want to shape our data into fixed-length time windows and reshape it into NumPy arrays to prepare it for TensorFlow models. We will do this in a way that is repeatable and does not overwrite our `train`, `validation`, and `test` variables so that we have the freedom to modify this window size later." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "id": "1959002c" }, "outputs": [], "source": [ "# Define a helper function to construct windowed datasets\n", "def create_dataset(dataset, window_size=1):\n", " data_x, data_y = [], []\n", " for i in range(len(dataset) - window_size - 1):\n", " window = dataset.iloc[i:(i + window_size), 0]\n", " target = dataset.iloc[i + window_size, 0]\n", " data_x.append(window)\n", " data_y.append(target)\n", " return np.array(data_x), np.array(data_y)\n", "\n", "# Set the desired window size\n", "window_size = 10\n", "\n", "# Construct train, validation, and test datasets\n", "X_train, y_train = create_dataset(train, window_size)\n", "X_validation, y_validation = create_dataset(validation, window_size)\n", "X_test, y_test = create_dataset(test, window_size)\n", "\n", "# Reshape into NumPy arrays\n", "X_train = np.reshape(X_train, (X_train.shape[0], 1, X_train.shape[1]))\n", "X_validation = np.reshape(X_validation, (X_validation.shape[0], 1, X_validation.shape[1]))\n", "X_test = np.reshape(X_test, (X_test.shape[0], 1, X_test.shape[1]))" ] }, { "cell_type": "markdown", "metadata": { "id": "14a5a3de" }, "source": [ "## Build and Train a Basic RNN Model" ] }, { "cell_type": "markdown", "metadata": { "id": "e230e893" }, "source": [ "Now comes the fun part! We've thoroughly prepared our data for modeling and now we need to build a TensorFlow model to make forecasts. Let's start with a `SimpleRNN` model." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "0d9a000a", "outputId": "ecfd3103-8af6-49ea-c416-70fe0fbb4a37" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"sequential\"\n", "_________________________________________________________________\n", " Layer (type) Output Shape Param # \n", "=================================================================\n", " simple_rnn (SimpleRNN) (None, 10) 210 \n", " \n", " dense (Dense) (None, 10) 110 \n", " \n", " dense_1 (Dense) (None, 1) 11 \n", " \n", "=================================================================\n", "Total params: 331\n", "Trainable params: 331\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ] } ], "source": [ "# Imports\n", "from tensorflow import keras\n", "from tensorflow.keras import layers\n", "\n", "# Build the model\n", "model = tf.keras.Sequential()\n", "model.add(tf.keras.layers.SimpleRNN(10, input_shape=(1, window_size), activation='relu'))\n", "model.add(tf.keras.layers.Dense(10, activation='relu'))\n", "model.add(tf.keras.layers.Dense(1))\n", "model.compile(optimizer='adam', loss='mean_squared_error')\n", "model.summary()" ] }, { "cell_type": "markdown", "metadata": { "id": "ca885bb7" }, "source": [ "Great! And now that we've built the model, let's train it on our training dataset and evaluate its performance using the validation dataset. Note that we are _not_ using the `test` dataset yet because we want to have a clean, untouched testing dataset for our final model evaluation. We may have skipped this step in previous lessons for simplicity, but it is a best practice to set aside an untouched testing set during the model optimization process." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "03c24e26", "outputId": "7e5c6c1f-e9e5-4f7a-ddb3-a1e9c6018295" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "29/29 [==============================] - 2s 5ms/step - loss: 0.0775\n", "14/14 [==============================] - 1s 3ms/step\n", "-2.7877363666898582\n" ] } ], "source": [ "# Import\n", "from sklearn.metrics import r2_score\n", "\n", "# Train the model\n", "model.fit(X_train, y_train)\n", "\n", "# Make predictions and evaluate\n", "y_pred = model.predict(X_validation)\n", "print(r2_score(y_validation, y_pred))" ] }, { "cell_type": "markdown", "metadata": { "id": "475190db" }, "source": [ "Yikes! That R-Squared score is not looking very good. Let's see if we can improve upon this model." ] }, { "cell_type": "markdown", "metadata": { "id": "0d2f80d5" }, "source": [ "## Build and Train an LSTM Model" ] }, { "cell_type": "markdown", "metadata": { "id": "c4211a13" }, "source": [ "Let's repeat the above steps for an LSTM model and see if we can improve performance." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "6dc34b29", "outputId": "cf5e5f9d-cb9f-4c33-c163-73bebc393ce8" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"sequential_1\"\n", "_________________________________________________________________\n", " Layer (type) Output Shape Param # \n", "=================================================================\n", " lstm (LSTM) (None, 10) 840 \n", " \n", " dense_2 (Dense) (None, 10) 110 \n", " \n", " dense_3 (Dense) (None, 1) 11 \n", " \n", "=================================================================\n", "Total params: 961\n", "Trainable params: 961\n", "Non-trainable params: 0\n", "_________________________________________________________________\n", "29/29 [==============================] - 3s 7ms/step - loss: 0.2656\n", "14/14 [==============================] - 1s 3ms/step\n", "-44.39475292905578\n" ] } ], "source": [ "# Build the model\n", "model = tf.keras.Sequential()\n", "model.add(tf.keras.layers.LSTM(10, input_shape=(1, window_size), activation='relu'))\n", "model.add(tf.keras.layers.Dense(10, activation='relu'))\n", "model.add(tf.keras.layers.Dense(1))\n", "model.compile(optimizer='adam', loss='mean_squared_error')\n", "model.summary()\n", "\n", "# Train the model\n", "model.fit(X_train, y_train)\n", "\n", "# Make predictions and evaluate\n", "y_pred = model.predict(X_validation)\n", "print(r2_score(y_validation, y_pred))" ] }, { "cell_type": "markdown", "metadata": { "id": "e82aa818" }, "source": [ "That didn't improve things like we hoped. For now, let's keep the LSTM in place until we're ready to fully optimize the model, and we can decide then whether it's worth keeping. For now, let's move onto the next section to see what other techniques we can try to improve this model." ] }, { "cell_type": "markdown", "metadata": { "id": "d265d334" }, "source": [ "## Add a Convolutional Layer" ] }, { "cell_type": "markdown", "metadata": { "id": "c46ad12c" }, "source": [ "Next, we are going to branch out a bit from the basic RNN or LSTM model and try adding a convolutional layer to see if this improves the model performance." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "9afe33e8", "outputId": "480766cd-3378-4dde-bcff-e93662070621" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"sequential_2\"\n", "_________________________________________________________________\n", " Layer (type) Output Shape Param # \n", "=================================================================\n", " conv1d (Conv1D) (None, 1, 64) 704 \n", " \n", " max_pooling1d (MaxPooling1D (None, 1, 64) 0 \n", " ) \n", " \n", " lstm_1 (LSTM) (None, 10) 3000 \n", " \n", " dense_4 (Dense) (None, 10) 110 \n", " \n", " dense_5 (Dense) (None, 1) 11 \n", " \n", "=================================================================\n", "Total params: 3,825\n", "Trainable params: 3,825\n", "Non-trainable params: 0\n", "_________________________________________________________________\n", "29/29 [==============================] - 3s 6ms/step - loss: 0.1485\n", "14/14 [==============================] - 1s 5ms/step\n", "0.1346845828075679\n" ] } ], "source": [ "# Build the model\n", "model = tf.keras.Sequential()\n", "model.add(tf.keras.layers.Conv1D(64, 1, activation=\"relu\", input_shape=(1, window_size)))\n", "model.add(tf.keras.layers.MaxPooling1D(1))\n", "model.add(tf.keras.layers.LSTM(10, activation='relu'))\n", "model.add(tf.keras.layers.Dense(10, activation='relu'))\n", "model.add(tf.keras.layers.Dense(1))\n", "model.compile(optimizer='adam', loss='mean_squared_error')\n", "model.summary()\n", "\n", "# Train the model\n", "model.fit(X_train, y_train)\n", "\n", "# Make predictions and evaluate\n", "y_pred = model.predict(X_validation)\n", "print(r2_score(y_validation, y_pred))" ] }, { "cell_type": "markdown", "metadata": { "id": "7c1aed13" }, "source": [ "These are still pretty terrible results. Let's go further and try modifying other model parameters to fully optimize this model, at which point we may or may not keep the convolutional layer." ] }, { "cell_type": "markdown", "metadata": { "id": "38621317" }, "source": [ "## Optimize the Model" ] }, { "cell_type": "markdown", "metadata": { "id": "6475171f" }, "source": [ "Now we can go further to optimize this model by adding layers, changing the number of nodes in each layer, increasing the number of training epochs, and modifying the window size." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "c43d8153", "outputId": "de7cb21f-2cef-4e2e-d0db-490251875dbc" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"sequential_3\"\n", "_________________________________________________________________\n", " Layer (type) Output Shape Param # \n", "=================================================================\n", " conv1d_1 (Conv1D) (None, 1, 128) 3328 \n", " \n", " max_pooling1d_1 (MaxPooling (None, 1, 128) 0 \n", " 1D) \n", " \n", " lstm_2 (LSTM) (None, 64) 49408 \n", " \n", " dense_6 (Dense) (None, 32) 2080 \n", " \n", " dense_7 (Dense) (None, 16) 528 \n", " \n", " dense_8 (Dense) (None, 1) 17 \n", " \n", "=================================================================\n", "Total params: 55,361\n", "Trainable params: 55,361\n", "Non-trainable params: 0\n", "_________________________________________________________________\n", "Epoch 1/35\n", "28/28 [==============================] - 2s 5ms/step - loss: 0.0563\n", "Epoch 2/35\n", "28/28 [==============================] - 0s 4ms/step - loss: 0.0037\n", "Epoch 3/35\n", "28/28 [==============================] - 0s 4ms/step - loss: 0.0018\n", "Epoch 4/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 0.0014\n", "Epoch 5/35\n", "28/28 [==============================] - 0s 4ms/step - loss: 0.0011\n", "Epoch 6/35\n", "28/28 [==============================] - 0s 4ms/step - loss: 9.4565e-04\n", "Epoch 7/35\n", "28/28 [==============================] - 0s 4ms/step - loss: 7.2098e-04\n", "Epoch 8/35\n", "28/28 [==============================] - 0s 4ms/step - loss: 5.9372e-04\n", "Epoch 9/35\n", "28/28 [==============================] - 0s 4ms/step - loss: 4.8971e-04\n", "Epoch 10/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 5.1772e-04\n", "Epoch 11/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 4.5120e-04\n", "Epoch 12/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 4.6070e-04\n", "Epoch 13/35\n", "28/28 [==============================] - 0s 4ms/step - loss: 3.6563e-04\n", "Epoch 14/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 3.6149e-04\n", "Epoch 15/35\n", "28/28 [==============================] - 0s 4ms/step - loss: 3.7301e-04\n", "Epoch 16/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 4.0208e-04\n", "Epoch 17/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 3.5503e-04\n", "Epoch 18/35\n", "28/28 [==============================] - 0s 5ms/step - loss: 3.4902e-04\n", "Epoch 19/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 3.9261e-04\n", "Epoch 20/35\n", "28/28 [==============================] - 0s 4ms/step - loss: 4.0408e-04\n", "Epoch 21/35\n", "28/28 [==============================] - 0s 4ms/step - loss: 3.9225e-04\n", "Epoch 22/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 4.0301e-04\n", "Epoch 23/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 3.6759e-04\n", "Epoch 24/35\n", "28/28 [==============================] - 0s 4ms/step - loss: 4.4122e-04\n", "Epoch 25/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 3.9261e-04\n", "Epoch 26/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 3.0763e-04\n", "Epoch 27/35\n", "28/28 [==============================] - 0s 4ms/step - loss: 3.1373e-04\n", "Epoch 28/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 3.0350e-04\n", "Epoch 29/35\n", "28/28 [==============================] - 0s 5ms/step - loss: 3.0890e-04\n", "Epoch 30/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 3.5386e-04\n", "Epoch 31/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 3.1993e-04\n", "Epoch 32/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 2.7276e-04\n", "Epoch 33/35\n", "28/28 [==============================] - 0s 4ms/step - loss: 3.3147e-04\n", "Epoch 34/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 3.2994e-04\n", "Epoch 35/35\n", "28/28 [==============================] - 0s 3ms/step - loss: 3.5087e-04\n", "14/14 [==============================] - 1s 2ms/step\n", "0.941325545898109\n" ] } ], "source": [ "# Set the desired window size\n", "window_size = 25\n", "\n", "# Construct train, validation, and test datasets\n", "X_train, y_train = create_dataset(train, window_size)\n", "X_validation, y_validation = create_dataset(validation, window_size)\n", "X_test, y_test = create_dataset(test, window_size)\n", "\n", "# Reshape into NumPy arrays\n", "X_train = np.reshape(X_train, (X_train.shape[0], 1, X_train.shape[1]))\n", "X_validation = np.reshape(X_validation, (X_validation.shape[0], 1, X_validation.shape[1]))\n", "X_test = np.reshape(X_test, (X_test.shape[0], 1, X_test.shape[1]))\n", "\n", "# Build the model\n", "model = tf.keras.Sequential()\n", "model.add(tf.keras.layers.Conv1D(128, 1, activation=\"relu\", input_shape=(1, window_size)))\n", "model.add(tf.keras.layers.MaxPooling1D(1))\n", "model.add(tf.keras.layers.LSTM(64, activation='relu'))\n", "model.add(tf.keras.layers.Dense(32, activation='relu'))\n", "model.add(tf.keras.layers.Dense(16, activation='relu'))\n", "model.add(tf.keras.layers.Dense(1))\n", "model.compile(optimizer='adam', loss='mean_squared_error')\n", "model.summary()\n", "\n", "# Train the model\n", "model.fit(X_train, y_train, epochs=35)\n", "\n", "# Make predictions and evaluate\n", "y_pred = model.predict(X_validation)\n", "print(r2_score(y_validation, y_pred))" ] }, { "cell_type": "markdown", "metadata": { "id": "3e6c545a" }, "source": [ "Overall, this is starting to look really good! We are achieving an R-Squared value between `0.90` and `0.95` on the validation set, which is impressive. As a final performance check, we should now compute and visualize the performance on the testing set." ] }, { "cell_type": "markdown", "metadata": { "id": "9802018a" }, "source": [ "## Evaluate Model Performance" ] }, { "cell_type": "markdown", "metadata": { "id": "a83ac44f" }, "source": [ "Finally! We've settled on a model that we can be satisfied with, so let's use this model to make predictions on the testing set and compute our final R-Squared. While we're at it, since we'll need them for plotting, let's make predictions on all three sets: training, validation, and testing." ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "dcdc7f91", "outputId": "59646323-152d-4165-b8d7-114aedfaa0db" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "28/28 [==============================] - 0s 2ms/step\n", "14/14 [==============================] - 0s 2ms/step\n", "14/14 [==============================] - 0s 3ms/step\n", "0.9947958915297632 --> Training Set\n", "0.941325545898109 --> Validation Set\n", "0.9421163908433968 --> Test Set\n" ] } ], "source": [ "# Make predictions on all three sets\n", "train_pred = model.predict(X_train)\n", "validation_pred = model.predict(X_validation)\n", "test_pred = model.predict(X_test)\n", "\n", "print(r2_score(y_train, train_pred), \" --> Training Set\")\n", "print(r2_score(y_validation, validation_pred), \" --> Validation Set\")\n", "print(r2_score(y_test, test_pred), \" --> Test Set\")" ] }, { "cell_type": "markdown", "metadata": { "id": "c1562f53" }, "source": [ "Excellent! The R-Squared value from validation seems to have held up in testing, which is a great sign. Now it's time to visualize this performance. First, we'll need to undo the scaling and windowing preprocessing we've done." ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "id": "b97cd8f1" }, "outputs": [], "source": [ "# Un-scale the predictions\n", "train_pred = scaler.inverse_transform(train_pred)\n", "validation_pred = scaler.inverse_transform(validation_pred)\n", "test_pred = scaler.inverse_transform(test_pred)\n", "\n", "# Un-window the training predictions\n", "plot_train_pred = np.empty((len(stock_data), 1))\n", "plot_train_pred[:] = np.nan\n", "plot_train_pred[window_size:len(train_pred) + window_size, :] = train_pred\n", "\n", "# Un-window the validation predictions\n", "plot_validation_pred = np.empty((len(stock_data), 1))\n", "plot_validation_pred[:] = np.nan\n", "plot_validation_pred[len(train_pred) + (window_size * 2) + 1:len(train_pred) + len(validation_pred) + (window_size * 2) + 1, :] = validation_pred\n", "\n", "# Un-window the test predictions\n", "plot_test_pred = np.empty((len(stock_data), 1))\n", "plot_test_pred[:] = np.nan\n", "plot_test_pred[len(train_pred) + len(validation_pred) + (window_size * 3) + 2:len(stock_data) - 1, :] = test_pred" ] }, { "cell_type": "markdown", "metadata": { "id": "0eaebce1" }, "source": [ "Finally, let's plot the un-scaled and un-windowed data on top of the original `stock_data` dataset." ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 257 }, "id": "b3667f57", "outputId": "f98e4bca-5a8b-4bb9-85c3-4fea7a1235b3" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAADwCAYAAADxckg9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydZ3hU1daA3zWTCgmEQOgdaQGSAKFLV5pIExVEBVEUG+q1Ybli79+1ISI2LCh48YpKUaSJAkqNlACGEjQQWnrPlP39OCeTSU+ASSj7fZ55OLPrOiE56+y1115LlFJoNBqNRlMalqoWQKPRaDTnP1pZaDQajaZMtLLQaDQaTZloZaHRaDSaMtHKQqPRaDRlopWFRqPRaMpEKwuNphIRkQEiElfVcmg0FUUrC43HEJHLRWSjiKSISKKIbBCRbm71LUTkFxFJE5EjInJzMWMoEckQkXQROSoi/xERawnzxYpIltk2XURWFqp/QESOi0iqiHwsIr5udc1FZK2IZIrIPhG5opT7mi8iz5/ZT8UziMg6Ecl2u/d0EelV1XKVxfn4s9QUj1YWGo8gIjWApcA7QDDQCHgGyHFr9iIQa9b3AKJLGC5cKRUADAZuAKaVMvXVSqkA8zPETZ6hwExzjGZAS1OePL4CdgC1gSeAxSISUq6bPX+4x+3eA5RSmyrSWUS8PCWY5sJHKwuNp2gDoJT6SinlUEplKaVWKqV2urWxAXFKKZtS6rhSamtpAyql9gG/Ah3PQJ7JwEdKqT1KqSTgOWAKgIi0AboAs0w5vwF2AdeUNai5IlEiMllE/haR0yLyhFu9v/n2nCQi0UC3Qv0bisg3InJKRA6LyAyzPFhE4kTkavN7gIgcKG71VYZ8FhF50ly5nRSRz0SkZiHZbxWRv4E1ZvlUEdlryvyTiDRzG6+DiPxsrhRPiMjjZnl3EdkkIskiEi8is0XEx6wTEXnDnD9VRHaJSEcRuR2YBDxiroR+qMi9aSoXrSw0nuIvwCEin4rIcBGpVUybLcBDIjKsPAOKSCjQF2MFUBILzAfvShEJdyvvAPzp9v1PoJ6I1DbrDiml0grVdyiPXCaXA20xVi5PiUh7s3wW0Mr8DMVQWnn3YwF+MOdqZPa9X0SGKqUSganAByJSF3gDiFJKfVYBmcBQiFOAgRirqQBgdqE2/YH2wFARGQ08DowDQjCU81emvIHAKuBHoCFwGbDaHMMBPADUAXqZ93KXWTcE6IfxAlETuA5IUErNAxYAr5oroasreG+aykQppT/645EPxgNoPhAH2IHvgXpmXR/gMMaDKg4YZpZfBpwGxPyugFQgCTgIPA9YSpivD+APVAMeA44DQWbdwbw5zO/e5tjNgZuA3wuN9QIwv4R55gPPm9fNzXEau9VvBiaY14cKzXs7xmoKDNPb34XGfgz4xO37OxirnKNA7VJ+1uuATCDZ/Gw3y1cDd7m1a4uxovNyk72lW/0K4Fa37xZz3GbARGBHOf/v7we+Na8HYbw89Cz8f+f+s9Sf8/ujVxYaj6GU2quUmqKUaoxhOmoIvGlW3wPMU0r9AowFPjdXGH2Atcp8kph0UUrVUkq1Uko9qZRyljDfBmWYkTKVUi9hPDT7mtXpQA235nnXacXU5dWnUX6Ou11nYrzBg3HP/7jVHXG7bgY0NE03ySKSjPFWX8+tzTyMn918pVRCGTLMUEoFmZ8ubvO7z3kEQ1G4z+EuXzPgLTd5EgHBWPk0wVC6RRCRNiKyNM+BAGM/qg6AUmoNxmrmXeCkiMwz97Q0FxBaWWgqBWXsN8wnf7/BC+PtHqXUFuB6YCHwNPDauZoW40EHsAdwN0uFAyfMB/AeoKVpZnGv33MOZIjHeMjm0dTt+h/gsNsDPkgpFaiUGgFgen3NAz4D7hKRy85g/mMYCsB9fjtwwq3MXTH/A9xRSCZ/pdRGs65lCfO8B+wDWiulamAovbyfPUqpt5VSXYFQDHPUw8XMrTmP0cpC4xFEpJ2IPCgijc3vTTDMGL+bTf4LzBCRfqbtPh7DM6o+xsOsovM1FZE+IuIjIn4i8jDGm+0Gs8lnwK0iEioiQcCTGMoLpdRfQBQwy+w7FggDvjmTey/E18BjIlLL/Fnc61a3GUgTkUfNjXCrufGbtwn+OMbDdCqGAv1MSnAbLoWvgAfEcFMOwHjjX6SUKulnPNeUtwOAiNQUkWvNuqVAAxG5X0R8RSRQRHqYdYEY5sJ0EWkH3Jk3oIh0E5EeIuINZADZQN7q8AQlKyDNeYRWFhpPkYZhk/9DRDIwlMRu4EEApdTXGK6s88y23wLvY7xxLhWRpsUNWgqBGG+3SRj2/WHA8DzTjVLqR+BVYC3wN4Y5ZpZb/wlApNn/ZWC8UupUBWUojmfMuQ4DK4HP8yqUUg5gJBBh1p8GPgRqikhX4F/AzWa7VzAUx8wKzv+xOed6c45sCiqsAiilvjXnWmiak3YDw826NOBK4GoMs1sMxsY5wEMYbs1pwAfAIrdha5hlSebPIoH81eNHQKhp9lpSwXvTVCJ5m4gajUaj0ZSIXlloNBqNpky0stBoNBpNmWhlodFoNJoy0cpCo9FoNGWilYVGo9FoyuSijTJZp04d1bx586oWQ6PRaC4Ytm3bdlopVWy05YtWWTRv3pytW0sNYqrRaDQaN0TkSEl12gyl0Wg0mjLRykKj0Wg0ZaKVhUaj0WjK5KLdsygOm81GXFwc2dnZVS2K5hLGz8+Pxo0b4+3tXdWiaDTl5pJSFnFxcQQGBtK8eXNEpOwOGs05RilFQkICcXFxtGjRoqrF0WjKzSVlhsrOzqZ27dpaUWiqDBGhdu3aenWrueC4pJQFoBWFpsrRv4MaTxCXlMmA19YSl5TpkfEvOWVR1cTFxTF69Ghat25Nq1atuO+++8jNzS227bFjxxg/fnyZY44YMYLk5OQzkufpp5/m9ddfL7a8UaNGRERE0Lp1a8aNG0d0dHSZ482fP59jx46dkSwajebMWbwtjtiETBZu/qfsxmeAVhaViFKKcePGMWbMGGJiYvjrr79IT0/niSeeKNLWbrfTsGFDFi9eXOa4y5cvJygo6JzL+8ADDxAVFUVMTAzXX389gwYN4tSp0vMBaWWh0VQNAb7GFnR6ToUTTZYLrSwqkTVr1uDn58ctt9wCgNVq5Y033uDjjz8mMzOT+fPnM2rUKAYNGsTgwYOJjY2lY0cjZXVmZibXXXcdoaGhjB07lh49erhOqDdv3pzTp08TGxtL+/btmTZtGh06dGDIkCFkZWUB8MEHH9CtWzfCw8O55ppryMys2FL1+uuvZ8iQIXz55ZcAPPvss3Tr1o2OHTty++23o5Ri8eLFbN26lUmTJhEREUFWVlax7TQazbnHIoLV6fDY39gl5Q3lzjM/7CH6WOo5HTO0YQ1mXd2hxPo9e/bQtWvXAmU1atSgadOmHDhwAIDt27ezc+dOgoODiY2NdbWbM2cOtWrVIjo6mt27dxMREVHsHDExMXz11Vd88MEHXHfddXzzzTfceOONjBs3jmnTpgHw5JNP8tFHH3HvvSVm1yyWLl26sG/fPgDuuecennrqKQBuuukmli5dyvjx45k9ezavv/46kZGRJba7+uqrKzSvRnMpEp+ShVWEujX8ytX+5+gTPL75M4L3+sHoL865PHplcZ5x5ZVXEhwcXKT8t99+Y8KECQB07NiRsLCwYvu3aNHCpUi6du3qUji7d++mb9++dOrUiQULFrBnz54Ky+b+xrJ27Vp69OhBp06dWLNmTYnjlbedRqMpSK+X1tD9xdXlbv/HwVNEnD7AAbuPR+S5ZFcWpa0APEVoaGiRPYjU1FT+/vtvLrvsMrZv30716tXPag5fX1/XtdVqdZmhpkyZwpIlSwgPD2f+/PmsW7euwmPv2LGDyMhIsrOzueuuu9i6dStNmjTh6aefLtYVtLztNBrN2RFzIo2uJ/ZTzZ7D/qCmHplDrywqkcGDB5OZmclnn30GgMPh4MEHH2TKlClUq1at1L59+vTh66+/BiA6Oppdu3ZVaO60tDQaNGiAzWZjwYIFFZb9m2++YeXKlUycONH1wK9Tpw7p6ekFFGBgYCBpaWkApbbTaDTnjoSMXFonG15QO0NaeWSOS3ZlURWICN9++y133XUXzz33HE6nkxEjRvDiiy+W2feuu+5i8uTJhIaG0q5dOzp06EDNmjXLPfdzzz1Hjx49CAkJoUePHq4Hemm88cYbfPHFF2RkZNCxY0fWrFlDSIgR6n7atGl07NiR+vXr061bN1efKVOmMH36dPz9/dm0aVOJ7TQazbljzrqDdM5KJtE3kDuuu9wjc8jF6p0SGRmpCuez2Lt3L+3bt68iic4Oh8OBzWbDz8+PgwcPcsUVV7B//358fDxjn9R4lgv5d1FTOVx12zu0SI1n9tcFXyadTsWLy/cyuXdzmgQbFomXl0QxeuZEDtdowIjNa854ThHZppSKLK7OYysLEfED1gO+5jyLlVKzRGQ+0B9IMZtOUUpFiXGs9S1gBJBplm83x5oMPGm2f14p9amn5D5fyczMZODAgdhsNpRSzJkzRysKjeYi5pUNc7EqJ87MJ7G4man/OpnGh78dZnNsIt/fY6wiWm0xFERLX4fH5PGkGSoHGKSUShcRb+A3EVlh1j2slCpswB4OtDY/PYD3gB4iEgzMAiIBBWwTke+VUkkelP28IzAwUGf+02guERxOhVU5jeukpALKwstibDUnZ9pcZX8lZtMRaDb7bY/J5LENbmWQbn71Nj+l2bxGA5+Z/X4HgkSkATAU+FkplWgqiJ+BYZ6SW6PRaKqaWYu2uK4dKSkF6uxOQ4lk5hqriMSMXNLjTwLg60HTpke9oUTEKiJRwEmMB/4fZtULIrJTRN4QkTxfz0aAe1CTOLOspPLi5rtdRLaKyNaywlJoNBrN+Upq1E7XdWFlkWNz0j9uB59/cS/Z+/aRnpZJ7ewUcqsHYvGgadqjykIp5VBKRQCNge4i0hF4DGgHdAOCgUfP4XzzlFKRSqnIPK8djUajudBonJ7/sutIKRhpItfhZOZWw/398JixZN56M43TT+Fo2MSjMlXKOQulVDKwFhimlIo3TU05wCdAd7PZUcD9bhubZSWVazQazUWJMz4/GKc94XSBusJRZdXBGKrbslE1anhUJo8pCxEJEZEg89ofuBLYZ+5DYHo/jQF2m12+B24Wg55AilIqHvgJGCIitUSkFjDELLugSEhIICIigoiICOrXr+8K/x0REVFiiPI8tm7dyowZM8qco3fv3udE1szMTCZNmkSnTp3o2LEjl19+Oenp6aX2Ke2sSPPmzenUqRNhYWEMGTKE48ePn7Fs7iHVn3rqKVatWlVi26ioKJYvX+76/v333/Pyyy+f8dwaTWXgcCpqppwmvlptcqze2P4uqBz8TxZ9V/Z15IKfv0fl8qQ3VAPgUxGxYiilr5VSS0VkjYiEAAJEAdPN9ssx3GYPYLjO3gKglEoUkeeAvB2fZ5VSiR6U2yPUrl2bqKgowHjgBQQE8NBDD7nq7XY7Xl7F/3dERka6AvOVxsaNG8+JrG+99Rb16tVznRLfv39/mfmiX3zxRR5//PES69euXUudOnV4/PHHefHFF3n77XyvDaUUSiksloq9uzz77LOl1kdFRbF161ZGjBgBwKhRoxg1alSF5tBoKpv0bDshWcmcrBaE3deX2n//XaA+8O+DAKT4VKdmbgYAQTnpiF/5Ag6eKZ70htqplOqslApTSnVUSj1rlg9SSnUyy27M85gyTVN3K6VamfVb3cb6WCl1mfn5xFMyVzZ5p5179OjBI488wubNm+nVqxedO3emd+/e7N+/H4B169YxcuRIwFA0U6dOZcCAAbRs2bLAQzcgIMDVfsCAAYwfP5527doxadIkVxDA5cuX065dO7p27cqMGTNc47oTHx9Po0b5PgRt27Z1xZz64osv6N69OxEREdxxxx04HA5mzpxJVlYWERERTJo0qdR77tevHwcOHCA2Npa2bdty880307FjR/755x9ee+01unXrRlhYGLNmzXL1eeGFF2jTpg2XX36562eS9/PLCyGyZcsWevfuTXh4ON27dyclJYWnnnqKRYsWERERwaJFi5g/fz733HMPALGxsQwaNIiwsDAGDx7M3+Yf5JQpU5gxYwa9e/emZcuWrvHj4+Pp168fERERdOzYkV9//bXM/1+N5kxIzbZRJyuZ5Oq1SPWuhiO14J5FtdPxAMzofx9/BTUGINCWhVzAK4vzmxUz4XjF4iuVSf1OMLxiZo64uDg2btyI1WolNTWVX3/9FS8vL1atWsXjjz/ON998U6TPvn37WLt2LWlpabRt25Y777yzyJv/jh072LNnDw0bNqRPnz5s2LCByMhI7rjjDtavX0+LFi2YOHFisTJNnTqVIUOGsHjxYgYPHszkyZNp3bo1e/fuZdGiRWzYsAFvb2/uuusuFixYwMsvv8zs2bNdK6fSWLp0KZ06dQKMcOqffvopPXv2ZOXKlcTExLB582aUUowaNYr169dTvXp1Fi5cSFRUFHa7nS5duhQJ856bm8v111/PokWL6NatG6mpqVSrVo1nn32WrVu3Mnv2bMBIzJTHvffey+TJk5k8eTIff/wxM2bMYMmSJYChGH777Tf27dvHqFGjGD9+PF9++SVDhw7liSeewOFwVDgfiEZTXlLSs6mdnUpOcB3Sj2XgLPS75puWQo6vPyerB/Nj8560iTJeaCz+nl1ZXLrK4jzh2muvxWq1ApCSksLkyZOJiYlBRLDZbMX2ueqqq/D19cXX15e6dety4sQJGjduXKBN9+7dXWURERHExsYSEBBAy5YtadGiBQATJ05k3rx5RcaPiIjg0KFDrFy5klWrVtGtWzc2bdrE6tWr2bZtmyvGU1ZWFnXr1i3XfQ4cOBCr1UpYWBjPP/88ycnJNGvWjJ49ewKwcuVKVq5cSefOnQFIT08nJiaGtLQ0xo4d6wq0WJwZaf/+/TRo0MAlV41ybPRt2rSJ//3vf4CRZ+ORRx5x1Y0ZMwaLxUJoaCgnTpwAoFu3bkydOhWbzcaYMWNKzCei0ZwtacfiCVJOpH4Dsk8eJ+V4HAkffkitm27C4utLtYwUsqobceHskm8csvrrlYVnqOAKwFO4hyT/97//zcCBA/n222+JjY1lwIABxfYpHIbcbi+aRrE8bUojICCAcePGMW7cOCwWC8uXL8fHx4fJkyfz0ksvVWgsyN+zyCM5ObnAvSuleOyxx7jjjjsK9HvzzTcrPNfZ4v6zyzPf9evXj/Xr17Ns2TKmTJnCv/71L26++eZKl01zCbDsewB6dW/HL3v34Zd0mpOv/x/KZqPOnXfS9dA2TjVqyZK7+/DJ0/nnMSweVhY6RPl5REpKimuvwN1kcq5o27Ythw4dciVEWrRoUbHtNmzYQFKSEU0lNzeX6OhomjVrxuDBg1m8eDEnTxqnRRMTEzly5AgA3t7eJa6EysPQoUP5+OOPXV5XR48e5eTJk/Tr148lS5aQlZVFWloaP/zwQ7H3FR8fz5Ythg9EWloadru9QLj0wvTu3ZuFCxcCsGDBAvr27VuqfEeOHKFevXpMmzaN2267je3bt5/xvWrObxypqRwaNZqscphVPYHthPH3Ve/ynsTVrJ8vV0oqa+YYGfBCjh4iokkQNQLyTU9WD5uhtLI4j3jkkUd47LHH6Ny5c4VXAuXB39+fOXPmMGzYMLp27UpgYGCxYc4PHjxI//796dSpE507dyYyMpJrrrmG0NBQnn/+eYYMGUJYWBhXXnkl8fHGZtvtt99OWFhYmRvcJTFkyBBuuOEGevXqRadOnRg/fjxpaWl06dKF66+/nvDwcIYPH15smHMfHx8WLVrEvffeS3h4OFdeeSXZ2dkMHDiQ6Oho1wa3O++88w6ffPIJYWFhfP7557z11lulyrdu3TrCw8Pp3LkzixYt4r777juj+9Sc/2Tt2EHOX38RO2EiWRXMG3O2OGx2Qtb/iE2s1AqpRYYlfy9S/P1o8PYLBdr7+uWf2Paq5tmVhQ5RfomRnp5OQEAASinuvvtuWrduzQMPPFDVYl1y6N/F85eYDz/D/rphag0YPpwmb/ynUua1OZx8P2IioUd28kujcKavXsgLV93BuIPrAQi6626S57wLwLywMbzx9UvMfv4TBn/xKgD+z71E82vHnJUMpYUo1yuLS4wPPviAiIgIOnToQEpKSpE9Ao3mUmfRtxtc1xlbtpTS8tzy7Y6jhB4x9iCiRk8FYFOD/PTPmXv3uq7Xt+4FgMU7f9vZS3tDac4lDzzwgF5JaDSl0CrzFIdr1CfVpzphUnrkgnPJiRNJdAI+bT+MF28dAMDuOvkpUnPXGjkr5rcf7jqtrdwCB3qXkZr5bNErC41GozHJ3L6DiPi9BNky+SewLionu1LmzbU74U3DnDRt8hCCq5ccPfaUfxAzh7cDQLkdxPOtrr2hNBqNxmOkrlhBzMBBpK9fz4J53wKQXqcBORZvyMmpFBk2HjhFtxP7OF6tFtX69y9Q91bEeI5Wz3c7T/WtxsiwhoBWFhqNRlNpHPjqG+zx8fxz+x0cPZoAwM67ZpFr9YbcHCrDCSjx6Alq5mawpFU/vLysBep+bN6T/7Ye6Pp+dd9Q/H2MNk63sxX6nIVGo9F4CLvDyam9Ma7vPY7v4aR/EGFtGmD188WiFJzF+aHykhVnhCTv2iOUOgH5Jqj7BrdmTERDjgbkryyuv6JTfke3lYX46nMWFw0DBw7kp58KRld/8803ufPOO0vsM2DAAFfu7REjRpCcnFykjXvY7pJYsmQJ0dHRru9lhfcuLzqcueZCJiEpjQbpCfxpbiQ3zjhNbvVArmhfj8AAcxPZ4fC4HDnHjbAy44dEYGRvMHjgyjY8N6ajscox8QnJVxx2n3wFYfHLjzzgCbSyqEQmTpzoOjWcx8KFC0sM6FeY5cuXExQUdEZzF1YWzz77LFdcccUZjeWOezjz3bt389FHH5UrnHlprF27lp07dxIZGVmkrVIKp5mDuCKUdb+FlcWoUaOYOXNmhefRXFgkbovCWzlY3ygcAC+nA5/AACwWATNmm/LAAdnC2M000N716hWpC/TzJsdNWbibmz7bkp/bwmJGnfYUWllUIuPHj2fZsmWuZEexsbEcO3aMvn37cueddxIZGUmHDh0KhOd2p3nz5pw+bWTNKils9wcffEC3bt0IDw/nmmuuITMzk40bN/L999/z8MMPExERwcGDBwuE9169ejWdO3emU6dOTJ06lRxzU6958+bMmjWLLl260KlTJ/bt21dEJh3OXIczv5DJ+OB9ANY3yg8M6fQ3XFCV1ThZUBnKwnoyHrvFilft2sXWO8RQXCf9C74spmTlm8gsHs5n4bFzFiLiB6wHfM15FiulZonIAiASsAGbgTuUUjYRGQB8Bxw2h/hfXg4MERkGvAVYgQ+VUmdtH3hl8yvsSyz68Dsb2gW349HuJacUDw4Opnv37qxYsYLRo0ezcOFCrrvuOkSEF154geDgYBwOB4MHD2bnzp2EhYUVO862bdtKDNs9btw4pk2bBsCTTz7JRx99xL333suoUaMYOXIk48ePLzBWdnY2U6ZMYfXq1bRp04abb76Z9957j/vvvx+AOnXqsH37dubMmcPrr7/Ohx9+WKC/Dmeuw5lfqCilqL57BwAtWzZgWfOeXBX7O5KXhMz8V3lwzyLX7kQEguNjSazTMH/uQiT7GquGHYOuxd1XSgS+bj2QPsd24el4AJ5cWeQAg5RS4UAEMMxMl7oAaAd0AvyB29z6/KqUijA/eYrCCrwLDAdCgYkiEupBuT2KuynK3QT19ddf06VLFzp37syePXsKmIwK8+uvv7rCdteoUaNA2O7du3fTt29fOnXqxIIFC9izZ0+p8uzfv58WLVrQpk0bACZPnsz69etd9ePGjQOga9eurgCE7uSFM3/44YdJTEykW7du7N27t0A484iICFavXs2hQ4fK9TMaOHAgERERpKam8thjjwGUGM68S5cu7Nu3j5iYmFJ/Lu73WziceUkZCvPYtGkTN9xwA2CEM//tt99cdSWFM//kk094+umn2bVrF4GBgeW6b03lsm2TEfdpdvg4ereqw97g5gD45mQBoPJ+Lzy4sgh75idGzlxIu3+iya1Z/KoCILRNY4aPeZ2/uw8qUG4V4ZMOV3HblZ43mXpsZaEMf7O8nU5v86OUUi7DsIhsBhoX092d7sABpdQhs89CYDRQ8tO0HJS2AvAko0eP5oEHHmD79u1kZmbStWtXDh8+zOuvv86WLVuoVasWU6ZMITv7zA4DTZkyhSVLlhAeHs78+fNZt27dWcmbZ1IqLcy5DmduoMOZXzikZ9tImvko1YFjzdpRIyuXBH8jqKa3qSykEsxQ2TYn7WKNEB/29h1KbDc+sjGbYxONvRQ3Pp7SjZs/3sxNPZt5TMY8PLpnISJWEYkCTgI/K6X+cKvzBm4CfnTr0ktE/hSRFSKS95NrBLhnLI8zyy5IAgICGDhwIFOnTnWtKlJTU6levTo1a9bkxIkTrFixotQxSgvbnZaWRoMGDbDZbCxYsMBVXlK47rZt2xIbG8uBAwcA+Pzzz+lf6FBQaehw5kXR4czPb5TDwe8jxtL4ZCwHajbilQfHYHco4qqHAJDS70qjnZdnlUW2zUGHhMNM3/UdAB3uuKXEtk6n8SJilYLKol+bEGJfvornxnT0iIzueDQ2lFLKAUSISBDwrYh0VErtNqvnAOuVUnm7f9uBZkqpdBEZASwBWldkPhG5HbgdoGnTpufkHjzBxIkTGTt2rOsBlBf6ul27djRp0oQ+ffqU2t89bHfdunULhO1+7rnn6NGjByEhIfTo0cP1AJwwYQLTpk3j7bffdm3EAvj5+fHJJ59w7bXXYrfb6datG9OnTy/3vRw8eJA777zT5aV01VVXcc011yAirnDmTqcTb29v3n33XZo1a+YKZ96lS5cCCq28DBkyhL1799KrlxFMLSAggC+++KLUn0se7uHMs7Ky8Pf3Z9WqVQwcOJCXX36ZiIgIl+krj3feeYdbbrmF1157jZCQED75pPQ08OvWreO1117D29ubgIAAPvvsswrfo7jAiKYAACAASURBVMZz7Hz1LRodO0i6lx8P9ruHv+oFEujnzelqQVxz1XO8OrI3AOJhb6ijyVlM2J/vzh3UMKTEtjl2wwPQyyoltvE4SqlK+QBPAQ+Z17MwlIGllPaxQB2gF/CTW/ljwGNlzde1a1dVmOjo6CJlGk1VoH8Xq4bsg4fU7rbt1ZrOvdRlD32rmj26VCmlVEaOTTV7dKlq9uhS9VvMKaWUUq//+30V3badytqz54zny8q1q6mfbFb7j6cWqftpd7yKbttORbdtp2YPvqHUcdbsPaGaPbpUrYo+fsaylAdgqyrhmeoxM5SIhJgrCkTEH7gS2CcitwFDgYlKKadb+/pinkYRke4YJrIEYAvQWkRaiIgPMAH43lNyazSai5eNS3/BguLVrpOwWb1Z/aBhcq3m40WPFsEAhAQa+1ByDsxQf/6TzOp9J3nsf0WTKP3yeP4Zov/rcn2p4wxsV5cNMwcxuH3RcxiVhSfNUA2AT01vJgvwtVJqqYjYgSPAJlM35LnIjgfuNOuzgAmmprOLyD3ATxiusx8rpUp38dFoNJpiOLx1F7UtVpxt27NoXDitQvIPsr01oTPR8Sm0rmuUiffZKwtfb8OUlZVb8BS4w6m4LmYtYIQkd1qsRfoWplGQZ2M/lYUnvaF2Ap2LKS92TqXUbGB2CXXLgeXF1Wk0Gk15UE4nvTYbj5GfHhpUpL5+TT/q18w/2OZaWdjOXFnk2AwlkWMvqCyybA62121Dz+PRXPniTJ6+rE5x3c8rdPIjjUZzSXBk0TcAJNVtUr4OXkaIDWU/c8+9RVsMR85cR8EQNQlHjtLzeDTJ4d2r1LRUEXS4D41Gc0lw9LfNAKS+9WEZLQ3kHBzKc5hnb5rXzj8ndDQ5i7RRw40vPiUnOTrf0MpCo9Fc1CjzgZ144DB/BTWmV7sG5ernMkOdRdRZu8OY2z2S7K0f/YEVozyjcYszHruy0cqikkhISCAiIoKIiAjq169Po0aNXN/zAguWxrp169i4cWOxdSdOnGDkyJGEh4cTGhrKiBEjSh0rOTmZOXPmlFhvtVpdQfCuvfbas4pt5B7A77bbbis1jEnhe5w7d64+o6A5K2zHj7OvfSh727XnsiN7iK9em0Df8lnfLd5nv2eRZ35yuaDm5uIXbcRFy/Ty5eiI0r2gzie0sqgkateuTVRUFFFRUUyfPp0HHnjA9d2nHEvR0pTFU089xZVXXsmff/5JdHR0mXkYylIW/v7+REVFsXv3bnx8fJg7d26B+pLCfpTFhx9+SGhoyWG9Ct/j9OnTdZgMzVmx9ouCp/gbjLm6SMiMksh3nT3zPQu7qSwcTsXpd95hX1g4L20wIt2+EjmJpnUvnLhhWllUIdu2baN///507dqVoUOHEh8fD8Dbb79NaGgoYWFhTJgwgdjYWObOncsbb7xBREREkZDX8fHxNG6cH2LLPVptcaG8Z86cycGDB4mIiODhhx8uVca+ffty4MAB1q1bR9++fRk1ahShoaE4HA4efvhh19jvv2/8ASiluOeee2jbti1XXHGFK9wHFEzk9OOPP9KlSxfCw8MZPHhwsffonrwoKiqKnj17EhYWxtixY10hRgYMGMCjjz5K9+7dadOmjetns2fPHld49LCwMGJiYtBcehzeuM11/Ue99th7lz+UjSUvL8tZ7FnYnYpRB39jyOovSPqqYC6b0PDWDGxb94zHrmwuWW+o4y++SM7ecxui3Ld9O+o//ni52iqluPfee/nuu+8ICQlh0aJFPPHEE3z88ce8/PLLHD58GF9fX5KTkwkKCmL69OkEBATw0EMPFRnr7rvv5vrrr2f27NlcccUV3HLLLTRs2LDEUN4vv/wyu3fvLjNMuN1uZ8WKFQwbNgyA7du3s3v3blq0aMG8efOoWbMmW7ZsIScnhz59+jBkyBB27NjB/v37iY6O5sSJE4SGhjJ16tQC4546dYpp06axfv16WrRoQWJiIsHBwUXucfXq1a4+N998M++88w79+/fnqaee4plnnnEFDrTb7WzevJnly5fzzDPPsGrVKubOnct9993HpEmTyM3NxVEJ2c405w9KKY5MupF+0UZcrj/rtOK/bQbxUPXybyjnnbP4buvfDBuUTd3AiueLyLU7uXOXEc4+J6BGgQfuk1PKr7jOBy5ZZVHV5OTksHv3bq680gha5nA4aNDA2HgLCwtj0qRJjBkzhjFjxpQ51tChQzl06BA//vgjK1asoHPnzuzevbtAKG+A9PR0YmJiyoyblZecCIyVxa233srGjRvp3r07LVoYG3IrV65k586drv2IlJQUYmJiWL9+PRMnTsRqtdKwYUMGDSrqz/7777/Tr18/11jBwcGlypOSkkJycrIrwOHkyZO59tprXfXFhVHv1asXL7zwAnFxcYwbN47WrSsUZkxzgbPjtyj8zQCOPzbrzludr+OyugH0a1Ny/KXC+JtpSjf9dYL1/9vNh5MjKyzHH4cTXdde6amu6/3NO9G+Vq0Kj1eVXLLKorwrAE+hlKJDhw5s2rSpSN2yZctYv349P/zwAy+88AK7dhUNFVCY4OBgbrjhBm644QZGjhzJ+vXrSwzlXVxeCnfy9iwKUzhM+DvvvMPQoUMLtHFPTVpZFBdG/YYbbqBHjx4sW7aMESNG8P777xeruDQXJ38tX0M4cPeABzgUZASpfuKq9ljLuV8BULeW8fvupRxkOCqeylcphb+t+FQDLUYOLbb8fEbvWVQRvr6+nDp1yqUsbDYbe/bswel08s8//zBw4EBeeeUVUlJSSE9PLzWE9po1a1weS2lpaRw8eJCmTZuWGMq7tLHKy9ChQ3nvvfdcocb/+usvMjIy6NevH4sWLcLhcBAfH8/atWuL9O3Zsyfr16/n8GEjKWJiovH2VZJcNWvWpFatWq79iPKEUT906BAtW7ZkxowZjB49mp07d57V/WouHHIOHSb8W+MsxaGaDV3l7epXbDPZ29cwWVmdTpf7bYXksDtpkJFQbJ1f9aoN3XEmXLIri6rGYrGwePFiZsyYQUpKCna7nfvvv582bdpw4403kpKSglKKGTNmEBQUxNVXX8348eP57rvveOeddwrkVNi2bRv33HMPXl5eOJ1ObrvtNld47uJCebdq1Yo+ffrQsWNHhg8fzmuvvVZh+W+77TZiY2Pp0qULSilCQkJYsmQJY8eOZc2aNYSGhtK0aVPX3O6EhIQwb948xo0bh9PppG7duvz8889F7tGdTz/9lOnTp5OZmUnLli3LDBP+9ddf8/nnn+Pt7U39+vV5vIpXkprKY/mr79MO+KlFT764rScvrdjLnmOpBFdgvwLAasZ1sioHDmfFlUW2zUGT9JPF1vkGVKvweFWNlKUxRaQa8CDQVCk1TURaA22VUksrQ8AzJTIyUuV53uSxd+9e2rf3dKZajaZs9O+iZ0hOzeSvPpeT7BuA7eNFXBXWgNPpOUQfS63QfgXAL3uOUveaK4zrLkOZ/mXFMjEeS87itdtmcfvuokm4av3fG9S/aliFxqsMRGSbUqrYzZnymKE+wcinnfeKeBR4/hzJptFoNOeM995eTKAti4R+w7gqzHAYqRPgW2FFAWA1Y0MB9N/+U4X7p+fYqZ2dv6ltk/zIsn4X4MqiPMqilVLqVcAGoJTKBKowXZNGo9GUwF4je8HYx24/66EsXme3pZuSZaNeZr43VLZXvhnM27/ibrhVTXl+Grlm8iIFICKtMFYaGo1Gc845k81kAKfdzphtRl40v5CzD/ldON91RTl8OoNG6afYXb8N+2o15dXIG1x1Fr+LU1nMAn4EmojIAmA18IhHpfIgZ/qLqNGcK/TvYMl8ufdLhn0zjPTc9Ar3/WzRLwBEBzc7J7JUxM22OB5ZvJPg7DSO+NXmgf4zUN17u+osgRdOmI88ylQWSqmfgXHAFOArIFIpta6sfiLiJyKbReRPEdkjIs+Y5S1E5A8ROSAii8xUqYiIr/n9gFnf3G2sx8zy/SJyxg7Kfn5+JCQk6D9WTZWhlCIhIQG/C/DN0pPYHDay7dm8tPkljmUc40DygQqPYV36LQDpd9x/TmQqbwyp4jiSkIHV6aBmbgaJfoZiuGfgZa56a40aZy1fZVOm66yI9AGilFLLRORG4HEReUspdaSMrjnAIKVUuoh4A7+JyArgX8AbSqmFIjIXuBV4z/w3SSl1mYhMAF4BrheRUIy82x2AhsAqEWmjlKpw/IbGjRsTFxfHqVOnKtpVozln+Pn5FYjldamTkJXEgK/7FSg7kXmi2LZZUVFk7dxJcKEAk9nHTxC5wwgPc9P4vsV1rTBeZ6EsMnMd1M5OASDZ10jT2iTYn+e73sBVsZtoV7PmOZGxMinPOYv3gHARCcd40H8EfAaUeirKzJ+dt5b0Nj8KGATkGe8+BZ425xhtXgMsBmaLEQR+NLBQKZUDHBaRA0B3oOjR5zLw9vZ2hZjQaDTnB7N+XlSkLCUnpUhZRo6dr9/9mp6/fsuJF1+i+aKF+IeHA3B4wAAAjnboTnu3SANng+Us9iwycx10PmkEr9xXyzCLeVstrGvShXVNunCt14V3xK08exZ288E/GnhXKfUuUC6Dm4hYRSQKOAn8DBwEkpVSeWEc44BG5nUj4B8Asz4FqO1eXkwfjUZzAfP97j38kvRukfLTbl5Eeew6msLOpPwIsAkfGwcz936Qn/OkyTtvnTPZCu9ZKFv5Q5Vn2xyEnz6As3aI6xR5/Zp+DGwbwnNjOp4zGSuT8qi3NBF5DLgJ6CsiFoxVQpmYpqIIEQkCvgXanbGk5UBEbgduB8oMlqfRaKqWzFw7D62Yj29dsGc2w57WAWduXfwbfcnJzKQi7Y+nZLOqaSSBuZm0TzxCxE8/kfTVVxyd+wGNgJl97uCbeuduL8BqEdxt3Y7kZLxCyndeIzPXQYOM09CiOWsfHsiptBx8vax8ckv3cyZfZVOelcX1GPsPU5VSx4HGQIXiQyilkoG1GAf7gkQkT0k1xjjkh/lvEwCzviaQ4F5eTJ/C88xTSkUqpSJDyvmfqtFoqoaTqTlY/WNx5tQh68id2BL74Uhvh7JXIyGroLJIybLx7NJo0nyqk3XTNHbVaQnA8WeepUZuBj+06E34qCvwsp67cHeFzVD2pKIKrCQyc2w0Sj+NtVEjWtSpTvcWpUdWvhAojzfUcWABUFNERgLZSqkyc12KSIi5osA8p3ElsBdDaYw3m00GvjOvvze/Y9avMc1f3wMTTG+pFkBrYHM570+j0ZynnErLxuIfhyOrKddFGhv+I8MaoBzVScpOLtB23f6TJGbkcnV4Qy5vXYfvW17uqgu0ZZHqU51nRnU4p/L5+1gLfHdkl+942Y6/k9j95HME2rLwadiw7A4XCGUqCxG5DuPhfC1wHfCHiIwvvRcADYC1IrIT2AL8bMaTehT4l7lRXRtjwxzz39pm+b+AmQBKqT3A10A0xnmPu8/EE0qj0VQOvx39jXHfj+OjXR+V2ObAyXT2J/yNxSudO3oO5NXx4Rx6cQS39GmBclQjJaegskjNNvYq/j2yPd2aB5PuU41/AvKtB7VC2yJneYiuMDX9C1rb7eXcs/jyj78Zd3A9AH4XkddbefYsngC6KaVOgrFiAFZheCyViFJqJ9C5mPJDGN5MhcuzMRRScWO9ALxQDlk1mksKm9PGskPLGNJsCNW8qz7ekFKKO1fdCcCbSW8ypcMUrJaCb+j7jqcy7M1f8a61Ab/60KOh8ZiwWISa/l4oRwApuccK9MnMMZRFdR8vqvt68dnU7jh/yt/snnx113N+L9ULrSyUrXzpVf+7LY4hASE0Tj9F7VEjz7lcVUV5DHyWPEVhklDOfhqNxsP8FPsT/97wb3p82YMxS8Zgc5bfY8cTzNtSMOBeSm5RF9jTabl419qEX/0fUA4/ujfs5Kqr4eeNsgeSlHuCTp92IibJcD/NyDWMCf5m2PCIpkH4OfLNQoH1653zeym8UnGUsrKYvSaGzzbFGv2Uk8bpxlkuX99y+QJdEJTnof+jiPwkIlNEZAqwDKj8dGgajaYIH2/JTy51MOUgh5IPVaE08ObW9wDIjjdS3SZlF90UTsu24Vff2KoUa3aBTeka/t4oR/7p9nHfj2PTkUO8vdpQGnmnqv29raxsZhgovmt5OT4tmp/zewGQavmrNWUv2fr9+sq/eOq7PYya/Rvhpw8C4HXZxZXKtzwb3A8D84Aw8zNPKfWopwXTaDRFybU72XgkhqiTUUSdjCIme2WB+tjU2CqR61j6MTp92gmv6odwZDfAmWt4/yRmFz0vcSw5y3Vd1xpRoM7P24pVCoZCmfzf2UXG8LZa+CR0BGNGvsjcsDHnfL8ij5bff8/xvkaEIWc59iy8tm/mpQ3vAxA0dapHZKoqynWMUCn1DfCNh2XRaDRlMGvpH/yQdDdiyX9wOe3VyTjwKIHtnuL5319gaPPKz+/89u/5p7A7+d7GFoeRHnfz8c10q9+tQNtDiYZVO/v4SB4ZeV+RsZoEBRHvXiDFP6SVWMjxqlj2u4ri07gRJwaOpP6vP+EsZWVhCKR4YeMHrq91Rl/tUdkqmxJXFiKSJiKpxXzSRCS1pH4ajcZz/HTsswKKwpFdj4wDj4IyHprJOUnFmn48yfKYDSw7+jEAOacHcHfvgYzq2AaAuX/OLRC4UynF6pRZALwyajBDQovuNVgdQQW+21PDAOjVsrZH5C8T00ym7KWvLOoWOkgoVmsJLS9MSlQWSqlApVSNYj6BSqkLL2SiRnMRYPU5DYAjx3AbtSX14uEhnZh7Yxeyjhke7Rm2jEqV6eG1TxoyZdfjs7FP069NCDW88wPlfbH3C9f1DQvfJkMZnk5DL+tVrPnIy1HQ3dSZU5+WIdX56vaexc4f6OfZOEtiZsxz5uSW2q5mbuX+3Cub0lYW3URkeDHlw0Xk3PupaTTnIUop/jqRRq7dWdWisP14FNne+/DP7YItsS/O3No08Y/g5l7NaF0vEJzG6iLbnl1pMuXanSgE5fSirfPf9DTf/v19vcmOHwPAq1te5cu9X2J3ONmVtQCAVtX6UN27+IB/x5O8yT7u7nJq4fDpog/iuwe24sEr2/Dbo4PO7U0V4rBUx4nw58Y/i9TtO57Kuv0nqZ6bRc/4PR6Vo6opTSW/AtxSTHk0Rl5uz/4PaTTnAb8fSmTqB+u4qfdlPD4qvEpk2Je4j5OZJ3lw7cMAXFYjnI0H22JL7s7y54bh520l0M+bgW0b8XsGZDsqT1l89+ffWHwSIbkf827Nf/NvVz8QtS0/3uhLm1+iRu4AxJqFNac1SybPLXHMlEwbNsflODLagMVwjy0uBc3DQz0aas5FTKqDXKsXR+KLmvfGvLuBbJuTdzbM5bKUYqMQXTSUpiwCi8tZoZQ6IiJnn7NQo7kAOJmWzYfe/0e7qBMw6mClz59rd3LtDwXPqj4/aBqnuxv2cz/vfLu4v9XwIsqyZ+Fp0rKzeX7Du2SltEbEwazhgwkJ9HXVjw5vxEPfF1w5HE0xPKNu61rEYFGAXIexinPm1nWVXR1edWEzRASHWLE4im5wZ9sMWd0VxWm/Ghxu04X2lSZh5VCasqhVSl3VHxXVaCqB9OxcRlv3gBPISgL/0v4szh05dgefbzqCb+DhAuWZ/0yhSXB1WoYUtSD7msqiMsxQ0759jz2587E4A8ACHeq0KVBvsQgN/VtzKrkL4p2EV/XDHErbC0DjGhU/QHdF+7plN/IQFgG7xYLFWfAEd7ateO+oyUOfZPaki89SX9o5i1Ui8oK47UCJwbPAGs+LptFUPeu2uNmpj26rtHm/3hrHq5v/wyvbH0U589/pVk6fjHcJkVV9vfyByllZ7E3YD4DTko7VUY82tdoUaZOZI2THX0dugmGx3pBkeEzVDyj9wX9H/5YFvoc3rklY46ASWnseEYyVhbPgvtUbP//lul7XyDgv8u9et+IUCw1rXXzv06WtLB4EPgQOmAmMAMKBrcBtnhZMo6lqlFLY4ndDnit/8j+ltj+XZNsz8K3zCwCOrEY4c+rhHbSdVnVKtgD7exkri4xczyqLXLsdZ+AG13efnPZF4j8BpJnxnByZRqa4FEccACH+pacPeGx4ex4b3p5V0Sfo3DSI2gG+pbb3NILgsFiwOAuuJFKy8l1p+xzbxf6gJmytZxifalW7eMJ85FGislBKZQATRaQlRv5rgD1mIECN5qInM9fB1dbfsSsLXuIkMfEUlZGV4FDCKd4+eL3rex2flvwdO4Ls+GLjbLrI27N4atOT5DizmNBuAn8nZKJQNKt99qlGbU4b/93/X46fLviWP7pdv2LbuzzIlA9KCSLGLnUd//JteV5RzBmMqsBiAbtYsRbas8gzung7bHgrB7UiOtGmXgB/nUgvErH2YqBMB2VTOWgFobnk+Gj9AW62bGdTwGB6pa8mJ61yDrtds2SS6y8zK+4GJkeOhUZC63oBpfbzN81QAP/Z9gYT2k2g32tr8QrczbaHplPTt2Ypvcvmi13f8Z+olwAKPPwj6hYJLg1AZLNabD2SxDd39mbi4v741lkHQA2fC+uYlojgsFiL7FnkZV0NsBkrubAB3fjiqh5sOZxEUDXPniyvCi68rOEaTSWxas1KZvhmENRpOKmbNkFOCYELlDIM2+eAZ1f/D7uXEewiN+Fy7GlhTOnTgjrlMMX4e+fHVMqyZ+JwOrD4HsW/8Re8vjWd5/o8d8ZyLdsbxX+innF9V/YaZP4zGav/P/RpUXwK409u6captBwa1fJH2QzHAFFWj8Vx8hQWERxiKbJnkZdJr36G4eXlVbcetQL9uCqsQaXLWBnoUOMaTTFsPHiasdbfAFAt+pGmqiE5BcNtJ2bk8sOcmeS8Hgq2insg7T+exvBPnufqb68mISsBgC+iDN+RPoEPs2rK/7Fx5qByKQoAv0JxkpYeWoq1umEUyLEbp4+/3voPzWcuK2BvLwu7w8m/fny3QFmQT23u7zeA14fdUeJbdKCfNy1DAvD1sqIcxqrHIheeeUYwzFDuexap2TY+/904WdAi1TiR7teubVWIV2mUdoI7uLRPWQOLSBMRWSsi0SKyR0TuM8sXiUiU+YnN2zwXkeYikuVWN9dtrK4isktEDojI23KhvZpoLji+Wb2RW7x+IkkFYAmoSxKB+GScACApI5fmM5fx+sr9XH3yPXwzjsHp/RUaPzkzl6FvrudQ0lFiU2P5cNeHAFj9/8GR3YC5426mSXA1Ggb5lzFSPoW9pNJt6Vh8jLwKJ009N+fXP6je+gX+jD9Q7nGvfn8J3rU2Y0uJIDfROHgX7FebGYNbMzqiUbnGUE5j1WPlwjPPWAScIojKX1mcTM1/OaidnYpTBK/69atCvEqjtJXFNgzPp23AKeAvIMa8Lo8PoR14UCkVCvQE7haRUKXU9UqpCKVUBEYk2/+59TmYV6eUmu5W/h4wDSP/dmtgWPluT6M5M9qnbQQgrt/r+HlbOKAa4ZtuvElGx6cywLKDZX9E53eooFvtvPXmG//JkShl5Yu9X5CYlYTVLw5HZvMzktlqKZSsx+nAy8tYQeQlIUrx2ojFK41fj/9YrjGTMnI5kL4ZESeP93oQi8MwJ1U0K1/PZka8J4tceJZvi0VQIoibGSrH3Lz3s+dww/5VWJRCLBe3oaa0QIItlFItMVKoXq2UqqOUqg2MBFaW1M+tf7xSart5nQbsBVyvIebq4Drgq9LGEZEGQA2l1O/KCF/5GTCmzDvTaM6CkNQ9pFpr0WnQBGr4e5OlfHDkZIHTQZv19zLf5zX+9Lvd1d6+Y2GFxreZp5TnTOqCPdXIFPfKbwsQay5XtY08I5mTMnOxJXfBbiqbLHsOXla7eW2EDM/iOAB+1vI97BMycvEK2EeAVy1u7haBvxj2eIeqmNmtUYDxp1/Lq/j9jfMZQXCKBXGLOWJzGNfdj++tKrEqnfKowp5KKVdmPKXUCqB3RSYRkeYY+bj/cCvuC5xQSsW4lbUQkR0i8ouI9DXLGgFxbm3icFM6Gs25Zt22XYyW9ezzagci1A30Q7z98cYGe74l5O+CiSKPq1o4jv1ZfACjEjiemkNwdR+Gd6xPzgkjaN6Wk78CMKJ9xzOS2+5QZMdfR9YRQ4n9cHApzmq7ADias5P+i/qDGHb3rHKe8n7iu81Yqx3iMv8BANSScGzJXRna6KYKyVbLvxaZf9+KT+KNFep3vuAQSwEzVJ5bcGQ9w0y4bMiUqhCrUimPsjgmIk+aewrNReQJ4FiZvUxEJADD3HS/UsrdnWQiBVcV8UBTpVRn4F/AlyJSIR87EbldRLaKyNZTp05VpKtG4yL+ZyMzW7X+97rK/KtVw1vl4vyhaLKeDc4O+KpsyCjld86ei/N/d7Bi7S/sikvhhz+P0blJECLCVR2M9Jun7IZZq0NIizOS22aaSbwsVpTTypG0gh7vidmJiJlIKD6j7KB38zccZuuxPYgoxncYAMAV7RqQHX8t3RtWLKjiXQMuw5HRmlv7dCi78XnGQ0PboESo43bQLk9ZNPY2Vm772nWvEtkqk/Ioi4lACPAtxv5CiFlWJiLijaEoFiil/udW7gWMA1zptZRSOUqpBPN6G3AQaAMcBdwD3Dc2y4qglJqnlIpUSkWGhJR+SlSjKYmgnDiyLNXo2PsqV5ldfPDCgSU3vUj7hLrmQjux5ONISz58DsvOhQz/ZRTXzV4FYIQVB4Z2KLgxWr/6mW2U+nkZp6gHtA1BLPmeOzmnB7iuxTsZgAxb0fsozNM/7MYaYCz8O9VrDsCjw9qx5YkrCG9SsfAbNf29iX35KsZ2blx24/OMBjX9UWLBSv7KIs+M2MqRQq6PP/++tltJ3S8aypODO1EpdR9wuVKqi1LqfqVU0cS6hTD3JD4C9iql/lOo+gpgn1Iqzq19iIhYzeuWGBvZh5RS8UCqiPQ0x7wZ+K68N6jRVIRsm4P6juOcDCz4BpxlNR7ssKx2rQAAIABJREFUiarowbiVac0BcJyKKVKXx4m4/Ii1N1l/BmB8V+PBeSI1m+wTw3FkNSJt37NnfA7hxp7NeGRYW+7o38pV5shqRO7pwThtgSinDxYvY6M70162svCttxTfOmsBqFvNiOdksUiB6LKXCk4gN9dNAZsrC+/UFAKaNKRZSOkHJi8GylQWItJbRKIxNqgRkXARmVOOsfsANwGD3NxhR5h1Eyi6sd0P2Gm60i4Gprsppbsw41RhrDhWlGN+jabCHE3OopkcxxnUvEB5orcReiJY0lnp6MqX9vx0LpnehofQqRP51tnF//sanq4JJ/eSa3fSQBI5pWoyLfdffOgwVizVfIyVwOiIRtgS+5MZey8D25z5dpyPl4W7BlyGv7cVR6axkRzkHwDKG1tyd8SSi1iN8xYZ9tKzuu2KS8G7RpTre4DPxf8wLA2HWEjOyOafxEwgP4y6JTcbi59faV0vGspjhnoDGArkmYj+xHiwl4pS6jellCilwtzcYZebdVOUUnMLtf9GKdXBbNdFKfWDW91WpVRHpVQrpdQ9SlVgJ1GjKQe204dw/P4+Id+MJ1jSUcEFI58ezsx/m/5b1WWOYzQAiQFteGx0JLnKijMzf8Gdtd2wsDpjVpGSZaOBJJAe2IpXnngMp/lnl/eGnvdvoK8XH04+e3OGt9WCM9cww1b3rkaXpkGuQ3F5JGWVcBrd5Mut0YhXJqT24en/b+++w6Mss8aPf8+0dEhCkxpAmsAqTQUBBQuioiJ2XRW7gqu+qCu4tpV1dV/birqiq9iw7c/+WlYRXVwLKiAiRaSJBJGWQGLKJJk5vz+eJ8mEdMwkJHM+18XlzP2UuScmc+auZ8jTv7lOzV1YPHhUyxYzFrstCwkGkYTYCBZ1mvSsqpv2aBpXvZG7Mc3Urw8dTprkUjqjIpBecYrnymwvuPFiYK8eZH7fjjHB+7hhwhG08/vIIwGCbtdOUR7n+ZxxCc+8m0lY/wUHe34gs/VxpCeVL0qLXET3yhUj6JqeWGmtxN7weQVVp9USIo/nLxnOUY9/Sq57PFySQpD8Gu+xvdCZYvv3k0/nqG4tLzdDfakIHg2XTXgrbVlIsBBP6m/bc6u5qEuw2CQihwHqDlhfg9slZUxLsG13Ae0lt0JZfPeK3/DztPzbY1q6k2f6R+3IsYP3Z/nPOeQRj7c4D976Ayx5tsK1yeveAcAbcL7dz7/uCHx7BIVh3RtuP1ufR5yUpGlfIaIkBLy0S0wrCxaeYDeKfatR1UrjI3nBEn7JKSQ7uA2Ajkktc5+j+vKGQ/TZlUnRz5nQpXXZADd5eXi6xMZM/roEiyuAB3HWNmzGWZA3JZqVMqYxLXn+FsYDT6RM4S/bR9ExJcB/O/WucE4B5d1Q3oRU3pg6kp+y8vF5PQS8HvI1jlbFebDk9Wpfp2CssxHf/lEeDFUFLXEW3Yk4H2qJvhRwvxUn0ZM8VhAMBYn3VexCufTZRXy+bicJ3d7Bl7T3M7NamoO2O9ujeB+5Hw55kpU/O9144ayd+Nq0acqqNZq6BIu+qnpuZIGIjAQ+q+Z8Y5qN/IICxm/7JwCTTvs9F3frj6oz6ydSYcSeRgkpafTqmsogd/powOfhV+Los7nmjQ16dmuc1cslYS3biymMsw4gyZcC7t6BfnECycXvX8rzJzxX4drFG7PxxG/Gl+Skc02La5w0ss2GOF2HL329CV+4BM3Jwds2NoJFXQa4H6pjmTHNytMvvEDi35xvzjOKLyalywGISKVAARCkfEFWh/YV04LG+Tzka+VBzhOCd/JduHvDVroOerZNQsNOSyjB4yQamr+ifPaT3+MEi2U7lrLnXBEFPIFtZc9tz86K1N3/aUzfdqS6Y1S+NnVL5tTcVduyEJERONt6tBORaRGHWgGVcyga08zEr3y57C/gT9NvrTa3NcD9ZwyCt5zHvrSKLYSAz0NASipds0J7cGLRX7nf/w9yM8ZxQYPVvGYejzDn3PFc+lomU05ysusd2LEzq7aNp+TX3sSnZ4Eb29btWkevtF7lF0sxCZ2dmVz7B/c+/0WL5fHwa7CEpDgff1jntCS9rVKauFKNo6aWRQBIxvlzSon4lwOcFv2qGRM9+UUlZMg2tmg6FyfOIjml5hXJPdpGpCVNy6hwLOD10Fsy2dM9px1Il7QEphVPYVf34ysdj6ax/drz6dRpnDDQGXuZe8mhFO0cw5E9hzC0U3nehWdWPlPhOvHvAKC1pwdPnnNc41V4X+e2sJZv+ZWBt73PO8u2cMjarwDwJNZvB97mqqYc3AuABSLytKpuBBARD5C8xx5PxjQ7GzdvZoR3Je8mnswtF9X+3cfn8TApeDtBAryzx7GAz0PpbPLfF81gvOcrUsZczenDunL6sK4s+Smb/h0bP5Xofq3Lu8ZS4v18P3M8fq8Hjwzlyhfb8Fnx1LLdYMFZvR7yOMHisePuok0dky7Fgl1xybQpzGFXYXkLMugLEFdSRNLhtS47axHqMmZxl4i0EpEkYDmwUkRuiHK9jGl4wVzY8i0AyW9eAkCP4SfTPbLVUA2PB5ZoH1bSvdKxgM9Dsfu9a3W4CzeXXIynXZ+y40O6pRHvb/qe23i/F69HEBFS41JR9RIMBcuOf/9LLhJwMvZ1SWl+ezhF0y53BXtcqDzDoIqQctz4mBnXqctsqP6qmiMi5+JsszEdJ/nRPVGtmTENKW8Hu546g9QdTpKirsC80FD6DaxbV0tp/oLfda68AMvnESYX3cj5vg/YgXP80J4Nt24iasJ+NmU7e0UVh8JMfOQz4vbbSZKvFa3jYmOhWV0V+N0JA+7W7mmFOcQXB4k/oH9TVqtR1aVl4XcX400E3lLVYspmbBuzj1Mlf/NyuGf/skBRduio2+iaXrf+5tItqQNVDIKLCEu1F9OKp6B4mH5cP9qn7NtbQJxwYEc0lMDWfCdV7JZdzoegx7+TjonWqtjT5iRn+5Rij49AqJi0QmeJYyAjo6bLWpS6tCweA34EvgU+EZEMnEFuY/Z5X3/wEgd/4WToLVQ/hwQfYbz3a0Lq5e7Ro2u5ulxHt/9//MCqF6k9cf4wFv+UzaP/WceoXvv+VMp2KXGEi9qxs9AJFr/kFIInH1/yWrqnWtbiPT3TfzzH/vQVizv0483/m8HPSc7aCk9i3XOkN3e1BgtVnQXMiijaKCJjo1clYxpIOETGFzMAKFYvc0Z+xPBfivjXymQmH9a9xqmye+qansjSW4+hdYK/yuNH9+/A0f07cMO4vlWu09jXxPm8aDiOwpDTDfVLTiEpfe8AYESnlp+bob6KPM7/99LUqp3ynLEdT4IFizIicms1h+5o4LoY0yBUlUWv3MPBK+6kPfBkyXE8WHIKy8YdyOZdBQBcf2zfmm9ShdTEQK3nNIdAAc5CQsIBgiHn57F1d/nGgqf3Ob2pqrXPUncQO3KAG2Jn2izUrRsqcuP7eGACtpGg2Yet3ppLxvJHQOCW4skMP+NGnkp1upE6pybw+PnDmriGTS/O50HDceSXOEHiBzfL382H3hwzs3vqI+T+TM5ZPa9Cub+RtnDZF9SlG+q+yOcici/wftRqZMxvtHnNUvrJLmb5L+LPt97fbL7tN6b0pAAaDhDCmTr77+zp4IFOyZ2auGb7JnX3hAqEK67Uj6VuqLp32pZLpGJO7CqJSFcR+VhEVorIChG5xi2/XUQ2V5E9DxGZISJrRWS1iBwbUT7eLVsrItP3os4mhoS+fJIifEyZeoMFimqICOkJySgl7MzLB4/TvXJIx0OauGb7JqWa3yPP3nyENk91GbP4jvKpsl6gHXUbrygBrlPVJSKSAiwWkdI23AOqeu8er9MfJ93qAKAT8KGIlK5segQ4BsgEvhaRt1R1ZR3qYGKIbl1J9tcvMSr3Pb5KHM2oVPuWXJOAN548YO4qZ+fZZG86cV5btV2VcDVdc7HUZVeXMYsJEY9LgK2qWnnXtD2o6hZgi/s4V0RW4eTEqM7JwEuqGgQ2iMhaoPRrzlpVXQ8gIi+551qwMGWys7MofvR42pMNAmmjL2vqKu3z4jxOF8oTKx4GoESLazo9pmkMBYXqVNuGEpHSzWxyI/4VAK1EJE1E6rx/gYh0BwYDX7pFV4nIMhGZIyKlG+Z3BjZFXJbpllVXXtXrXCYii0Rk0fbt2+taPdOc5O2EooopQcNh5d37LqE92awKd+Pm4gsZMMI2watNnLdif7tXrFVRnXB13VAxpKaWxQs4rYrFON1Qe/60kkXkn6p6U00vICLJwKvAte62IY8CM917zgTuAy7ay/pXoKqPA48DDBs2zFaZtxS/bqNkwb14l84lHA4h/gQ8N25wdgJd+yGFX8zhSO83bPJ0ZsOpHzDG6ynbJdRUL8Fbcdrn2LTrm6gmzYD9PtW46+wE9789qjrutiyWA9UGC3ebkFeB51X1Nfd+WyOO/xN42326GWfLnlJd3DJqKDctXGFxiPh7e5f9onoBQoXw51SYtoqtL19Dh+JMEgW2djmB439nOaPrKsGfWJY9ryDzbNp27FXzBSam1dQNNaSmf6oaUtUDarhegCeBVap6f0R55F/zKTgBB5zUMmeJSJyI9AB6A18BXwO9RaSHiARwBsHf2ts3bJqX1a/fVfZ4dsmJXF10Vdnz8DcvEFeUXfZ8x5BrGrVuzV2Sr7xloeonrNYYr84Llx7a1FVocjV1Q5Wur4gHhuHsDSXAgcAiYEQt9x4JnAd8JyJL3bKbgLNFZBBON9SPwOUAqrpCRP6FM3BdAkxV1RCAiFyFs7bDC8xR1RX1eI+mmSgqCfPKy09xUvcQyYNOJRhozUErnc2NV5y/kgu67Mes1+bD986ArOfjmaQKPF0yjidCx/Pa/gc1ZfWbnQR/gjMKCaB+whYrquURIcefSKvi/NpPbqFq6oYaCyAirwFDVPU79/lA4Pbabqyqn1J5nAPg3RquuRO4s4ryd2u6zjRvWz99hnaf3c522nJOwQ+wBph3A0VdjiAO+KLbFYzo6cxp2B3XkQdLTuEa3+sA5Gsc4ZHTmH/MIcT5mj5nRHOSHIjI4xH2k1HHHXhjkSqsT+3M4O1rmroqTaYuK0r6lgYKAFVdDlTb/WRMfQSLi+nw4dV4CrLoXPBDhWMpmQsAyBhTnr06zufhgZLTWRF2toY+KeUlLjh2uAWKvZASESz+OnEwZx7ctYazY5uqEpLYWYBXlbqss1gmIk8Ac93n5wLLolclE0tCc88oe3x10VW8FT6MeIK8GLiTwZ61fBXuy4Fdyzf9u25cX9qlxHH6v28jkSBf3zY2phZGNaSUQHlLYlhGB/s51iCsFRfmZTw/FwnUvrFkS1KXYHEhcCVQOnq4AHg0ajUyMeOHFUvos/EjABYc8TJ/GjqGWa3i6T79HU4puoMU8vEmtmZpRErS5Dgfl4zqSX4wxNh+7ewD7jdIiviwi/ft28mampqihCKWlvm7dMXfoX0T1qjx1dquUtVCVX1AVU9R1VNw1l3cX9t1xlRlx6bVbPn7GLi9NX3+n5MW5Ur/TAYdehQdWjkfWMtuH8etE/qTSyKvXHFYpXsEfB6uP7YvQzOaQerSfdhRB3Qoe2zbfNRsz5aFxOCeY3VpWSAig4GzgTOADcBr0ayUaV6e+O96Xpq/kA+nT4Cfl8AXj8AZz4G//NuqqpI1+wTabv2swrVrPD149E9XVyhrFe/nolE9OOuQriQG6vQravZC2+TyABHvtZZFTcJ7jlnE0AaCpar9S3Q38Tvb/bcDeBmQ0llSxpSa++5H/CfuOrj7yvLC79+GA04EbwBEWLB0NWPcQPFcydEE8XOJ7z3Wpx9B72rua4Eiuvze8m/HcT5rWdQkJc5HyBMxiSIGg0VN7/h74EhggqqOUtWHgFDjVMs0F6rKZG/l9CYlH90Ff2kP9/aBonxyNzlrL19MvoDTbn6RF0NH8n5oGIUHndfYVTauyPEev6fqdLHGMax7OjnJabWf2ILVFCwm4ewa+7GI/FNEjqLqdRMmht31wH1M9n0AOC2GC4puBMCXvdY5IW8b/LUjPX56BYCzLrmehPgA67QzlxdP47gRQ5qk3sbUV5fe5VnxJAZbFjUtynsDeENEknC2BL8WaO9uBPi6qn7QSHU0+6h3Fq/lppyZADzS5ykWFnSmqCQMW/4GwNLw/qTyK909Wxm44z2yNJn01s5c/oDPQ1FJmIAv9v7o9iX5P16BJ2FT7ScawvERu/RasKhMVfNwdqB9wd1O/HTgRsCCRYxb+tp9nOCHlcP+wtQJk5gKLFy/kzH/vI+/+//Bbf5p/JTn5Zv4KwA4Nvi/fO12fXxyw1gKiq1Xs6mN6nowXdIOb+pqNAsVgkUMLtCr1wiiqmbjbAH+eHSqY5qL3IIgZ3gX8FNif/pP+ENZ+eBuqQRb9eCS0P/y0mXDmb9qK3d+cA67SeLpq8vzaO3X2mbf7AueucjSqNZVOKF8EWMsLu+x6SZmr/z8w1L6ejazss/kCuVxPi9fzDiq7PlPWXncFZrA6N5tGdCpdSPX0piGo9YNZUz9fTf/OXqrkHDQpBrPO7JfBz6bfiQdUmxqpmneNKJlYcHCmBqEwsrON6bTftlsTgPWejLo1aP2hDmdUxNqPceYfV3kmIX4Yu+jM/besakfVVj+KqRm8NFPIY5ZNrvs0De9r8Zyq5lYMbhv57LH4o29XY6jFixEpCvwLNABJ9HR46r6oIjcA5wIFAHrgAtVdZeIdAdWAavdWyxU1Svcew0FngYScPJaXKNqab2iLj+L4tXv43/Tmc10jFscUuGuknO46Phzm65uxjSyrp3bkF37aS1WNFsWJcB1qrpERFKAxSIyD5gHzFDVEhH5GzADZyouwDpVHVTFvR4FLgW+xAkW44H3olj3mLV043Z2zH+YodvfIK3gR9ys1xX0KX6epyYfQifrXjIxJJAU28mhohYsVHULzgpwVDVXRFYBnfdYzLcQOK2m+7g5u1up6kL3+bPARCxYNLiSUJjkJw9nkOfnsjI/Ib4M9+OsopsZ51nE/PAQVswcT7w/9prhJrYFYvx3vlGG9N0upsE4LYNIF1HxQ7+HiHwjIgtEZLRb1hnIjDgn0y0zDWzx+l/o5QaKV0Oj+SrsJB1KbteNBTccRUmfCbwy9QgLFCYmBbwetiS24fP9BjR1VZpE1IOFiCQDrwLXqmpORPmfcLqqnneLtgDdVHUwMA1nxXirer7WZSKySEQWbd++vWHeQAz5edN6AO5LvJax019jWbincyB9f7q1SeTJyQczqGtqE9bQmKbj9woXjZvBzOEXNnVVmkRUg4WI+HECxfOq+lpE+WRgAnBu6UC1qgZVdaf7eDHO4HcfYDPQJeK2XdyySlT1cVUdpqrD2rVrF4V31LJlbfkRgGmTDic9KcDyATdwedG1hEZf16T1MmZfULpL7/gB+zVxTZpGNGdDCfAksEpV748oHw/8EThCVfMjytsBWaoaEpGeQG9gvapmiUiOiAzH6cY6H3goWvWOSUV5sPJN2uxeAYCkZgBw/5lDKJh0EElxNsPaGIBvbjkmZv8eovmuRwLnAd+JyFK37CZgFhAHzHMjdekU2cOBO0SkGAgDV6hqlnvdFMqnzr6HDW43qCVvPcKQ5XcyEQgjeFKdnWE9HonZPwxjqpKWFKj9pBYqmrOhPqXq/BfvVnP+qzhdVlUdWwQMbLjatSDhMORugdYRY/6qdd7prDgUJufbt8tmyGZ729LGsqYZY/ZgXxubs//8Df7z17KnesIDFH/zEr6sH/D8z3cQl1LrLT7+76eM835b9nxT+gjaRKWyxpjmzIJFM6OqBN+7mfivHq50TN75H8oayTk/s2H1t3TPX44Ec2DCA5VaG4XFITI+uhI8sKDPTczP6cyZE46L/pswxjQ7FiyaEc3eiDx4IJGZID4KDWJq8dU86H+Ecd7FZeXLln7FgZ9dVX7imnlw9gvQ8aCyok/nzuRoTybbNJXDTruWIwLW/WSMqVrs7bPbjL380PRKZa2OmMqKv05i3gEzmVl8LqcE/wxA7prPK56Yk0nx3DOhYBfkZ7FxZx69NrzADm1FwcUL8FugMMbUwFoWzURmdj4dizPBCwtPW8RVb2ykfd5q3j3qdBDhnnNGAiOZ8/YCWAQjt71Q6R7+vC3wN2da7JaJnzHcs5V395vC8d26N+6bMcY0OxYsmokVm3YwyrOaTfufw/CBvfm0b09Ux1Yahwgntq3wfHTwAbI1hVn+hznSu7SsPH3l0wAMGdgv6nU3xjR/1g21LyrKh9xfKhRlf/h3kiRI+kAnZWm830tCoPIeTWFf+YjGO8mn8sqMc/j6jkl8Fnb2s8nTODK1LRlrngMgJT02V6MaY+rHWhb7mnCInIdG0yp3Ldy+u6z4iN1vgEBSn7E1Xt6jbTL9Cp+iBC9rbz+5rLzr+GsZ/+7v6Nh7EAetf5xrxdl9JSm1fXTehzGmRbFg0RSCv4I/ATx7tAx+3Qb39qZ098R/f76Ye/5vMe+N3UJH2UlIfHiTal4FcUz/Dnx+ywSKQ+EK5ZNH9+HEwRmkJga4c/Yq2OZu1ZVoqyqMMbWzYFGFB+b9wML1O3n58hENfu9tWbtJm9UTPyUUnfMagT5HlR3bOudsOkScm/rhdcyP+wbciU1eLanTa6RXsyVBm2RnxlPXPgfBNrdwjzEOY4ypio1Z7OH9Fb/w4Pw1fLkhi9kL1jX4/Xe9dRN+nA/9wAuTKA6FKQ6F2ZlbSPLO7wB4NTSKfI1jePibCte+c9AjDVKH/Tp0Kn8SiO3sX8aYurFgEWF3fjHTXi6fMXT3e987D3K3sur1u8n88Ye9v/nmJWz7fC5JGyrugfjNn4fjn5nGqtWrSJIgS/pdT/8pL7Io3KfCeQ8Un8qY48/a+9ePMKBTa84rms7drW9rkPsZY1o+64aK0DrRz9tXj2bsvf8pK+s+/R0WZszmgK2fwLd3VRh0rqsNP66jx9NjaQ8g8GbaZJbvjuNP4cc4xLMaAP/SZwDo1r0XyW2TmB0azeHe78ruccGRBzbYDrDd2iRy49QptE22hXjGmLqxlsUeerRN4rSh5bmWTvH8l/22flJ+QubiKq6qRmEO3D+AkqcnVigec+jB5PQ+pULZoZlPAZDYrhvxfi+9j76IUcEHy477Exs2Q93Azq3Zr3V87ScaYwwWLKrk95b/WM71za948Ikj63yfZYs+gZxMevNThfKkjr24ddLB3FF8XqVrEtKdrcavOrI3My84vqw8LtnSmRpjmo4Fiypce3RvAj7nR7NF0yscK0zoUNUlVSr+4rEKz98ODWe7tsbX8XckxfkYOPE6ZnSdy+DC2Xwf7so7oUPKstRBxaDV0C0LY4ypj6gFCxHpKiIfi8hKEVkhIte45ekiMk9E1rj/TXPLRURmichaEVkmIkMi7nWBe/4aEbkgWnUu1aFVPItvPhqAn9VZh7C0wySWhvdnZ2LPut2kYBdD88q7rzYnDeBfGbfx+UkLIJAEwKSDe3DXxSfy+vUn8Y8DnmPkjHcqbN/ROS2h7LEkW05xY0zTieYAdwlwnaouEZEUYLGIzAMmA/NV9W4RmQ5MB24EjsPJu90bOBR4FDhURNKB24BhgLr3eUtVs6NYdxIDzo8mU50P6U6JYVZoMul5O+t0/etzH+IU4IvUCex/0h/p1KkHz8a3qvLc7m2TmHX24ErlPdomsTHcngzPNmjVqYorjTGmcUStZaGqW1R1ifs4F1gFdAZOBp5xT3sGKB39PRl4Vh0LgVQR6QgcC8xT1Sw3QMwDxker3qW8Hucb/i9uN1RKSTbZpCD5dYhRK9/klM33ArB91F9o3/MgpJpAUZvLiqdxTdEUSEjbq+uNMaYhNMqYhYh0BwYDXwIdVHWLe+gXKFu03BnYFHFZpltWXXnUPXruED4LD+TLcD884+5gN8mkeX6t9bqs124oe3zi0O6/qQ6rtRtvhkf9pnsYY8xvFfV1FiKSDLwKXKuqORLRJ6+qKiLagK91GXAZQLdu3X7z/Y77XUeOuONk8oITiEuJo337TiTvLICSIvBVvaXGjtwC2pZsjazTb6rDY+cNRRvsJ2SMMXsnqi0LEfHjBIrnVdXduY6tbvcS7n9LdynaDHSNuLyLW1ZdeSWq+riqDlPVYe3aNcyAcGLAR7sUZ/FaKN6dkVRQfVfU8pUrAfg23JPHOs38za9/7ID9GD/QthE3xjStaM6GEuBJYJWq3h9x6C2gdEbTBcCbEeXnu7OihgO73e6q94FxIpLmzpwa55Y1ulC8O25QkFXtOdnbMwH4uMOFnPX7KxujWsYYE3XR7IYaCZwHfCcipRsu3QTcDfxLRC4GNgJnuMfeBY4H1gL5wIUAqpolIjOBr93z7lDV6j+to0gTnMHukl934KsmDYQ335kt9YcTR+BN9DdW1YwxJqqiFixU9VOgug77o/YsUFUFplZzrznAnIar3d6RRCdYBHN3VvmD+/j7bZy08n8A8Cbb1t/GmJbDVnDXg8dNPLRgqbMb7Zy3P4LbW5Pzw2cA/PHpeeUnp3Rs9PoZY0y0WLCoB1+yEyyW/bABAFk4G4DgwsdZuy2XhXFOw+j24vPBb5v0GWNaDgsW9ZCUlALAdP9LrPk5i4CbxGiTduCBWffjdWcBn3nqGdXewxhjmiMLFvUwolf5OMRd//qITrIDgA1rV3KXb3bZsQOGjG70uhljTDRZsKiHOJ+Xh0tOBmDOrosZ6/0WgPGer1icdLhz0i112zvKGGOaEwsW9fSt9KvwfFO4HUkSpHNwPTu87cFryQeNMS2PBYt6uvyiyys8n15yCQA9i9dS6EtpiioZY0zUWbCop6Hdy5MhPVQykaOPGAOAT8IEfa2bqFbGGBNdFizqSUR4dPRCzi2agf/omxlx4AFkazIAu1r3q+VqY4xpnqyDfS+cN6o3u4smMnlkTwqKQkwu+iPn+z5g4lm3NXXVjDEmKixY7IVw5qVFAAADV0lEQVTkOB/Tj3NaEfF+L99qL64r7sWprWx3WGNMy2TdUMYYY2plLYsG8OqVh7Fue+0Z9IwxprmyYNEAhmakMTTDcmQbY1ou64YyxhhTKwsWxhhjahXNtKpzRGSbiCyPKHtZRJa6/34szaAnIt1FpCDi2OyIa4aKyHcislZEZrnpWo0xxjSiaI5ZPA08DDxbWqCqZ5Y+FpH7gN0R569T1UFV3OdR4FLgS5zUq+OB96JQX2OMMdWIWstCVT8BqsyV7bYOzgBerOkeItIRaKWqC920q88CExu6rsYYY2rWVGMWo4GtqromoqyHiHwjIgtEpDQhRGcgM+KcTLfMGGNMI2qqqbNnU7FVsQXopqo7RWQo8IaIDKjvTUXkMuAygG7dujVIRY0xxjRBsBARHzAJGFpapqpBIOg+Xiwi64A+wGagS8TlXdyyKqnq48Dj7utsF5GNDf4GjDGm5cqo7kBTtCyOBr5X1bLuJRFpB2SpakhEegK9gfWqmiUiOSIyHGeA+3zgobq8iKq2i0LdjTEmJkVz6uyLwBdAXxHJFJGL3UNnUXlg+3BgmTuV9hXgClUtHRyfAjwBrAXWYTOhjDGm0YkzycgYY4ypnq3gNqYBiEjIXVC6QkS+FZHrRKTGvy93Meo5jVVHY34LCxbGNIwCVR2kqgOAY4DjgNqyYXUHLFiYZsG6oYxpACLyq6qbX9d53hP4GmiLM8PkOSDJPXyVqn4uIguBA4ANwDPALOBuYAwQBzyiqo812pswpgYWLIxpAHsGC7dsF9AXyAXCqlooIr2BF1V1mIiMAa5X1Qnu+ZcB7VX1LyISB3wGnK6qGxr1zRhTBctnYUz0+YGHRWQQEMJZQ1SVccCBInKa+7w1zjRyCxamyVmwMCYK3G6oELANZ+xiK3AQzjhhYXWXAX9Q1fcbpZLG1IMNcBvTwNxFprOBh90NMFsDW1Q1DJwHeN1Tc4GUiEvfB64UEb97nz4ikoQx+wBrWRjTMBLcRaV+oARnQPt+99g/gFdF5Hzg30CeW74MCInItzhb+j+IM0Nqibsz83Zsl2Wzj7ABbmOMMbWybihjjDG1smBhjDGmVhYsjDHG1MqChTHGmFpZsDDGGFMrCxbGGGNqZcHCGGNMrSxYGGOMqdX/Bwx3UzTjDk40AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Plot the original data\n", "plt.plot(stock_data, label=\"Original Data\")\n", "\n", "# Plot the predictions\n", "plt.plot(plot_train_pred, label=\"Training Set Predictions\")\n", "plt.plot(plot_validation_pred, label=\"Validation Set Predictions\")\n", "plt.plot(plot_test_pred, label=\"Test Set Predictions\")\n", "\n", "# Add title, axis labels, and a legend\n", "plt.title('S&P 500 Index Forecast')\n", "plt.xlabel('Date')\n", "plt.xticks(rotation=45)\n", "plt.gca().xaxis.set_major_locator(mdates.YearLocator())\n", "plt.gcf().autofmt_xdate()\n", "plt.ylabel('Adjusted Close')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "32ce7009" }, "source": [ "The visualization looks as great as the R-Squared suggested it might. The real test, of course, would be to make some predictions for the future and do some trades (or pretend to do some trades — \"paper trading\" as we call it), then see if we can make any money. Investment advice is way outside the scope of this course, and the stock market has a way of being a \"harsh teacher,\" so please be cautious and remember that this project was only meant to be fun and educational! If successfully predicting the market trends was \"this easy,\" everyone would do it." ] } ], "metadata": { "colab": { "provenance": [] }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 5 }