{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Is Fandango Still Inflating Ratings?\n",
    "\n",
    "In October 2015, Walt Hickey from Fandango published [a popular article](https://fivethirtyeight.com/features/fandango-movies-ratings/) where he presented strong evidence which suggest that Fandango's movie rating system was biased and dishonest. In this project, we'll analyze more recent movie ratings data to determine whether there has been any change in Fandango's rating system after Hickey's analysis.\n",
    "\n",
    "\n",
    "# Understanding the Data\n",
    "\n",
    "We'll work with two samples of movie ratings: the data in one sample was collected previous to Hickey's analysis, while the other sample was collected after. Let's start by reading in the two samples (which are stored as CSV files) and getting familiar with their structure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>FILM</th>\n",
       "      <th>RottenTomatoes</th>\n",
       "      <th>RottenTomatoes_User</th>\n",
       "      <th>Metacritic</th>\n",
       "      <th>Metacritic_User</th>\n",
       "      <th>IMDB</th>\n",
       "      <th>Fandango_Stars</th>\n",
       "      <th>Fandango_Ratingvalue</th>\n",
       "      <th>RT_norm</th>\n",
       "      <th>RT_user_norm</th>\n",
       "      <th>Metacritic_norm</th>\n",
       "      <th>Metacritic_user_nom</th>\n",
       "      <th>IMDB_norm</th>\n",
       "      <th>RT_norm_round</th>\n",
       "      <th>RT_user_norm_round</th>\n",
       "      <th>Metacritic_norm_round</th>\n",
       "      <th>Metacritic_user_norm_round</th>\n",
       "      <th>IMDB_norm_round</th>\n",
       "      <th>Metacritic_user_vote_count</th>\n",
       "      <th>IMDB_user_vote_count</th>\n",
       "      <th>Fandango_votes</th>\n",
       "      <th>Fandango_Difference</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Avengers: Age of Ultron (2015)</td>\n",
       "      <td>74</td>\n",
       "      <td>86</td>\n",
       "      <td>66</td>\n",
       "      <td>7.1</td>\n",
       "      <td>7.8</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>3.70</td>\n",
       "      <td>4.3</td>\n",
       "      <td>3.30</td>\n",
       "      <td>3.55</td>\n",
       "      <td>3.90</td>\n",
       "      <td>3.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>3.5</td>\n",
       "      <td>3.5</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1330</td>\n",
       "      <td>271107</td>\n",
       "      <td>14846</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cinderella (2015)</td>\n",
       "      <td>85</td>\n",
       "      <td>80</td>\n",
       "      <td>67</td>\n",
       "      <td>7.5</td>\n",
       "      <td>7.1</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>4.25</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.35</td>\n",
       "      <td>3.75</td>\n",
       "      <td>3.55</td>\n",
       "      <td>4.5</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.5</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.5</td>\n",
       "      <td>249</td>\n",
       "      <td>65709</td>\n",
       "      <td>12640</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ant-Man (2015)</td>\n",
       "      <td>80</td>\n",
       "      <td>90</td>\n",
       "      <td>64</td>\n",
       "      <td>8.1</td>\n",
       "      <td>7.8</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>4.00</td>\n",
       "      <td>4.5</td>\n",
       "      <td>3.20</td>\n",
       "      <td>4.05</td>\n",
       "      <td>3.90</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>627</td>\n",
       "      <td>103660</td>\n",
       "      <td>12055</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             FILM  RottenTomatoes  RottenTomatoes_User  \\\n",
       "0  Avengers: Age of Ultron (2015)              74                   86   \n",
       "1               Cinderella (2015)              85                   80   \n",
       "2                  Ant-Man (2015)              80                   90   \n",
       "\n",
       "   Metacritic  Metacritic_User  IMDB  Fandango_Stars  Fandango_Ratingvalue  \\\n",
       "0          66              7.1   7.8             5.0                   4.5   \n",
       "1          67              7.5   7.1             5.0                   4.5   \n",
       "2          64              8.1   7.8             5.0                   4.5   \n",
       "\n",
       "   RT_norm  RT_user_norm  Metacritic_norm  Metacritic_user_nom  IMDB_norm  \\\n",
       "0     3.70           4.3             3.30                 3.55       3.90   \n",
       "1     4.25           4.0             3.35                 3.75       3.55   \n",
       "2     4.00           4.5             3.20                 4.05       3.90   \n",
       "\n",
       "   RT_norm_round  RT_user_norm_round  Metacritic_norm_round  \\\n",
       "0            3.5                 4.5                    3.5   \n",
       "1            4.5                 4.0                    3.5   \n",
       "2            4.0                 4.5                    3.0   \n",
       "\n",
       "   Metacritic_user_norm_round  IMDB_norm_round  Metacritic_user_vote_count  \\\n",
       "0                         3.5              4.0                        1330   \n",
       "1                         4.0              3.5                         249   \n",
       "2                         4.0              4.0                         627   \n",
       "\n",
       "   IMDB_user_vote_count  Fandango_votes  Fandango_Difference  \n",
       "0                271107           14846                  0.5  \n",
       "1                 65709           12640                  0.5  \n",
       "2                103660           12055                  0.5  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "pd.options.display.max_columns = 100  # Avoid having displayed truncated output\n",
    "\n",
    "previous = pd.read_csv('fandango_score_comparison.csv')\n",
    "after = pd.read_csv('movie_ratings_16_17.csv')\n",
    "\n",
    "previous.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movie</th>\n",
       "      <th>year</th>\n",
       "      <th>metascore</th>\n",
       "      <th>imdb</th>\n",
       "      <th>tmeter</th>\n",
       "      <th>audience</th>\n",
       "      <th>fandango</th>\n",
       "      <th>n_metascore</th>\n",
       "      <th>n_imdb</th>\n",
       "      <th>n_tmeter</th>\n",
       "      <th>n_audience</th>\n",
       "      <th>nr_metascore</th>\n",
       "      <th>nr_imdb</th>\n",
       "      <th>nr_tmeter</th>\n",
       "      <th>nr_audience</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10 Cloverfield Lane</td>\n",
       "      <td>2016</td>\n",
       "      <td>76</td>\n",
       "      <td>7.2</td>\n",
       "      <td>90</td>\n",
       "      <td>79</td>\n",
       "      <td>3.5</td>\n",
       "      <td>3.80</td>\n",
       "      <td>3.60</td>\n",
       "      <td>4.5</td>\n",
       "      <td>3.95</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>13 Hours</td>\n",
       "      <td>2016</td>\n",
       "      <td>48</td>\n",
       "      <td>7.3</td>\n",
       "      <td>50</td>\n",
       "      <td>83</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2.40</td>\n",
       "      <td>3.65</td>\n",
       "      <td>2.5</td>\n",
       "      <td>4.15</td>\n",
       "      <td>2.5</td>\n",
       "      <td>3.5</td>\n",
       "      <td>2.5</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A Cure for Wellness</td>\n",
       "      <td>2016</td>\n",
       "      <td>47</td>\n",
       "      <td>6.6</td>\n",
       "      <td>40</td>\n",
       "      <td>47</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.35</td>\n",
       "      <td>3.30</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.35</td>\n",
       "      <td>2.5</td>\n",
       "      <td>3.5</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 movie  year  metascore  imdb  tmeter  audience  fandango  \\\n",
       "0  10 Cloverfield Lane  2016         76   7.2      90        79       3.5   \n",
       "1             13 Hours  2016         48   7.3      50        83       4.5   \n",
       "2  A Cure for Wellness  2016         47   6.6      40        47       3.0   \n",
       "\n",
       "   n_metascore  n_imdb  n_tmeter  n_audience  nr_metascore  nr_imdb  \\\n",
       "0         3.80    3.60       4.5        3.95           4.0      3.5   \n",
       "1         2.40    3.65       2.5        4.15           2.5      3.5   \n",
       "2         2.35    3.30       2.0        2.35           2.5      3.5   \n",
       "\n",
       "   nr_tmeter  nr_audience  \n",
       "0        4.5          4.0  \n",
       "1        2.5          4.0  \n",
       "2        2.0          2.5  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "after.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below we isolate only the columns that provide information about Fandango so we make the relevant data more readily available for later use. We'll make copies [to avoid any `SettingWithCopyWarning`](https://www.dataquest.io/blog/settingwithcopywarning/) later on. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>FILM</th>\n",
       "      <th>Fandango_Stars</th>\n",
       "      <th>Fandango_Ratingvalue</th>\n",
       "      <th>Fandango_votes</th>\n",
       "      <th>Fandango_Difference</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Avengers: Age of Ultron (2015)</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>14846</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cinderella (2015)</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>12640</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ant-Man (2015)</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>12055</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             FILM  Fandango_Stars  Fandango_Ratingvalue  \\\n",
       "0  Avengers: Age of Ultron (2015)             5.0                   4.5   \n",
       "1               Cinderella (2015)             5.0                   4.5   \n",
       "2                  Ant-Man (2015)             5.0                   4.5   \n",
       "\n",
       "   Fandango_votes  Fandango_Difference  \n",
       "0           14846                  0.5  \n",
       "1           12640                  0.5  \n",
       "2           12055                  0.5  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fandango_previous = previous[['FILM', 'Fandango_Stars', 'Fandango_Ratingvalue', 'Fandango_votes',\n",
    "                             'Fandango_Difference']].copy()\n",
    "fandango_after = after[['movie', 'year', 'fandango']].copy()\n",
    "\n",
    "fandango_previous.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movie</th>\n",
       "      <th>year</th>\n",
       "      <th>fandango</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10 Cloverfield Lane</td>\n",
       "      <td>2016</td>\n",
       "      <td>3.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>13 Hours</td>\n",
       "      <td>2016</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A Cure for Wellness</td>\n",
       "      <td>2016</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 movie  year  fandango\n",
       "0  10 Cloverfield Lane  2016       3.5\n",
       "1             13 Hours  2016       4.5\n",
       "2  A Cure for Wellness  2016       3.0"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fandango_after.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our goal is to determine whether there has been change in Fandango's rating system after Hickey's analysis. The population of interest is made of all the movie ratings stored on Fandango's website, regardless of the releasing year. Because we're interested about whether the parameters of this population changed after Hickey's analysis, we're interested to sample it at two different periods in time — previous and after Hickey's analysis — so we can compare the two states.\n",
    "\n",
    "The data we're working with was sampled at the moments we want: one sample was taken previous to the analysis, and the other after the analysis. We want to describe the population, so we need to make sure that the samples are representative, otherwise we should expect a large sampling error and, ultimately, wrong conclusions.\n",
    "\n",
    "From Hickey's article and from the `README.md` of [the data set's repository](https://github.com/fivethirtyeight/data/tree/master/fandango), we can see that he used the following sampling criteria:\n",
    "\n",
    "* The movie must have had at least 30 fan ratings on Fandango's website at the time of sampling (Aug. 24, 2015).\n",
    "* The movie must have had tickets on sale in 2015.\n",
    "\n",
    "The sampling was clearly not random because not every movie had the same chance to be included in the sample — some movies didn't have a chance at all (like those with under 30 fan reviews or those without tickets on sale in 2015). It's questionable whether this sample is representative of the entire population we're interested to describe. It's much more likely that it isn't, mostly because this sample is subject to *temporal trends* — e.g. movies in 2015 might have been outstandingly good or bad compared to other years.\n",
    "\n",
    "The sampling conditions for our other sample were (as it can be read in the `README.md` of [the data set's repository](https://github.com/mircealex/Movie_ratings_2016_17)):\n",
    "\n",
    "* The movie must have been released in 2016 or later.\n",
    "* The movie must have had a considerable number of votes and reviews (unclear how many from the `README.md` or from the data).\n",
    "\n",
    "This second sample is also subject to temporal trends and it's unlikely to be representative of our population of interest.\n",
    "\n",
    "Both these authors had certain research questions in mind when they sampled the data, and they used a set of criteria to get a sample that would fit their questions. Their sampling method is called [**purposive sampling**](https://youtu.be/CdK7N_kTzHI) (or judgmental/selective/subjective sampling)]. While these samples were good enough for their research, they don't seem too useful for us.\n",
    "\n",
    "# Changing the Goal of our Analysis\n",
    "\n",
    "At this point, we can either collect new data or change our the goal of our analysis. We choose the latter and place some limitations on our initial goal.\n",
    "\n",
    "Instead of trying to determine whether there has been any change in Fandango's rating system after Hickey's analysis, our new goal is to determine whether there's any difference between Fandango's ratings for popular movies in 2015 and Fandango's ratings for popular movies in 2016. This new goal should also be a fairly good proxy for our initial goal.\n",
    "\n",
    "# Isolating the Samples We Need\n",
    "\n",
    "With this new research goal, we have two populations of interest:\n",
    "\n",
    "1. All Fandango's ratings for popular movies released in 2015.\n",
    "2. All Fandango's ratings for popular movies released in 2016.\n",
    "\n",
    "We need to be clear about what counts as popular movies. We'll use Hickey's benchmark of 30 fan ratings and count a movie as popular only if it has 30 fan ratings or more on Fandango's website.\n",
    "\n",
    "Although one of the sampling criteria in our second sample is movie popularity, the sample doesn't provide information about the number of fan ratings. We should be skeptical once more and ask whether this sample is truly representative and contains popular movies as we defined them.\n",
    "\n",
    "One quick way to check the representativity of this sample is to sample randomly 10 movies from it and then check the number of fan ratings ourselves on Fandango's website. Ideally, at least 8 out of the 10 movies have 30 fan ratings or over."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movie</th>\n",
       "      <th>year</th>\n",
       "      <th>fandango</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>108</th>\n",
       "      <td>Mechanic: Resurrection</td>\n",
       "      <td>2016</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>206</th>\n",
       "      <td>Warcraft</td>\n",
       "      <td>2016</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>106</th>\n",
       "      <td>Max Steel</td>\n",
       "      <td>2016</td>\n",
       "      <td>3.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>107</th>\n",
       "      <td>Me Before You</td>\n",
       "      <td>2016</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>Fantastic Beasts and Where to Find Them</td>\n",
       "      <td>2016</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>Cell</td>\n",
       "      <td>2016</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59</th>\n",
       "      <td>Genius</td>\n",
       "      <td>2016</td>\n",
       "      <td>3.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>152</th>\n",
       "      <td>Sully</td>\n",
       "      <td>2016</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>A Hologram for the King</td>\n",
       "      <td>2016</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>Captain America: Civil War</td>\n",
       "      <td>2016</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                       movie  year  fandango\n",
       "108                   Mechanic: Resurrection  2016       4.0\n",
       "206                                 Warcraft  2016       4.0\n",
       "106                                Max Steel  2016       3.5\n",
       "107                            Me Before You  2016       4.5\n",
       "51   Fantastic Beasts and Where to Find Them  2016       4.5\n",
       "33                                      Cell  2016       3.0\n",
       "59                                    Genius  2016       3.5\n",
       "152                                    Sully  2016       4.5\n",
       "4                    A Hologram for the King  2016       3.0\n",
       "31                Captain America: Civil War  2016       4.5"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fandango_after.sample(10, random_state = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above we used a value of 1 as the random seed. This is good practice because it suggests that we weren't trying out various random seeds just to get a favorable sample.\n",
    "\n",
    "As of April 2018, these are the fan ratings we found:\n",
    "\n",
    "<style type=\"text/css\">\n",
    ".tg  {border-collapse:collapse;border-spacing:0;}\n",
    ".tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:black;}\n",
    ".tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;border-color:black;}\n",
    ".tg .tg-baqh{text-align:center;vertical-align:top}\n",
    ".tg .tg-amwm{font-weight:bold;text-align:center;vertical-align:top}\n",
    ".tg .tg-yw4l{vertical-align:top}\n",
    "</style>\n",
    "<table class=\"tg\">\n",
    "  <tr>\n",
    "    <th class=\"tg-amwm\">Movie</th>\n",
    "    <th class=\"tg-amwm\">Fan ratings</th>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td class=\"tg-baqh\">Mechanic: Resurrection</td>\n",
    "    <td class=\"tg-baqh\">2247</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td class=\"tg-baqh\">Warcraft</td>\n",
    "    <td class=\"tg-baqh\">7271</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td class=\"tg-baqh\">Max Steel</td>\n",
    "    <td class=\"tg-baqh\">493</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td class=\"tg-baqh\">Me Before You</td>\n",
    "    <td class=\"tg-baqh\">5263</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td class=\"tg-baqh\">Fantastic Beasts and Where to Find Them</td>\n",
    "    <td class=\"tg-baqh\">13400</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td class=\"tg-yw4l\">Cell</td>\n",
    "    <td class=\"tg-yw4l\">17</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td class=\"tg-yw4l\">Genius</td>\n",
    "    <td class=\"tg-yw4l\">127</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td class=\"tg-yw4l\">Sully</td>\n",
    "    <td class=\"tg-yw4l\">11877</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td class=\"tg-yw4l\">A Hologram for the King\t</td>\n",
    "    <td class=\"tg-yw4l\">500</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td class=\"tg-yw4l\">Captain America: Civil War</td>\n",
    "    <td class=\"tg-yw4l\">35057</td>\n",
    "  </tr>\n",
    "</table>\n",
    "\n",
    "\n",
    "90% of the movies in our sample are popular. This is enough and we move forward with a bit more confidence.\n",
    "\n",
    "Let's also double-check the other data set for popular movies. The documentation states clearly that there're only movies with at least 30 fan ratings, but it should take only a couple of seconds to double-check here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(fandango_previous.Fandango_votes < 30)  # `fandango_previous.Fandango_votes` is the same\n",
    "                                             # as `fandago_previous['Fandango_votes']`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you explore the data sets, you'll notice that there are movies that weren't released in 2015 or 2016. For our purposes, we'll need to isolate only the movies released in 2015 and 2016."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>FILM</th>\n",
       "      <th>Fandango_Stars</th>\n",
       "      <th>Fandango_Ratingvalue</th>\n",
       "      <th>Fandango_votes</th>\n",
       "      <th>Fandango_Difference</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Avengers: Age of Ultron (2015)</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>14846</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cinderella (2015)</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>12640</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             FILM  Fandango_Stars  Fandango_Ratingvalue  \\\n",
       "0  Avengers: Age of Ultron (2015)             5.0                   4.5   \n",
       "1               Cinderella (2015)             5.0                   4.5   \n",
       "\n",
       "   Fandango_votes  Fandango_Difference  \n",
       "0           14846                  0.5  \n",
       "1           12640                  0.5  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fandango_previous.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There's no special column for the releasing year, but we can extract it from the strings in the `FILM` column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>FILM</th>\n",
       "      <th>Fandango_Stars</th>\n",
       "      <th>Fandango_Ratingvalue</th>\n",
       "      <th>Fandango_votes</th>\n",
       "      <th>Fandango_Difference</th>\n",
       "      <th>Year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Avengers: Age of Ultron (2015)</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>14846</td>\n",
       "      <td>0.5</td>\n",
       "      <td>2015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cinderella (2015)</td>\n",
       "      <td>5.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>12640</td>\n",
       "      <td>0.5</td>\n",
       "      <td>2015</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             FILM  Fandango_Stars  Fandango_Ratingvalue  \\\n",
       "0  Avengers: Age of Ultron (2015)             5.0                   4.5   \n",
       "1               Cinderella (2015)             5.0                   4.5   \n",
       "\n",
       "   Fandango_votes  Fandango_Difference  Year  \n",
       "0           14846                  0.5  2015  \n",
       "1           12640                  0.5  2015  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fandango_previous['Year'] = fandango_previous.FILM.str[-5:-1]\n",
    "fandango_previous.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2015    129\n",
       "2014     17\n",
       "Name: Year, dtype: int64"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fandango_previous.Year.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2015    129\n",
       "Name: Year, dtype: int64"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fandango_2015 = fandango_previous[fandango_previous.Year == '2015'].copy()\n",
    "fandango_2015.Year.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great, now let's isolate the movies in the other data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movie</th>\n",
       "      <th>year</th>\n",
       "      <th>fandango</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10 Cloverfield Lane</td>\n",
       "      <td>2016</td>\n",
       "      <td>3.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>13 Hours</td>\n",
       "      <td>2016</td>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 movie  year  fandango\n",
       "0  10 Cloverfield Lane  2016       3.5\n",
       "1             13 Hours  2016       4.5"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fandango_after.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2016    191\n",
       "2017     23\n",
       "Name: year, dtype: int64"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fandango_after.year.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2016    191\n",
       "Name: year, dtype: int64"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fandango_2016 = fandango_after[fandango_after.year == 2016].copy()\n",
    "fandango_2016.year.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Comparing Distribution Shapes for 2015 and 2016\n",
    "\n",
    "Once again, our aim is to figure out whether there's any difference between Fandango's ratings for popular movies in 2015 and Fandango's ratings for popular movies in 2016. One way to go about is to analyze and compare the distributions of movie ratings for the two samples.\n",
    "\n",
    "We'll start with comparing the shape of the two distributions using kernel density plots."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 53964 characters]\n",
      "text/plain": [
       "<Figure size 576x396 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from numpy import arange\n",
    "%matplotlib inline\n",
    "plt.style.use('fivethirtyeight')\n",
    "\n",
    "fandango_2015.Fandango_Stars.plot.kde(label = '2015', legend = True, figsize = (8,5.5))\n",
    "fandango_2016.fandango.plot.kde(label = '2016', legend = True)\n",
    "\n",
    "plt.title(\"Comparing distribution shapes for Fandango's ratings\\n(2015 vs 2016)\",\n",
    "          y = 1.07) # the `y` parameter pads the title upward\n",
    "plt.xlabel('Stars')\n",
    "plt.xlim(0,5) # because ratings start at 0 and end at 5\n",
    "plt.xticks(arange(0,5.1,.5))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Two aspects are striking on the figure above:\n",
    "* Both distributions are strongly left skewed.\n",
    "* The 2016 distribution is slightly shifted to the left.\n",
    "\n",
    "The left skew suggests that movies on Fandango are given mostly high and very high fan ratings. Coupled with the fact that Fandango sells tickets, the high ratings are a bit dubious. It'd be really interesting to investigate this further — ideally in a separate project, since this is quite irrelevant for the current goal of our analysis.\n",
    "\n",
    "The slight left shift of the 2016 distribution is very interesting for our analysis. It shows that ratings were slightly lower in 2016 compared to 2015. This confirms that there was a difference indeed between Fandango's ratings for popular movies in 2015 and Fandango's ratings for popular movies in 2016. We can also see the direction of the difference: the ratings in 2016 were slightly lower than in 2015.\n",
    "\n",
    "\n",
    "# Comparing Relative Frequencies\n",
    "\n",
    "It seems we're following a good thread so far, but we need to analyze more granular information. Let's examine the frequency tables of the two distributions to analyze some numbers. Because the data sets have different numbers of movies, we normalize the tables and show percentages instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2015\n",
      "----------------\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "3.0     8.527132\n",
       "3.5    17.829457\n",
       "4.0    28.682171\n",
       "4.5    37.984496\n",
       "5.0     6.976744\n",
       "Name: Fandango_Stars, dtype: float64"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('2015' + '\\n' + '-' * 16) # To help us distinguish between the two tables immediately and\n",
    "                                # avoid silly mistakes as we read to and fro\n",
    "fandango_2015.Fandango_Stars.value_counts(normalize = True).sort_index() * 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2016\n",
      "----------------\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "2.5     3.141361\n",
       "3.0     7.329843\n",
       "3.5    24.083770\n",
       "4.0    40.314136\n",
       "4.5    24.607330\n",
       "5.0     0.523560\n",
       "Name: fandango, dtype: float64"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('2016' + '\\n' + '-' * 16)\n",
    "fandango_2016.fandango.value_counts(normalize = True).sort_index() * 100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In 2016, very high ratings (4.5 and 5 stars) had significantly lower percentages compared to 2015. In 2016, under 1% of the movies were given a perfect rating of 5 stars, compared to 2015 when the percentage was close to 7%. Ratings of 4.5 were also more popular in 2015 — there were approximately 13% more movies rated with a 4.5 in 2015 compared to 2016.\n",
    "\n",
    "The minimum rating is also lower in 2016 — 2.5 instead of 3 stars, the minimum of 2015. There clearly is a difference between the two frequency distributions.\n",
    "\n",
    "For some other ratings, the percentage went up in 2016. There was a greater percentage of movies in 2016 that received 3.5 and 4 stars, compared to 2015. 3.5 and 4.0 are high ratings and this challenges the direction of the change we saw on the kernel density plots.\n",
    "\n",
    "# Determining the Direction of the Change\n",
    "\n",
    "Let's take a couple of summary metrics to get a more precise picture about the direction of the change. In what follows, we'll compute the mean, the median, and the mode for both distributions and then plot the values with a bar graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>2015</th>\n",
       "      <th>2016</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>4.085271</td>\n",
       "      <td>3.887435</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>median</th>\n",
       "      <td>4.000000</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mode</th>\n",
       "      <td>4.500000</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            2015      2016\n",
       "mean    4.085271  3.887435\n",
       "median  4.000000  4.000000\n",
       "mode    4.500000  4.000000"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_2015 = fandango_2015.Fandango_Stars.mean()\n",
    "mean_2016 = fandango_2016.fandango.mean()\n",
    "\n",
    "median_2015 = fandango_2015.Fandango_Stars.median()\n",
    "median_2016 = fandango_2016.fandango.median()\n",
    "\n",
    "mode_2015 = fandango_2015.Fandango_Stars.mode()[0] # the output of Series.mode() is a bit uncommon\n",
    "mode_2016 = fandango_2016.fandango.mode()[0]\n",
    "\n",
    "summary = pd.DataFrame()\n",
    "summary['2015'] = [mean_2015, median_2015, mode_2015]\n",
    "summary['2016'] = [mean_2016, median_2016, mode_2016]\n",
    "summary.index = ['mean', 'median', 'mode']\n",
    "summary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 26832 characters]\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.style.use('fivethirtyeight')\n",
    "summary['2015'].plot.bar(color = '#0066FF', align = 'center', label = '2015', width = .25)\n",
    "summary['2016'].plot.bar(color = '#CC0000', align = 'edge', label = '2016', width = .25,\n",
    "                         rot = 0, figsize = (8,5))\n",
    "\n",
    "plt.title('Comparing summary statistics: 2015 vs 2016', y = 1.07)\n",
    "plt.ylim(0,5.5)\n",
    "plt.yticks(arange(0,5.1,.5))\n",
    "plt.ylabel('Stars')\n",
    "plt.legend(framealpha = 0, loc = 'upper center')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The mean rating was lower in 2016 with approximately 0.2. This means a drop of almost 5% relative to the mean rating in 2015."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.048426835689519929"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(summary.loc['mean'][0] - summary.loc['mean'][1]) / summary.loc['mean'][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While the median is the same for both distributions, the mode is lower in 2016 by 0.5. Coupled with what we saw for the mean, the direction of the change we saw on the kernel density plot is confirmed: on average, popular movies released in 2016 were rated slightly lower than popular movies released in 2015.\n",
    "\n",
    "# Conclusion\n",
    "\n",
    "Our analysis showed that there's indeed a slight difference between Fandango's ratings for popular movies in 2015 and Fandango's ratings for popular movies in 2016. We also determined that, on average, popular movies released in 2016 were rated lower on Fandango than popular movies released in 2015.\n",
    "\n",
    "We cannot be completely sure what caused the change, but the chances are very high that it was caused by Fandango fixing the biased rating system after Hickey's analysis."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
