Regression Week 3: Assessing Fit (polynomial regression)

In this notebook you will compare different regression models in order to assess which model fits best. We will be using polynomial regression as a means to examine this topic. In particular you will:

  • Write a function to take an SArray and a degree and return an SFrame where each column is the SArray to a polynomial value up to the total degree e.g. degree = 3 then column 1 is the SArray column 2 is the SArray squared and column 3 is the SArray cubed
  • Use matplotlib to visualize polynomial regressions
  • Use matplotlib to visualize the same polynomial degree on different subsets of the data
  • Use a validation set to select a polynomial degree
  • Assess the final fit using test data

We will continue to use the House data from previous notebooks.

Fire up graphlab create

In [210]:
import graphlab

Next we're going to write a polynomial function that takes an SArray and a maximal degree and returns an SFrame with columns containing the SArray to all the powers up to the maximal degree.

The easiest way to apply a power to an SArray is to use the .apply() and lambda x: functions. For example to take the example array and compute the third power we can do as follows: (note running this cell the first time may take longer than expected since it loads graphlab)

In [211]:
tmp = graphlab.SArray([1., 2., 3.])
tmp_cubed = tmp.apply(lambda x: x**3)
print tmp
print tmp_cubed
[1.0, 2.0, 3.0]
[1.0, 8.0, 27.0]

We can create an empty SFrame using graphlab.SFrame() and then add any columns to it with ex_sframe['column_name'] = value. For example we create an empty SFrame and make the column 'power_1' to be the first power of tmp (i.e. tmp itself).

In [212]:
ex_sframe = graphlab.SFrame()
ex_sframe['power_1'] = tmp
print ex_sframe
+---------+
| power_1 |
+---------+
|   1.0   |
|   2.0   |
|   3.0   |
+---------+
[3 rows x 1 columns]

Polynomial_sframe function

Using the hints above complete the following function to create an SFrame consisting of the powers of an SArray up to a specific degree:

In [213]:
def polynomial_sframe(feature, degree):
    # assume that degree >= 1
    # initialize the SFrame:
    poly_sframe = graphlab.SFrame()
    # and set poly_sframe['power_1'] equal to the passed feature
    poly_sframe['power_1'] = feature

    # first check if degree > 1
    if degree > 1:
        # then loop over the remaining degrees:
        # range usually starts at 0 and stops at the endpoint-1. We want it to start at 2 and stop at degree
        for power in range(2, degree+1): 
            # first we'll give the column a name:
            name = 'power_' + str(power)
            # then assign poly_sframe[name] to the appropriate power of feature
            poly_sframe[name] = feature.apply(lambda x: x**power)

    return poly_sframe

To test your function consider the smaller tmp variable and what you would expect the outcome of the following call:

In [214]:
p =  polynomial_sframe(tmp, 3)
print str(type(p))
p
<class 'graphlab.data_structures.sframe.SFrame'>
Out[214]:
power_1 power_2 power_3
1.0 1.0 1.0
2.0 4.0 8.0
3.0 9.0 27.0
[3 rows x 3 columns]

Visualizing polynomial regression

Let's use matplotlib to visualize what a polynomial regression looks like on some real data.

In [215]:
sales = graphlab.SFrame('kc_house_data.gl/')

As in Week 3, we will use the sqft_living variable. For plotting purposes (connecting the dots), you'll need to sort by the values of sqft_living. For houses with identical square footage, we break the tie by their prices.

In [216]:
sales = sales.sort(['sqft_living', 'price'])

Let's start with a degree 1 polynomial using 'sqft_living' (i.e. a line) to predict 'price' and plot what it looks like.

In [217]:
print poly1_data
poly1_data = polynomial_sframe(sales['sqft_living'], 1)
poly1_data['price'] = sales['price'] # add price to the data since it's the target
print poly1_data[:2]
print len(poly1_data)
+---------+----------+
| power_1 |  price   |
+---------+----------+
|  290.0  | 142000.0 |
|  370.0  | 276000.0 |
|  380.0  | 245000.0 |
|  384.0  | 265000.0 |
|  390.0  | 228000.0 |
|  390.0  | 245000.0 |
|  410.0  | 325000.0 |
|  420.0  | 229050.0 |
|  420.0  | 280000.0 |
|  430.0  | 80000.0  |
+---------+----------+
[21613 rows x 2 columns]
Note: Only the head of the SFrame is printed.
You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.
+---------+----------+
| power_1 |  price   |
+---------+----------+
|  290.0  | 142000.0 |
|  370.0  | 276000.0 |
+---------+----------+
[2 rows x 2 columns]

21613

NOTE: for all the models in this notebook use validation_set = None to ensure that all results are consistent across users.

In [218]:
model1 = graphlab.linear_regression.create(poly1_data, target = 'price', features = ['power_1'], validation_set = None)
Linear regression:
--------------------------------------------------------
Number of examples          : 21613
Number of features          : 1
Number of unpacked features : 1
Number of coefficients    : 2
Starting Newton Method
--------------------------------------------------------
+-----------+----------+--------------+--------------------+---------------+
| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |
+-----------+----------+--------------+--------------------+---------------+
| 1         | 2        | 0.021837     | 4362074.696077     | 261440.790724 |
+-----------+----------+--------------+--------------------+---------------+
SUCCESS: Optimal solution found.

In [219]:
#let's take a look at the weights before we plot
model1.get("coefficients")
Out[219]:
name index value stderr
(intercept) None -43579.0852515 4402.68969743
power_1 None 280.622770886 1.93639855513
[2 rows x 4 columns]
In [220]:
import matplotlib.pyplot as plt
%matplotlib inline
In [221]:
fig = plt.figure()
plt.xlabel('sqft')
plt.ylabel('Price ($)')
plt.plot(poly1_data['power_1'],poly1_data['price'],'.',
        poly1_data['power_1'], model1.predict(poly1_data),'-')
Out[221]:
[<matplotlib.lines.Line2D at 0x7f8be958c590>,
 <matplotlib.lines.Line2D at 0x7f8be9417290>]

Let's unpack that plt.plot() command. The first pair of SArrays we passed are the 1st power of sqft and the actual price we then ask it to print these as dots '.'. The next pair we pass is the 1st power of sqft and the predicted values from the linear model. We ask these to be plotted as a line '-'.

We can see, not surprisingly, that the predicted values all fall on a line, specifically the one with slope 280 and intercept -43579. What if we wanted to plot a second degree polynomial?

In [222]:
poly2_data = polynomial_sframe(sales['sqft_living'], 2)
my_features = poly2_data.column_names() # get the name of the features
poly2_data['price'] = sales['price'] # add price to the data since it's the target
model2 = graphlab.linear_regression.create(poly2_data, target = 'price', features = my_features, validation_set = None)
Linear regression:
--------------------------------------------------------
Number of examples          : 21613
Number of features          : 2
Number of unpacked features : 2
Number of coefficients    : 3
Starting Newton Method
--------------------------------------------------------
+-----------+----------+--------------+--------------------+---------------+
| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |
+-----------+----------+--------------+--------------------+---------------+
| 1         | 2        | 0.018608     | 5913020.984255     | 250948.368758 |
+-----------+----------+--------------+--------------------+---------------+
SUCCESS: Optimal solution found.

In [223]:
model2.get("coefficients")
Out[223]:
name index value stderr
(intercept) None 199222.496445 7058.00483552
power_1 None 67.9940640677 5.28787201316
power_2 None 0.0385812312789 0.000898246547032
[3 rows x 4 columns]
In [224]:
fig = plt.figure()
plt.xlabel('Quadratic Function (sqft)')
plt.ylabel('Price ($)')
plt.plot(poly2_data['power_1'],poly2_data['price'],'.',
        poly2_data['power_1'], model2.predict(poly2_data),'-')
Out[224]:
[<matplotlib.lines.Line2D at 0x7f8be94dbe90>,
 <matplotlib.lines.Line2D at 0x7f8be93690d0>]

The resulting model looks like half a parabola. Try on your own to see what the cubic looks like:

In [225]:
poly3_data = polynomial_sframe(sales['sqft_living'], 3)
my_features = poly3_data.column_names() # get the name of the features
poly3_data['price'] = sales['price'] # add price to the data since it's the target
model3 = graphlab.linear_regression.create(poly3_data, target = 'price', features = my_features, validation_set = None)
Linear regression:
--------------------------------------------------------
Number of examples          : 21613
Number of features          : 3
Number of unpacked features : 3
Number of coefficients    : 4
Starting Newton Method
--------------------------------------------------------
+-----------+----------+--------------+--------------------+---------------+
| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |
+-----------+----------+--------------+--------------------+---------------+
| 1         | 2        | 0.032930     | 3261066.736007     | 249261.286346 |
+-----------+----------+--------------+--------------------+---------------+
SUCCESS: Optimal solution found.

In [226]:
fig = plt.figure()
plt.xlabel('Cubic Function (sqft)')
plt.ylabel('Price ($)')
plt.plot(poly3_data['power_1'],poly3_data['price'],'.',
        poly3_data['power_1'], model3.predict(poly3_data),'-')
Out[226]:
[<matplotlib.lines.Line2D at 0x7f8be9330650>,
 <matplotlib.lines.Line2D at 0x7f8be92acc10>]

Now try a 15th degree polynomial:

In [227]:
poly15_data = polynomial_sframe(sales['sqft_living'], 15)

my_features = poly15_data.column_names() # get the name of the features

poly15_data['price'] = sales['price'] # add price to the data since it's the target

model15 = graphlab.linear_regression.create(poly15_data, target = 'price', features = my_features, validation_set = None)
Linear regression:
--------------------------------------------------------
Number of examples          : 21613
Number of features          : 15
Number of unpacked features : 15
Number of coefficients    : 16
Starting Newton Method
--------------------------------------------------------
+-----------+----------+--------------+--------------------+---------------+
| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |
+-----------+----------+--------------+--------------------+---------------+
| 1         | 2        | 0.041314     | 2662308.584338     | 245690.511190 |
+-----------+----------+--------------+--------------------+---------------+
SUCCESS: Optimal solution found.

In [228]:
fig = plt.figure()
plt.xlabel('15th Polynomial Function (sqft)')
plt.ylabel('Price ($)')
plt.plot(poly15_data['power_1'],poly15_data['price'],'.',
        poly15_data['power_1'], model15.predict(poly15_data),'-')
Out[228]:
[<matplotlib.lines.Line2D at 0x7f8be94aab10>,
 <matplotlib.lines.Line2D at 0x7f8be91f1bd0>]

What do you think of the 15th degree polynomial? Do you think this is appropriate? If we were to change the data do you think you'd get pretty much the same curve? Let's take a look.

Changing the data and re-learning

We're going to split the sales data into four subsets of roughly equal size. Then you will estimate a 15th degree polynomial model on all four subsets of the data. Print the coefficients (you should use .print_rows(num_rows = 16) to view all of them) and plot the resulting fit (as we did above). The quiz will ask you some questions about these results.

To split the sales data into four subsets, we perform the following steps:

  • First split sales into 2 subsets with .random_split(0.5, seed=0).
  • Next split the resulting subsets into 2 more subsets each. Use .random_split(0.5, seed=0).

We set seed=0 in these steps so that different users get consistent results. You should end up with 4 subsets (set_1, set_2, set_3, set_4) of approximately equal size.

In [229]:
ss1,ss2 = sales.random_split(.5,seed=0)
set_1, set_2 = ss1.random_split(.5, seed=0)
set_3, set_4 = ss2.random_split(.5, seed=0)

Fit a 15th degree polynomial on set_1, set_2, set_3, and set_4 using sqft_living to predict prices. Print the coefficients and make a plot of the resulting model.

In [230]:
fig = plt.figure()
fig.suptitle('SET 1')
plt.xlabel('15th Polynomial Function (sqft)')
plt.ylabel('Price ($)')

set1_poly15_data = polynomial_sframe(set_1['sqft_living'], 15)
my_features = set1_poly15_data.column_names() # get the name of the features
set1_poly15_data['price'] = set_1['price'] # add price to the data since it's the target
model_set1_15 = graphlab.linear_regression.create(set1_poly15_data, target = 'price', features = my_features, validation_set = None)
model_set1_15.get("coefficients").print_rows(num_rows=16)

plot = plt.plot(set1_poly15_data['power_1'],set1_poly15_data['price'],'.',
        set1_poly15_data['power_1'], model_set1_15.predict(set1_poly15_data),'-')
Linear regression:
--------------------------------------------------------
Number of examples          : 5404
Number of features          : 15
Number of unpacked features : 15
Number of coefficients    : 16
Starting Newton Method
--------------------------------------------------------
+-----------+----------+--------------+--------------------+---------------+
| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |
+-----------+----------+--------------+--------------------+---------------+
| 1         | 2        | 0.006622     | 2195218.932305     | 248858.822200 |
+-----------+----------+--------------+--------------------+---------------+
SUCCESS: Optimal solution found.

+-------------+-------+--------------------+-------------------+
|     name    | index |       value        |       stderr      |
+-------------+-------+--------------------+-------------------+
| (intercept) |  None |    223312.75025    |   512219.445212   |
|   power_1   |  None |   118.086127585    |   1834.21083609   |
|   power_2   |  None |  -0.0473482011328  |   2.58684418296   |
|   power_3   |  None | 3.25310342466e-05  |  0.00198429469675 |
|   power_4   |  None | -3.3237215257e-09  | 1.23319909264e-06 |
|   power_5   |  None | -9.75830457466e-14 | 7.44504110666e-10 |
|   power_6   |  None |  1.154403034e-17   | 3.18599704653e-13 |
|   power_7   |  None | 1.05145869407e-21  | 8.70800820489e-17 |
|   power_8   |  None |  3.4604961643e-26  | 1.48821209458e-20 |
|   power_9   |  None | -1.09654453965e-30 | 1.44400106918e-24 |
|   power_10  |  None | -2.42031812032e-34 |        nan        |
|   power_11  |  None | -1.99601206862e-38 |        nan        |
|   power_12  |  None | -1.07709903826e-42 | 5.89920225456e-37 |
|   power_13  |  None | -2.72862818311e-47 | 4.82878052961e-41 |
|   power_14  |  None | 2.44782693215e-51  | 2.01784431957e-45 |
|   power_15  |  None | 5.01975232933e-55  | 3.72749025797e-50 |
+-------------+-------+--------------------+-------------------+
[16 rows x 4 columns]

In [231]:
fig = plt.figure()
fig.suptitle('SET 2')
plt.xlabel('15th Polynomial Function (sqft)')
plt.ylabel('Price ($)')

set2_poly15_data = polynomial_sframe(set_2['sqft_living'], 15)

my_features = set2_poly15_data.column_names() # get the name of the features

set2_poly15_data['price'] = set_2['price'] # add price to the data since it's the target

model_set2_15 = graphlab.linear_regression.create(set2_poly15_data, target = 'price', features = my_features, validation_set = None)

model_set2_15.get("coefficients").print_rows(num_rows=16)

plt.plot(set2_poly15_data['power_1'],set2_poly15_data['price'],'.',
        set2_poly15_data['power_1'], model_set2_15.predict(set2_poly15_data),'-')
Linear regression:
--------------------------------------------------------
Number of examples          : 5398
Number of features          : 15
Number of unpacked features : 15
Number of coefficients    : 16
Starting Newton Method
--------------------------------------------------------
+-----------+----------+--------------+--------------------+---------------+
| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |
+-----------+----------+--------------+--------------------+---------------+
| 1         | 2        | 0.011268     | 2069212.978548     | 234840.067186 |
+-----------+----------+--------------+--------------------+---------------+
SUCCESS: Optimal solution found.

+-------------+-------+--------------------+-------------------+
|     name    | index |       value        |       stderr      |
+-------------+-------+--------------------+-------------------+
| (intercept) |  None |   89836.5077404    |   1013516.72287   |
|   power_1   |  None |   319.806946747    |    4537.0695403   |
|   power_2   |  None |  -0.103315397028   |   6.76673277359   |
|   power_3   |  None | 1.06682476025e-05  |        nan        |
|   power_4   |  None | 5.75577097743e-09  |        nan        |
|   power_5   |  None | -2.54663464588e-13 |        nan        |
|   power_6   |  None | -1.09641345103e-16 |        nan        |
|   power_7   |  None | -6.36458441346e-21 |        nan        |
|   power_8   |  None | 5.52560416846e-25  |        nan        |
|   power_9   |  None |  1.3508203898e-28  |        nan        |
|   power_10  |  None | 1.18408188222e-32  |        nan        |
|   power_11  |  None | 1.98348000648e-37  |        nan        |
|   power_12  |  None | -9.92533590279e-41 |        nan        |
|   power_13  |  None | -1.60834847057e-44 |        nan        |
|   power_14  |  None | -9.12006024172e-49 |        nan        |
|   power_15  |  None | 1.68636658318e-52  | 1.90480211954e-49 |
+-------------+-------+--------------------+-------------------+
[16 rows x 4 columns]

Out[231]:
[<matplotlib.lines.Line2D at 0x7f8be918bb90>,
 <matplotlib.lines.Line2D at 0x7f8be900bb50>]
In [232]:
fig = plt.figure()
fig.suptitle('SET 3')
plt.xlabel('15th Polynomial Function (sqft)')
plt.ylabel('Price ($)')

set3_poly15_data = polynomial_sframe(set_3['sqft_living'], 15)

my_features = set3_poly15_data.column_names() # get the name of the features

set3_poly15_data['price'] = set_3['price'] # add price to the data since it's the target

model_set3_15 = graphlab.linear_regression.create(set3_poly15_data, target = 'price', features = my_features, validation_set = None)

model_set3_15.get("coefficients").print_rows(num_rows=16)

plt.plot(set3_poly15_data['power_1'],set3_poly15_data['price'],'.',
        set3_poly15_data['power_1'], model_set3_15.predict(set3_poly15_data),'-')
Linear regression:
--------------------------------------------------------
Number of examples          : 5409
Number of features          : 15
Number of unpacked features : 15
Number of coefficients    : 16
Starting Newton Method
--------------------------------------------------------
+-----------+----------+--------------+--------------------+---------------+
| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |
+-----------+----------+--------------+--------------------+---------------+
| 1         | 2        | 0.018740     | 2269769.506520     | 251460.072754 |
+-----------+----------+--------------+--------------------+---------------+
SUCCESS: Optimal solution found.

+-------------+-------+--------------------+-------------------+
|     name    | index |       value        |       stderr      |
+-------------+-------+--------------------+-------------------+
| (intercept) |  None |   87317.9795578    |   2179777.37575   |
|   power_1   |  None |   356.304911039    |    11498.265568   |
|   power_2   |  None |  -0.164817442804   |   25.2975478512   |
|   power_3   |  None | 4.40424992681e-05  |  0.0306824961494  |
|   power_4   |  None | 6.48234876532e-10  | 2.27897581962e-05 |
|   power_5   |  None | -6.75253226667e-13 | 1.08093295853e-08 |
|   power_6   |  None | -3.36842592529e-17 | 3.24366337913e-12 |
|   power_7   |  None |  3.6099970419e-21  | 5.30543475078e-16 |
|   power_8   |  None | 6.46999725571e-25  |        nan        |
|   power_9   |  None |  4.2363938878e-29  |        nan        |
|   power_10  |  None | -3.62149425413e-34 |        nan        |
|   power_11  |  None | -4.27119527348e-37 |        nan        |
|   power_12  |  None | -5.61445971603e-41 |        nan        |
|   power_13  |  None | -3.87452772885e-45 |        nan        |
|   power_14  |  None | 4.69430358462e-50  |        nan        |
|   power_15  |  None | 6.39045886044e-53  |  3.2110710848e-49 |
+-------------+-------+--------------------+-------------------+
[16 rows x 4 columns]

Out[232]:
[<matplotlib.lines.Line2D at 0x7f8be917bd10>,
 <matplotlib.lines.Line2D at 0x7f8be8f4a650>]
In [233]:
fig = plt.figure()
fig.suptitle('SET 4')
plt.xlabel('15th Polynomial Function (sqft)')
plt.ylabel('Price ($)')

set4_poly15_data = polynomial_sframe(set_4['sqft_living'], 15)

my_features = set4_poly15_data.column_names() # get the name of the features

set4_poly15_data['price'] = set_4['price'] # add price to the data since it's the target

model_set4_15 = graphlab.linear_regression.create(set4_poly15_data, target = 'price', features = my_features, validation_set = None)

model_set4_15.get("coefficients").print_rows(num_rows=16)

plt.plot(set4_poly15_data['power_1'],set4_poly15_data['price'],'.',
        set4_poly15_data['power_1'], model_set4_15.predict(set4_poly15_data),'-')
Linear regression:
--------------------------------------------------------
Number of examples          : 5402
Number of features          : 15
Number of unpacked features : 15
Number of coefficients    : 16
Starting Newton Method
--------------------------------------------------------
+-----------+----------+--------------+--------------------+---------------+
| Iteration | Passes   | Elapsed Time | Training-max_error | Training-rmse |
+-----------+----------+--------------+--------------------+---------------+
| 1         | 2        | 0.024827     | 2314893.173831     | 244563.136754 |
+-----------+----------+--------------+--------------------+---------------+
SUCCESS: Optimal solution found.

+-------------+-------+--------------------+-------------------+
|     name    | index |       value        |       stderr      |
+-------------+-------+--------------------+-------------------+
| (intercept) |  None |   259020.879448    |   1387621.91195   |
|   power_1   |  None |   -31.7277161897   |   8565.53943718   |
|   power_2   |  None |   0.109702769602   |   21.5864074204   |
|   power_3   |  None | -1.58383847282e-05 |  0.0288007775087  |
|   power_4   |  None | -4.47660623765e-09 |  2.1200402023e-05 |
|   power_5   |  None | 1.13976573437e-12  | 4.18640964244e-09 |
|   power_6   |  None | 1.97669120598e-16  |        nan        |
|   power_7   |  None | -6.15783677844e-21 |        nan        |
|   power_8   |  None | -4.88012304188e-24 |        nan        |
|   power_9   |  None | -6.6218678137e-28  |        nan        |
|   power_10  |  None | -2.70631583288e-32 |        nan        |
|   power_11  |  None | 6.72370411518e-36  | 9.98061269507e-31 |
|   power_12  |  None | 1.74115646316e-39  |        nan        |
|   power_13  |  None | 2.09188375694e-43  |        nan        |
|   power_14  |  None | 4.78015565998e-48  |        nan        |
|   power_15  |  None |  -4.745353331e-51  |        nan        |
+-------------+-------+--------------------+-------------------+
[16 rows x 4 columns]

Out[233]:
[<matplotlib.lines.Line2D at 0x7f8be90ba850>,
 <matplotlib.lines.Line2D at 0x7f8be9794950>]

Some questions you will be asked on your quiz:

Quiz Question: Is the sign (positive or negative) for power_15 the same in all four models?

Quiz Question: (True/False) the plotted fitted lines look the same in all four plots

Selecting a Polynomial Degree

Whenever we have a "magic" parameter like the degree of the polynomial there is one well-known way to select these parameters: validation set. (We will explore another approach in week 4).

We split the sales dataset 3-way into training set, test set, and validation set as follows:

  • Split our sales data into 2 sets: training_and_validation and testing. Use random_split(0.9, seed=1).
  • Further split our training data into two sets: training and validation. Use random_split(0.5, seed=1).

Again, we set seed=1 to obtain consistent results for different users.

In [234]:
training_and_validation, testing = sales.random_split(.9,seed=1)
training, validation = training_and_validation.random_split(0.5, seed=1)
print 'sales data: ' + str(len(sales))
print 'training_and_validation data: ' + str(len(training_and_validation))
print 'testing data: ' + str(len(testing))
print 'training data: '+ str(len(training))
print 'validation data: ' + str(len(validation))
sales data: 21613
training_and_validation data: 19396
testing data: 2217
training data: 9761
validation data: 9635

Next you should write a loop that does the following:

  • For degree in [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] (to get this in python type range(1, 15+1))
    • Build an SFrame of polynomial data of train_data['sqft_living'] at the current degree
    • hint: my_features = poly_data.column_names() gives you a list e.g. ['power_1', 'power_2', 'power_3'] which you might find useful for graphlab.linear_regression.create( features = my_features)
    • Add train_data['price'] to the polynomial SFrame
    • Learn a polynomial regression model to sqft vs price with that degree on TRAIN data
    • Compute the RSS on VALIDATION data (here you will want to use .predict()) for that degree and you will need to make a polynmial SFrame using validation data.
  • Report which degree had the lowest RSS on validation data (remember python indexes from 0)

(Note you can turn off the print out of linear_regression.create() with verbose = False)

In [236]:
def get_residual_sum_of_squares(model, data, outcome):
    # First get the predictions
    predictions = model.predict(data)

    # Then compute the residuals/errors
    residuals = predictions - outcome

    # Then square and add them up
    
    RSS = (residuals * residuals).sum()

    return(RSS)    
In [251]:
rssDic = {}
modelDic = {}
for i in range(1,15+1):
    poly_train_data  = polynomial_sframe(training['sqft_living'], i)
    poly_features = poly_train_data.column_names()
    poly_train_data['price'] = training['price']
    modelDic[i] = graphlab.linear_regression.create(poly_train_data, target='price', features=poly_features, validation_set=None, verbose=False)

    poly_valid_data = polynomial_sframe(validation['sqft_living'], i)
    poly_valid_data['price'] = validation['price']
    predictions = modelDic[i].predict(poly_valid_data)
    rssDic[i] = get_residual_sum_of_squares(modelDic[i], poly_valid_data, validation['price'])
    
minRSS =min(rssDic.values())
minKey = rssDic.keys()[rssDic.values().index(minRSS)]
print 'model degree with least RSS: ' + str(minKey)
print 'least RSS: ' + str(rssDic[minKey])
model degree with least RSS: 6
least RSS: 5.8918247781e+14

Quiz Question: Which degree (1, 2, …, 15) had the lowest RSS on Validation data?

Model 6

price = intercept + w1(stft_living) + w2(sqft_living)*2 + w3(sqft_living)*3 + w4(sqft_living)*4 + w5(sqft_living)*5 + w6(sqft_living)*6

Now that you have chosen the degree of your polynomial using validation data, compute the RSS of this model on TEST data. Report the RSS on your quiz.

In [249]:
# first way of calculating RSS is using matrix dot product as learned in earlier lessons
poly_testing_data = polynomial_sframe(testing['sqft_living'], minKey)
poly_testing_data['price'] = testing['price']
predictions = modelDic[minKey].predict(poly_testing_data)
print get_residual_sum_of_squares(modelDic[minKey], poly_testing_data, poly_testing_data['price'])

# second way is to just ask graphlab model for rmse and then RSS = sq(rmse)*N
print modelDic[minKey].evaluate(poly_testing_data)['rmse']**2 * len(poly_testing_data)
1.25529337848e+14
1.25529337848e+14

Quiz Question: what is the RSS on TEST data for the model with the degree selected from Validation data?

1.25529337848e+14

In [ ]: