{"id":563,"date":"2015-07-19T13:25:36","date_gmt":"2015-07-19T17:25:36","guid":{"rendered":"http:\/\/sites.berry.edu\/vbissonnette\/?page_id=563"},"modified":"2016-06-23T10:25:52","modified_gmt":"2016-06-23T14:25:52","slug":"regression-solution","status":"publish","type":"page","link":"https:\/\/sites.berry.edu\/vbissonnette\/index\/stats-homework\/documentation\/regression\/regression-solution\/","title":{"rendered":"Regression Solution"},"content":{"rendered":"

Example homework problem:<\/h3>\n

You work for an automotive magazine, and you are investigating the relationship between a car’s gas mileage (in miles-per-gallon) and the amount of horsepower produced by a car’s engine. You collect the following data:<\/p>\n\n\n\n\n\n
Automobile:<\/td>\n1<\/td>\n2<\/td>\n3<\/td>\n4<\/td>\n5<\/td>\n6<\/td>\n7<\/td>\n8<\/td>\n9<\/td>\n10<\/td>\n<\/tr>\n
Horsepower:<\/td>\n95<\/td>\n135<\/td>\n120<\/td>\n167<\/td>\n210<\/td>\n146<\/td>\n245<\/td>\n110<\/td>\n160<\/td>\n130<\/td>\n<\/tr>\n
MPG:<\/td>\n37<\/td>\n19<\/td>\n26<\/td>\n20<\/td>\n24<\/td>\n30<\/td>\n15<\/td>\n32<\/td>\n23<\/td>\n33<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Compute the best-fitting regression line (Y’ = a + bx) for predicting a car’s gas mileage from its horsepower. Test to see if the slope of this regression line deviates significantly from zero (alpha = .05).<\/p>\n

\n

The goal of your analysis is to predict<\/u> the value of the outcome variable (Y: gas milage) given a value for the predictor variable (X: horsepower). To do this, we will compute the best-fitting regression line:<\/p>\n

\"reg1\"<\/a><\/p>\n

where the predicted value of Y is equal to the Y intercept (a<\/i>) plus the product of a given value of X and the slope of the regression line (b<\/i>).<\/p>\n

We will begin our computations by computing the Variance\/Covariance Matrix<\/u>:<\/p>\n\n\n\n\n\n
Variable:<\/td>\nHorsepower
\n(X)<\/td>\n		MPG
\n(Y)<\/td>\n<\/tr>\n
Horsepower (X):<\/td>\n2127.5111<\/td>\n-243.4667<\/td>\n<\/tr>\n
MPG (Y):<\/td>\n<\/td>\n48.9889<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

which presents the variance of X (2127.51), the variance of Y (48.99), and the covariance between X and Y (-243.47). The variable means have also been added. For help computing the variance and covariance, see the documentation for descriptive statistics<\/a> and the documentation for correlation<\/a>, respectively.<\/p>\n

The slope of the regression line is equal to:<\/p>\n

\"reg2\"<\/a><\/p>\n

The slope of the regression line (b<\/i>) is equal to the covariance between the two variables divided by the variance of the predictor variable (X).<\/p>\n

Next, find the value of the Y intercept:<\/p>\n

\"reg3\"<\/a><\/p>\n

If you would like help computing the means, then see the documentation for descriptive statistics<\/a>. Here is the completed regression equation:<\/p>\n

\"reg4\"<\/a><\/p>\n

Conduct hypothesis test<\/b>. We would like to know if the value of b<\/i> is significantly different from zero. There are several approaches. For example, we can use the t<\/i> test for the slope:<\/p>\n

\"reg5\"<\/a><\/p>\n

where b<\/i> is the slope of the regression line, S(b)<\/i> is the Standard Error <\/u>of the slope, and n<\/i> is the number of pairs of scores. The standard error of the slope is equal to:<\/p>\n

\"reg6\"<\/a><\/p>\n

The numerator of this formula refers to the Variance of the Estimate<\/u>. This is computed with this formula:<\/p>\n

\"reg7\"<\/a><\/p>\n

Now that you have computed the Variance of the Estimate, you can compute
\nthe Standard Error of the Slope:<\/p>\n

\"reg8\"<\/a><\/p>\n

And now that you have the standard error of the slope, you can compute
\nthe value of your t<\/i> test:<\/p>\n

\"reg9\"<\/a><\/p>\n

Recall that the df<\/i> for this t<\/i> test is equal to n – 2<\/i>, where n <\/i>is the number of pairs of scores. So, our df<\/i> is equal to 8.<\/p>\n

Alpha was set at .05 and we will conduct a two-tailed test. When you consult your table<\/a>
\n of critical values<\/a> for t<\/i>, you will find that if the absolute value of our obtained t<\/i> is greater than 2.306, then we would conclude that the relationship between horsepower and gas mileage is significant — i.e., that the slope of the regression line is significantly different from zero.<\/p>\n

Since the obtained t<\/i> (-3.25) is greater in absolute value than the critical t<\/i> (2.306), we would conclude that there is a significant negative relationship between the horsepower level of the automobile and its gas mileage — i.e., the greater the horsepower, the lower the gas mileage.<\/p>\n

The Standardized Regression Coefficient<\/b>. The problem with b<\/i> is that it will vary greatly in value depending on the scales of X and Y. It is not a useful descriptive statistic at all. So, we will typically convert b<\/i> to the Standardized Regression Coefficient<\/u> (\u03b2):<\/p>\n

\"reg10\"<\/a><\/p>\n

The standardized regression coefficient is equal to the slope of the regression line predicting Y from X after you have converted both variables to Standard Scores<\/u> (Z scores). When you have one predictor, \u03b2 = r<\/i>, the correlation coefficient. Think about it — the correlation coefficient is the slope of the regression line for predicting the Z of Y from the Z of X (in that case, the Y intercept will be 0). Don’t believe me? Convert both of your variables to Z scores and repeat your regression analysis — see how it turns out!<\/p>\n

Now that you have r<\/i>, you can easily compute r<\/i>\u00b2. r<\/i>\u00b2 = -.7541\u00b2 = .57. This is the Coefficient of Determination<\/u>; r<\/i>\u00b2 represents the proportion of variance in Y that can be accounted for by variation in X.<\/p>\n

The Analysis of Variance<\/b>. We will now cover another approach that you can use to see if the slope (b<\/i>) of your regression line is significantly different from zero — the Analysis of Variance (ANOVA).<\/p>\n

Our test statistic will be the value of the F<\/i> statistic:<\/p>\n

\"reg11\"<\/a><\/p>\n

The F<\/i> statistic is equal to the ratio of the Mean Square Due to Regression<\/i> divided by the Mean Square Error<\/i>. n is equal to the number of observations (i.e., the number of automobiles), and k is equal to the number of predictor variables (i.e., one: gas mileage).<\/p>\n

Compute the test statistic<\/b>. When we conduct an ANOVA, we typically construct a Source Table<\/i> that details the sources of your variance and the corresponding degrees
\nof freedom:<\/p>\n\n\n\n\n\n\n
Source<\/td>\nSS<\/td>\ndf<\/td>\nMS<\/td>\nF<\/td>\n<\/tr>\n
Regression:<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Error:<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
Total:<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

To facilitate your ANOVA, you should compute the following statistics: a) the sum of all of your outcome scores (\u03a3Y), the sum of all squared outcome scores (\u03a3Y\u00b2), and the\u00a0 number of observations (n). In our case, \u03a3Y=259, \u03a3Y\u00b2=7149, and n=10.<\/p>\n

Also, you should keep handy the value of r<\/i>\u00b2 that we computed above (r<\/i>\u00b2 = .5687).<\/p>\n

Now, you are ready to conduct an Analysis of Variance. The Sum of Squares<\/i> (SS<\/i>) Total is equal to:<\/p>\n

\"reg12\"<\/a><\/p>\n

The SS<\/i> due to regression is equal to:<\/p>\n

\"reg13\"<\/a><\/p>\n

The SS<\/i> error can be found by subtraction:<\/p>\n

\"reg14\"<\/a><\/p>\n

Add these values to your Source Table<\/i>. The df<\/i> Total is equal to the total number of observations minus one (n – 1). In our case this is: 10-1=9. The df<\/i> Regression is equal to the number of predictors (k). In our case this is: 1. The df<\/i> Error is equal to the number of observations minus two (n – 2). In our case this is: 10-2=8.<\/p>\n

Each Mean Square (MS<\/i>) is equal to the SS for that source divided by the df<\/i> for that source. In our case, MS<\/i> Regression = 250.76 \/ 1 = 250.76, and MS<\/i> Error = 190.15 \/ 8 = 23.77. Note that MS<\/i> Error is exactly<\/u> equal to the Variance of the Estimate that we computed earlier — 23.678. That’s what MS<\/i> Error means — error variance in our predictions.<\/p>\n

Now you are ready to compute your test statistic. F<\/i> is equal to:<\/p>\n

\"reg15\"<\/a><\/p>\n

Here is your completed Source Table<\/i>:<\/p>\n\n\n\n\n\n\n
Source<\/td>\nSS<\/td>\ndf<\/td>\nMS<\/td>\nF<\/td>\n<\/tr>\n
Regression:<\/td>\n250.755<\/td>\n1<\/td>\n250.755<\/td>\n10.55<\/td>\n<\/tr>\n
Error:<\/td>\n190.145<\/td>\n8<\/td>\n23.768<\/td>\n<\/td>\n<\/tr>\n
Total:<\/td>\n440.900<\/td>\n9<\/td>\n<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Conduct hypothesis test<\/b>. In our study, the F<\/i> statistic will have 1 and 8 df<\/i>, and alpha = .05. If you look up the critical value of F<\/i> in your table of critical values<\/a>, you will see that if your obtained F<\/i> is greater than 5.32, you would conclude that there is a significant relationship between X and Y (i.e., that a car’s horsepower is a significant predictor of the car’s gas mileage.<\/p>\n

Since the obtained F<\/i> (10.55) is greater than the critical value for F<\/i> (5.32), you would conclude that there was<\/i> a significant linear relationship between horsepower and gas mileage.<\/p>\n

Effect Size<\/b>. We already know that the correlation between horsepower and gas mileage (r<\/i>) is equal to -.75, and that r<\/i>\u00b2 = .57. And, you know that r<\/i>\u00b2 represents the proportion of variance in Y that can be accounted for by variance in X. So, we can say that knowing the horsepower of cars allows us to account for 57% of the variance in the cars’ gas mileage scores.<\/p>\n

However, the problem with r<\/i>\u00b2 is that it will systematically over-estimate the variance accounted for in your outcome variable; the smaller the sample size in your study, the more biased r<\/i>\u00b2 will be. So, we typically prefer to compute an unbiased estimate <\/u>of the variance accounted for. There is a simple solution — we will compute Adjusted R<\/i>\u00b2<\/u>:<\/p>\n

\"reg16\"<\/a><\/p>\n

The numerator of this formula is equal to the variance in your outcome variable (Y) minus the Variance of the Estimate (or MS<\/i> Error). Then, divide this difference by the variance of Y.<\/p>\n

Notice that Adjusted R<\/i>\u00b2 will always be a smaller value than was r<\/i>\u00b2. We would conclude that using the horsepower of cars as a predictor variable can account for 51% of the variance in cars’ gas mileage.<\/p>\n

\n

Return to Regression Procedure<\/a><\/p>\n

Return to Table of Contents<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Example homework problem: You work for an automotive magazine, and you are investigating the relationship between a car’s gas mileage (in miles-per-gallon) and the amount of horsepower produced by a […]<\/p>\n","protected":false},"author":34,"featured_media":0,"parent":558,"menu_order":0,"comment_status":"open","ping_status":"open","template":"","meta":{"site-container-style":"default","site-container-layout":"default","site-sidebar-layout":"default","site-transparent-header":"default","disable-article-header":"default","disable-site-header":"default","disable-site-footer":"default","disable-content-area-spacing":"default","footnotes":""},"class_list":["post-563","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/sites.berry.edu\/vbissonnette\/wp-json\/wp\/v2\/pages\/563","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sites.berry.edu\/vbissonnette\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/sites.berry.edu\/vbissonnette\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/sites.berry.edu\/vbissonnette\/wp-json\/wp\/v2\/users\/34"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.berry.edu\/vbissonnette\/wp-json\/wp\/v2\/comments?post=563"}],"version-history":[{"count":9,"href":"https:\/\/sites.berry.edu\/vbissonnette\/wp-json\/wp\/v2\/pages\/563\/revisions"}],"predecessor-version":[{"id":977,"href":"https:\/\/sites.berry.edu\/vbissonnette\/wp-json\/wp\/v2\/pages\/563\/revisions\/977"}],"up":[{"embeddable":true,"href":"https:\/\/sites.berry.edu\/vbissonnette\/wp-json\/wp\/v2\/pages\/558"}],"wp:attachment":[{"href":"https:\/\/sites.berry.edu\/vbissonnette\/wp-json\/wp\/v2\/media?parent=563"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}