Quantitative Analysis - Regression

Regression
Analysis

A tool for building mathematical and statistical
models that characterize relationships between a dependent variable and one or more independent variables, all in numerical form.

Regression

One dependent variable

and one independent

variable

SIMPLE LINEAR
REGRESSION

MULTIPLE
REGRESSION

Multiple independent

variables

Types of Regression

Simple Linear Regression

Finds a linear relationship between

-One dependent variable (Y)
-One Independent variable (X)

To predict or forecast Y, we must use this…

Simple Linear Regression

𝛽0 = Intercept
𝛽1 = Slope of Independent Variable
X = Value of Independent Variable
Y = Value of Dependent Variable

Simple Linear Regression Model

𝑌 = 𝛽0 + 𝛽1𝑋

Simple Linear Regression

Excel Example

Step 1: Go to Data

Step 2: Do to Data Analysis

Step 3: Go to Regression

Step 4: Select Y (Dependent)

Step 5: Select X (Independent)

Step 6: Check for Labels if selected

Step 7: Ok!

Simple Linear Regression

Excel Example

Regression

Correlation Coefficient.
Ranges from -1 to +1.

Adjusted R
Squared

Multiple R

Standard Error

Adjusts the R2 for sample size and
adds potential observations to make
it more reliable.

Variability between observed and
predicted Y values.

R Squared

Level of reliability of model that explains how much % of independent variable explains change in dependent variable.

Simple Linear Regression

Excel Example

Test for Significance of Regression:

H0= 0 (Is not significant)

H1≠ 0(is significant)

Rejecting H0 indicates significance.

Regression

ANOVA Test

P > 0.1

Insignificant

Null Hypothesis not rejected

0.05 > P > 0.1

Significant

Between 90% to 95% significant

0.01 > P > 0.05

Significant

Between 95% to 99% significant

0.01 > P

Significant

More than 99% significant

Regression

Check P-Values to Test for Significance of Variables

Simple Linear Regression

Excel Example

P-Value is 3.798 × 10−8

This can be considered
highly significant above
99%, as the p-value is
lower than 0.01

This means we can reject
the null hypothesis H0

Simple Linear Regression

Excel Example

𝑌 = 𝛽0 + 𝛽1 X

Let’s say the X value is
10…, what will Y be?

When using the formula

𝑌 = 𝛽0 + 𝛽1𝑋

We can help predict or forecast the value of Y using

the X such as…

For every % change in Y, the X would increase or

decrease...

Simple Linear Regression

Multiple Regression

𝛽0= Intercept
𝛽n = Slope of Independent Variables
Xn= Value of Independent Variables
Y= Value of Dependent Variable

Multiple Linear Regression Model

𝑌 = 𝛽0 + 𝛽1X1 + 𝛽2X2 + 𝛽3X3 + ⋯

Multiple Regression

​Excel Example

Simple! Just remove the variable and regress

again! Try different combinations…

Dropping a variable is the most common fix for

regression errors.

Insignificant P-Value?

Multiple Regression

The middle value when data is arranged from

least to greatest. (Half above, half below)

Excel Formula
=MEDIAN(data)

Median

Descriptive Statistics

Regression analysis requires numerical data;

This can be fixed by converting the categorical

data (Yes/No) into codes such as 0 and 1.

Categorical Variables

Multiple Regression

Multiple Regression

The regression model would be:

𝑌 = 𝛽0 + 𝛽1𝑋1 + 𝛽2𝑋2

Where;

Y = Salary
X1 = Age
X2 = MBA Indicator (0 or 1)

​Excel Example

Multiple Regression

If MBA = 0

𝑆𝑎𝑙𝑎𝑟𝑦 = 893.59 + 1044.15 × 𝐴𝑔𝑒 + (14767.23 ∗ 0)

​Excel Example

𝑆𝑎𝑙𝑎𝑟𝑦 = 893.59 + 1044.15 × 𝐴𝑔𝑒 + 14767.23 ∗ MBA

A short summary guide to regression

X and Y

Figure out your dependent
and independent
variable(s) and ensure you
know your X and Ys!

Check R2

You must check the R-
Squared to see the
reliability of the model!

Predict Y

Use the coefficients of
each significant variable,
including the Intercept as
shown in the formula to
find values of Y!

Regression

Use the regression function
in Excel and correctly input
your dependent and
independent variables

Check
P-values

Important to check for
significance by looking at
the p-values and noting
any variables that may be
insignificant (and dropping
them)

Different
Combos

When dealing with multiple
independent variables,
don’t be afraid to try out
different combinations by
dropping / adding
variables!

What to Look out for

Now lets play an
interesting
game!