Appendix Regression Analysis

This appendix describes estimation of the regression equation, several commonly used regression statistics, and how to choose among cost functions that have been estimated by regression analysis. We use the data for Elegant Rugs presented in Exhibit 10-3 and displayed here again for easy reference.

Estimating the Regression Line

The least-squares technique for estimating the regression line minimizes the sum of the squares of the vertical deviations from the data points to the estimated regression line (also called residual term in Exhibit 10-6). The objective is to find the values of a and b in the linear cost function $y=a+bX\mathrm{,}$  where y is the predicted cost value as distinguished from the observed cost value, which we denote by Y. We wish to find the numerical values of a and b that minimize $\mathrm{\Sigma }(Y-y{)}^2\mathrm{,}$ the sum of the squares of the vertical deviations between Y and y. Generally, these computations are done using software packages such as R. For the data in our example,^*

Goodness of Fit

Goodness of fit measures how well the predicted values, y, based on the cost driver, X, match actual cost observations, Y. The regression analysis method computes a measure of goodness of fit, called the coefficient of determination (r²). The coefficient of determination measures the percentage of variation in Y explained by X (the independent variable). It is more convenient to express the coefficient of determination as 1 minus the proportion of total variance that is not explained by the independent variable—that is, 1 minus the ratio of unexplained variation to total variation. The unexplained variance arises because of differences between the actual values, Y, and the predicted values, y. In the Elegant Rugs example, goodness of fit is given by^*

The calculations indicate that $r^2$ increases as the predicted values, y, more closely approximate the actual observations, Y. The range of $r^2$ is from 0 (implying no explanatory power) to 1 (implying perfect explanatory power). Generally, an $r^2$ of 0.30 or higher passes the goodness-of-fit test. However, do not rely exclusively on goodness of fit. It can lead to the indiscriminate inclusion of independent variables that increase $r^2$ but have no economic plausibility as cost drivers. Goodness of fit has meaning only if the relationship between the cost drivers and costs is economically plausible.

An alternative and related way to evaluate goodness of fit is to calculate the standard error of the regression. The standard error of the regression is the standard deviation of the residuals. It is equal to

Degrees of freedom equal the number of observations, 12, minus the number of coefficients estimated in the regression (in this case two, a and b). The standard error of $170.54 is an estimate of the variation of the observed indirect manufacturing labor costs about the regression line. It is in the same unit of measurement (dollars) as indirect manufacturing labor costs, the dependent variable. For comparison, note that $\overline{Y}\mathrm{,}$ the average value of Y, is $1,041.75. The smaller the standard error of the regression, the better the fit and the better the predictions for different values of X.

Significance of Independent Variables

Exhibit 10-15 shows a convenient format for summarizing the regression results for number of machine-hours and indirect manufacturing labor costs. Do changes in the economically plausible independent variable result in significant changes in the dependent variable? Or, alternatively stated, is the slope coefficient, $b=\mathrm{\$}10.31\mathrm{,}$ of the regression line statistically significant (that is, different from $0)? Recall that in the regression of number of machine-hours and indirect manufacturing labor costs in the Elegant Rugs’ illustration, b is estimated from a sample of 12 weekly observations. The estimate, b, is subject to random factors, as are all sample statistics. That is, a different sample of 12 data points would give a different estimate of b. The standard error of the estimated coefficient indicates how much the estimated value, b, is likely to be affected by random factors.

The t-value of a coefficient measures how large the value of the estimated coefficient is relative to its standard error. The t-value (also called t Stat) for the slope coefficient b is the value of the estimated coefficient, $\mathrm{\$}\text{10.31}\hspace{0.17em}÷$ the standard error of the estimated coefficient, $\mathrm{\$}\text{3.12}=\text{3.30}.$ This is compared to a critical or cutoff value to ensure that a relationship exists between the independent variable and the dependent variable that cannot be attributed to random chance alone. The cutoff t-value for making inferences is a function of the number of degrees of freedom and the significance level. It is typical to look for a 5% level of significance, which indicates that there is less than a 5% probability that random factors could have affected the coefficient b. The cutoff t-value at the 5% significance level and 10 degrees of freedom is 2.228. Because the t-value for the slope coefficient b is 3.30, which exceeds 2.228, we can conclude that there is a statistically significant relationship between machine-hours and indirect manufacturing labor costs.^*

An alternative way to test that the coefficient b is significantly different from zero is in terms of a confidence interval: There is less than a 5% chance that the true value of the machine-hours coefficient lies outside the range $\mathrm{\$}10.31\pm (2.228\times \mathrm{\$}3.12)\mathrm{,}$ or $\$\text{10.31}\pm \$\text{6.95},$ or from $3.36 to $17.26. Because zero does not appear in the confidence interval, we can conclude that changes in the number of machine-hours do affect indirect manufacturing labor costs. Similarly, using data from Exhibit 10-15, the t-value for the constant term a is $\mathrm{\$}300.98÷229.75=1.31\mathrm{,}$ which is less than 2.228. This t-value indicates that, within the relevant range, the constant term is not significantly different from zero. The Durbin-Watson statistic in Exhibit 10-15 will be discussed in the following section.

Specification Analysis of Estimation Assumptions

Specification analysis is the testing of the assumptions of regression analysis. If the assumptions of (1) linearity within the relevant range, (2) constant variance of residuals, (3) independence of residuals, and (4) normality of residuals all hold, then the simple regression procedures give reliable estimates of coefficient values. This section provides a brief overview of specification analysis. When these assumptions are not satisfied, more-complex regression procedures are necessary to obtain the best estimates.^*

1. LINEARITY WITHIN THE RELEVANT RANGE. A common assumption—and one that appears to be reasonable in many business applications—is that a linear relationship exists between the independent variable X and the dependent variable Y within the relevant range. If a linear relationship is assumed when the relationship is nonlinear, the coefficient estimates obtained will be inaccurate.^*

1. When there is only one independent variable, the easiest way to check for linearity is to study the data plotted in a scatter diagram, a step that often is unwisely skipped. Exhibit 10-6 presents a scatter diagram for the indirect manufacturing labor costs and machine-hours variables of Elegant Rugs. The scatter diagram reveals that linearity appears to be a reasonable assumption for these data. 
2. The learning-curve models discussed in this chapter (see section Nonlinear Cost Functions) are examples of nonlinear cost functions. Costs increase when the level of production increases, but by lesser amounts than would occur with a linear cost function. In this case, the analyst should estimate a nonlinear cost function that incorporates learning effects.

2. CONSTANT VARIANCE OF RESIDUALS. The vertical deviation of the observed value Y from the regression line estimate y is called the residual term, disturbance term, or error term, $u=Y-y\mathrm{.}$ The assumption of constant variance implies that the residual terms are unaffected by the level of the cost driver. The assumption also implies that there is a uniform scatter, or dispersion, of the data points about the regression line as in Exhibit 10-16, Panel A. This assumption is likely to be violated, for example, in cross-sectional estimation of costs in operations of different sizes. For example, suppose Elegant Rugs has production areas of varying sizes. The company collects data from these different production areas to estimate the relationship between machine-hours and indirect manufacturing labor costs. It is possible that the residual terms in this regression will be larger for the larger production areas that have higher machine-hours and higher indirect manufacturing labor costs. There would not be a uniform scatter of data points about the regression line (see Exhibit 10-15, Panel B). Constant variance is also known as homoscedasticity. Violation of this assumption is called heteroscedasticity.

1. Heteroscedasticity does not affect the accuracy of the regression estimates a and b. It does, however, reduce the reliability of the estimates of the standard errors and thus affects the precision with which inferences about the population parameters can be drawn from the regression estimates.

3. INDEPENDENCE OF RESIDUALS. The assumption of independence of residuals is that the residual term for any one observation is not related to the residual term for any other observation. The problem of serial correlation (also called autocorrelation) in the residuals arises when there is a systematic pattern in the sequence of residuals such that the residual in observation n conveys information about the residuals in observations $n+1,\hspace{0.17em}n+2\mathrm{,}$ and so on. Consider another production cell at Elegant Rugs that has, over a 20-week period, seen an increase in production and hence machine-hours. Exhibit 10-17, Panel B, is a scatter diagram of machine-hours and indirect manufacturing labor costs. Observe the systematic pattern of the residuals in Panel B—positive residuals for extreme (high and low) quantities of machine-hours and negative residuals for moderate quantities of machine-hours. One reason for this observed pattern at low values of the cost driver is the “stickiness” of costs. When machine-hours are below 50 hours, indirect manufacturing labor costs do not decline. When machine-hours increase over time as production is ramped up, indirect manufacturing labor costs increase more as managers at Elegant Rugs struggle to manage the higher volume. How would the plot of residuals look if there were no auto-correlation? Like the plot in Exhibit 10-17, Panel A, that shows no pattern in the residuals.

1. As with nonconstant variance of residuals, serial correlation does not affect the accuracy of the regression estimates a and b. It does, however, affect the standard errors of the coefficients and, therefore, the precision with which inferences about the population parameters can be drawn from the regression estimates. 
2. The Durbin-Watson statistic is one measure of serial correlation in the estimated residuals. For samples of 10 to 20 observations, a Durbin-Watson statistic in the 1.10–2.90 range indicates that the residuals are independent. The Durbin-Watson statistic for the regression results of Elegant Rugs in Exhibit 10-15 is 2.05. Therefore, an assumption of independence in the estimated residuals is reasonable for this regression model.

4. NORMALITY OF RESIDUALS. The normality of residuals assumption means that the residuals are distributed normally around the regression line. The normality of residuals assumption is frequently satisfied when using regression analysis on real cost data. Even when the assumption does not hold, accountants can still generate accurate estimates based on the regression equation, but the resulting confidence interval around these estimates is likely to be inaccurate.

Using Regression Output to Choose Cost Drivers of Cost Functions

Consider the two choices of cost drivers we described earlier in this chapter for indirect manufacturing labor costs (y):

Exhibits 10-6 and 10-8 show plots of the data for the two regressions. Exhibit 10-15 reports regression results for the cost function using number of machine-hours as the independent variable. Exhibit 10-18 presents comparable regression results for the cost function using number of direct manufacturing labor-hours as the independent variable.

On the basis of the material presented in this appendix, which regression is better? Exhibit 10-19 compares these two cost functions in a systematic way. On the basis of several criteria, the cost function based on machine-hours is preferable to the cost function based on direct manufacturing labor-hours. The economic plausibility criterion is especially important.

Do not assume that any one cost function will perfectly satisfy all the criteria in Exhibit 10-19. A cost analyst must often make a choice among “imperfect” cost functions, in the sense that the data of any particular cost function will not perfectly meet one or more of the assumptions underlying regression analysis. For example, both of the cost functions in Exhibit 10-18 are imperfect because inferences drawn from only 12 observations are generally not reliable.

Multiple Regression and Cost Hierarchies

In some cases, a satisfactory estimation of a cost function may be based on only one independent variable, such as number of machine-hours. In many cases, however, basing the estimation on more than one independent variable (that is, multiple regression) is more economically plausible and improves accuracy. The most widely used equations to express relationships between two or more independent variables and a dependent variable are linear in the form

where,

Example: Consider the Elegant Rugs data in Exhibit 10-20. The company’s ABC analysis indicates that indirect manufacturing labor costs include large amounts incurred for setup and changeover costs when a new batch of carpets is started. Management believes that in addition to number of machine-hours (an output unit-level cost driver), indirect manufacturing labor costs are also affected by the number of batches of carpet produced during each week (a batch-level driver). Elegant Rugs estimates the relationship between two independent variables—number of machine-hours and number of production batches of carpet manufactured during the week—and indirect manufacturing labor costs.

Exhibit 10-21 presents results for the following multiple regression model, using data in columns B, C, and E of Exhibit 10-20:

where X₁ is the number of machine-hours and X₂ is the number of production batches. It is economically plausible that both number of machine-hours and number of production batches would help explain variations in indirect manufacturing labor costs at Elegant Rugs. The $r^2$ of 0.52 for the simple regression using number of machine-hours (Exhibit 10-15) increases to 0.72 with the multiple regression in Exhibit 10-21. The t-values suggest that the independent variable coefficients of both number of machine-hours ($7.60) and number of production batches ($37.77) are significantly different from zero ($t=2.74$ is the t-value for number of machine-hours, and $t=2.48$ is the t-value for number of production batches, compared to the cutoff t-value of 2.26). The multiple regression model in Exhibit 10-21 satisfies both economic plausibility and statistical criteria, and explains much greater variation (that is, $r^2$ of 0.72 versus $r^2$ of 0.52) in indirect manufacturing labor costs than the simple regression model using only number of machine-hours as the independent variable.^* The standard error of the regression equation that includes number of batches as an independent variable is

which is lower than the standard error of the regression with only machine-hours as the independent variable, $170.54. That is, even though adding a variable reduces the degrees of freedom in the denominator, it substantially improves fit so that the numerator, $\mathrm{\Sigma }\hspace{0.17em}{(Y-y)}^2\mathrm{,}$ decreases even more. Number of machine-hours and number of production batches are both important cost drivers of indirect manufacturing labor costs at Elegant Rugs.

In Exhibit 10-21, the slope coefficients—$7.60 for number of machine-hours and $37.77 for number of production batches—measure the change in indirect manufacturing labor costs associated with a unit change in an independent variable (assuming that the other independent variable is held constant). For example, indirect manufacturing labor costs increase by $37.77 when one more production batch is added, assuming that the number of machine-hours is held constant.

An alternative approach would create two separate cost pools for indirect manufacturing labor costs: one for costs related to number of machine-hours and another for costs related to number of production batches. Elegant Rugs would then estimate the relationship between the cost driver and the costs in each cost pool. The difficult task under this approach is to properly subdivide the indirect manufacturing labor costs into the two cost pools.

Multicollinearity

A major concern that arises with multiple regression is multicollinearity. Multicollinearity exists when two or more independent variables are highly correlated with each other. Generally, a coefficient of correlation between independent variables greater than 0.70 indicates multicollinearity. Multicollinearity increases the standard errors of the coefficients of the individual variables. That is, variables that are economically and statistically significant will appear not to be significantly different from zero.

The matrix of correlation coefficients of the different variables described in Exhibit 10-20 are as follows:

These results indicate that multiple regressions using any pair of the independent variables in Exhibit 10-20 are not likely to encounter multicollinearity problems.

When multicollinearity exists, try to obtain new data that do not suffer from multicollinearity problems. Do not drop an independent variable (cost driver) that should be included in a model because it is correlated with another independent variable. Omitting such a variable will cause the estimated coefficient of the independent variable included in the model to be biased away from its true value.