# Deriving Ols Estimator
**Source:** https://hi.youtuber.hist.in/blog/deriving-ols-estimator
**Language:** Hindi

---

लेखक
:   - नाम
      :   टेल्स एज़िमुथ

      ट्विटर
      :   @ट्विटर

# suruaat

Parsing and display of math equations is included in this blog template. Parsing of math is enabled by `remark-math`और `rehype-katex`. KaTeX and its associated font is included in `_document.js` so feel free to use it on any page. 1

Inline math symbols can be included by enclosing the term between the `$` symbol.

Math code blocks are denoted by `$$`.

If you intend to use the `$` sign instead of math, you can escape it (`\$`), or specify the HTML entity (`$`) 2

Inline or manually enumerated footnotes are also supported. Click on the links above to see them in action.

# ओएलएस अनुमानक प्राप्त करना

Using matrix notation, let nnn denote the number of observations and kkk denote the number of regressors.

The vector of outcome variables Y\mathbf{Y}Y is a n×1n \times 1n×1 matrix,

```
\mathbf{Y} = \left[\begin{array}
  {c}
  y_1 \\
  . \\
  . \\
  . \\
  y_n
\end{array}\right]
```

Y=[y1...yn]\mathbf{Y} = \left[\begin{array} {c} y\_1 \\ . \\ . \\ . \\ y\_n \end{array}\right]Y=​y1​...yn​​​

The matrix of regressors X \mathbf{X}X  is a n×kn \times kn×k matrix (or each row is a k×1k \times 1k×1 vector),

```
\mathbf{X } = \left[\begin{array}
  {ccccc}
  x_{11} & . & . & . & x_{1k} \\
  . & . & . & . & .  \\
  . & . & . & . & .  \\
  . & . & . & . & .  \\
  x_{n1} & . & . & . & x_{nn}
\end{array}\right] =
\left[\begin{array}
  {c}
  \mathbf{x }'_1 \\
  . \\
  . \\
  . \\
  \mathbf{x }'_n
\end{array}\right]
```

X =[x 11...x 1k...............x n1...x nn]=[x 1′...x n′]\mathbf{X} = \left[\begin{array} {ccccc} x\_{11} & . & . & . & x\_{1k} \\ . & . & . & . & . \\ . & . & . & . & . \\ . & . & . & . & . \\ x\_{n1} & . & . & . & x\_{nn} \end{array}\right] = \left[\begin{array} {c} \mathbf{x}'\_1 \\ . \\ . \\ . \\ \mathbf{x}'\_n \end{array}\right]X =​x 11​...x n1​​.....​.....​.....​x 1k​...x nn​​​=​x 1′​...x n′​​​

The vector of error terms U\mathbf{U}U is also a n×1n \times 1n×1 matrix.

At times it might be easier to use vector notation. For consistency, I will use the bold small x to denote a vector and capital letters to denote a matrix. Single observations are denoted by the subscript.

## Least Squares

**Start**:  
yमैं =x मैं ′β+uमैं y\_i = \mathbf{x}'\_i \beta + u\_iyमैं ​=x मैं ′​β+uमैं ​

**Assumptions**:

1. Linearity (given above)
2. E(U∣X )=0E(\mathbf{U}|\mathbf{X}) = 0E(U∣X )=0 (conditional independence)
3. rank(X \mathbf{X}X ) = kkk (multi-collinearity i.e. full rank)
4. Var(U∣X )=σ2मैं nVar(\mathbf{U}|\mathbf{X}) = \sigma^2 I\_nVar(U∣X )=σ2मैं n​ (Homoskedascity)

**Aim**:  
Find β\betaβ that minimises the sum of squared errors:

Q=∑मैं =1nuमैं 2=∑मैं =1n(yमैं −x मैं ′β)2=(Y−X β)′(Y−X β)Q = \sum\_{i=1}^{n}{u\_i^2} = \sum\_{i=1}^{n}{(y\_i - \mathbf{x}'\_i\beta)^2} = (Y-X\beta)'(Y-X\beta)Q=मैं =1∑n​uमैं 2​=मैं =1∑n​(yमैं ​−x मैं ′​β)2=(Y−Xβ)′(Y−Xβ)

**Solution**:  
Hints: QQQ is a 1×11 \times 11×1 scalar, by symmetry ∂b′Ab∂b=2Ab\frac{\partial b'Ab}{\partial b} = 2Ab∂b∂b′Ab​=2Ab.

Take matrix derivative w.r.t β\betaβ:

```
\begin{aligned}
  \min Q           & = \min_{\beta} \mathbf{Y}'\mathbf{Y} - 2\beta'\mathbf{X }'\mathbf{Y} +
  \beta'\mathbf{X }'\mathbf{X }\beta \\
                   & = \min_{\beta} - 2\beta'\mathbf{X }'\mathbf{Y} + \beta'\mathbf{X }'\mathbf{X }\beta \\
  \text{[FOC]}~~~0 & =  - 2\mathbf{X }'\mathbf{Y} + 2\mathbf{X }'\mathbf{X }\hat{\beta}                  \\
  \hat{\beta}      & = (\mathbf{X }'\mathbf{X })^{-1}\mathbf{X }'\mathbf{Y}                              \\
                   & = (\sum^{n} \mathbf{x }_i \mathbf{x }'_i)^{-1} \sum^{n} \mathbf{x }_i y_i
\end{aligned}
```

min⁡Q=min⁡βY′Y−2β′X ′Y+β′X ′X β=min⁡β−2β′X ′Y+β′X ′X β[FOC]   0=−2X ′Y+2X ′X β^β^=(X ′X )−1X ′Y=(∑nx मैं x मैं ′)−1∑nx मैं yमैं \begin{aligned} \min Q & = \min\_{\beta} \mathbf{Y}'\mathbf{Y} - 2\beta'\mathbf{X}'\mathbf{Y} + \beta'\mathbf{X}'\mathbf{X}\beta \\ & = \min\_{\beta} - 2\beta'\mathbf{X}'\mathbf{Y} + \beta'\mathbf{X}'\mathbf{X}\beta \\ \text{[FOC]}~~~0 & = - 2\mathbf{X}'\mathbf{Y} + 2\mathbf{X}'\mathbf{X}\hat{\beta} \\ \hat{\beta} & = (\mathbf{X}'\mathbf{X})^{-1}\mathbf{X}'\mathbf{Y} \\ & = (\sum^{n} \mathbf{x}\_i \mathbf{x}'\_i)^{-1} \sum^{n} \mathbf{x}\_i y\_i \end{aligned}minQ[FOC]   0β^​​=βmin​Y′Y−2β′X ′Y+β′X ′X β=βmin​−2β′X ′Y+β′X ′X β=−2X ′Y+2X ′X β^​=(X ′X )−1X ′Y=(∑n​x मैं ​x मैं ′​)−1∑n​x मैं ​yमैं ​​

## फुटनोट्स

1. For the full list of supported TeX functions, check out the KaTeX documentation ↩
2. $10 and $20. ↩

ट्विटर पर चर्चा करें  • गिटहब पर देखें