What are dot products, anyway?

The dot product (or inner product, or scalar product) is ubiquitous throughout different areas of math, most importantly but not limited to linear algebra. It's an operation that takes in two vectors and returns a single number. Functionally, the dot product is calculated in two different ways:

$\bold{a} \cdot \bold{b} = \sum_{i=0}^n a_i \cdot b_i$

$\bold{a} \cdot \bold{b} = |\bold{a}||\bold{b}|\cos{\theta}$

Personally, I've always wondered - what does the dot product actually mean? Is there a visual representation for what the purpose of a dot product is?

To answer this question, we can start with our second formula,

$\bold{a} \cdot \bold{b} = |\bold{a}||\bold{b}|\cos{\theta}$

Looking at the diagram, the geometric meaning of the dot product by our second formula seems to become apparent. We can imagine dropping a perpendicular from the end of a to b:

We get a right triangle defined by the perpendicular, a, and b. We also know that the cosine of our angle θ is equal to the ratio of the magnitude of the vector project of a onto b (let's call it c) to the magnitude of a.

$\cos{\theta} = \frac{|\bold{c} |}{|\bold{a}|}$

Multiplying |a| and |b| onto both sides gives us

$|\bold{a}||\bold{b}|\cos{\theta} = |\bold{c}||\bold{b}|$

This is an interesting result; apparently, the dot product gives us the component of a along b multiplied with the length of b itself. Note that we can replace c with the actual definition of a component (or the magnitude of the projection of one vector onto another).

$\text{comp}_{\bold{b}} \bold{a} = \bold{a} \cdot \frac{\bold{b}}{|\bold{b}|} = |\bold{c}|$

Plugging this into our original expression,

$|\bold{a}||\bold{b}|\cos{\theta} = \bold{a} \cdot \frac{\bold{b}}{|\bold{b}|} = \bold{a} \cdot \bold{b}$

It works out cleanly in the end;

nothing's blown up, which is always good.

However, I'm more interested in our first expression: how does the sum of the componentwise products of our vectors relate in any way to the product of the magnitudes of the vectors and the cosine of the angle between them? What is the relationship between these two expressions?

To start, recall the law of cosines:

$|\bold{c}|^2 = |\bold{a}|^2 + |\bold{b}|^2 - 2|\bold{a}||\bold{b}|\cos{\theta}$

Notice how we can express c in terms of a and b: namely,

$\bold{c} = \bold{a} - \bold{b}$

Thus, we know the following about the magnitude of c:

$|\bold{c}| = |\bold{a} - \bold{b}| = \sqrt{\sum_{i=0}^n (a_i - b_i)^2}$

Thus, we can use the law of cosines:

$|\bold{a} - \bold{b}|^2 = |\bold{a}^2| + |\bold{b}|^2 - 2|\bold{a}||\bold{b}|\cos{\theta}$

We know that

$\bold{a} \cdot \bold{b} = |\bold{a}||\bold{b}|\cos{\theta}$

Therefore, the equation becomes

$|\bold{a} - \bold{b}|^2 = |\bold{a}|^2 + |\bold{b}|^2 - 2(\bold{a} \cdot \bold{b})$

$2(\bold{a} \cdot \bold{b}) = |\bold{a}|^2 + |\bold{b}|^2 - |\bold{a} - \bold{b}|^2$

$\bold{a} \cdot \bold{b} = \frac{1}{2} \left(\sum_{i = 0}^n a_i^2 + \sum_{i=0}^n b_i^2 - \sum_{i=0}^n (a_i - b_i)^2\right)$

We can see a pattern here. When

$(a_i - b_i)^2$

is multiplied out, we see a result of something along the lines of

$a_i^2 - 2a_ib_i + b_i^2$

This matches up with each extraneous term in our sum, which ends up canceling out:

$\bold{a} \cdot \bold{b} = \frac{1}{2} \left(\sum_{i = 0}^n a_i^2 + \sum_{i=0}^n b_i^2 - \sum_{i=0}^n (a_i^2 - 2a_ib_i + b_i^2) \right)$

$\bold{a} \cdot \bold{b} = \frac{1}{2} \left(-\sum_{i=0}^n (- 2a_ib_i) \right)$

$\bold{a} \cdot \bold{b} = \sum_{i=0}^n a_ib_i$

And we are done with our proof.