Flow Lab

Syllabus / Module 1: Linear Algebra, Reloaded

Lesson 1.1

Vectors, Linear Combinations & Span

Vectors as arrows and as lists of numbers; linear combinations; span, linear independence and basis — seen geometrically.

Written byClaude Opus

🌊
Why this matters for simulation: A velocity field assigns a vector to every grid cell; everything in a fluid solver is built from vector operations.

By the end of this lesson you can…

  • Move fluently between the arrow picture and the list-of-numbers picture of a vector.
  • Add vectors, scale them, and compute lengths and dot products by hand.
  • Write a vector as a linear combination of others by solving a small linear system.
  • Describe the span of a set of vectors geometrically (point, line, plane, all of space).
  • Test vectors for linear independence and explain what a basis and dimension are.
  • Explain how a fluid simulator stores a velocity field as a big array of vectors.
On this page
·
  1. Two personalities of a vector
  2. Adding and scaling
  3. Length and the dot product
  4. Linear combinations
  5. Worked example: finding coefficients
  6. Span
  7. Linear independence
  8. Worked example: three vectors in space
  9. Basis and dimension
  10. Vectors in code
  11. Fields: a vector at every point
  12. Key Takeaways

Every simulation you will ever write stores its state as numbers, and almost always those numbers come grouped as vectors: a position (x,y,z)(x, y, z), a velocity (u,v)(u, v), the temperatures at a thousand grid points, the six numbers that describe an aircraft’s orientation and spin. Your first linear algebra course probably treated vectors as columns to push through Gaussian elimination. In this module we rebuild the subject around the geometry of vectors, because that’s the picture that makes differential equations, stability, and fluid solvers make sense.

▶ Video Vectors, linear combinations and spancaptions only
Transcript
  1. Welcome to Linear Algebra, Reloaded. We start with the single most important object in simulation: the vector.
  2. A vector has two personalities. Geometrically, it is an arrow: it has a length and a direction, and it starts at the origin.
  3. Numerically, it is a list of numbers. This arrow is the list three, two: walk three units along x, then two units along y.
  4. Physicists love the arrow picture. Computers only understand the list. Being fluent in linear algebra means switching between them without thinking.
  5. To add two vectors, place the second arrow at the tip of the first. The sum points from the start of the first to the tip of the second.
  6. In the list picture, you just add matching entries: three plus negative one, and two plus two, gives two, four.
  7. Multiplying by a number, a scalar, stretches the arrow. Two times v is twice as long. A half shrinks it. A negative number flips it around.
  8. Now combine both operations. Scale v by a, scale w by b, and add. The result, a v plus b w, is called a linear combination of v and w.
  9. Turn the two dials, a and b, and the green arrow sweeps around. Which points can it reach?
  10. Every one of them. Every point in the plane is some combination of v and w. The set of all linear combinations is called the span, and here the span is the whole plane.
  11. But watch what happens if w points along the same line as v. Now every combination stays on that line. The span collapses from a plane to a line.
  12. When one vector is just a scaled copy of the other, it adds no new direction. We say the vectors are linearly dependent.
  13. Two independent vectors in the plane form a basis: every point can be reached, in exactly one way. The standard basis is i hat and j hat, the unit steps along each axis.
  14. So the list three, two secretly means three i hat plus two j hat. The numbers in a vector are always coefficients of a linear combination of basis vectors.
  15. Why does this matter for simulation? A velocity field puts one of these vectors at every point of space. A fluid solver stores two numbers per grid cell in 2D, three in 3D.
  16. Every operation a solver performs, advecting, diffusing, projecting, is built from the two moves you just learned: scaling vectors and adding them.
  17. Next time: what happens when we let a matrix move every vector at once.

Two personalities of a vector

A vector has two descriptions, and you need both.

The arrow. Geometrically, a vector is an arrow with a length and a direction. In linear algebra we almost always draw it starting at the origin. Physicists like this picture: a velocity has a speed (length) and a heading (direction), and a force pushes some amount in some direction.

The list. Numerically, a vector is an ordered list of numbers, its components:

v⃗=[32]∈R2,u⃗=[1−42.5]∈R3.\vec v = \begin{bmatrix} 3 \\ 2 \end{bmatrix} \in \mathbb{R}^2, \qquad \vec u = \begin{bmatrix} 1 \\ -4 \\ 2.5 \end{bmatrix} \in \mathbb{R}^3 .

The list is a set of walking instructions: go 3 units along xx, then 2 along yy. The symbol Rn\mathbb{R}^n means “the set of all lists of nn real numbers”. The plane is R2\mathbb{R}^2 and space is R3\mathbb{R}^3. The state of a 100×100 temperature grid is a single vector in R10000\mathbb{R}^{10000}.

We’ll write vectors as columns (or as (3,2)(3, 2) in running text) and use v⃗\vec v, v\mathbf v or just vv depending on context.

Adding and scaling

Only two operations are needed to build all of linear algebra.

Addition. Add matching components. Geometrically, put the tail of w⃗\vec w at the tip of v⃗\vec v (tip-to-tail); the sum goes from the start of v⃗\vec v to the end of w⃗\vec w:

[32]+[−12]=[24].\begin{bmatrix} 3 \\ 2 \end{bmatrix} + \begin{bmatrix} -1 \\ 2 \end{bmatrix} = \begin{bmatrix} 2 \\ 4 \end{bmatrix}.

If you walk along v⃗\vec v and then along w⃗\vec w, you end up at v⃗+w⃗\vec v + \vec w. Order doesn’t matter: v⃗+w⃗=w⃗+v⃗\vec v + \vec w = \vec w + \vec v (the two routes form a parallelogram).

Scalar multiplication. Multiply every component by a number cc (a scalar):

2[32]=[64],−1[32]=[−3−2].2\begin{bmatrix} 3 \\ 2 \end{bmatrix} = \begin{bmatrix} 6 \\ 4 \end{bmatrix}, \qquad -1\begin{bmatrix} 3 \\ 2 \end{bmatrix} = \begin{bmatrix} -3 \\ -2 \end{bmatrix}.

Geometrically, cv⃗c\vec v stretches the arrow by a factor ∣c∣|c|, and flips it if c<0c \lt 0. Subtraction is addition of −1-1 times a vector: v⃗−w⃗\vec v - \vec w is the arrow from the tip of w⃗\vec w to the tip of v⃗\vec v.

Length and the dot product

Two more tools will show up constantly. The length (or norm) of a vector comes from Pythagoras:

∥v⃗∥=v12+v22+⋯+vn2,∥[34]∥=5.\|\vec v\| = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2}, \qquad \left\| \begin{bmatrix} 3 \\ 4 \end{bmatrix} \right\| = 5 .

The dot product multiplies matching components and adds:

v⃗⋅w⃗=v1w1+v2w2+⋯+vnwn.\vec v \cdot \vec w = v_1 w_1 + v_2 w_2 + \cdots + v_n w_n .

Geometrically, v⃗⋅w⃗=∥v⃗∥ ∥w⃗∥cos⁡θ\vec v \cdot \vec w = \|\vec v\|\,\|\vec w\| \cos\theta, where θ\theta is the angle between them. So:

  • v⃗⋅w⃗=0\vec v \cdot \vec w = 0 exactly when the vectors are perpendicular (orthogonal);
  • v⃗⋅v⃗=∥v⃗∥2\vec v \cdot \vec v = \|\vec v\|^2;
  • v⃗⋅w⃗∥w⃗∥\dfrac{\vec v \cdot \vec w}{\|\vec w\|} is the signed length of the “shadow” of v⃗\vec v on the line through w⃗\vec w (the projection).
✏️ Pen & paperLengths, angles and projections●○○

Let v⃗=(3,2)\vec v = (3, 2) and w⃗=(−1,2)\vec w = (-1, 2).

  1. Compute ∥v⃗∥\|\vec v\|, ∥w⃗∥\|\vec w\| and v⃗⋅w⃗\vec v \cdot \vec w.
  2. Find the angle between them.
  3. Find the component of v⃗\vec v along the unit vector w^=w⃗/∥w⃗∥\hat w = \vec w / \|\vec w\|.
Show worked solution
  1. ∥v⃗∥=9+4=13≈3.606\|\vec v\| = \sqrt{9+4} = \sqrt{13} \approx 3.606, ∥w⃗∥=1+4=5≈2.236\|\vec w\| = \sqrt{1+4} = \sqrt5 \approx 2.236, and v⃗⋅w⃗=(3)(−1)+(2)(2)=1\vec v\cdot\vec w = (3)(-1) + (2)(2) = 1.
  2. cos⁡θ=1135=165≈0.124\cos\theta = \dfrac{1}{\sqrt{13}\sqrt5} = \dfrac{1}{\sqrt{65}} \approx 0.124, so θ≈82.9∘\theta \approx 82.9^\circ: nearly perpendicular, which matches the picture in the video.
  3. v⃗⋅w^=v⃗⋅w⃗∥w⃗∥=15≈0.447\vec v \cdot \hat w = \dfrac{\vec v\cdot \vec w}{\|\vec w\|} = \dfrac{1}{\sqrt5} \approx 0.447. Only a small part of v⃗\vec v points along w⃗\vec w.

Linear combinations

Put the two operations together. Given vectors v⃗1,…,v⃗k\vec v_1, \dots, \vec v_k and scalars c1,…,ckc_1, \dots, c_k, the vector

c1v⃗1+c2v⃗2+⋯+ckv⃗kc_1 \vec v_1 + c_2 \vec v_2 + \cdots + c_k \vec v_k

is a linear combination of the v⃗i\vec v_i. The scalars are its coefficients or weights. “Linear” means we only scale and add: no squaring, no multiplying vectors together.

The basic question of this whole module is: which vectors can you build as linear combinations of a given set?

Worked example: finding coefficients

Can b⃗=(4,7)\vec b = (4, 7) be written as a combination of v⃗=(1,2)\vec v = (1, 2) and w⃗=(1,1)\vec w = (1, 1)?

We need numbers a,ba, b with av⃗+bw⃗=b⃗a\vec v + b\vec w = \vec b:

a[12]+b[11]=[47]⟺{a+b=42a+b=7a\begin{bmatrix} 1 \\ 2 \end{bmatrix} + b\begin{bmatrix} 1 \\ 1 \end{bmatrix} = \begin{bmatrix} 4 \\ 7 \end{bmatrix} \quad\Longleftrightarrow\quad \begin{cases} a + b = 4 \\ 2a + b = 7 \end{cases}

Subtracting the first equation from the second gives a=3a = 3, then b=1b = 1. Check: 3(1,2)+1(1,1)=(4,7)3(1,2) + 1(1,1) = (4, 7). ✓

Look at what happened. Asking “is b⃗\vec b a combination of these vectors?” is the same as asking “does this linear system have a solution?” The columns of the system are the vectors themselves. This is what Gaussian elimination was doing all along, and the idea comes back as Ax⃗=b⃗A\vec x = \vec b in 1.3 Linear Systems, Kernel & Column Space.

◆ Interactive Span explorer: build a·v + b·w

Loading simulation…

Try these in the explorer:

  1. Use the sliders to land the green arrow on a few grid intersections. The violet grid is the set of combinations with whole-number a,ba, b. It’s just a skewed version of the ordinary coordinate grid.
  2. Hit Random challenge and find the coefficients that reach the target. Then hit it again and solve for a,ba, b on paper first.
  3. Click Make w depend on v, or drag w⃗\vec w until it lies along v⃗\vec v. Watch the grid collapse. No choice of a,ba, b can leave the green line.

Span

The span of a set of vectors is the set of all their linear combinations:

span⁡{v⃗1,…,v⃗k}={ c1v⃗1+⋯+ckv⃗k  :  c1,…,ck∈R }.\operatorname{span}\{\vec v_1, \dots, \vec v_k\} = \{\, c_1\vec v_1 + \cdots + c_k \vec v_k \;:\; c_1,\dots,c_k \in \mathbb{R} \,\}.

In the plane and in space, spans are easy to picture:

VectorsTheir span
{0⃗}\{\vec 0\}just the origin
one nonzero vector v⃗\vec vthe line through the origin along v⃗\vec v
two vectors in R2\mathbb{R}^2 pointing in different directionsthe whole plane R2\mathbb{R}^2
two vectors in R2\mathbb{R}^2 along the same lineonly that line
two non-parallel vectors in R3\mathbb{R}^3a plane through the origin
three vectors in R3\mathbb{R}^3 not all in one planeall of R3\mathbb{R}^3

Spans always pass through the origin (take all coefficients zero), and they are always “flat”: points, lines, planes, and their higher-dimensional cousins called subspaces.

✅ Concept check

The vectors $(2, -1)$ and $(-6, 3)$ span which set?

✅ Concept check

Two non-parallel vectors in $\mathbb{R}^3$ span…

Linear independence

In the explorer, when w⃗\vec w fell onto the line of v⃗\vec v it became redundant: it added no new direction. The precise version of “no redundancy” is:

Why does this capture redundancy? If c1v⃗1+⋯+ckv⃗k=0⃗c_1\vec v_1 + \cdots + c_k\vec v_k = \vec 0 with, say, ck≠0c_k \neq 0, solve for v⃗k\vec v_k: v⃗k=−1ck(c1v⃗1+⋯+ck−1v⃗k−1)\vec v_k = -\tfrac{1}{c_k}(c_1\vec v_1 + \cdots + c_{k-1}\vec v_{k-1}). So v⃗k\vec v_k already lives in the span of the others, and dropping it doesn’t shrink the span.

Quick tests.

  • Two vectors are dependent exactly when one is a multiple of the other (they’re parallel).
  • In R2\mathbb{R}^2, (v1,v2)(v_1, v_2) and (w1,w2)(w_1, w_2) are dependent exactly when v1w2−v2w1=0v_1 w_2 - v_2 w_1 = 0. You’ll meet this number again as the determinant in 1.4 Determinants as Volume Scaling.
  • More than nn vectors in Rn\mathbb{R}^n are always dependent. Three arrows in the plane can’t all point in genuinely new directions.

Worked example: three vectors in space

Are a⃗=(1,0,1)\vec a = (1, 0, 1), b⃗=(0,1,1)\vec b = (0, 1, 1), c⃗=(1,1,2)\vec c = (1, 1, 2) independent?

Look for xa⃗+yb⃗+zc⃗=0⃗x\vec a + y\vec b + z\vec c = \vec 0:

{x+z=0y+z=0x+y+2z=0\begin{cases} x + z = 0 \\ y + z = 0 \\ x + y + 2z = 0 \end{cases}

From the first two, x=−zx = -z and y=−zy = -z. Substituting into the third gives −z−z+2z=0-z - z + 2z = 0, which holds for every zz. Take z=1z = 1: −a⃗−b⃗+c⃗=0⃗-\vec a - \vec b + \vec c = \vec 0, i.e. c⃗=a⃗+b⃗\vec c = \vec a + \vec b. Dependent. All three lie in the plane spanned by a⃗\vec a and b⃗\vec b.

✏️ Pen & paperIndependent or not?●●○

Decide whether each set is linearly independent. If it’s dependent, write one vector as a combination of the others.

  1. (1,2)(1, 2), (3,5)(3, 5)
  2. (1,2,3)(1, 2, 3), (2,4,6)(2, 4, 6), (0,1,0)(0, 1, 0)
  3. (1,0,0)(1, 0, 0), (1,1,0)(1, 1, 0), (1,1,1)(1, 1, 1)
  4. (1,1)(1, 1), (2,−1)(2, -1), (0,3)(0, 3)
Show worked solution
  1. 1⋅5−2⋅3=−1≠01\cdot 5 - 2 \cdot 3 = -1 \neq 0, so they’re independent.
  2. (2,4,6)=2(1,2,3)(2,4,6) = 2(1,2,3), so they’re dependent. Any set containing two parallel vectors is dependent.
  3. Solve x(1,0,0)+y(1,1,0)+z(1,1,1)=0x(1,0,0) + y(1,1,0) + z(1,1,1) = 0. The third component forces z=0z = 0, the second then forces y=0y = 0, and the first forces x=0x = 0. Only the trivial solution exists, so they’re independent. (This triangular structure is why back-substitution works.)
  4. That’s three vectors in R2\mathbb{R}^2, so they’re dependent automatically. To find the relation, solve a(1,1)+b(2,−1)=(0,3)a(1,1) + b(2,-1) = (0,3): a+2b=0a + 2b = 0 and a−b=3a - b = 3 give b=−1b = -1, a=2a = 2. So (0,3)=2(1,1)−(2,−1)(0,3) = 2(1,1) - (2,-1).

Basis and dimension

A basis of a space is a set of vectors that

  1. spans the space (every vector can be built), and
  2. is linearly independent (no redundancy).

Together these guarantee that every vector can be built in exactly one way. If there were two ways, ∑civ⃗i=∑div⃗i\sum c_i \vec v_i = \sum d_i \vec v_i, subtracting would give ∑(ci−di)v⃗i=0⃗\sum (c_i - d_i)\vec v_i = \vec 0, and independence forces ci=dic_i = d_i.

The standard basis of R2\mathbb{R}^2 is ı^=(1,0)\hat\imath = (1, 0), ȷ^=(0,1)\hat\jmath = (0, 1), and in R3\mathbb{R}^3 we add k^=(0,0,1)\hat k = (0,0,1). Writing v⃗=(3,2)\vec v = (3, 2) really means

v⃗=3 ı^+2 ȷ^.\vec v = 3\,\hat\imath + 2\,\hat\jmath .

The components of a vector are its coefficients in a basis. Change the basis and the same arrow gets different numbers. That idea is the heart of 1.6 Change of Basis & Diagonalization, and it’s why eigenvectors simplify differential equations.

Every basis of a given space has the same number of vectors. That number is the dimension: dim⁡Rn=n\dim \mathbb{R}^n = n, a line through the origin has dimension 1, and a plane through the origin has dimension 2.

✏️ Pen & paperCoordinates in a skewed basis●●○

The vectors b⃗1=(2,1)\vec b_1 = (2, 1) and b⃗2=(−1,1)\vec b_2 = (-1, 1) form a basis of R2\mathbb{R}^2.

  1. Verify that they’re independent.
  2. Find the coordinates of x⃗=(1,5)\vec x = (1, 5) in this basis, i.e. find c1,c2c_1, c_2 with x⃗=c1b⃗1+c2b⃗2\vec x = c_1 \vec b_1 + c_2 \vec b_2.
  3. Sketch the “grid” formed by the basis and locate x⃗\vec x on it.
Show worked solution
  1. 2⋅1−1⋅(−1)=3≠02\cdot 1 - 1\cdot(-1) = 3 \neq 0, so they’re independent.
  2. Solve 2c1−c2=12c_1 - c_2 = 1 and c1+c2=5c_1 + c_2 = 5. Adding the equations gives 3c1=63c_1 = 6, so c1=2c_1 = 2 and c2=3c_2 = 3. Check: 2(2,1)+3(−1,1)=(4−3,2+3)=(1,5)2(2,1) + 3(-1,1) = (4-3, 2+3) = (1,5). ✓
  3. The grid lines run along b⃗1\vec b_1 and b⃗2\vec b_2. Starting at the origin, take 2 steps along b⃗1\vec b_1 to reach (4,2)(4, 2), then 3 steps along b⃗2\vec b_2 to reach (1,5)(1, 5). In the skewed grid, x⃗\vec x sits at grid coordinates (2,3)(2, 3). You can reproduce this in the span explorer by dragging vv to (2,1)(2,1) and ww to (−1,1)(-1,1).
✅ Concept check

In the basis $\vec b_1 = (1,1)$, $\vec b_2 = (1,-1)$, what is the first coordinate $c_1$ of the vector $(5, 1)$?

Vectors in code

NumPy arrays are vectors. Addition and scaling are elementwise, @ (or np.dot) is the dot product, and np.linalg.norm is the length. Run this cell and experiment with it:

🐍 PythonVector operations in NumPy
⌘/Ctrl + Enter

Now write two small functions yourself.

🧪 Coding exerciseExercise: combinations and independence
⌘/Ctrl + Enter

Fields: a vector at every point

A vector field assigns a vector to every point in space. A fluid’s velocity field u⃗(x,y)\vec u(x, y) tells you how fast, and in which direction, the fluid moves at each location. You’ll study vector fields properly in 2.6 Vector Fields, Flow Lines & Streamlines, but here’s the connection to what you just learned: a 2D grid-based solver (like the one in Stam’s Art of Fluid Animation) stores the velocity as two arrays, u[i, j] and v[i, j], which is one 2-component vector per cell.

Everything the solver does is built from linear combinations:

  • Advection moves the field along itself.
  • Diffusion replaces each vector with a weighted average (a linear combination!) of its neighbors.
  • Projection subtracts off a gradient to make the flow incompressible.
🐍 PythonA velocity field is an array of vectors
⌘/Ctrl + Enter

That cell is a linear combination of two whole fields. Superposing simple flows like this is exactly how potential-flow aerodynamics builds the flow around an airfoil (11.7 Potential Flow & Stream Functions).

✏️ Pen & paperSuperposing flows●●●

A 2D flow is the sum of a uniform stream U⃗=(1,0)\vec U = (1, 0) and a “source” field S⃗(x,y)=(x,y)x2+y2\vec S(x, y) = \dfrac{(x, y)}{x^2 + y^2}, which points directly away from the origin.

  1. Compute the total velocity at the points (−1,0)(-1, 0), (1,0)(1, 0) and (0,1)(0, 1).
  2. At which point on the xx-axis is the total velocity zero (a stagnation point)?
  3. Explain in words why a stagnation point appears upstream (to the left) of the source.
Show worked solution
  1. At (−1,0)(-1,0): S⃗=(−1,0)/1=(−1,0)\vec S = (-1, 0)/1 = (-1, 0), so the total is (0,0)(0, 0). At (1,0)(1, 0): S⃗=(1,0)\vec S = (1, 0), so the total is (2,0)(2, 0). At (0,1)(0, 1): S⃗=(0,1)\vec S = (0, 1), so the total is (1,1)(1, 1).
  2. On the xx-axis, S⃗=(1/x,0)\vec S = (1/x, 0). The total xx-velocity is 1+1/x=01 + 1/x = 0, so x=−1x = -1. The stagnation point is (−1,0)(-1, 0), as part 1 already showed.
  3. Upstream of the source, the source pushes fluid to the left, against the stream. At exactly one point the two cancel. Downstream they point the same way and add. This flow is the classic “Rankine half-body”: the streamline through the stagnation point wraps around the source like the nose of an object in a wind tunnel.

Key Takeaways

  • A vector is both an arrow (length + direction) and a list of components. Fluency means switching between the two pictures.
  • All of linear algebra is built from adding and scaling. A linear combination c1v⃗1+⋯+ckv⃗kc_1\vec v_1 + \cdots + c_k \vec v_k uses only those two operations.
  • Finding coefficients of a linear combination is solving a linear system whose columns are the vectors.
  • The span is the set of all linear combinations. It is always a flat subspace through the origin: a point, line, plane, and so on.
  • Vectors are linearly independent when none is redundant: only the trivial combination gives 0⃗\vec 0.
  • A basis spans with no redundancy, so every vector has unique coordinates. The number of basis vectors is the dimension.
  • A simulation’s state, including a whole velocity field, is one big vector, and solver steps are linear combinations.
Finished this lesson?

Mark it complete to track your progress (saved in this browser). Reading every section does this for you.

Where this leads

These later lessons build directly on this one: