Every simulation you will ever write stores its state as numbers, and almost always those numbers come grouped as vectors: a position , a velocity , the temperatures at a thousand grid points, the six numbers that describe an aircraft’s orientation and spin. Your first linear algebra course probably treated vectors as columns to push through Gaussian elimination. In this module we rebuild the subject around the geometry of vectors, because that’s the picture that makes differential equations, stability, and fluid solvers make sense.
Transcript
- Welcome to Linear Algebra, Reloaded. We start with the single most important object in simulation: the vector.
- A vector has two personalities. Geometrically, it is an arrow: it has a length and a direction, and it starts at the origin.
- Numerically, it is a list of numbers. This arrow is the list three, two: walk three units along x, then two units along y.
- Physicists love the arrow picture. Computers only understand the list. Being fluent in linear algebra means switching between them without thinking.
- To add two vectors, place the second arrow at the tip of the first. The sum points from the start of the first to the tip of the second.
- In the list picture, you just add matching entries: three plus negative one, and two plus two, gives two, four.
- Multiplying by a number, a scalar, stretches the arrow. Two times v is twice as long. A half shrinks it. A negative number flips it around.
- Now combine both operations. Scale v by a, scale w by b, and add. The result, a v plus b w, is called a linear combination of v and w.
- Turn the two dials, a and b, and the green arrow sweeps around. Which points can it reach?
- Every one of them. Every point in the plane is some combination of v and w. The set of all linear combinations is called the span, and here the span is the whole plane.
- But watch what happens if w points along the same line as v. Now every combination stays on that line. The span collapses from a plane to a line.
- When one vector is just a scaled copy of the other, it adds no new direction. We say the vectors are linearly dependent.
- Two independent vectors in the plane form a basis: every point can be reached, in exactly one way. The standard basis is i hat and j hat, the unit steps along each axis.
- So the list three, two secretly means three i hat plus two j hat. The numbers in a vector are always coefficients of a linear combination of basis vectors.
- Why does this matter for simulation? A velocity field puts one of these vectors at every point of space. A fluid solver stores two numbers per grid cell in 2D, three in 3D.
- Every operation a solver performs, advecting, diffusing, projecting, is built from the two moves you just learned: scaling vectors and adding them.
- Next time: what happens when we let a matrix move every vector at once.
Two personalities of a vector
A vector has two descriptions, and you need both.
The arrow. Geometrically, a vector is an arrow with a length and a direction. In linear algebra we almost always draw it starting at the origin. Physicists like this picture: a velocity has a speed (length) and a heading (direction), and a force pushes some amount in some direction.
The list. Numerically, a vector is an ordered list of numbers, its components:
The list is a set of walking instructions: go 3 units along , then 2 along . The symbol means “the set of all lists of real numbers”. The plane is and space is . The state of a 100×100 temperature grid is a single vector in .
We’ll write vectors as columns (or as in running text) and use , or just depending on context.
Adding and scaling
Only two operations are needed to build all of linear algebra.
Addition. Add matching components. Geometrically, put the tail of at the tip of (tip-to-tail); the sum goes from the start of to the end of :
If you walk along and then along , you end up at . Order doesn’t matter: (the two routes form a parallelogram).
Scalar multiplication. Multiply every component by a number (a scalar):
Geometrically, stretches the arrow by a factor , and flips it if . Subtraction is addition of times a vector: is the arrow from the tip of to the tip of .
Length and the dot product
Two more tools will show up constantly. The length (or norm) of a vector comes from Pythagoras:
The dot product multiplies matching components and adds:
Geometrically, , where is the angle between them. So:
- exactly when the vectors are perpendicular (orthogonal);
- ;
- is the signed length of the “shadow” of on the line through (the projection).
Let and .
- Compute , and .
- Find the angle between them.
- Find the component of along the unit vector .
Show worked solution
- , , and .
- , so : nearly perpendicular, which matches the picture in the video.
- . Only a small part of points along .
Linear combinations
Put the two operations together. Given vectors and scalars , the vector
is a linear combination of the . The scalars are its coefficients or weights. “Linear” means we only scale and add: no squaring, no multiplying vectors together.
The basic question of this whole module is: which vectors can you build as linear combinations of a given set?
Worked example: finding coefficients
Can be written as a combination of and ?
We need numbers with :
Subtracting the first equation from the second gives , then . Check: . ✓
Look at what happened. Asking “is a combination of these vectors?” is the same as asking “does this linear system have a solution?” The columns of the system are the vectors themselves. This is what Gaussian elimination was doing all along, and the idea comes back as in 1.3 Linear Systems, Kernel & Column Space.
Try these in the explorer:
- Use the sliders to land the green arrow on a few grid intersections. The violet grid is the set of combinations with whole-number . It’s just a skewed version of the ordinary coordinate grid.
- Hit Random challenge and find the coefficients that reach the target. Then hit it again and solve for on paper first.
- Click Make w depend on v, or drag until it lies along . Watch the grid collapse. No choice of can leave the green line.
Span
The span of a set of vectors is the set of all their linear combinations:
In the plane and in space, spans are easy to picture:
| Vectors | Their span |
|---|---|
| just the origin | |
| one nonzero vector | the line through the origin along |
| two vectors in pointing in different directions | the whole plane |
| two vectors in along the same line | only that line |
| two non-parallel vectors in | a plane through the origin |
| three vectors in not all in one plane | all of |
Spans always pass through the origin (take all coefficients zero), and they are always “flat”: points, lines, planes, and their higher-dimensional cousins called subspaces.
Linear independence
In the explorer, when fell onto the line of it became redundant: it added no new direction. The precise version of “no redundancy” is:
Why does this capture redundancy? If with, say, , solve for : . So already lives in the span of the others, and dropping it doesn’t shrink the span.
Quick tests.
- Two vectors are dependent exactly when one is a multiple of the other (they’re parallel).
- In , and are dependent exactly when . You’ll meet this number again as the determinant in 1.4 Determinants as Volume Scaling.
- More than vectors in are always dependent. Three arrows in the plane can’t all point in genuinely new directions.
Worked example: three vectors in space
Are , , independent?
Look for :
From the first two, and . Substituting into the third gives , which holds for every . Take : , i.e. . Dependent. All three lie in the plane spanned by and .
Decide whether each set is linearly independent. If it’s dependent, write one vector as a combination of the others.
- ,
- , ,
- , ,
- , ,
Show worked solution
- , so they’re independent.
- , so they’re dependent. Any set containing two parallel vectors is dependent.
- Solve . The third component forces , the second then forces , and the first forces . Only the trivial solution exists, so they’re independent. (This triangular structure is why back-substitution works.)
- That’s three vectors in , so they’re dependent automatically. To find the relation, solve : and give , . So .
Basis and dimension
A basis of a space is a set of vectors that
- spans the space (every vector can be built), and
- is linearly independent (no redundancy).
Together these guarantee that every vector can be built in exactly one way. If there were two ways, , subtracting would give , and independence forces .
The standard basis of is , , and in we add . Writing really means
The components of a vector are its coefficients in a basis. Change the basis and the same arrow gets different numbers. That idea is the heart of 1.6 Change of Basis & Diagonalization, and it’s why eigenvectors simplify differential equations.
Every basis of a given space has the same number of vectors. That number is the dimension: , a line through the origin has dimension 1, and a plane through the origin has dimension 2.
The vectors and form a basis of .
- Verify that they’re independent.
- Find the coordinates of in this basis, i.e. find with .
- Sketch the “grid” formed by the basis and locate on it.
Show worked solution
- , so they’re independent.
- Solve and . Adding the equations gives , so and . Check: . ✓
- The grid lines run along and . Starting at the origin, take 2 steps along to reach , then 3 steps along to reach . In the skewed grid, sits at grid coordinates . You can reproduce this in the span explorer by dragging to and to .
Vectors in code
NumPy arrays are vectors. Addition and scaling are elementwise, @ (or np.dot) is the dot product, and np.linalg.norm is the length. Run this cell and experiment with it:
Now write two small functions yourself.
Fields: a vector at every point
A vector field assigns a vector to every point in space. A fluid’s velocity field tells you how fast, and in which direction, the fluid moves at each location. You’ll study vector fields properly in 2.6 Vector Fields, Flow Lines & Streamlines, but here’s the connection to what you just learned: a 2D grid-based solver (like the one in Stam’s Art of Fluid Animation) stores the velocity as two arrays, u[i, j] and v[i, j], which is one 2-component vector per cell.
Everything the solver does is built from linear combinations:
- Advection moves the field along itself.
- Diffusion replaces each vector with a weighted average (a linear combination!) of its neighbors.
- Projection subtracts off a gradient to make the flow incompressible.
That cell is a linear combination of two whole fields. Superposing simple flows like this is exactly how potential-flow aerodynamics builds the flow around an airfoil (11.7 Potential Flow & Stream Functions).
A 2D flow is the sum of a uniform stream and a “source” field , which points directly away from the origin.
- Compute the total velocity at the points , and .
- At which point on the -axis is the total velocity zero (a stagnation point)?
- Explain in words why a stagnation point appears upstream (to the left) of the source.
Show worked solution
- At : , so the total is . At : , so the total is . At : , so the total is .
- On the -axis, . The total -velocity is , so . The stagnation point is , as part 1 already showed.
- Upstream of the source, the source pushes fluid to the left, against the stream. At exactly one point the two cancel. Downstream they point the same way and add. This flow is the classic “Rankine half-body”: the streamline through the stagnation point wraps around the source like the nose of an object in a wind tunnel.
Key Takeaways
- A vector is both an arrow (length + direction) and a list of components. Fluency means switching between the two pictures.
- All of linear algebra is built from adding and scaling. A linear combination uses only those two operations.
- Finding coefficients of a linear combination is solving a linear system whose columns are the vectors.
- The span is the set of all linear combinations. It is always a flat subspace through the origin: a point, line, plane, and so on.
- Vectors are linearly independent when none is redundant: only the trivial combination gives .
- A basis spans with no redundancy, so every vector has unique coordinates. The number of basis vectors is the dimension.
- A simulation’s state, including a whole velocity field, is one big vector, and solver steps are linear combinations.