Linear transformations

Welcome to the most fun part of the linear algebra course! This section can be somewhat demanding for the beginner, but with the right introduction and the right tempo, this section can also be made really enjoyable. The course in linear algebra is almost exclusively about the following equation:

So far, we have considered the equation as a linear system of equations and determined the solution sets for the three cases: a unique solution, infinite solutions and no solution. Now we will look at the matrix from another perspective, namely as a linear transformation. We will focus on the properties of the matrix from the perspective of seeing as a function. Let us again take the same equation, but instead name the vectors and .

Now we have no unknowns and can instead see as a function that transforms the vector to the vector . This section is about what can look like, how the transformation takes place and what geometric interpretations can be made. We start by going through what generally applies to the definition of functions, that is, something that also applies outside this course.

This section is a bit difficult to digest for some, so it is okay if it doesn't come naturally at first. But it is important for the beginner to practice abstract thinking, so we start with an explanation of the analogy between linear transformations and functions that one has learned in courses in calculus.

Look at the picture! We have two quantities, one on the left and one on the right. Let be a function that relates elements from the left set to the right set. Then the left set is called the domain of , while the right set is called the codomain of . Let's see the domain as an input to . We then have a set of elements on the right that we can see as an output of . If we note the input to as , then the output of is called , which is the value of or the transformation of . Furthermore, we say that maps to . It is common to note outputs with a single variable , as in . However, does not have to be the entire target set, and it is common for to cover a subset of the target set, called range or map.

Three classes of functions (and linear transformations for that matter) are injection, surjection and bijection. We summarize with:

A nice observation that can be made by the beginner is that we can define a bijective mapping as both injective and surjective.

Let be . Then we have the following:

Let be . Then we have the following:

Let be . Then we have the following:

Let's again revisit the equation:

and do not focus too much on the fact that and are usually noted as unknown vectors, since it is common to use the same notation in these contexts. Now remember that the matrix maps to the vector . But what then is a mapping? A mapping is a function whose input and output are vectors and can also be called a transformation. A mapping is usually noted with a capital letter such as , or . If is a mapping that transforms the vector from the subspace (noted as and read as " in ") onto from the subspace , this relation is noted or sometimes as:

which reads as " maps onto ".
In a linear algebra course, the spaces for the domain and codomain are usually defined, so let be and be , such that:

which is usually a more familiar notation in this course.

A linearity or linear system outside of mathematics is usually defined as something that has a proportional relationship between input and output. A system meets the condition of linearity, if, when you modify its input, it returns a correspondingly modified output. For a real-world example, we can look to personal finance. Imagine a savings account without interest. Let's assume that we have a monthly savings of 10 USD. This means that we have saved 120 USD in one year and 1,200 USD in ten years. If we were to save double, i.e increase our savings rate by a factor of 2, we will have saved 2,400 USD in ten years. Should we instead save in two savings accounts separately with 10 USD per month each, we will then also receive the same total amount. These are examples of linear systems, and as well, an introduction to the definition of a linear transformation.

This definition leads to the following properties.
If:

is a linear mapping, then it applies that:

Before we go into what a standard matrix is, we must first determine that all linear transformations, or mappings, from to are matrix transformations. First we show that a matrix transformation is a linear transformation, then we show that a linear transformation must be a matrix transformation.

The -matrix noted above as is called the standard matrix. It is often noted in the literature as or for the respective linear transformations and . It can also be easily noted as . The two previously mentioned notes are written:

and can appear in a sentences as the standard matrix for , is the transformation of or is the transformation represented by .
The insight above is so interesting, that we summarize it again here. The proof is already clear from the previous statement.

Let be a transformation that maps to with the help of the matrix :

We then have:

Then we can express the transformation as:

with mapping to :

Summarizing the previous section, we have:

The fact that the range is a plane usually needs a further explanation, but looks like the following:

Note the last line! Surely it can be compared to the plane's parametric form? Namely:

This is usually a point of further confusion for the beginner, but we introduce this aspect early, so the knowledge has time to mature to insight before the exam. All content in linear algebra is interconnected, which makes the course conceptually difficult. When you see the context for what it is without being confused, then you know that the insight is in place!

Let be a transformation via the matrix:

We then have:

We say that collapses onto . In this particular case, any and in the point in collapse onto the point in .

Three types of linear transformations everyone must handle, understand and be able to derive are rotation, projection, and reflection.

By rotation as a linear transformation, we mean a matrix , called a rotation matrix, which maps each vector to a rotation around the origin at a given angle . The beginner is expected to learn this for both and , and as usual it is recommended to first try to understand instead of first learning formulas by heart.

Let be the result vector of an arbitrary vector that has been rotated counterclockwise around the origin by the angle . Then we have that the standard matrix returns as follows:

According to the general formula of the standard matrix for a linear transformation, we can express for:

as:

Thus, the transformed vector can be expressed as:

The last row is a linear combination of two vectors with the scalars and . Graphically, this can be deduced with the help of the unit circle. The beginner is expected to know both it and the values of cosine, sine and tangent for the standard angles:

Rotation around the origin in 3 dimensions raises the follow-up questions: around which axis? and with which orientation? We get the answer to the second question with the help of the right-hand rule that defines the direction of a selected angle (in two dimensions it becomes as simple as counterclockwise or clockwise). The vector can be rotated about the -axis, -axis or -axis. If it is rotated around several, it is easiest to produce the rotation matrix for each of the axes and then multiply them together (called composite transformation). The rotation matrix in is derived using the rotation matrix in that we derived in the previous section. Let's start with an example where the vector is rotated about the -axis. This means that we lock the -coordinates for and apply the rotation matrix only for the - and -coordinates. We start from scratch and have:

where the marked elements can be recognized as the -rotation matrix in a part of the -rotation matrix that we note . Note that in the last row of the result vector, the -coordinate is locked with and does not affect the - and -coordinates. There we find instead the recognized expression for the mapping of the -rotation matrix. Analogously, we can arrive at the corresponding -rotation matrices and with rotation around the - and - axes, respectively. We define all three:

Note, however, how the minus sign changes places for . This is associated with being consistent with the orientation of the three axes and is linked to the right-hand rule. For now, the recommendation is to buy into it, as complete understanding may require quite a lot of time that can be spent on other parts of the material in the course.

The projection matrix takes a vector and orthogonally projects it on a line in , or on a line or plane in , using the projection theorem. We go through both standard matrices here.

Let be the projection of the vector onto the vector . Then it is true that the standard matrix is:

To derive the standard matrix, we remember the projection theorem for projected onto as:

but where we instead let the vectors:

Then we can derive the standard matrix as:

where the last line shows the formula for the standard matrix .

Let's take the linear transformation:

which projects the vector onto the vector:

From the derivation of the -projection matrix, we have that the standard matrix is:

Thus, the standard matrix for the linear transformation of the projection onto a vector in is:

Let:

be the linear transformation that projects a vector onto the plane with the normal vector . Then the standard matrix is :

where is the identity matrix in , and the right-most matrix we recognize as the standard matrix for vector projection in . Unfortunately, the standard matrix cannot be noted more beautifully, but the easiest way is to insert numbers from the given normal vector first and then do the subtraction. The derivation is quite simple, so much so that we learned this already at the beginning of the course, namely the summation of two vectors. See the picture! Where we define the projected vector onto the plane and as the sum:

where is the projection of on the plane's normal vector .

This leads us to:

where is the standard matrix for projection onto the vector . Thus, we have derived the definition of the standard matrix above.

Once we have understood the projection onto an arbitrary plane, we can reuse the formula for projection onto the coordinate plane. We have three coordinate planes in , namely, the -plane, the -plane and the -plane. We take the example of the -plane. Let:

be a linear transformation, with standard matrix projecting any vector onto the -plane. This means that:

While the coordinates and exist, the coordinate becomes 0, since the vector is projected onto the -plane. We can both algebraically and geometrically derive the standard matrix to:

and we can also confirm using the formula for the standard matrix for a projection onto an arbitrary plane. The same applies to the other two standard matrices for projection onto the coordinate plane. We summarize all three:

Reflection of a vector always refers to the transformation of a line or plane whose result vector is a mirror image on the other side.

We have the linear transformation:

which reflects the vector with respect to the line with the direction vector . Its standard matrix is:

where is the -projection matrix that projects an arbitrary vector onto the line of the direction vector . We name the result vector of the projection . (Remember that the projection formula requires two vectors, so even if we project onto a line, we need a direction vector). According to the transformation, we can write the result vector , which is the reflection vector of around the line as the sum:

which connects to the definition of the standard matrix above. The reflection matrix for a line in and for a plane in can be derived analogously.

This is a very technical section, and therefore we have placed extra emphasis on keeping it short. It is a concise section, but it may appear on the vocabulary section that each exam in linear algebra offers.

A linear operator can refer to different definitions. In the basic course in linear algebra, it is fairly conventional that a linear operator is referred to as a linear transformation with the special case that the dimensions between the domain and the codomain are the same. Namely that:

Quite simply, we have dimensions both to the left and right of the arrow. If this were not the case, we would not have a linear operator, only a linear transformation (provided that the definition of linear transformation is met). Examples of linear operators are projection, reflection and rotation. The latter two are also called linear isomorphism, which means that the image is invertible, which also means that it is a bijection.
If a linear operator has a length-preserving property, that is:

the operator is called an orthogonal operator. The following theorem applies to these:

An orthogonal operator gives rise to the definition of an orthogonal matrix.

Which in turn leads to the following theorem:

And finally, we connect all the theorems and definitions with the following: