Compositions of linear transformations is about treating multiple linear transformations in sequence. For instance, let's say that $T$, $R$ and $S$ are linear transformations. A composition is then, for example, the output $y$ of the vector $x$ for
        
      $$T \circ R \circ S(x) = y$$

Compositions of linear transformations

A basis is a set of linearly independent vectors (for instance $\vec{v_1}, \ldots \vec{v}_n$) that span a vector space or subspace. That means that any vector $\vec{x}$ belonging to that space can be expressed as a linear combination of the basis for a unique set of constants $k_1, \ldots k_n$, such as:
      $$
      \vec{x} = k_1\vec{v}_1 + \ldots + k_n\vec{v}_n
      $$
      The dimension of the vector space corresponds to the number of vectors required to form a basis (the basis is not unique). In this example, $n$.

Basis and dimension

The null space (or commonly referred to as kernel) and column space (or commonly referred to as image) are spaces related to a certain matrix $A$. 
            
      The null space is plain and simple the name of the solution space for the homogeneous equation $A\vec{x} = \vec{0}$. 
      
      The column space (or commonly referred to as image) is the range of the linear transformation with the standard matrix $A$, meaning all the possible vectors $\vec{y}$ that can be mapped to via a multiplication with $A$, such that $A\vec{x} = \vec{y}$.

Null space and column space

Kernel and image are subspaces relating to a linear transformation L represtend by the standard matrix A. The kernel refers to the solution space of the homogenous system of linear equations that the matrix A represents, meaning the solutions to
      
      $$Ax = 0$$
      
      The image refers to the subspace of all resulting vectors y from multiplying the matrix A with all the possible vectors x.
     
      $$Ax = y$$

Kernel and image

What does linear transformations mean?

All matrix multiplications are linear transformations. For the general case, a transformation could be seen as a function, or a black box, that for any given input has an output. The definition for the transform to be linear is that the operation is consistent for two input elements, in our case, two vectors. Let L be a transformation, x and y be vectors, and c and k be scalars. Then L is a linear transformation if, and only if,
         
      $$L(cx + ky) = cL(x) + kL(y)$$

What are linear transformations used for in practise?

What are the requirements for linear transformations?

Linear transformations

A vector in physics represents a force with a given direction, magnitude and origin. But in linear algebra, a vector has the first two properties but lacks an origin. Therefore, it can be moved if you want.

Vectors

The equation for a straight line is only defined for two dimensions. It does not however stop them from existing in higher dimensions, where we define them in parametric or vector form.

Lines and planes in space

Bunching together several equations creates a system of equations, and a solution for the system must solve each of the system's equations. The system is said to be linear if each equation is of the form:
      $$
      a_1x_1 + a_2x_2 + \ldots + a_nx_n
      $$

Linear systems of equations

Why does Gauss Jordan work?

Gauss-Jordan is a method for solving a linear system of equations. Each system falls into one of three cases; unique solution, no solutions or infinitely many solutions. 

Why is it called the Gauss Jordan method?

How is Gaussian eliminationn used?

Gauss-Jordan

Matrix arithmetics are defined for addition, subtraction and multiplication. For the two former the matrices must have equal dimensions and the operations are commutative. Multiplication is however not commutative, and is only defined for when the number of rows of the left matrix is equal to the number of columns of the right matrix.  

Matrix arithmetics

The inverse to a matrix is another matrix, and the product of the matrix and its inverse is the identity matrix.

Inverse matrices

If a vector can be expressed as a linear combination of other vectors, the bunch is then considered as linearly dependent. If not, the vectors are considered to be linearly independent.

Linear dependence

A solution space is the vector space containing all solutions to a given system. It does follow the properties of vector spaces, so if $\vec{x}_1$ and $\vec{x}_1$ are solutions to 
      $$A\vec{x} = \vec{b}$$ 
      then we have that 
      $$k_1\vec{x}_1 + k_2\vec{x}_2$$ 
      is a solution for any values of $k_1,k_2$.

Solution space

Three types of matrices of special form are diagonal matrices ($D$), triangular matrices ($T$) and symmetric ($S$) and skew-symmetric matrices ($S_k$).
      $$
      \begin{aligned}
          D &= \left[\begin{array}{cc}
               d_1 & 0  \\
               0 & d_2 
          \end{array}\right]
          ,\quad
          T &&= \left[\begin{array}{cr}
               t_1 & \phantom{-}0  \\
               t_2 & t_3 
          \end{array}\right]
          \\
          S &= \left[\begin{array}{cc}
               s_1 & s_2  \\
               s_2 & s_3 
          \end{array}\right]
          ,\quad
          S_k &&= \left[\begin{array}{cr}
               s_1 & -s_2  \\
               s_2 & s_3 
          \end{array}\right]
      \end{aligned}
      $$ 

Matrices of special form

What is the definition of the determinant?

Determinant is a scalar representation of a matrix, defined by a specific calculation. The geometric interpretation is that it is a scale factor for the linear transformation the matrix represents. It also talks about whether the system of linear equations that the matrix represents has a unique solution or not.

Why is it called Cramer's rule?

How to find the determinant of a square matrix?

Determinant

The cross product is a calculation between two vectors in three dimensions, and the result is a third vector that is unique and orthogonal to the first two. The length of the resulting vector is equal to the area of the parallelogram that the two vectors span. The cross product can also be used to calculate the volume of a parallelepiped, as part of the calculation called triple product. Given the volume spanning the three vectors, one calculates a cross product between two of them and then makes a point product between the resulting vector and the third vector.

Cross product, area and volume

What does an eigenvector mean?

Eigenvalues and eigenvectors are related to a given square matrix A. An eigenvector is a vector which does not change its direction when multiplied with A, it can however change its length. When applicable, the length is changed by a scalar, which is the corresponding eigenvalue to the eigenvector.

How are eigenvectors used in practice?

What is the mathematical definition of an eigenvector?

Eigenvalues and eigenvectors

The dimension theorem applies to a matrix and its rank (dimension of column space) and nullity (dimension of null space). Let $A$ be a $m \times n$ matrix, then we have according to the dimension theorem that:
      $$
      \operatorname{rank}(A) = \operatorname{nullity}(A) = n
      $$

Dimension theorem

By projection one is typically referring to orthogonal projection of one vector onto another. The result is the representative contribution of the one vector along the other vector projected on. Imagine having the sun in zenit, casting a shadow of the first vector strictly down (orthogonally) onto the second vector. That shadow is then the ortogonal projection of the first vector to the second vector.

Projection theorem

Least squares is a method applied for data analysis and statistics. It is used to describe the relationship by a number of observations and their explanatory variables. Let’s say you have a number of observations (xn,yn), of which you’d like to model the relationship by a linear equation
      $$c_1x + c_2 = y$$
      where $c_1$ and $c_2$ are constants we'd like to decide so that we get the optimal fit for our line to the data. This is done by minimizing the sum of all errors, meaning the distance to the line and each of the observed data points. This is actually easily calculated, by first transforming the system of linear equations to the matrix equation
      $$Ac = y$$
      and then multiplying with the transpose of matrix A from the left on both sides
      $$A^TAc = A^Ty$$
      which results in a square matrix system which has a unique solution. The solution is the vector c, consisting of our searched constants $c_1$ and $c_2$.

Least squares

Gram-Schmidt is an algorithm for finding an ON basis to a given subspace. The input to the algorithm is a known, non-ON basis, and when applying the projection theorem in a sequence will find the basis vectors one by one.

Gram-Schmidt

A base is a set of vectors that are linearly independent and span a subspace. A vector is an element of a subspace, where its coordinates is the scalar representatives of the linear combination that can be expressed by the base vectors. Since a base is not unique for a subspace, each vector to that subspace can be expressed with coordinates for each and one of its bases.

Change of basis

A linear transformation is always with respect to a given basis. A common challenge is to determine the standard matrix A for a linear transformation given another basis.

Linear transformations and bases

Diagonlization is a process for decomposing a square $n$ x $n$ matrix $A$ into the product of three matrices; $D$, $P$ and $P^{-1}$ such as
      $$A = PDP^{-1}$$
      where $D$ is a diagonal matrix consisting of the eigenvalues to $A$ and $P$ is a square matrix which columns are the eigenvectors to $A$. Note that not all square matrices can be diagonalized, only those of which eigenvectors span the space Rn

Diagonalization

Orthogonal diagonalization is the same as regular diagonlization, with the extended requirement of the eigenvectors needed to form an ON basis for $R^n$. Only symmetric matrices are orthogonal diagonalizable. The process of deciding the vectors for the matrix $P$ is by applying Gram-Schmidt. Then, by the property of symmetric matrices, you have that
     
      $$A = PDP^{-1} = PDP^T$$

Orthogonal diagonalization

Equations of the form
      $$a_1x_1^2 + a_2x_2^2 + ... + a_nx_n^2$$
      + all possible cross terms $a_kx_ix_j$ with distinct $x_ix_j$
      are called quadratic forms, and can be expressed with a unique matrix $A$
      $$x^TAx$$
      and its geometrical shape can be determined by stuying the eigenvalues of the matrix $A$.

Quadratic form

Vector spaces does not need to be built up by numbers, they could be applied from a far more general approach. A popular application in a course of linear algebra is to cover polynomial spaces, where each element in the space is a polynomial. Confused? it can be at start, but it’s not as hard as it seems at first glance. The crucial thing is to know the axioms defining a vector space V, and sticking to them. Those axioms are
      <br /><br />
      <ul>
      <li>Space V is closed under addition and scalar multiplication</li>
      <li>Addition is both commutative and associative</li>
      <li>Scalar multiplication is both commutative and associative</li>
      <li>Existence of an identity and inverse for addition </li>
      <li>Existence of an identity and inverse for scalar multiplication</li>
      </ul>

Compositions of linear transformations

Intro

Concept

Math

Compositions

Example

Good outline for linear algebra and short to-do list

Get exam problems for old linear algebra exams divided into chapters