Intro
Vectors are involved in a large portion of the work in high-tech companies like Google, Facebook, and Spotify, forming the basis of the fascinating technical solutions we interact with on the daily.
From Apple's algorithms for face recognition to Tesla's Autopilot feature, vectors are everywhere!
In 2006, Netflix announced "The Netflix Prize", a competition for people outside of the company to improve the accuracy of their recommendation engine.
With official data from Netflix, consisting of how their users had previously rated movies, would anyone be able to come up with an algorithm to predict what other movies a user would enjoy? The answer was yes, and it involved vectors.
By applying machine learning techniques to vectors consisting of the following elements:
A specific user
A specific movie
The grade given
The date of grading
A team of researchers calling themselves "BellKor's Pragmatic Chaos" were announced the winners of the competition having significantly improved Netflix's ability to figure out your next favorite movie.
Concept
In contrast to scalars that only consist of a single number, a vector is composed by a set of numbers. Just like in the example with Netflix user data.
A vector is composed by a set of numbers
Imagine that you are a broker and you are listing a house for sale. Certain properties of the house, such as the total living area, can be represented by a single number. However, multiple parameters are needed to describe the whole house.
By representing the house as a vector with a living area, year of construction, number of rooms, starting price, and so on, buyers can get an overview of the object and compare it to other houses.
Math
There are five elementary vector operations: addition, subtraction, scalar multiplication, dot product and cross product.
To perform vector addition, the components of each vector are simply added together separately. Consider as an example the vectors:
By using vector addition:
Subtraction of vectors works in the same fashion. Vectors can also be multiplied by scalars, which multiplies each component by that scalar.
The last two elementary vector operations are called dot product and cross product. We will not discuss them here in detail, but they explain the two different ways in which vectors can be multiplied together.
Point
A single point is often noted as , and and has coordinates. The number of coordinates always corresponds to the number of dimensions in the space, such as in , or in . A continuous collection of infinitely many points can form a geometric shape, such as a line or a plane.
Properties of points
A point in space...
lacks length and width,
and has a fixed position in the space,
and has the same number of coordinates as the dimension of the space,
and is usually noted , , and if its known,
and is usually noted , , and if its unknown.
Vector
A vector in physics is drawn as an arrow and represents the starting point, direction and magnitude of a force. In the world of mathematics, the practical perspective is abstracted, where the biggest difference is that vectors are usually considered free, which means that they are not tied to a fixed position. Each vector is created based on two points (and thus has a direction and length). If one of the points of a vector is the origin, that is, the point , the vector is called a local vector. If both points are at the origin, the vector is called the zero vector and is noted .
Properties of vectors
A vector in space...
has a given length and direction,
and lacks fixed position (can be moved around in space),
and has the same number of coordinates as the dimension of the space,
and is usually noted , , and if its known,
and is usually noted , , if its unknown.
However, a vector can be noted in printed material as . Therefore, it is convenient to use the notation in a handwritten text with an arrow. On the other hand, it is a rule rather than an exception to note the vector in handwritten text .
In linear algebra, vectors lack position and can be moved around
Arithmetic rules
The following laws, or arithmetic rules, apply to vectors and often feel natural even to the novice:
Summary
(commutative law)
(associative law)
(identity)
Scalar multiplication
(commutative law)
(distributive law)
(identity)
Creation of vectors
A vector has a length and direction and is derived from a known starting point and end point . Therefore it can be created using two points in space by the formula:
Let us take the example with the points:
Let be the starting point and be the end point. The formula applies as follows:
and as a counterexample, we instead let be the end point and be the starting point:
Note that reversing the start and end points results in the vector retaining the length but having the opposite direction. We have:
Vector scaling
Scaling a vector means multiplying it by a scalar . This changes the length of . If , then is extended. If , then is abbreviated, and if , then gets the opposite direction.
Vector addition
The sum of two vectors and constitutes a new vector that we can call , and the notation is as follows:
Geometrically, the vector can be interpreted as the diagonal of the parallelogram stretched by and .
Algebraically, the respective coordinates are summed with each other, similar to how we create a vector from two points. Let for example:
Then it applies that:
Vector subtraction
Subtracting two vectors and is most easily likened to the sum of a positive and a negative vector in the following ways:
Geometrically, the vector can be interpreted in the same way as in vector addition:
Algebraically, it works analogously as in addition. Let for example:
Then it applies that:
Length of a vector
The definition of the length, or norm, of a vector can be complicated in mathematics, but the basic course in Linear Algebra in principle, always deals only with the most practical one based on the Pythagorean theorem - the Euclidean norm. Let the vector in have the coordinates:
whose Euclidean length is then defined as:
which can be recognized from Pythagoras' theorem.
The formula similarly follows in higher dimensions. In the space , the length of then becomes:
The length of the vector can also be considered as the distance between its two points because a vector is not really more than a relative spatial difference, or as we can put it to ordinary people: an arrow between two points!
The Euclidean norm is the length of a vector and is based on the Pythagorean theorem
More about distance in a later section, but we will treat ourselves to a derivation of how the length of the vector relates to the distance between its start and end points and , respectively. Let:
where:
then we have what we need to derive the distance formula between and .
Unit vector
A vector with length (norm) 1 is called a unit vector. Each vector in can be made into a unit vector by multiplying by the scalar , that is, dividing it by its own length. This is called normalizing the vector . Let be the normalized vector of , then:
One often encounters the expression above as:
which is considered a compressed expression of a normalized vector .
Generally, a unit vector with one coordinate of 1 and the rest of 0 is called a default unit vector and is noted as . The composition of the following vector fragments in is called the standard unit vectors of space:
The astute can see that each vector can be written as a combination of the standard unit vectors.
Dot product
Multiplication of two vectors, and , is not defined, but there are two operations to understand; dot (or scalar) product and cross product. The dot product results in a scalar (a number) and is usually noted either or , while the cross product results in a brand new vector . Let and be two vectors in :
then the dot product is algebraically defined as:
and the dot product has the following geometric definition:
where refers to the length of the vector and refers to the angle between and .
The scalar product is also defined for with the same geometric relationship as above for . The calculation works analogously:
Orthogonality
Orthogonality means that two vectors, lines or planes are perpendicular. Thus, the angle between them is .
Note that the dot product between two orthogonal vectors is always 0 because the angle is then :
whereupon each angle between two vectors can be found by:
We are ready for the following definition.
If the dot product between two vectors and is 0, it means that the angle between them is , and the vectors are said to be orthogonal to each other. This applies to for each pair of vectors based on a set of vectors:
If:
the vectors are said to constitute a orthogonal set. In addition, if the length of all vectors is 1, it is said to be a orthonormal set.
Furthermore, we can say that if is an arbitrary set of vectors and is orthogonal to each of these vectors in , then is said to be orthogonal to .
Standard angles
It is a common challenge for beginners to leave the concept of degrees as a unit for angles and fully embrace radians instead. It is important that the student manages this transition because it is expected that the student knows the angular values of sine, cosine and tangent according to the angle table above. This lecture note is intended to support this transition in life, and we do so in three steps:
remember the unit circle
remember the five standard angles
remember the angle table
Let's start!
1: The unit circle
According to the image, the -coordinate corresponds to the value of and the -coordinates correspond to the value of . Remember that:
and that:
Also important to remember are the trigonometric laws:
2: The five standard angles
All angular values for sine, cosine and tangent that the student is expected to memorize are some form of multiple of the five standard angles , , , and .
3: The angle table
Begin each exam by writing down the angle table as it appears above. It can be daunting, but if you learn the method behind it, you do not need to memorize the content. The first step is to draw the structure of the table with the standard angles filled in from step 2:
From step 1, we remember the - and -values for the angles and . But instead of filling in 0 and 1 in the table, we write and , respectively.
The worst is over because now we simply fill in the angular values for sine , and from left to right.
This is a beautiful pattern! And the cosine follows the same pattern, just the opposite. We've filled in the first two rows!
Only the tangent remains, which is the sine divided by the cosine. By reading the table's values, we simply fill in the last row.
To answer the angular values for or then the angle table is used together with the unit circle and the trigonometric laws.
Orthogonal projection
Orthogonal, or perpendicular, projection between two vectors and is an operation (and linear transformation, but more on that later) that results in a new vector. If is projected on we note the new vector , which has the same direction as .
According to the image, we see how is projected towards at a right angle and is parallel to . With the projection theorem we decide :
The denominator in the projection theorem is needed to normalize , i.e. scale it so that it has length 1, otherwise the vector does not get the correct length and thus does not become an orthogonal projection. Should already be normalized, the formula is reduced to:
And to derive completely from the general formula, let's insert the normalized vector:
and get:
which ends the derivation.