Projection.md (3096B)
1 # Projection 2 3 ML D5 / Khan 4 5 ## Notes (mostly ml) 6 7 **Definition:** Projection is the process of moving an element from higher dimensional space to a lower dimensional space. 8 9 Projection is often used to reduce dimensionallity of datasets as data becomes more sparse in higher dimensions. 10 11 ## Linear Algebra Specifics 12 13 The Proj_L(x) = vector \in L where x - Proj_L(x) is orthogonal to L. 14 15 This states the difference between the projection and the original vector is orthogonal to the line (dot produce = 0). 16 17 Let's define v as some vector on L. We then know the projection of x onto L is cv for some coeficcient c. This coefficient can be found as described below. 18 19 Finding: 20 21 1. Take dot product between vector v on line L and our original vector x 22 2. Divide result by ||v|| to find c 23 3. Find the vector p which is v scaled to length of c 24 25 Example: 26 27 L = <2,1> 28 x = <2,3> 29 v = <2,1> (arbitrary on L) 30 31 Find dot product: 32 x dot v = 4 + 3 = 7 33 34 Find length of v: 35 ||v|| = sqrt(5) 36 37 Find length of projected vector (dot product / ||v||): 38 l = 7 / sqrt(5) = 3.13 39 40 Scale v to be projection: 41 //it is not necessary to create a unit vector, but easier conceptually 42 vhat = <2,1> x 1/sqrt(5) 43 vhat = <.894, .447> 44 45 p = vhat x l 46 = <.894,.447> x 3.13 47 = <2.798, 1.399> 48 49 Code: 50 51 ```python3 52 53 # Define vector class with init, length, standardize, multiply by scalar, and print 54 import math 55 56 class Vector3d: 57 def __init__(self, x, y, z): 58 self.x = x 59 self.y = y 60 self.z = z 61 62 def length(self): 63 return math.sqrt(self.x * self.x + self.y * self.y + self.z * self.z) 64 65 def standardized(self): 66 len = self.length() 67 scaleFactor = 1/len 68 return Vector3d(self.x * scaleFactor, self.y * scaleFactor, self.z * scaleFactor) 69 70 def __mul__(self, scalar): 71 return Vector3d(self.x * scalar, self.y*scalar, self.z * scalar) 72 73 def __repr__(self): 74 return('x: ' + str(self.x) + '\ny: ' + str(self.y) + '\nz: ' + str(self.z)) 75 76 # Initialize vector to project 77 v = Vector3d(2,3,1) 78 79 # Initialize vector to project onto 80 u = Vector3d(3,10,2) 81 82 # Find DP 83 dp = v.x * u.x + v.y * u.y + v.z * u.z 84 85 # Find length of projected onto 86 uLen = u.length() 87 88 # Find projected vector's length 89 projLen = dp / uLen 90 91 # Scale u to length projLen 92 standardU = u.standardized() 93 projected = standardU * projLen 94 95 print(projected) 96 97 # Output: 98 99 # x: 1.008849557522124 100 # y: 3.362831858407079 101 # z: 0.6725663716814159 102 103 ``` 104 105 ## Matrix Representation 106 107 A projection is a L.T. and thus it can be expressed by a matrix. 108 109 Additionally, given our knowledge about L.Ts we know that Proj(a + b) = Proj(a) + Proj(b) and that Proj(ca) = cProj(a). 110 111 To find the matrix we use the same identity matrix strategy where we find the columns based on their mapping from the identity matrix to the output of the L.T. 112 113 Example: 114 115 // Projection onto 116 u = <2,1> 117 118 // Identity matrix 119 [1 0] 120 [0 1] 121 122 // This can be calculated using the above strategy, but I used my code to find this 123 Proj_u(<1,0>) = <.8,.4> 124 Proj_u(<0,1>) = <.4,.2> 125 126 // Standard Matrix of L.T. based on identity matrix column inputs 127 [.8 .4] 128 [.4 .2]