commit e5f74412ec08e0a87badcf8e40966179f6d1072f
parent 0d6b7393784a3a0932493960c9a6287159966c75
Author: AndrewLockVI <andrew@laack.co>
Date: Wed, 21 May 2025 12:43:16 -0500
Took notes, started taking notes w/ latex.
Diffstat:
18 files changed, 311 insertions(+), 22 deletions(-)
diff --git a/.gitignore b/.gitignore
@@ -0,0 +1,4 @@
+*.pdf
+*.aux
+*.log
+*.toc
diff --git a/docs/AES.md b/docs/AES.md
@@ -0,0 +1,13 @@
+# AES
+
+**Source:** Computer and Network Security
+
+**Chapter:** 6
+
+**Definition:** AES is the advanced encryption algorithm, currently the most widely used encryption algorithm for symmetric encryption.
+
+## Performing AES Encryption/Decryption
+
+Decryption is done by reversing the order of encryption and thus I will only take notes on the encryption portion of AES.
+
+
diff --git a/docs/Broadcasting.md b/docs/Broadcasting.md
@@ -4,8 +4,6 @@
**Chapter:** 2
-
-
**Definition:** Broadcasting is the process of iteratively applying a lower dimensional operation on higher dimensional structures.
-An example of broadcasting is adding a vector to a matrix where each column in the matrix adds the corresponding coordinate in the vector to itself.
+Something interesting; when we say $D = B + c$ where $c \in \mathbb{R^n}$, and $D,B \in R^{m,n}$, then $D_{i,j} = B_{i,j} + c_{j}$, meaning the implicit addition of a vector to a matrix adds the vector to each row, not column.
diff --git a/docs/CircuitTechnology.md b/docs/CircuitTechnology.md
@@ -1,7 +1,3 @@
# Circuit Technology
Discussion of materials, gates, and things of that sort.
-
-
-
-
diff --git a/docs/ComputerSecurity.md b/docs/ComputerSecurity.md
@@ -81,3 +81,12 @@ Main index for notes related to CSCI 370, Computer Security
### 3.1 - Symmetric Cipher Model
- [Single Key](SingleKey.md)
+
+### 6.1
+
+- [AES](AES.md)
+- [Finite Field](FiniteField.md)
+
+### 9.1
+
+- [RSA](RSA.md)
diff --git a/docs/DistributedRepresentation.md b/docs/DistributedRepresentation.md
@@ -4,8 +4,6 @@
**Chapter:** 1
-
-
**Definition:** Distributed learning is the idea of using multiple neurons to learn one thing.
As an example, we might want to distribute the idea of a red bus to be represented by two neurons one that picks out color and the other that picks out shape.
diff --git a/docs/FiniteField.md b/docs/FiniteField.md
@@ -2,6 +2,4 @@
Ch 5
-
-
**Definition:** A finite field in abstract algebra is a set where addition, subtraction, multiplication, and division are defined and behave in a way similar to real numbers (field) that also contains a finite number of elements.
diff --git a/docs/GeneralizedNondeterministicFiniteAutomaton.md b/docs/GeneralizedNondeterministicFiniteAutomaton.md
@@ -0,0 +1,17 @@
+# Generalized Nondeterministic Finite Automaton
+
+**Source:** Theory of Computation
+
+**Chapter:** 1
+
+**Definition:** Generalized nondeterministic finite automaton (GNFA) are NFAs where the transitions may have regular expressions as labels.
+
+By this definition we see that GNFAs can read in blocks of symbols, not just one as is the case with NFAs and DFAs.
+
+GNFAs are useful because they can be used as an intermediary when converting DFAs to equivalent regular expressions.
+
+DFA -> GNFA -> Regular Expression
+
+---
+
+We can easily convert a DFA into a GNFA in the special form. We simply add a new start state with an ε arrow to the old start state and a new accept state with ε arrows from the old accept states. If any arrows have multiple labels (or if there are multiple arrows going between the same two states in the same direction), we replace each with a single arrow whose label is the union of the previous labels. Finally, we add arrows labeled ∅between states that had no arrows. This last step won’t change the language recognized because a transition labeled with ∅ can never be used. From here on we assume that all GNFAs are in the special form.
diff --git a/docs/MatrixMultiplication.md b/docs/MatrixMultiplication.md
@@ -2,8 +2,6 @@
Khan U2
-
-
**Definition:** The product of A and B is defined as AB where each column of AB is Axb_n where n is the number of the column.
Idea:
diff --git a/docs/MulticlassClassifier.md b/docs/MulticlassClassifier.md
@@ -2,6 +2,4 @@
ML D2
-
-
**Definition:** A multiclass classifier is a classifier that classifies items into more than two classes (not binary classification).
diff --git a/docs/ProbabilisticGraph.md b/docs/ProbabilisticGraph.md
@@ -4,6 +4,4 @@
**Chapter:** 1
-
-
**Definition:** A probabilistic graph in ML is a graph that goes from input to output where each step is a probabilistic action being taken upon the inputs of the step.
diff --git a/docs/RSA.md b/docs/RSA.md
@@ -0,0 +1,7 @@
+# RSA (Rivest, Shamir, Adleman)
+
+**Source:** Computer and Network Security
+
+**Chapter:** 9.1
+
+**Definition:** RSA is a public key cryptosystem that uses large numbers and the fact that factoring large numbers is prohibitively expensive to ensure it is hard to derive the private key from the public key.
diff --git a/docs/RepresentationLearning.md b/docs/RepresentationLearning.md
@@ -2,8 +2,6 @@
ML P722
-
-
**Definition:** Representation learning is the iterative process of learning a representation of some value.
Think of embedding where values are represented in higher dimensional space and iteratively moved to positions relative to other values.
diff --git a/docs/Singular.md b/docs/Singular.md
@@ -4,8 +4,6 @@
**Chapter:** 2
-
-
**Definition:** For a matrix to be singular it must be a square matrix with a deteminant of zero.
Given this definition, we also see this means the matrix must not be invertible. I think this is likely where the term 'singular' is derived from.
diff --git a/docs/TheoryOfComputation.md b/docs/TheoryOfComputation.md
@@ -16,4 +16,8 @@ Lecture 01/27
Lecture 01/28
- [Deterministic Finite Automata](DeterministicFiniteAutomata.md)
-- [Non Deterministic Finite Automata](NonDeterministicFiniteAutomata.md)
+- [NonDeterministic Finite Automata](NonDeterministicFiniteAutomata.md)
+
+Reading 02/19
+
+- [Generalized Nondeterministic Finite Automaton](GeneralizedNondeterministicFiniteAutomaton.md)
diff --git a/latex/DeepLearning.tex b/latex/DeepLearning.tex
@@ -0,0 +1,183 @@
+\documentclass[12pt, letterpaper]{article}
+\usepackage{enumitem}
+\usepackage{graphicx}
+\usepackage{amsthm}
+\usepackage{listings}
+\usepackage{caption}
+\usepackage{tcolorbox}
+\usepackage{datetime}
+\usepackage{amsfonts}
+\usepackage{amsmath}
+
+\newenvironment{definition}{
+ \begin{quote}
+ \textbf{Definition:}
+ }{
+ \end{quote}
+}
+
+
+\newenvironment{explanation}{
+ \begin{quote}
+ \textbf{Explanation:}
+ }{
+ \end{quote}
+}
+
+\newenvironment{example}{
+ \begin{quote}
+ \textbf{Example:}
+ }{
+ \end{quote}
+}
+
+\lstnewenvironment{code}{
+ \hspace{.45cm}\textbf{Code:}
+ \lstset{
+ basicstyle=\ttfamily,
+ columns=fullflexible,
+ breaklines=true
+ }
+}{
+}
+
+
+\begin{document}
+
+\noindent{\large \textbf{Deep Learning by Ian Goodfellow}}
+
+\noindent Notes by Andrew Laack
+
+\tableofcontents
+
+\section{Linear Algebra}
+
+
+\subsection{Broadcasting}
+
+\begin{definition}
+Broadcasting is the process of iteratively applying a lower dimensional operation on higher dimensionalt structures.
+\end{definition}
+
+\begin{explanation}
+
+Assume \[ C = A + b\]
+
+and \[ C,A \in \mathbb{R}^{m,n}, b \in \mathbb{R}^n. \]
+
+With this, we see we are adding an $n$ dimentional vector to an $m \times n$ matrix. For this to be well defined, we know the $n$ indicies must relate to each other. In the case of the matrix, we have $n$ columns and thus we add each $b_i$ to $A_{:,i}$.
+
+\end{explanation}
+
+\begin{example}
+
+Assume
+
+\begin{align*}
+ A = \begin{bmatrix}
+ 3 & 4 & 1 \\
+ 5 & 7 & 8
+ \end{bmatrix},
+ b = \begin{bmatrix} 1 & 2 & 3 \end{bmatrix}^\top
+\end{align*}
+
+Compute $C = A + b$
+
+\begin{align*}
+&= \begin{bmatrix}
+ 3 & 4 & 1 \\
+ 5 & 7 & 8
+ \end{bmatrix} + \begin{bmatrix}1 & 2 & 3\end{bmatrix}\\
+&= \begin{bmatrix}
+ 4 & 6 & 4 \\
+ 6 & 9 & 11
+\end{bmatrix}
+\end{align*}
+
+\end{example}
+
+\begin{code}
+ import numpy as np
+
+ B = np.array([[6,2,8], [2,9,8]])
+ a = [2, 1, 2]
+ D = np.empty((len(B), len(B[0])))
+
+ for i in range(len(B)):
+ for j in range(len(B[i])):
+ D[i][j] = B[i][j] + a[j]
+
+ # Alternatively
+ D = B + a
+
+\end{code}
+
+\subsection{Matrix Multiplication}
+
+\begin{definition}
+ The product of A and B, C, is defined as \[C_{i,j} = A_{i,:}^\top \cdot B_{:,j}.\]
+\end{definition}
+
+\begin{explanation}
+ If $C = AB$ then the index $i,j$ is the dot product between the transpose of the $i$'th row vector of A and the $j$'th column vector of B. The row vector must be transposed, or vice versa, as dot products are defined for like vectors.
+
+ Additionally, we note that the number of \textbf{columns} of A must be equal to the number of \textbf{rows} in B. The result will be a matrix that has as many rows as A and as many columns as B.
+\end{explanation}
+
+\begin{example}
+ Compute $C = A \cdot B$
+ \begin{align*}
+ A = \begin{bmatrix}
+ 4 & 3 & 4 \\
+ 1 & 1 & 5 \\
+ 2 & 3 & 6
+ \end{bmatrix},
+ B = \begin{bmatrix}
+ 3 & 1 & 7 \\
+ 1 & 2 & 1 \\
+ 2 & 4 & 2
+ \end{bmatrix}
+ \end{align*}
+
+ \begin{align*}
+ C &= AB \\
+ &= \begin{bmatrix}
+ 4 & 3 & 4 \\
+ 1 & 1 & 5 \\
+ 2 & 3 & 6 \\
+ \end{bmatrix}
+ \cdot
+ \begin{bmatrix}
+ 4 & 3 & 4 \\
+ 1 & 1 & 5 \\
+ 2 & 3 & 6 \\
+ \end{bmatrix} \\
+ & =
+ \begin{bmatrix}
+ 23 & 26 & 39 \\
+ 14 & 23 & 18 \\
+ 21 & 32 & 29 \\
+ \end{bmatrix} \\
+ \end{align*}
+
+\end{example}
+
+\begin{code}
+ import numpy as np
+
+ A = np.array([[4,3,4],[1,1,5],[2,3,6]])
+ B = np.array([[3,1,7],[1,2,1],[2,4,2]])
+ C = A @ B # matrix multiplication
+ C = A.__matmul__(B) # more explicit
+
+ def matMul(A, B):
+ C = np.empty((len(A), len(B[0])))
+ for i in range(0, len(A)):
+ for j in range(0, len(B[0])):
+ C[i][j] = A[i].dot(B[:,j])
+ return C
+
+ C = matMul(A,B)
+\end{code}
+
+\end{document}
diff --git a/work/deep-learning/Broadcasting.py b/work/deep-learning/Broadcasting.py
@@ -0,0 +1,56 @@
+# How is it we broadcast a vector to a matrix?
+
+# We first consider the vector to be a row vector.
+# This assumption seems kinda stupid, but alright then.
+
+# assume $a \in R^n$.
+
+# assume $B \in R^{m,n}$.
+
+# By this assumption, B has the same number of columns as a has coordinates.
+# This makes it easy to broadcast across the rows of the matrix. Observe the notion:
+
+# B := [6 2 8]
+# [2 9 8]
+
+# Since $B \in R^{m,n}$, we see $m = 2$ and $n = 3$.
+# Since $n = 3, a \in R^3$.
+
+# a := [2, 1, 2]
+# We a has the same coordinates as $B_{:,i}$, this makes it prime for broadcasting.
+# NOTE: $B_{:,i}$ is the slice (column) where the second index of the entry must be i.
+
+# Let's do this in math!!!
+
+import numpy as np
+
+B = np.array([[6,2,8], [2,9,8]])
+a = [2, 1, 2]
+
+print("B:")
+print()
+print(str(B))
+print()
+print("a:")
+print()
+print(str(a))
+print()
+
+# this does it exactly how we thought it would!!!
+# BUT.... that's kinda boring to just do, so let's do it ourselves!
+
+D = B + a
+
+print()
+print(D)
+print()
+
+#########
+
+D = np.empty((len(B), len(B[0]))) # Create empty array of same size as B (n x m)
+
+for i in range(len(B)):
+ for j in range(len(B[i])):
+ D[i][j] = B[i][j] + a[j]
+
+print(D)
diff --git a/work/deep-learning/matmul.py b/work/deep-learning/matmul.py
@@ -0,0 +1,16 @@
+import numpy as np
+
+A = np.array([[4,3,4],[1,1,5],[2,3,6]])
+B = np.array([[3,1,7],[1,2,1],[2,4,2]])
+C = A @ B # matrix multiplication
+C = A.__matmul__(B) # more explicit
+
+def matMul(A, B):
+ C = np.empty((len(A), len(B[0])))
+ for i in range(0, len(A)):
+ for j in range(0, len(B[0])):
+ C[i][j] = A[i].dot(B[:,j])
+ return C
+
+C = matMul(A,B)
+print(C)
