commit 8f4dbc6ae670f2238e79babbda0a1fcf57c3fd5d
parent 605819c866571627904beabc0ba1d309ac5a9439
Author: Andrew <andrewlaack1@gmail.com>
Date: Tue, 2 Jul 2024 09:45:10 -0500
took notes on data augmentation adjacent stuff
Diffstat:
9 files changed, 127 insertions(+), 1 deletion(-)
diff --git a/CNN.md b/CNN.md
@@ -1,5 +1,5 @@
:ml:
-# Convolutional Neural Network
+# Convolutional Neural Network (CNN)
ML SS
@@ -14,3 +14,41 @@ CNNs are good for image detection because they retain information about pixels a
When using CNNs it is a good idea to start at around 32 filters (or higher) and increase (often double) the number of filters as the layers progress.
Additionally, don't forget to add [[MaxPooling.md]] to ensure features are compressed and complexity of the model is minimized.
+
+### Typical Form
+
+Early on have few filters and later more. This is to capture general stuff early and more complex stuff later.
+
+Use 'same' dimension instead of 'valid' when trying to maintain dimensions (small images) and normally use it early on.
+
+Double filters after each pooling layer (generally)
+
+General Form:
+
+Conv
+Relu
+Conv
+Relu
+Pooling
+Conv
+Relu
+Conv
+Relu
+Pooling
+...
+...
+...
+Flatten
+Dense
+(DROPOUT???)
+Dense
+(DROPOUT???)
+Dense (output)
+
+You can have a few more convs stacked together before pooling, but the idea is a few convs with relus right after then pooling. At the end there should then be a few dense layers.
+
+It is a good idea to have a larger kernel early on to decrease dimensionallity early on. Then later use smaller kernels that require less computations and have better fine grained accuracy.
+
+NOTE:
+
+When using keras you should specify relu in line with the conv for the activation.
diff --git a/DataAugmentation.md b/DataAugmentation.md
@@ -0,0 +1,10 @@
+:ml:
+# Data Augmentation
+
+ML P773
+
+## Notes
+
+**Definition:** Data augmentation is the process of changing training data in such a way to make the training data set larger and more robust.
+
+In CNNs this often involves rotation, lighting, flipping, and other augmentations.
diff --git a/Embedding.md b/Embedding.md
@@ -0,0 +1,12 @@
+:ml:
+# Embedding
+
+ML P722
+
+## Notes
+
+**Definition:** Embeddings are a high dimensional dense representation of data.
+
+When using one hot encoding we get a sparse output with only one 1 and the rest 0s. However, when using embeddings all representations are high dimensional and don't have sparsity.
+
+Embeddings are generally trainable so while they are initialized, over time they will become more representative of the underlying data and how it relates to other embeddings.
diff --git a/MCTS.md b/MCTS.md
@@ -0,0 +1,8 @@
+:ml:
+# MCTS (Monte Carlo Tree Search)
+
+ML SS
+
+## Notes
+
+**Definition:**
diff --git a/MachineLearning.md b/MachineLearning.md
@@ -155,3 +155,12 @@ Concepts:
[[Tensor.md]]
[[Transpose.md]]
[[CNN.md]]
+[[NaiveBayes.md]]
+[[Embedding.md]]
+[[RepresentationLearning.md]]
+[[PoolingLayers.md]]
+[[DataAugmentation.md]]
+[[SMOTE.md]]
+
+TODO:
+[[MCTS.md]]
diff --git a/NaiveBayes.md b/NaiveBayes.md
@@ -0,0 +1,17 @@
+:ml:
+# Naive Bayes
+
+ML SS
+
+## Notes
+
+**Definition:** Naive Bayes is an algorithm used to find the probabilities of text being part of a given class.
+
+This is often used for spam classification. Here are the steps:
+
+1. Find percent of classification messages that contain each token (word/phrase)
+2. Using this percent, multiply all percents together for each token in a given message.
+3. Multiply this final percent with a known probability of any given item being part of the current class.
+4. Find the class with the highest percent and assume it is of that class.
+
+Often for this we want to add a pseudo-count to each token count for the class. This ensures that if a class has none of a given token the output is not 0% instead it would simply be lower.
diff --git a/PoolingLayers.md b/PoolingLayers.md
@@ -0,0 +1,10 @@
+:ml:
+# Pooling Layers
+
+ML P762
+
+## Notes
+
+**Definition:** Pooling layers are layers of a CNN that 'pool' together surrounding values to pass through a singular representative value.
+
+This representative value is normally either max or mean, but max has grown in favor of late.
diff --git a/RepresentationLearning.md b/RepresentationLearning.md
@@ -0,0 +1,12 @@
+:ml:
+# Representation Learning
+
+ML P722
+
+## Notes
+
+**Definition:** Representation learning is the iterative process of learning a representation of some value.
+
+Think of embedding where values are represented in higher dimensional space and iteratively moved to positions relative to other values.
+
+Additionally, autoencoders are also this as they learn lower dimensional representations (compressions) of higher dimension data.
diff --git a/SMOTE.md b/SMOTE.md
@@ -0,0 +1,10 @@
+:ml:
+# Synthetic Minority Oversampling Technique (SMOTE)
+
+ML P775
+
+## Notes
+
+**Definition:** SMOTE is the process of manipulating minority samples in the dataset to increase their representation and improve a model's classification of them.
+
+If you only have a few images of a specific type of flower maybe you augment them (rotate or something) to get 100s of instances so the model is trained on more instances of it and will subsequently be better at classifying them.
