commit 4363194e243cbec5874ac3ee0f4feab9ccb24ce1
parent d5020aa881d8e183d03fa6122e0b4f5cdeefa0f8
Author: Andrew <andrewlaack1@gmail.com>
Date: Wed, 4 Sep 2024 11:12:08 -0500
Took discrete notes
Diffstat:
9 files changed, 239 insertions(+), 160 deletions(-)
diff --git a/Combinatorics.md b/Combinatorics.md
@@ -0,0 +1,10 @@
+:discrete:
+# Combinatorics
+
+Ch 6.1
+
+## Notes
+
+**Definition:** Combinatorics is the study of counting.
+
+Combinatorics is commonly used for enumeration in probability theory and sometimes computer science.
diff --git a/DiscreteMath.md b/DiscreteMath.md
@@ -107,12 +107,13 @@ Unit 2.4 (integers and division):
- [LinearCongruence](LinearCongruence.md)
Unit 6.1 (The Basics of Counting 8th edition)
- - Combinatorics
- - Product Rule (also known as [[CountingPrinciple.md]])
- - SumRule
- - TreeDiagram
- - SubtractionRule (principle of inclusion-exclusion)
- - DivisionRule
+ - [Combinatorics](Combinatorics.md)
+ - [SumRule](SumRule.md)
+ - [TreeDiagram](TreeDiagram.md)
+ - [SubtractionRule](SubtractionRule.md)
+ - [DivisionRule](DivisionRule.md)
+ - [SumOfGeometricSeries](SumOfGeometricSeries.md)
+
Unit 6.2 (Pigeonhole principle)
- - PigeonholePrinciple
+ - [PigeonholePrinciple](PigeonholePrinciple.md)
-
diff --git a/DivisionRule.md b/DivisionRule.md
@@ -0,0 +1,10 @@
+:discrete:
+# Division Rule
+
+Ch 6.1
+
+## Notes
+
+**Definition:** The division rule is a rule that describes the total size of the outcome space of some function.
+
+A good way to think of this is as a function. Consider the function f A -> B where there are d values such that f(a) = b for some b in B. Knowing this, there are |A|/d total possible outcomes.
diff --git a/MachineLearning.md b/MachineLearning.md
@@ -33,166 +33,170 @@ h(x) = this is the function with an input of x this should be about the correct
Introduction to Statistical Learning (Python):
Ch 2:
- - [[Inference.md]]
- - [[Prediction.md]]
+- [[Inference.md]]
+- [[Prediction.md]]
Math for Machine Learning:
Ch 2.2
- - [MatrixMultiplication](MatrixMultiplication.md)
- - [HadamardProduct](HadamardProduct.md)
- - [IdentityMatrix](IdentityMatrix.md)
- - [Associative](Associative.md)
- - [Distributive](Distributive.md)
- - [Commutative](Commutative.md)
- - [InverseTransformation](InverseTransformation.md)
- - [Transpose](Transpose.md)
- - [SymmetricMatrix](SymmetricMatrix.md)
- - [LinearCombination](LinearCombination.md)
- - [ParticularSolution](ParticularSolution.md)
- - [GeneralSolution](GeneralSolution.md)
- - [ElementaryTransformations](ElementaryTransformations.md)
- - [RowEchelonForm](RowEchelonForm.md)
- - [BasicVariables](BasicVariables.md)
- - [FreeVariables](FreeVariables.md)
- - [ReducedRowEchelonForm](ReducedRowEchelonForm.md)
- - [GaussianElimination](GaussianElimination.md)
- - [MinusOneTrick](MinusOneTrick.md)
+- [MatrixMultiplication](MatrixMultiplication.md)
+- [HadamardProduct](HadamardProduct.md)
+- [IdentityMatrix](IdentityMatrix.md)
+- [Associative](Associative.md)
+- [Distributive](Distributive.md)
+- [Commutative](Commutative.md)
+- [InverseTransformation](InverseTransformation.md)
+- [Transpose](Transpose.md)
+- [SymmetricMatrix](SymmetricMatrix.md)
+- [LinearCombination](LinearCombination.md)
+- [ParticularSolution](ParticularSolution.md)
+- [GeneralSolution](GeneralSolution.md)
+- [ElementaryTransformations](ElementaryTransformations.md)
+- [RowEchelonForm](RowEchelonForm.md)
+- [BasicVariables](BasicVariables.md)
+- [FreeVariables](FreeVariables.md)
+- [ReducedRowEchelonForm](ReducedRowEchelonForm.md)
+- [GaussianElimination](GaussianElimination.md)
+- [MinusOneTrick](MinusOneTrick.md)
+
+Ch 2.4
+- MoorePenrosePseudoInverse (approach for solving system of linear equations)
+- Group
+- AbelianGroup (group + commutative)
+- GeneralLinearGroup (group matricies under multiplication think determinants GL(n,R))
+- RegularMatricies (invertible)
+- InnerOperation (+ : GxG -> G)
+- OuterOperation ($\cdot$ : RxV -> V - Two different sets in domain)
ML Categories:
-[[SupervisedLearning.md]]
-[[SemiSupervisedLearning.md]]
-[[SelfSupervisedLearning.md]]
-[[UnsupervisedLearning.md]]
-[[ReinforcementLearning.md]]
-[[InstanceBasedLearning.md]]
-[[ModelBasedLearning.md]]
+- [[SupervisedLearning.md]]
+- [[SemiSupervisedLearning.md]]
+- [[SelfSupervisedLearning.md]]
+- [[UnsupervisedLearning.md]]
+- [[ReinforcementLearning.md]]
+- [[InstanceBasedLearning.md]]
+- [[ModelBasedLearning.md]]
Concepts:
-[[RegressionProblem.md]]
-[[TransferLearning.md]]
-[[VisualizationAlgorithm.md]]
-[[DimensionalityReduction.md]]
-[[AnomalyDetection.md]]
-[[NoveltyDetection.md]]
-[[RuleLearning.md]]
-[[LinearRegression.md]]
-[[GradientDescent.md]]
-[[ClassificationProblem.md]]
-[[SupportVectorMachine.md]]
-[[ClusteringAlgorithms.md]]
-[[EigenVector.md]]
-[[NLP.md]]
-[[NLU.md]]
-[[Feature.md]]
-[[OfflineLearning.md]]
-[[OnlineLearning.md]]
-[[KNearestNeighbor.md]]
-[[Overfitting.md]]
-[[Underfitting.md]]
-[[GeneralizationError.md]]
-[[RMSE.md]]
-[[MAE.md]]
-[[StratifiedSampling.md]]
-[[CorrelationCoefficient.md]]
-[[LogisticRegression.md]]
-[[Imputation.md]]
-[[OneHotEncoding.md]]
-[[LabelEncoding.md]]
-[[TargetEncoding.md]]
-[[Hyperparameter.md]]
-[[FeatureScaling.md]]
-[[Standardization.md]]
-[[MinMaxScaling.md]]
-[[OrdinaryLeastSquares.md]]
-[[RadialBasisFunction.md]]
-[[KMeans.md]]
-[[StochasticAlgorithm.md]]
-[[Ensembles.md]]
-[[ConfusionMatrix.md]]
-[[CrossValidation.md]]
-[[Precision.md]]
-[[TruePositiveRate.md]]
-[[Recall.md]]
-[[HarmonicMean.md]]
-[[Accuracy.md]]
-[[DecisionThreshold.md]]
-[[ROC.md]]
-[[MulticlassClassifier.md]]
-[[OneVersusAll.md]]
-[[OneVersusOne.md]]
-[[MultilabelClassification.md]]
-[[MultioutputClassification.md]]
-[[PartialDerivative.md]]
-[[RidgeRegression.md]]
-[[LassoRegression.md]]
-[[ElasticNetRegression.md]]
-[[EarlyStopping.md]]
-[[SoftmaxRegression.md]]
-[[SVM.md]]
-[[DecisionTrees.md]]
-[[SimilarityFeature.md]]
-[[CART.md]]
-[[RandomForest.md]]
-[[VotingClassifiers.md]]
-[[Bagging.md]]
-[[Pasting.md]]
-[[Bias.md]]
-[[Variance.md]]
-[[OutOfBag.md]]
-[[RandomPatches.md]]
-[[RandomSubspaces.md]]
-[[ExtraTrees.md]]
-[[AdaBoost.md]]
-[[GradientBoosting.md]]
-[[HistogramBasedGradientBoosting.md]]
-[[Stacking.md]]
-[[Projection.md]]
-[[Subspace.md]]
-[[ManifoldLearning.md]]
-[[PCA.md]]
-[[RandomProjection.md]]
-[[LLE.md]]
-[[Affinity.md]]
-[[Segmentation.md]]
-[[DBSCAN.md]]
-[[GaussianMixtureModels.md]]
-[[NeuralNetworks.md]]
-[[Perceptrons.md]]
-[[Backpropagation.md]]
-[[MLP.md]]
-[[WideAndDeepNN.md]]
-[[CategoricalCrossEntropy.md]]
-[[VanishingGradients.md]]
-[[ExplodingGradients.md]]
-[[UnstableGradients.md]]
-[[LeakyReLU.md]]
-[[GradientClipping.md]]
-[[BatchNormalization.md]]
-[[PretrainedModels.md]]
-[[UnsupervisedPretraining.md]]
-[[Autoencoder.md]]
-[[Optimizer.md]]
-[[Momentum.md]]
-[[NAG.md]]
-[[AdaGrad.md]]
-[[Adam.md]]
-[[Dropout.md]]
-[[MaxNormRegularization.md]]
-[[Tensor.md]]
-[[Transpose.md]]
-[[CNN.md]]
-[[NaiveBayes.md]]
-[[Embedding.md]]
-[[RepresentationLearning.md]]
-[[PoolingLayers.md]]
-[[DataAugmentation.md]]
-[[SMOTE.md]]
-[[LatentSpace.md]]
-
-TODO:
-[[T-SNE.md]]
-[[UMAP.md]]
-[[MCTS.md]]
+- [[RegressionProblem.md]]
+- [[TransferLearning.md]]
+- [[VisualizationAlgorithm.md]]
+- [[DimensionalityReduction.md]]
+- [[AnomalyDetection.md]]
+- [[NoveltyDetection.md]]
+- [[RuleLearning.md]]
+- [[LinearRegression.md]]
+- [[GradientDescent.md]]
+- [[ClassificationProblem.md]]
+- [[SupportVectorMachine.md]]
+- [[ClusteringAlgorithms.md]]
+- [[EigenVector.md]]
+- [[NLP.md]]
+- [[NLU.md]]
+- [[Feature.md]]
+- [[OfflineLearning.md]]
+- [[OnlineLearning.md]]
+- [[KNearestNeighbor.md]]
+- [[Overfitting.md]]
+- [[Underfitting.md]]
+- [[GeneralizationError.md]]
+- [[RMSE.md]]
+- [[MAE.md]]
+- [[StratifiedSampling.md]]
+- [[CorrelationCoefficient.md]]
+- [[LogisticRegression.md]]
+- [[Imputation.md]]
+- [[OneHotEncoding.md]]
+- [[LabelEncoding.md]]
+- [[TargetEncoding.md]]
+- [[Hyperparameter.md]]
+- [[FeatureScaling.md]]
+- [[Standardization.md]]
+- [[MinMaxScaling.md]]
+- [[OrdinaryLeastSquares.md]]
+- [[RadialBasisFunction.md]]
+- [[KMeans.md]]
+- [[StochasticAlgorithm.md]]
+- [[Ensembles.md]]
+- [[ConfusionMatrix.md]]
+- [[CrossValidation.md]]
+- [[Precision.md]]
+- [[TruePositiveRate.md]]
+- [[Recall.md]]
+- [[HarmonicMean.md]]
+- [[Accuracy.md]]
+- [[DecisionThreshold.md]]
+- [[ROC.md]]
+- [[MulticlassClassifier.md]]
+- [[OneVersusAll.md]]
+- [[OneVersusOne.md]]
+- [[MultilabelClassification.md]]
+- [[MultioutputClassification.md]]
+- [[PartialDerivative.md]]
+- [[RidgeRegression.md]]
+- [[LassoRegression.md]]
+- [[ElasticNetRegression.md]]
+- [[EarlyStopping.md]]
+- [[SoftmaxRegression.md]]
+- [[SVM.md]]
+- [[DecisionTrees.md]]
+- [[SimilarityFeature.md]]
+- [[CART.md]]
+- [[RandomForest.md]]
+- [[VotingClassifiers.md]]
+- [[Bagging.md]]
+- [[Pasting.md]]
+- [[Bias.md]]
+- [[Variance.md]]
+- [[OutOfBag.md]]
+- [[RandomPatches.md]]
+- [[RandomSubspaces.md]]
+- [[ExtraTrees.md]]
+- [[AdaBoost.md]]
+- [[GradientBoosting.md]]
+- [[HistogramBasedGradientBoosting.md]]
+- [[Stacking.md]]
+- [[Projection.md]]
+- [[Subspace.md]]
+- [[ManifoldLearning.md]]
+- [[PCA.md]]
+- [[RandomProjection.md]]
+- [[LLE.md]]
+- [[Affinity.md]]
+- [[Segmentation.md]]
+- [[DBSCAN.md]]
+- [[GaussianMixtureModels.md]]
+- [[NeuralNetworks.md]]
+- [[Perceptrons.md]]
+- [[Backpropagation.md]]
+- [[MLP.md]]
+- [[WideAndDeepNN.md]]
+- [[CategoricalCrossEntropy.md]]
+- [[VanishingGradients.md]]
+- [[ExplodingGradients.md]]
+- [[UnstableGradients.md]]
+- [[LeakyReLU.md]]
+- [[GradientClipping.md]]
+- [[BatchNormalization.md]]
+- [[PretrainedModels.md]]
+- [[UnsupervisedPretraining.md]]
+- [[Autoencoder.md]]
+- [[Optimizer.md]]
+- [[Momentum.md]]
+- [[NAG.md]]
+- [[AdaGrad.md]]
+- [[Adam.md]]
+- [[Dropout.md]]
+- [[MaxNormRegularization.md]]
+- [[Tensor.md]]
+- [[Transpose.md]]
+- [[CNN.md]]
+- [[NaiveBayes.md]]
+- [[Embedding.md]]
+- [[RepresentationLearning.md]]
+- [[PoolingLayers.md]]
+- [[DataAugmentation.md]]
+- [[SMOTE.md]]
+- [[LatentSpace.md]]
diff --git a/PigeonholePrinciple.md b/PigeonholePrinciple.md
@@ -0,0 +1,8 @@
+:discrete:
+# Pigeonhole Principle
+
+Ch 6.2
+
+## Notes
+
+**Definition:** The pigeonhole principle states that if there are n pigeons and z nests, if z is smaller than n there then must be at least one z such that z contains multiple pigeons.
diff --git a/SubtractionRule.md b/SubtractionRule.md
@@ -0,0 +1,8 @@
+:discrete:
+# Subtraction Rule
+
+Ch 6.1
+
+## Notes
+
+**Definition:** The subtraction rule (inclusion-exclusion principle) is the idea that the cardinality of the union of two sets is the individual cardinalities minus the elements in both sets (ensure not double counting).
diff --git a/SumOfGeometricSeries.md b/SumOfGeometricSeries.md
@@ -0,0 +1,14 @@
+:discrete:
+# Sum Of Geometric Series
+
+Ch 6.1
+
+## Notes
+
+**Definition:** The sum of the geometric series is the formula to solve a sequence of the form ab^0 + ab^1 .... ab^n.
+
+The formula is as follows:
+
+$S_n = \frac{a(r^n-1)}{r-1}$
+
+Where we have S_n the sum, a the constant, r the base of the exponential (common ratio), and n the total number of iterations.
diff --git a/SumRule.md b/SumRule.md
@@ -0,0 +1,14 @@
+:discrete:
+# Sum Rule
+
+Ch 6.1
+
+## Notes
+
+**Definition:** The sum rule states that the total number of possible choices is the sum of all choices.
+
+Example:
+
+There are 5 ways to paint a fence and 3 ways to paint a wall. How many ways are there to paint fences and walls?
+
+8
diff --git a/TreeDiagram.md b/TreeDiagram.md
@@ -0,0 +1,10 @@
+:discrete:
+# Tree Diagram
+
+Ch 6.1
+
+## Notes
+
+**Definition:** A tree diagram is a diagram that shows all possible choices (outcomes) along with their branching.
+
+Think of 2^n where we split into 2 paths n times as a horizontal diagram.
