commit 6985bb3d3dc307670f55cfda005098eed5166355
parent 22b4ca88b254449b52ac3736b99fe229bdce2199
Author: Andrew Laack <andrew@laack.co>
Date: Wed, 14 Jan 2026 15:57:23 -0600
Took OS notes, nlp notes, and IR notes
Diffstat:
24 files changed, 408 insertions(+), 3 deletions(-)
diff --git a/docs/BM25.md b/docs/BM25.md
@@ -0,0 +1,25 @@
+# BM-25
+
+**Source:** https://en.wikipedia.org/wiki/Okapi_BM25
+
+**Definition:** BM25 is a ranking function that attempts to concretize the notion that going from 1 -> 2 occurrences of a word is more important than going from 2000 -> 2001, and that simply repeating the same words over and over does not improve document relevance (keyword stuffing).
+
+## Equation
+
+$score(D,Q) = \Sigma_{i=1}^n IDF(q_i) \cdot\frac{f(q_i,D) \cdot (k_1 + 1)}{f(q_i, D) + k_1 \cdot (1 - b + b \cdot \frac{|D|}{avgdl})}$
+
+Where the IDF of $q_i$ is $\text{log(}\frac{\text{\# of documents}}{\text{\# of documents conaining containing }q_i})$ and both $k_1$ and $b$ are free variables.
+
+NOTE: There are different variants of IDF.
+
+Often, $k_1 \in [1.2,2.0]$ and $b = 0.75$.
+
+## Intuition
+
+The score of a given document D wrt the query Q is the sum of the scores for each query term.
+
+The second part of the equation for a given term is representative of the notion that we want documents to be shorter (dictated by the $b$ term), and that going from 1 -> 2 instances of a term is more important than going from 200 -> 201, dictated by the relative term frequency appearing in both the numerator and the denominator.
+
+## History
+
+This equation is rooted in empiricism and statistics. BM25 is called best matching 25 because it was the 25th iteration of the best matching formula. As such, there were (probably) 24 prior iterations of the equation, supporting the conclusion that while it does have some statistical significance, it is not grounded entirely in statistics.
diff --git a/docs/BlockDeviceDriver.md b/docs/BlockDeviceDriver.md
@@ -0,0 +1,7 @@
+# Block Device Driver
+
+**Source:** CS 6200
+
+**Chapter:** P1L2
+
+**Definition:** A block device driver is a basic OS service that provides abstracted access to block storage devices to processes.
diff --git a/docs/ComputerScience.md b/docs/ComputerScience.md
@@ -10,6 +10,7 @@ This is the index for my Computer Science related notes.
- [Computer Security](ComputerSecurity.md)
- [Probabilistic Robotics](ProbabilisticRobotics.md)
- [Information Retrieval](InformationRetrieval.md)
+- [Operating Systems](OperatingSystems.md)
## Personal Interest
diff --git a/docs/DeepLearning.md b/docs/DeepLearning.md
@@ -8,7 +8,7 @@ This index tracks deep learning-related content. While much of my deep learning
## Links by Resource
-### **Deep Learning** - Goodfellow, Bengio, Courville
+### Deep Learning - Goodfellow, Bengio, Courville
Chapter 1
@@ -37,3 +37,20 @@ Chapter 2
- EigenDecomposition
- EigenValue
- EigenVector
+
+### Personal Research
+
+- PEFT
+ - [LoRA](LoRA.md)
+ - https://arxiv.org/abs/2106.09685
+ - PLoP
+ - https://arxiv.org/abs/2506.20629
+ - QLoRA
+ - https://arxiv.org/abs/2305.14314
+- Prompt Optimization
+ - APE
+ - https://arxiv.org/abs/2211.01910
+ - APO / ProTeGi
+ - https://arxiv.org/abs/2305.03495
+ - OPRO
+ - https://arxiv.org/abs/2309.03409
diff --git a/docs/Filesystem.md b/docs/Filesystem.md
@@ -0,0 +1,7 @@
+# Filesystem
+
+**Source:** CS 6200
+
+**Chapter:** P1L2
+
+**Definition:** A filesystem is often a basic OS service that provides abstracted access to files and directories.
diff --git a/docs/InformationRetrieval.md b/docs/InformationRetrieval.md
@@ -4,5 +4,6 @@
- [TF-IDF](TF-IDF.md)
- [Cosine Similarity](CosineSimilarity.md)
-- Stemming
-- BM25
+- [Stemming](Stemming.md)
+- [Lemmatization](Lemmatization.md)
+- [BM25](BM25.md)
diff --git a/docs/KernelMode.md b/docs/KernelMode.md
@@ -0,0 +1,13 @@
+# Kernel Mode (Privileged Mode)
+
+**Source:** CS 6200
+
+**Chapter:** P1L2
+
+**Definition:** Kernel mode allows for direct hardware access, this differs from user mode where access to hardware is mediated by the OS.
+
+## Specifics
+
+Most modern hardware supports this divide natively. When in kernel mode, a special bit in the CPU is set which allows any instruction that directly manipulates the hardware to execute. If such an instruction is attempted to be executed from user mode, a trap will be caused, interrupting the process and giving control to the operating system to check what is happening, and execute the instruction if permitted (otherwise it can terminate the process).
+
+The mode bit is set to 0 for kernel mode.
diff --git a/docs/Lemmatization.md b/docs/Lemmatization.md
@@ -0,0 +1,15 @@
+# Lemmatization
+
+**Source:** https://www.ibm.com/think/topics/stemming-lemmatization
+**Definition:** Lemmatization is an NLP approach to simplifying terms that reduces terms to a dictionary base form.
+
+## Examples Using the WordNet Lemmatizer
+
+computer -> computer
+news -> news
+dogs -> dog
+dog -> dog
+doormen -> doorman
+doorman -> doorman
+
+See [stemming](Stemming.md) for a comparison between stemming and lemmatization.
diff --git a/docs/Linux.md b/docs/Linux.md
@@ -0,0 +1,35 @@
+# Linux
+
+**Source:** CS 6200
+
+**Chapter:** P1L2
+
+## Top Down Architecture
+
+### User Mode
+
+- User developed processes
+ - this includes user interfaces
+- Standard utilities
+ - shell, editor, compilers, etc.
+- Standard library
+ - fork, exec, etc.
+
+### Kernel Mode
+
+- Linux operating system
+ - Scheduler, memory manager, file system, etc
+- Hardware
+
+## Kernel Makeup
+
+The kernel is made up of the following components:
+
+- I/O management
+ - File systems, sockets, device drivers, etc.
+- Memory management
+ - Virtual memory, paging replacement, page cache
+- Process management
+ - Signal handling, process / thread creation, scheduler
+
+This differs from the MacOS X architecture because MacOS X has a [Microkernel](Microkernel.md) called Mach, and then implements a BSD CLI interface for BSD compatability and POSIX API support. This BSD CLI interface is what user processes interface with.
diff --git a/docs/LoRA.md b/docs/LoRA.md
@@ -0,0 +1,5 @@
+# Low-Rank Adaptation (LoRA)
+
+**Source:** https://arxiv.org/abs/2106.09685
+
+**Definition:** Low-Rank adaptation for LLMs is a fine tuning approach that freezes the pretrained model weights and adds additional trainable rank decomposition matrices to each layer of the Transformer, effectively reducing the number of trainable parameters.
diff --git a/docs/MachineLearning.md b/docs/MachineLearning.md
@@ -4,6 +4,8 @@ Links to ML Notes
**Definition:** Field of study that gives computers the ability to learn without being explicitly programmed.
+- [Deep Learning](DeepLearning.md)
+
## Deep Learning With Python (Chollet)
#### Ch 1 (What is DL)
diff --git a/docs/MemoryManager.md b/docs/MemoryManager.md
@@ -0,0 +1,7 @@
+# Memory Manager
+
+**Source:** CS 6200
+
+**Chapter:** P1L2
+
+**Definition:** The memory manager is responsible for allocating physical memory to process.
diff --git a/docs/MemoryPage.md b/docs/MemoryPage.md
@@ -0,0 +1,19 @@
+# Memory Page
+
+**Source:** CS 6200
+
+**Chapter:** P1L2
+
+**Definition:** A memory page is an abstraction on top of a data storage medium that allows processes to use bytes associated with the memory page.
+
+In the case of swap, processes are not aware they are using swap, but this might be where their information is being stored by the OS.
+
+## Mechanisms
+
+- Allocate
+- Map to a process's address space
+
+## Policies
+
+- Least recently used to disk
+ - LRU pages are evicted from physical memory (swap)
diff --git a/docs/Microkernel.md b/docs/Microkernel.md
@@ -0,0 +1,19 @@
+# Microkernel
+
+**Source:** CS 6200
+
+**Chapter:** P1L2
+
+**Definition:** A microkernel is a minimal kernel that only supplies primitives like an address space for processes, the context of the process (thread), and IPC.
+
+Everything else runs at user level, even things like device drivers and filesystems. This forces lots of IPC.
+
+## Pros and Cons
+
+- Smaller
+- (Often more) verifiable
+- Less portable
+- Software complexity
+ - lots of IPC to manage
+- Frequent user kernel crossing
+ - Adds performance overhead
diff --git a/docs/ModularOS.md b/docs/ModularOS.md
@@ -0,0 +1,21 @@
+# Modular OS
+
+**Source:** CS 6200
+
+**Chapter:** P1L2
+
+**Definition:** A modular OS is one where every OS service can be added to the base as a module.
+
+## Details
+
+The OS defines an interface where specific operations must be implemented by the modules. You can then put any module into said position that satisfies these requirements.
+
+The Linux operating system is considered a modular OS.
+
+## Pros and Cons
+
+- Less memory usage
+ - Don't need to load things not in use
+- Dynamic installation of new modules
+- Smaller codebase
+- Adds indirection using the interface, decreasing optimization opportunities
diff --git a/docs/MonolithicOS.md b/docs/MonolithicOS.md
@@ -0,0 +1,18 @@
+# Monolithic OS
+
+**Source:** CS 6200
+
+**Chapter:** P1L2
+
+**Definition:** A monolithic operating system is one where every service the processes can require is part of the operating system.
+
+## Pros and Cons
+
+- Everything is packaged at the same time
+ - Compile time optimizations
+- Can be quite large
+- Lots of state
+- Lots of code
+- Large memory requirements
+
+See also: [Modular OS](ModularOS.md)
diff --git a/docs/OSAbstractions.md b/docs/OSAbstractions.md
@@ -0,0 +1,13 @@
+# Operating System Abstractions
+
+**Source:** CS 6200
+
+## Examples
+
+Here is a non-exhaustive list of abstractions provided by Operating Systems
+
+- Process
+- Thread
+- File
+- Socket
+- Memory page
diff --git a/docs/OSMechanisms.md b/docs/OSMechanisms.md
@@ -0,0 +1,13 @@
+# Operating System Mechanisms
+
+**Source:** CS 6200
+
+## Examples
+
+Here is a non-exhaustive list of mechanisms provided by Operating Systems to operate upon [OS Abstractions](OSAbstractions.md)
+
+- Create
+- Schedule
+- Open
+- Write
+- Allocate
diff --git a/docs/OSPolicies.md b/docs/OSPolicies.md
@@ -0,0 +1,12 @@
+# Operating System Policies
+
+**Source:** CS 6200
+
+- We want a separation between mechanisms and policies
+- We want policies to optimize for the common case for mechanism usage
+
+## Examples
+
+- Max # of files open per process
+- Max memory usage per process
+- Max # of sockets per process
diff --git a/docs/OperatingSystems.md b/docs/OperatingSystems.md
@@ -0,0 +1,42 @@
+# Operating Systems
+
+**Definition:** An operating system is a layer of systems software that:
+
+- has privileged access to underlying hardware
+- hides the hardware complexity
+- manages hardware on behalf of applications accordingto a policy
+- Ensures applications are isolated and protected from each other
+
+- Direct operational resources
+ - Control of CPU, memory, peripherals, etc
+- Enforce working policies
+ - Fair resource access, resource usage limits, etc
+- Mitigate difficulty of complex tasks
+ - Abstracts hardware details with syscalls
+ - Don't want to worry about disk sectors / blocks in software
+
+
+## Links
+
+### CS6200: Graduate Introduction to Operating Systems
+
+- OS Elements
+ - [OS Abstractions](OSAbstractions.md)
+ - [Memory Page](MemoryPage.md)
+ - [OS Mechanisms](OSMechanisms.md)
+ - Operate on abstractions
+ - [OS Policies](OSPolicies.md)
+
+- [Kernel Mode](KernelMode.md)
+- [Trap Instruction](TrapInstruction.md)
+- [System Call](SystemCall.md)
+- Signals
+- Basic OS Services
+ - [Scheduler](Scheduler.md)
+ - [Memory Manager](MemoryManager.md)
+ - [Block Device Driver](BlockDeviceDriver.md)
+ - [Filesystem](Filesystem.md)
+- [Monolithic OS](MonolithicOS.md)
+- [Modular OS](ModularOS.md)
+- [Microkernel](Microkernel.md)
+- [Linux](Linux.md)
diff --git a/docs/Scheduler.md b/docs/Scheduler.md
@@ -0,0 +1,7 @@
+# Scheduler
+
+**Source:** CS 6200
+
+**Chapter:** P1L2
+
+**Definition:** An operating system scheduler controls access to the CPU by giving processes execution time.
diff --git a/docs/Stemming.md b/docs/Stemming.md
@@ -0,0 +1,28 @@
+# Stemming
+
+**Source:** https://www.nltk.org/howto/stem.html
+
+**Definition:** Stemming is an NLP approach to simplify terms that convey the same meaning into the same form by stripping affixes.
+
+## Examples With Porter Stemmer
+
+caresses -> caress
+caressing -> caress
+stemming -> stem
+stem -> stem
+flies -> fli
+flying -> fli
+
+NOTE: Porter stemmer is the most popular stemmer for English
+
+## Usage
+
+In practice, stemming is very useful in English for keyword matching because it is closer to capturing the meaning of words, making keyword searches more representative of semantics.
+
+Additionally, in practice, it can be more performant than lemmatization.
+
+## Limitations
+
+Different languages employ different approaches to word tense and combination such that the generalization of stemming is generally limited to a specific language, and there may not be a simple approach that can be applied to all languages (even individually).
+
+Additionally, terms in English like 'news' are changed to 'new' (using Porter stemming) which can result in lost meaning. Such limitations don't necessarily apply to lemmatization which uses a dictionary based approach, but tends to be slower.
diff --git a/docs/SystemCall.md b/docs/SystemCall.md
@@ -0,0 +1,71 @@
+# System Call (syscall)
+
+**Source:** CS 6200
+
+**Chapter:** P1L2
+
+**Definition:** A system call is performed by a user mode process to interface directly with the operating system.
+
+## Details
+
+An operating system exports a set of system call operations that can be invoked by user mode processes. These (often) include opening of files, sending data over a socket, and allocating memory.
+
+To make a system call:
+
+- Write arguments
+ - These can be passed directly to the system call (register) or indirectly by specifying an address
+- Save relevant data to well-defined location
+ - Allows OS to determine which arguments, how many, and where they are based on the system call number
+- Make system call
+
+### Performance
+
+At 2GHz there is ~50-100ns of latency imposed by the whole process. Additionally, this also impacts the hardware cache because there will probably be cache eviction during the context swap.
+
+### Modes
+
+- Synchronous mode
+ - The process waits until the system call completes before continuing execution
+- Asynchronous mode
+
+## Flow
+
+```mermaid
+
+---
+title: System call flowchart
+---
+
+stateDiagram-v2
+ direction LR
+
+ state UserProcess{
+ A:User Process Executing
+ B:Calls system call
+ C:Returns from system call
+ }
+
+ state Kernel{
+ D: Execute system call
+ }
+
+ A --> B
+ B --> D : trap mode bit = 0
+ D --> C : return mode bit = 1
+
+```
+
+NOTE: Kernel (privileged) mode is denoted with mode bit = 0.
+
+## System call definitions
+
+### Linux 64bit
+
+- kill
+ - send a signal to a process
+- setgid
+ - set group id for a process
+- mount
+ - mount a file system
+- sysctl
+ - read / write system parameters
diff --git a/docs/TrapInstruction.md b/docs/TrapInstruction.md
@@ -0,0 +1,7 @@
+# Trap Instruction
+
+**Source:** CS 6200
+
+**Chapter:** P1L2
+
+**Definition:** Trap instructions are caused by user mode processes attempting to execute privileged instructions. When this occurs, the process is interrupted, and control is given to the operating system to see what is happening, and determine what to do next.
