notes

Personal notes
git clone git://git.laack.co/notes.git
Log | Files | Refs

commit 9c7435c9b0bfc826d26a89ea66396b109d28c2ca
parent 632a6d3091f3be06f3d92f2b98db9735223546aa
Author: Andrew Laack <andrew@laack.co>
Date:   Sun, 31 May 2026 19:36:14 -0500

Looking into LLM productivity.

Diffstat:
Adocs/CodeCompletionTools.md | 11+++++++++++
Mdocs/ComputerScience.md | 1+
Adocs/DORAMetrics.md | 13+++++++++++++
Adocs/SPACEFramework.md | 16++++++++++++++++
Adocs/UnderstandingMentalModelsInAIDrivenCodeCompletion.md | 31+++++++++++++++++++++++++++++++
5 files changed, 72 insertions(+), 0 deletions(-)

diff --git a/docs/CodeCompletionTools.md b/docs/CodeCompletionTools.md @@ -0,0 +1,11 @@ +# Code Completion Tools (CCTs) + +**Source:** https://doi.org/10.1016/j.ijhcs.2025.103648 + +**Definition:** Code completion tools are a category of developer tooling that generate text based on suggestions. + +Code completion tools include LSP enabled tooling like Intellisense, snippet expanson tooling, and AI code completion tools, like Supermaven. + +## Links + +- [Understanding Mental Models in AI-Driven Code Completion](UnderstandingMentalModelsInAIDrivenCodeCompletion.md) diff --git a/docs/ComputerScience.md b/docs/ComputerScience.md @@ -33,6 +33,7 @@ This is the index for my Computer Science related notes. - [Code Verification](CodeVerification.md) - [Beam Search](BeamSearch.md) - [Cybersecurity](Cybersecurity.md) +- [Code Completion Tools](CodeCompletionTools.md) ## Forced to Take Notes on diff --git a/docs/DORAMetrics.md b/docs/DORAMetrics.md @@ -0,0 +1,13 @@ +# DORA Metrics + +**Source:** [https://dora.dev](https://dora.dev) + +**Definition:** The DORA metrics are a set of metrics, defined by the DevOps Research and Assessment team, run by Google cloud, that are meant to aid in understanding devops team productivity and success. + +## Metrics + +1. Deployment frequency +2. Lead time for changes + - How long does it take from code -> production +3. Change failure rate +4. Mean time to recovery diff --git a/docs/SPACEFramework.md b/docs/SPACEFramework.md @@ -0,0 +1,16 @@ +# SPACE Framework + +**Definition:** The SPACE framework is a framework from Microsoft Research, GitHub, and the University of Victoria that can be used to survey developer experience. + +## Dimensions + +1. Satisfaction and well-being +2. Performance + - Outcomes of work +3. Activity + - Volume of work items + - LoC, PRs, closed tickets, etc +4. Communication and collaboration + - Quality of information sharing and collaboration +5. Efficiency and flow + - Ability to focus diff --git a/docs/UnderstandingMentalModelsInAIDrivenCodeCompletion.md b/docs/UnderstandingMentalModelsInAIDrivenCodeCompletion.md @@ -0,0 +1,31 @@ +# Understanding Mental Models in AI-Driven Code Completion + +**Source:** https://doi.org/10.1016/j.ijhcs.2025.103648 + +## Summary + +The researchers performed an elicitation study with 56 developers to evaluate developer preferences wrt how suggestons should be displayed, when they should be displayed, and other implementation details. + +### Thoughts as I Read + +- What if you had a complementary explanation of suggestions? + - Perhaps this would lead to a hands off approach? + - Maybe that wouldn't be part of the CCT, something additional + - I could like that. A model is called with the AST and you ask a question like, "why is X assigned to Y here". +- Two modalities + - Acceleration + - Speed up time to location + - Exploration + - Unsure where you are going +- Should be possible to disable suggestions easily (toggle perhaps) + +## Related + +- Research on productivity with AI-based code completion tools + - http://dx.doi.org/10.2139/ssrn.4945566 + - http://dx.doi.org/10.48550/ARXIV.2302.06590 + - http://dx.doi.org/10.1145/3661145 +- Deskilling + - http://doi.org/10.1145/3491102.3517578 +- [DORA Metrics](DORAMetrics.md) +- [SPACE Framework](SPACEFramework.md)