Raw note · dev only 12 min read

Raw: eight questions about LLM systems

Asked in one sitting, answered only far enough to know what to read next.

Pratyush August 10, 2026

This is a raw note. Questions as asked, plus an overview written to point at what to read next. One of these — the Security Efficiency Paradox — I could not match to an established term, and the overview says so rather than inventing a definition. The page is noindex and only renders on the board in dev mode.

The questions

  1. Does AI necessarily mean a model, or is a model a way to achieve AI? Do we have any other way?
  2. Why did GPT-2 have coding errors only in the Python language?
  3. Why use YAML over JSON in LLMs?
  4. LLMs are not actually end-to-end language models — why?
  5. What is the Security Efficiency Paradox?
  6. How can a model's compression affect the safety alignment of the model?
  7. Things to keep in mind to improve local deployment of LLMs.
  8. How does continuous local adaptation introduce privacy leakage?

Overview

1. Model or AI. A model is one route, and historically the minority one. Search, constraint solving and symbolic reasoning are AI with no learned parameters anywhere: A* and its descendants, SAT and SMT solvers, planners in the STRIPS lineage, expert systems like MYCIN, logic programming. These still beat learned systems outright on problems with crisp structure — a modern SMT solver will decide things no transformer can. The interesting position is not either/or but the hybrid: AlphaGeometry pairs a language model with a symbolic deduction engine and reaches medallist-level olympiad geometry, where neither half does alone. Read Sutton's Bitter Lesson and then read a good critique of it, because the honest answer to your question is that scale has repeatedly beaten structure at the same time as the strongest current systems are structured search with a learned proposal distribution inside.

2. GPT-2 and Python. Two causes, and the specific one is a tokeniser fact. GPT-2's BPE was fitted on web text where runs of spaces are rare, so it has no efficient tokens for four, eight or twelve consecutive spaces — indentation gets shredded into many tokens, each carrying almost no information, and long stretches of the context are spent encoding whitespace. In most languages that is merely wasteful. In Python indentation is the block structure, so a representation that garbles it garbles the program's semantics. The Codex paper states the fix plainly: they added tokens representing whitespace runs, and it materially improved code generation. The second cause is mundane — WebText was not a code corpus, so GPT-2 saw comparatively little Python and almost none of it well-formatted. Worth checking the premise before building on it, though: "errors only in Python" is probably "errors most visibly in Python", since the failure mode there is syntactic and loud rather than silently wrong.

3. YAML over JSON. The honest ranking depends on which end you are on. For model output, JSON usually wins now, because constrained decoding against a JSON Schema gives you a validity guarantee no amount of prompting gives you, and YAML's indentation sensitivity interacts badly with the same whitespace tokenisation from question 2. For model input — long structured prompts, config, few-shot payloads — YAML is genuinely cheaper in tokens, because JSON spends a large fraction of its characters on quotes, braces and commas that each cost a token. The catch is that YAML is an ambiguous format: unquoted no, on and NO parse as booleans under YAML 1.1, version strings become floats, and a model producing YAML will produce those. And there is a cost to constraint itself — Tam et al. found that forcing rigid formats can reduce reasoning quality, which argues for letting the model think in prose and then formatting in a second, cheap call.

4. Not end-to-end. Correct, and the seam is exactly the tokeniser. It is fitted by a separate greedy counting algorithm, before training, against no loss the model ever sees, and then frozen — gradients stop at the embedding lookup and never reach the segmentation decision. Detokenisation is a second, non-learned stage. So is sampling: temperature, top-p and repetition penalties are hand-set decision rules operating outside the objective the model was trained on. So, usually, is retrieval. The current attempts to close the seam are the byte-level and learned-tokenisation lines — ByT5, MegaByte, Charformer, MANTa, and most recently the Byte Latent Transformer, whose dynamic patching is trained rather than fitted.

5. Security Efficiency Paradox. I cannot point you to a canonical definition, and you should treat any confident one-line answer here with suspicion. It most plausibly names one of two real things. In systems security it is the old observation that the mitigations are what cost you the performance — the Spectre and Meltdown patches being the standard case, where correctness against a side channel is bought directly in throughput. In LLM serving it names something newer and sharper: the optimisations that make inference cheap are the ones that break isolation. Prefix and KV-cache sharing across users creates a timing side channel that leaks whether someone else has asked a similar question; batching couples tenants' latency; speculative decoding leaks token-level timing. If the phrase came from a specific paper, find that paper before using the term, because as it stands it is a description in search of a citation.

6. Compression against alignment. This one has good evidence and a satisfying mechanism. Quantisation, pruning and distillation preserve average benchmark scores far better than they preserve tail behaviour, and safety is tail behaviour. Qi et al. showed alignment is shallow — concentrated in the first few generated tokens — which explains why a perturbation too small to move perplexity can still move refusal behaviour: you only have to disturb a small, specific region. Worse, it is attackable on purpose. Egashira et al. built models that behave benignly in full precision and turn malicious once quantised, meaning the fp16 evaluation you ran certifies nothing about the artefact you shipped. The operational rule that follows is blunt: re-run safety evaluation on the exact quantised weights you deploy, not on the parent.

7. Local deployment. The single most useful thing to internalise is that single-stream decoding is memory-bandwidth bound, not compute bound — tokens per second tracks how fast you can pull weights through memory, which is why quantisation buys speed and not only capacity. From there: pick the quantisation deliberately (GPTQ, AWQ and the K-quants have different damage profiles, and 4-bit weights with a poorly chosen group size lose more than the size suggests); budget the KV cache explicitly, since it grows with context length times batch and is what actually kills you at long context; quantise the cache if you must; use paged attention if you are serving more than yourself; consider speculative decoding with a small draft model; keep the prompt prefix stable so it can be cached; and evaluate the quantised build on your own task, before and after, rather than trusting the published table. Then reread question 6 before calling it done.

8. Continuous local adaptation and leakage. Training on your own data locally feels private and mostly is — until the artefact moves. Models memorise, verbatim, at rates that rise with repetition and model size; a LoRA adapter fitted on your notes is a lossy but real encoding of them, and shipping or syncing that adapter ships the data with it. If adaptation is federated, gradients alone are enough — Zhu et al. reconstructed training images from shared gradients. Even without exfiltration there is membership inference: an adapted model assigns systematically higher likelihood to what it was adapted on, so anyone who can query it can ask whether a document was in your corpus. Differential privacy is the principled defence and costs utility honestly. The underrated failure is the one Debenedetti et al. call a privacy side channel — the leak comes not from the model but from the pipeline around it: deduplication, caching, filters, retrieval indices, telemetry. For a local-first stack that is the part worth designing first.

Reading list

Every arXiv identifier below was checked against arXiv on 10 August 2026 and resolves to the paper named. The entries with no identifier — Newell and Simon, Russell and Norvig, Sutton, AlphaGeometry, the GPT-2 paper, the YAML specification and the llama.cpp repository — were not machine-checked. No identifier is a substitute for opening the paper.

Is a model the only route

  • Newell & Simon (1976), "Computer Science as Empirical Inquiry: Symbols and Search", CACM. The physical symbol system hypothesis, stated by the people who meant it.
  • Russell & Norvig, Artificial Intelligence: A Modern Approach. Chapters on search, logic, planning and constraint satisfaction — several hundred pages of AI with no gradient in sight.
  • Sutton (2019), "The Bitter Lesson". Short, and the argument every hybrid proposal has to answer.
  • Garcez & Lamb (2020), "Neurosymbolic AI: The 3rd Wave". The counter-position, surveyed. arXiv:2012.05876
  • Trinh, Wu, Le, He, Luong (2024), "Solving olympiad geometry without human demonstrations", Nature 625. AlphaGeometry — the concrete case for language model plus symbolic engine.
  • Chollet (2019), "On the Measure of Intelligence". Why benchmark scores and intelligence come apart, and the origin of ARC. arXiv:1911.01547

GPT-2, Python and formats

  • Chen, Tworek, Jun et al. (2021), "Evaluating Large Language Models Trained on Code". The Codex paper, including the whitespace-run tokens added because GPT-2's BPE handled Python indentation badly. arXiv:2107.03374
  • Radford, Wu, Child et al. (2019), "Language Models are Unsupervised Multitask Learners". The GPT-2 paper — read the section on WebText for the data half of the answer.
  • Tam et al. (2024), "Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models". Evidence that format constraints can cost reasoning quality. arXiv:2408.02442
  • Willard & Louf (2023), "Efficient Guided Generation for Large Language Models". The finite-state approach behind Outlines and most JSON-mode implementations. arXiv:2307.09702
  • The YAML 1.2 specification, §on booleans and the "Norway problem". Ten minutes that will decide the question for you on ambiguity grounds alone.

Where the pipeline stops being end-to-end

  • Pagnoni, Pasunuru, Rodriguez et al. (2024), "Byte Latent Transformer: Patches Scale Better Than Tokens". arXiv:2412.09871
  • Tay, Tran, Ruder et al. (2022), "Charformer: Fast Character Transformers via Gradient-based Subword Tokenization". arXiv:2106.12672
  • Xue, Barua, Constant et al. (2022), "ByT5: Towards a token-free future with pre-trained byte-to-byte models". arXiv:2105.13626
  • See also the companion raw note on tokenisation, which is the same seam from the other side.

Compression, alignment and the efficiency/security trade

  • Qi, Panda, Lyu et al. (2024), "Safety Alignment Should Be Made More Than Just a Few Tokens Deep". Why a small perturbation can undo alignment without touching benchmark scores. arXiv:2406.05946
  • Egashira, Vero, Staab, He, Vechev (2024), "Exploiting LLM Quantization", NeurIPS. Weights that pass evaluation in fp16 and misbehave once quantised. arXiv:2405.18137
  • Hong, Duan, Zhang et al. (2024), "Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression". Compression measured against trustworthiness dimensions rather than perplexity. arXiv:2403.15447
  • Jaiswal, Gan, Du et al. (2023), "Compressing LLMs: The Truth is Rarely Pure and Never Simple". Where the standard compression evaluations flatter themselves. arXiv:2310.01382
  • Kocher, Genkin, Gruss et al. (2019), "Spectre Attacks: Exploiting Speculative Execution", IEEE S&P. The canonical case of paying for security in performance, for the systems reading of question 5. arXiv:1801.01203

Local deployment

  • Frantar, Ashkboos, Hoefler, Alistarh (2023), "GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers", ICLR. arXiv:2210.17323
  • Lin, Tang, Tang et al. (2024), "AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration", MLSys. arXiv:2306.00978
  • Xiao, Lin, Seznec et al. (2023), "SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models", ICML. arXiv:2211.10438
  • Kwon, Li, Zhuang et al. (2023), "Efficient Memory Management for Large Language Model Serving with PagedAttention", SOSP. The KV cache as the real constraint. arXiv:2309.06180
  • Leviathan, Kalman, Matias (2023), "Fast Inference from Transformers via Speculative Decoding", ICML. arXiv:2211.17192
  • Dao, Fu, Ermon, Rudra, Ré (2022), "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness", NeurIPS. The clearest statement of why memory movement, not arithmetic, is the budget. arXiv:2205.14135
  • The llama.cpp repository, and its quantisation documentation. The K-quant damage profiles are documented nowhere better than in the project's own measurements.

Adaptation and leakage

  • Carlini, Tramèr, Wallace et al. (2021), "Extracting Training Data from Large Language Models", USENIX Security. arXiv:2012.07805
  • Carlini, Ippolito, Jagielski et al. (2022; ICLR 2023), "Quantifying Memorization Across Neural Language Models". Memorisation as a function of scale, duplication and context. arXiv:2202.07646
  • Zhu, Liu, Han (2019), "Deep Leakage from Gradients", NeurIPS. Why sharing gradients is sharing data. arXiv:1906.08935
  • Shokri, Stronati, Song, Shmatikov (2017), "Membership Inference Attacks against Machine Learning Models", IEEE S&P. arXiv:1610.05820
  • Abadi, Chu, Goodfellow et al. (2016), "Deep Learning with Differential Privacy", CCS. DP-SGD, and an honest account of what it costs. arXiv:1607.00133
  • Debenedetti, Severi, Carlini et al. (2023), "Privacy Side Channels in Machine Learning Systems". Leakage from the pipeline rather than the weights — the most relevant one for a local-first design. arXiv:2309.05610
  • Zeng, Zhang, He et al. (2024), "The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)". If local adaptation is done by retrieval rather than training, the leak moves here. arXiv:2402.16893