Model Extraction Attacks: Threat Models and Defences

A prediction API protects model files from direct download, but it still exposes information about the model through its outputs. With enough carefully chosen queries, an adversary may train a substitute that imitates the service, infer properties of the target, or—in narrower settings—recover parameters. These outcomes are often grouped under “model stealing”, although they are not equivalent. A surrogate that matches task accuracy may disagree with the target on many inputs. A high-fidelity copy need not contain the same parameters. Extracting memorised training records is a privacy attack, not proof that the model itself was copied. ...

September 15, 2024 · 1 min · Akshat Gupta

Memory in Large Language Models

“Memory” is an overloaded word in large language model systems. It can refer to information encoded in model parameters, tokens supplied in the current request, cached attention tensors, retrieved documents, recurrent state, or records stored by an application. These mechanisms have different lifetimes, costs, failure modes, and privacy implications. Treating them as one feature leads to architectural mistakes. A larger context window does not create cross-session memory. A key–value (KV) cache does not make a model remember a user. Retrieval does not update model parameters. A product that recalls a preference after a week is using storage and orchestration outside the base model unless the model itself has been retrained or edited. ...

July 10, 2025 · 2 min · Akshat Gupta

Speaker Anonymization: Privacy Beyond Changing a Voice

Speech carries more than words. It can reveal speaker identity, accent, age cues, emotion, health information, recording environment, and conversational context. Speaker anonymization transforms a recording to reduce identity disclosure while retaining the information required for an authorised use. That goal is narrower than making audio “anonymous”. A transformed utterance may still identify someone through names, places, background sounds, linguistic habits, or linkage across recordings. Even the voice itself may remain linkable under a stronger attacker than the one used during development. ...

October 15, 2024 · 11 min · Akshat Gupta

LLM Agents: From Model Output to Reliable Action

An LLM becomes an agent when its outputs can change an environment: querying a database, editing a record, sending a message, running code, or asking a person for approval. The model remains important, but the production system around it determines whether those actions are valid, authorised, repeatable, and observable. This distinction matters. A capable model inside a weak control loop can execute the wrong tool, repeat an irreversible action after a timeout, or follow malicious instructions hidden in retrieved content. A dependable agent therefore needs more than prompting. It needs an explicit action interface, an execution policy, state management, safety boundaries, and evaluation over complete trajectories. ...

May 5, 2025 · 10 min · Akshat Gupta