Llm

Fine-tuning is useful when a model repeatedly exhibits the wrong behaviour and high-quality examples can define the behaviour you want. It is not the default way to add changing facts, private documents, or authoritative records to an application. That distinction matters. Training a model on policy documents may make its language sound familiar, but it does not provide reliable document versioning, access control, citations, or guaranteed recall. Retrieval-augmented generation (RAG) or a deterministic tool is usually a better interface to knowledge that must remain inspectable. ...

A production prompt is not a clever sentence. It is one component in a system that includes model configuration, trusted instructions, untrusted input, tools, validators, retries, monitoring, and human review. The useful unit of design is therefore a testable contract: what the model may read; what it must produce; which tools it may request; which constraints are enforced outside the model; how failures are detected; and which evidence determines whether a change is safe to release. Prompt wording still matters, but wording alone cannot guarantee factuality, valid data, authorisation, or reliable execution. Those properties require controls around the model. ...

A public benchmark can help compare foundation models. It cannot tell you whether a document assistant retrieves the right policy, whether an agent calls a tool with safe arguments, or whether a release stays within its latency budget. For an applied AI system, evaluation is a decision process: Given a defined workload and risk tolerance, is this version safe and useful enough to release? That question changes the unit of evaluation. The object under test is not only the model. It is the complete system: prompts, retrieval, tools, orchestration, guardrails, model configuration, and user interface. ...

Retrieval-augmented generation (RAG) gives a language model access to external evidence at inference time. It is useful when answers depend on private, changing, or domain-specific information that should not be encoded only in model weights. RAG does not make an answer correct by construction. A system can retrieve the wrong document, omit a decisive table row, use an obsolete version, misunderstand accurate evidence, or attach a citation that does not support its claim. Retrieval changes the failure surface; it does not remove it. ...

A prediction API protects model files from direct download, but it still exposes information about the model through its outputs. With enough carefully chosen queries, an adversary may train a substitute that imitates the service, infer properties of the target, or—in narrower settings—recover parameters. These outcomes are often grouped under “model stealing”, although they are not equivalent. A surrogate that matches task accuracy may disagree with the target on many inputs. A high-fidelity copy need not contain the same parameters. Extracting memorised training records is a privacy attack, not proof that the model itself was copied. ...

“Memory” is an overloaded word in large language model systems. It can refer to information encoded in model parameters, tokens supplied in the current request, cached attention tensors, retrieved documents, recurrent state, or records stored by an application. These mechanisms have different lifetimes, costs, failure modes, and privacy implications. Treating them as one feature leads to architectural mistakes. A larger context window does not create cross-session memory. A key–value (KV) cache does not make a model remember a user. Retrieval does not update model parameters. A product that recalls a preference after a week is using storage and orchestration outside the base model unless the model itself has been retrained or edited. ...

An LLM becomes an agent when its outputs can change an environment: querying a database, editing a record, sending a message, running code, or asking a person for approval. The model remains important, but the production system around it determines whether those actions are valid, authorised, repeatable, and observable. This distinction matters. A capable model inside a weak control loop can execute the wrong tool, repeat an irreversible action after a timeout, or follow malicious instructions hidden in retrieved content. A dependable agent therefore needs more than prompting. It needs an explicit action interface, an execution policy, state management, safety boundaries, and evaluation over complete trajectories. ...