Forward noising and learned reverse denoising processes in a diffusion model

What Are Diffusion Models?

A diffusion model learns to generate data by reversing a controlled corruption process. During training, clean examples are mixed with noise at different intensities. A neural network learns what noise was added—or, in equivalent parameterizations, what clean sample or velocity produced the noisy input. During generation, the model begins with random noise and repeatedly transforms it into a structured sample. That description is simple, but three details matter: Training does not run the entire noising chain for every example. Any noise level can be sampled directly. The reverse transition is learned because the exact reverse distribution depends on the unknown data distribution. Generation is iterative. Computation within one step is parallelizable, but the denoising steps themselves are sequential. The formulation was introduced by Sohl-Dickstein et al. and made practical for high-quality image synthesis by Ho, Jain, and Abbeel. This article develops the discrete-time DDPM formulation first, then connects it to modern architectures and samplers. ...

February 15, 2024 · 9 min · Akshat Gupta

Fairness in Machine Learning: Metrics, Trade-offs, and Evaluation

Fairness is not a property that can be established by checking whether one metric exceeds a universal threshold. It is a system-level question about people, decisions, benefits, harms, institutions, and the evidence used to justify an intervention. Machine-learning metrics still matter. They can reveal differences in selection rates, error rates, calibration, and performance across groups. But choosing a metric is already a policy decision: it determines which differences count as harms and which trade-offs receive attention. ...

October 15, 2023 · 12 min · Akshat Gupta

Vector Databases: Indexes, Retrieval, and Production Trade-offs

A vector database stores vectors alongside identifiers and metadata, indexes them for similarity search, and provides database capabilities such as persistence, filtering, updates, access patterns, and operational management. Its purpose is broader than running a nearest-neighbour algorithm. That distinction separates a database from a library such as FAISS. FAISS provides efficient vector indexing and search primitives. It does not, by itself, provide the complete persistence, multi-tenancy, metadata, authorisation, backup, replication, and service-management layer expected from a production database. ...

December 15, 2023 · 13 min · Akshat Gupta

Ontologies in Knowledge Graphs: Reasoning and Validation

An ontology gives a knowledge graph an explicit, machine-readable account of a domain: the concepts that exist, the relationships between them, and the statements that follow from those definitions. It can align data from different systems, support inference, and make the intended meaning of a graph inspectable. That does not make an ontology a database schema, a data-quality checker, or a guarantee that every statement in the graph is true. Those responsibilities require different mechanisms. In particular, OWL reasoning and SHACL validation answer different questions: ...

January 15, 2024 · 11 min · Akshat Gupta

Fine-Tuning LLMs with LoRA

Fine-tuning is useful when a model repeatedly exhibits the wrong behaviour and high-quality examples can define the behaviour you want. It is not the default way to add changing facts, private documents, or authoritative records to an application. That distinction matters. Training a model on policy documents may make its language sound familiar, but it does not provide reliable document versioning, access control, citations, or guaranteed recall. Retrieval-augmented generation (RAG) or a deterministic tool is usually a better interface to knowledge that must remain inspectable. ...

April 20, 2025 · 2 min · Akshat Gupta

Knowledge Graphs: Data Models, Construction, and Graph RAG

A knowledge graph represents entities and the relationships between them in a graph-shaped data model. Its value is not that graphs automatically contain truth or make systems explainable. The value is that identity, relationships, provenance, and domain meaning can be made explicit and queried together. That distinction matters in applied AI. A graph built from noisy documents can preserve extraction errors. A graph query can return an irrelevant path. An ontology can support valid inference from false assertions. A language model can still misread correctly retrieved facts. Knowledge graphs create useful structure; reliability comes from the evidence, controls, and evaluation around that structure. ...

March 15, 2024 · 12 min · Akshat Gupta

Prompt Design for Production LLM Systems

A production prompt is not a clever sentence. It is one component in a system that includes model configuration, trusted instructions, untrusted input, tools, validators, retries, monitoring, and human review. The useful unit of design is therefore a testable contract: what the model may read; what it must produce; which tools it may request; which constraints are enforced outside the model; how failures are detected; and which evidence determines whether a change is safe to release. Prompt wording still matters, but wording alone cannot guarantee factuality, valid data, authorisation, or reliable execution. Those properties require controls around the model. ...

April 15, 2024 · 11 min · Akshat Gupta

How Do You Evaluate LLM Systems?

A public benchmark can help compare foundation models. It cannot tell you whether a document assistant retrieves the right policy, whether an agent calls a tool with safe arguments, or whether a release stays within its latency budget. For an applied AI system, evaluation is a decision process: Given a defined workload and risk tolerance, is this version safe and useful enough to release? That question changes the unit of evaluation. The object under test is not only the model. It is the complete system: prompts, retrieval, tools, orchestration, guardrails, model configuration, and user interface. ...

June 15, 2024 · 10 min · Akshat Gupta

Building Reliable RAG Systems

Retrieval-augmented generation (RAG) gives a language model access to external evidence at inference time. It is useful when answers depend on private, changing, or domain-specific information that should not be encoded only in model weights. RAG does not make an answer correct by construction. A system can retrieve the wrong document, omit a decisive table row, use an obsolete version, misunderstand accurate evidence, or attach a citation that does not support its claim. Retrieval changes the failure surface; it does not remove it. ...

July 15, 2024 · 11 min · Akshat Gupta

Understanding Attention in Transformers

Attention lets a model construct a new representation of each token by mixing information from other tokens. The mixing weights depend on the current input, so the same word can use different context in different sentences. That description is useful but incomplete. Attention does not inherently know token order. A causal mask is not a positional encoding. A key-value (KV) cache is not long-term memory. Retrieval-augmented generation (RAG) and mixture-of-experts (MoE) are not attention variants. Attention maps can also be informative without being faithful explanations of a prediction. ...

August 15, 2024 · 8 min · Akshat Gupta