English for Artificial Intelligence: Essential Vocabulary

"training data"

The labeled or unlabeled dataset used to fit a machine learning model by adjusting its internal parameters during the learning process

"The team spent three months curating high-quality training data before a single model was trained, recognising that data quality would determine the ceiling of performance."

"overfitting"

A modeling failure in which a model learns the noise and specific patterns of the training set so precisely that it performs poorly on new, unseen data

"The decision tree achieved 99% accuracy on training data but only 61% on the test set, a classic sign of overfitting that was addressed by pruning the tree depth."

"hyperparameter tuning"

The process of searching for the optimal configuration settings — such as learning rate, batch size, or regularisation strength — that are set before training and govern how a model learns

"Automated hyperparameter tuning via Bayesian optimisation reduced the validation loss by 18% compared to the hand-selected baseline configuration."

"cross-validation"

A model evaluation technique that partitions data into multiple folds and repeatedly trains and tests the model on different splits to produce a robust performance estimate

"Five-fold cross-validation was used because the dataset was too small to reserve a separate held-out test set without significantly reducing training data."

"feature engineering"

The craft of transforming raw input variables into informative representations — through scaling, encoding, combining, or extracting features — that improve a model's predictive accuracy

"The data scientist applied log transformation to the skewed income variable and created an interaction term between age and spending score during feature engineering."

"gradient descent"

An iterative optimisation algorithm that minimises a loss function by repeatedly adjusting model parameters in the direction opposite to the gradient, taking steps proportional to the learning rate

"Stochastic gradient descent with a cyclical learning rate schedule enabled the neural network to escape local minima and converge to a lower loss than standard SGD."

"confusion matrix"

A table that summarises the prediction results of a classification model by showing the counts of true positives, false positives, true negatives, and false negatives across all classes

"Inspecting the confusion matrix revealed that the classifier struggled most with distinguishing class 3 from class 5, which the summary accuracy metric had obscured."

"regularisation"

A family of techniques — including L1 (lasso), L2 (ridge), and dropout — that add a penalty to the loss function or randomly disable neurons to discourage overfitting and improve generalisation

"Adding L2 regularisation with a coefficient of 0.01 reduced the gap between training and validation accuracy from 23 percentage points to just 4."

"backpropagation"

The algorithm that computes gradients of the loss function with respect to each weight in a neural network by applying the chain rule of calculus from the output layer back to the input layer

"During backpropagation, vanishing gradients caused the weights in the early layers to receive updates so small that learning effectively stalled, prompting the use of residual connections."

"convolutional neural network"

A deep learning architecture that uses learnable convolutional filters to detect local spatial patterns in grid-structured data such as images, progressively building up complex representations

"The convolutional neural network learned to detect edges in early layers, textures in middle layers, and object parts in deeper layers without any hand-crafted feature design."

"attention mechanism"

A neural module that allows a model to dynamically weight the relevance of different positions in the input sequence when producing each element of the output, enabling long-range dependency capture

"Visualising the attention mechanism showed that when translating the pronoun "it", the model correctly attended to "the animal" rather than "the fence" in the source sentence."

"transformer architecture"

A neural network design relying entirely on self-attention and feed-forward layers — without recurrence or convolution — enabling parallelised training and exceptional scaling on sequence data

"The transformer architecture allowed training on sequences of 8,000 tokens in parallel across thousands of GPUs, a task that would have been infeasible with recurrent models."

"embedding"

A dense, low-dimensional vector representation that maps discrete objects — words, tokens, users, or products — into a continuous space where semantic or relational similarity is reflected by geometric proximity

"The word embedding space captured the analogy "king minus man plus woman equals queen" as a vector arithmetic relationship, demonstrating learned semantic structure."

"batch normalisation"

A layer that normalises the activations of each mini-batch during training to have zero mean and unit variance, then applies learnable scale and shift parameters to stabilise and accelerate training

"Adding batch normalisation after each convolutional layer reduced training time by 40% and allowed a higher learning rate without the loss diverging."

"dropout"

A regularisation technique that randomly deactivates a fraction of neurons during each training step, forcing the network to learn redundant representations and reducing reliance on any single neuron

"A dropout rate of 0.5 in the fully connected layers halved the test error on the benchmark dataset compared to an identical architecture trained without dropout."

"transfer learning"

The practice of initialising a model with weights pre-trained on a large source task and then fine-tuning it on a smaller target task, leveraging learned representations to reduce data and compute requirements

"By applying transfer learning from a model pre-trained on ImageNet, the medical imaging team achieved state-of-the-art tumour detection using only 2,000 labelled scans."

"tokenisation"

The process of splitting raw text into smaller units called tokens — which may be words, subwords, or characters — that serve as the discrete inputs fed into a language model

"The tokeniser split the word "unbelievably" into three subword tokens, enabling the model to handle rare words without an excessively large vocabulary."

"fine-tuning"

The process of continuing the training of a pre-trained foundation model on a smaller domain-specific dataset to adapt its knowledge and output style to a particular downstream task

"Fine-tuning the base language model on ten thousand customer service transcripts dramatically improved its ability to resolve support queries in the correct brand tone."

"prompt engineering"

The discipline of designing, structuring, and iterating on the text instructions given to a language model to elicit accurate, relevant, and well-formatted responses without modifying model weights

"Adding a chain-of-thought prompt that asked the model to reason step by step before giving an answer increased accuracy on multi-step arithmetic problems from 58% to 82%."

"hallucination"

A failure mode in which a language model confidently generates factually incorrect, fabricated, or contradictory information that is not grounded in its training data or context

"The legal team flagged a hallucination in which the model cited a court case with a plausible-sounding name and citation that did not exist in any legal database."

"context window"

The maximum number of tokens a language model can process in a single forward pass, spanning both the input prompt and the generated output

"The 128,000-token context window allowed the model to ingest an entire academic paper and answer specific questions about any section without losing track of earlier content."

"retrieval-augmented generation"

An architecture in which a language model's output is conditioned on documents retrieved from an external knowledge store at inference time, grounding responses in up-to-date or proprietary information

"Retrieval-augmented generation reduced hallucinations in the customer knowledge base chatbot by 71% because the model was forced to synthesise its answers from verified retrieved passages."

"temperature"

A sampling parameter that controls the randomness of a language model's token selection: low values make outputs more deterministic and focused, while high values increase diversity and creativity

"Setting temperature to 0.1 produced near-identical structured JSON outputs on repeated runs, which was essential for the automated data extraction pipeline."

"named entity recognition"

An NLP task that identifies and classifies named mentions in text — such as persons, organisations, locations, dates, and products — into predefined semantic categories

"The named entity recognition pipeline extracted all company names and monetary values from five years of earnings call transcripts in under twenty minutes."

"few-shot prompting"

A technique in which a small number of input-output demonstration examples are included in the prompt to guide the model's behaviour and output format without any gradient-based training

"Including three few-shot examples of correctly formatted JSON in the prompt eliminated the need to post-process the model's output, saving significant engineering effort."

"chain-of-thought prompting"

A prompting strategy that encourages a language model to articulate intermediate reasoning steps before producing a final answer, significantly improving performance on complex multi-step tasks

"Chain-of-thought prompting guided the model to decompose a compound word problem into four sub-calculations, each checked before proceeding, dramatically reducing arithmetic errors."

"RLHF"

Reinforcement Learning from Human Feedback — a training paradigm in which a reward model trained on human preference annotations is used to fine-tune a language model via reinforcement learning to produce more helpful and harmless outputs

"After RLHF training, the assistant was rated by evaluators as significantly more helpful and less likely to produce harmful content than the same model trained with supervised fine-tuning alone."

"constitutional AI"

An alignment technique in which a set of human-authored principles is used to guide a model to self-critique and revise its own outputs during training, reducing reliance on large-scale human labelling for harmlessness

"Constitutional AI allowed the team to iterate on safety properties by editing a plain-English document of principles rather than re-labelling thousands of examples each time."

"system prompt"

An initial instruction or persona specification, typically hidden from the end user, that shapes the language model's behaviour, tone, scope, and constraints throughout a conversation

"The system prompt instructed the assistant to respond only in formal English, never discuss competitor products, and always recommend consulting a professional for legal questions."

"guardrails"

Safety and content-filtering mechanisms layered around or built into an AI system to prevent harmful, offensive, or off-topic outputs from reaching end users

"The guardrails flagged and blocked a prompt injection attempt that tried to override the system prompt and extract the model's confidential instructions."

"reward model"

A neural network trained on human preference data — typically by learning which of two model responses a human rater preferred — that assigns scalar scores to model outputs for use in reinforcement learning

"The reward model learned that human evaluators consistently preferred responses with concrete examples over abstract explanations, a preference that would have been difficult to specify as a rule."

"red-teaming"

A structured adversarial testing process in which human testers or automated systems deliberately attempt to elicit harmful, unsafe, or policy-violating behaviour from an AI model before deployment

"Months of internal red-teaming uncovered seventeen categories of prompt injection that could bypass the content filter, all of which were patched before the public launch."

"model drift"

The gradual degradation of a deployed model's predictive performance over time as the statistical properties of incoming real-world data diverge from those of the training distribution

"Model drift was detected when the fraud detection system's precision dropped by 12 percentage points over six months as new scam patterns emerged that were absent from the training data."

"model registry"

A versioned repository that stores trained model artefacts alongside their metadata, metrics, hyperparameters, and lineage information to support reproducibility, auditing, and staged promotion to production

"The model registry showed that the current production model was three versions behind the latest candidate, which had consistently outperformed it on holdout evaluation over the past two weeks."

"inference latency"

The elapsed time between submitting an input to a deployed model and receiving its prediction, a critical performance metric for real-time and user-facing AI applications

"Quantising the model from 32-bit to 8-bit precision reduced inference latency from 340 milliseconds to 85 milliseconds, bringing it within the 100-millisecond product requirement."

"A/B testing"

A controlled experiment that randomly assigns users or requests to two or more model variants and statistically compares their downstream business or performance metrics to determine which version to promote

"A/B testing showed that the new recommendation model increased click-through rate by 6.3% with statistical significance at p < 0.01, justifying a full rollout."

"feature store"

A centralised data platform that computes, stores, and serves machine learning features consistently across training and serving pipelines, eliminating training-serving skew and reducing duplicated engineering work

"Introducing a feature store ensured that the "days since last purchase" feature was computed identically during model training and at real-time inference, resolving a subtle skew bug."

"containerisation"

The practice of packaging a model, its dependencies, and serving code into a portable, isolated container image — typically using Docker — so it can be deployed consistently across different infrastructure environments

"Containerisation of the inference service allowed the team to promote the exact same image from staging to production, eliminating "works on my machine" environment discrepancies."

"data pipeline"

An automated sequence of data ingestion, validation, transformation, and loading steps that prepares raw data for model training or serving in a reproducible and monitored workflow

"The data pipeline failed silently when an upstream source changed its schema, and the model was trained on six weeks of stale features before the issue was detected."

"shadow mode"

A deployment strategy in which a new model processes all live traffic in parallel with the production model — logging its predictions without serving them — to evaluate real-world performance risk-free before cutover

"Running the new credit-scoring model in shadow mode for four weeks revealed a 3% disagreement rate with the production model on high-value loan applications, prompting additional validation."

"algorithmic bias"

Systematic and unjust differences in a model's outcomes across demographic groups, arising from biased training data, flawed problem framing, or discriminatory proxies encoded in features

"Algorithmic bias in the hiring tool was discovered when an audit showed that resumes from women were scored 30% lower on average, because the training data reflected a historically male-dominated workforce."

"explainability"

The degree to which a model's predictions can be understood and communicated in human-interpretable terms, either through the model's inherent simplicity or through post-hoc explanation methods

"Regulators required explainability for every credit denial, so the team used SHAP values to generate personalised explanations showing which factors most influenced each individual decision."

"differential privacy"

A mathematical framework for training models or releasing statistics in a way that provides provable guarantees that individual records in the training data cannot be inferred from the model's outputs

"Differential privacy was applied to the health data model by adding calibrated noise during training, allowing the hospital to share a useful model without exposing any individual patient's records."

"data governance"

The organisational policies, processes, and standards that define who can access, use, modify, and share data, ensuring compliance with privacy regulations and responsible AI principles across the data lifecycle

"Weak data governance meant that engineers had direct access to raw, unmasked user data during model development, a practice the compliance audit flagged as a critical risk."

"model card"

A structured documentation artefact that accompanies a trained model, describing its intended use cases, performance across demographic subgroups, known limitations, and ethical considerations

"The model card disclosed that performance on speakers with non-native English accents was 14 percentage points lower than on native speakers, prompting the product team to add a fallback."

"AI Act"

The European Union's comprehensive regulatory framework — effective from 2024 — that classifies AI systems by risk level and imposes corresponding obligations on developers and deployers, including bans on certain high-risk applications

"Legal counsel determined that the real-time biometric surveillance product fell under the AI Act's prohibited category, requiring the feature to be redesigned before entering the EU market."

"dual-use risk"

The potential for an AI technology developed for beneficial purposes to be repurposed or misused to cause harm, either by bad actors or through unintended downstream applications

"The research team delayed publishing their protein-structure model's weights due to dual-use risk concerns that the technology could accelerate the design of biological toxins."

"responsible AI"

An organisational and engineering practice that proactively addresses fairness, safety, privacy, transparency, and accountability throughout the full lifecycle of AI system design, development, and deployment

"The company's responsible AI framework required every new model to pass demographic parity checks and a human rights impact assessment before being approved for external release."