Model Extraction Attacks: Threat Models and Defences
A prediction API protects model files from direct download, but it still exposes information about the model through its outputs. With enough carefully chosen queries, an adversary may train a substitute that imitates the service, infer properties of the target, or—in narrower settings—recover parameters. These outcomes are often grouped under “model stealing”, although they are not equivalent. A surrogate that matches task accuracy may disagree with the target on many inputs. A high-fidelity copy need not contain the same parameters. Extracting memorised training records is a privacy attack, not proof that the model itself was copied. ...