There has been significant amount of research in Stackelberg Security Games (SSG), and a common
assumption in that literature is that the adversary perfectly observes the defender’s mixed strategy.
However, in real-world settings the adversary can only observe a sequence of defender pure strategies sampled from the actual mixed strategy. Therefore, a key challenge is the modeling of adversary’s belief formation based on such limited observations. The SSG literature lacks a comparative
analysis of these models and a principled study of their strengths and weaknesses. In this paper, we
study the following shortcomings of previous work and introduce new models that address these
shortcomings. First, we address the lack of empirical evaluation or head-to-head comparison of
existing models by conducting the first-of-its-kind systematic comparison of existing and new proposed models on belief data collected from human subjects on Amazon Mechanical Turk. Second,
we show that assuming a homogeneous population of adversaries, a common assumption in the
literature, is unrealistic based on our experiments, which highlight four heterogeneous groups of
adversaries with distinct belief update mechanisms. We present new models that address this shortcoming by clustering and learning these disparate behaviors from data when available. Third, we
quantify the value of having historical data on the accuracy of belief prediction.