Evaluate on genuinely separate material
Hold out buildings, not merely image files. Near-duplicate photographs, repeated archive scans and multiple views of one landmark can leak familiar information into a test set. If the intended use includes unfamiliar regions, evaluate that condition explicitly. A random split from one city may say little about performance in another city with different materials, building traditions and survey practices.
Report results by category and by meaningful subgroups rather than relying only on overall accuracy. A common category can dominate a headline metric while rare categories fail. Precision asks how often a proposed label is appropriate within the evaluation labels; recall asks how much of that category the system retrieves. Confusion matrices help identify systematic substitutions. These measures remain dependent on the quality and scope of the reference annotations.
Test the explanation, not just its appearance
Heat maps and saliency overlays can suggest where a visual model responds, but they do not prove that it has learned an architectural concept. A highlighted window might matter because of its shape, a photographic artifact or its association with a familiar landmark. Compare multiple explanation methods where appropriate and test the prediction under controlled changes that preserve relevant architectural information.
A language model can produce an articulate explanation after receiving a label, even when the cited historical relationship is unsupported. Require verifiable source references for factual claims. Separate a generated description of visible features from statements about authorship, dates or significance. If the system invents a source or cannot locate the claimed passage, reject that part of the explanation rather than treating fluency as corroboration.
Review the outliers deliberately
Unusual cases are valuable tests of the category system. A modern building may reproduce a historic facade language; a medieval structure may contain a later classical entrance. The Palace of Westminster illustrates revival architecture, while buildings altered over centuries resist a single period label. The lesson is not that these examples are inherently unclassifiable, but that the unit, category and historical claim must be precise.
Check underrepresented regions, modest buildings, damaged structures and images taken in poor conditions. Training collections can privilege accessible monuments and familiar Western categories. Review whether apparently low performance reflects missing examples, unsuitable terminology or genuinely uncertain evidence. Do not solve a representational gap simply by forcing unfamiliar material into the nearest established class.
Make abstention a valid outcome
A system should be able to return “insufficient evidence,” “outside scope” or several competing labels. Thresholds for review must be chosen using evaluation evidence and the consequences of error, not visual preference. Lowering a threshold may retrieve more candidates while increasing false suggestions. Document that trade-off and revisit it when the dataset, category vocabulary or intended use changes.
Calibration can help align scores with observed performance on a specified evaluation population, but it does not guarantee reliability elsewhere. Monitor shifts in photography, geography and source quality. If a deployment begins receiving interior details after being tested on exterior elevations, treat that change as a new evaluation problem. A polished interface does not expand the model’s tested scope.
Record a human decision with reasons
Expert review is not a ceremonial approval step. Give reviewers the original evidence, known limitations and plausible alternatives. Ask them to accept, revise, reject or defer the proposed label and explain why. Where possible, obtain an initial interpretation before exposing the model suggestion to reduce anchoring. Preserve disagreement between reviewers when it reveals a genuine interpretive issue.
Frameworks such as the NIST AI Risk Management Framework can help structure governance questions, while documentation practices such as model cards can clarify intended use and limitations. Neither replaces architectural or archival expertise. The final record should state who reviewed which evidence, what they concluded and what new information would trigger reconsideration.