Severe sensitivity to Square Attack

Summary

A Square Attack test was performed on resnet-50, in which a 40% failure rate was observed. In at least one case, the model's prediction changed -0.52. This caused the label to change from snowplow, snowplough to bobsled, bobsleigh, bob.

Test Information

This test measures the robustness of the model to Square attacks. It does this by taking a sample input, applying a Square attack, and measuring the performance of the model on the perturbed input. See the paper "Square Attack: a query-efficient black-box adversarial attack via random search" by Andriushchenko, Croce, et al. (https://arxiv.org/abs/1912.00049) for more details.

Why is this important?

Malicious actors can perturb input images to alter model behavior in unexpected ways. It is important that Computer Vision models are robust to such attacks.

Model Information

• Model name: resnet-50
• Model package URL: pkg:huggingface/microsoft/resnet-50@f5104f67a0a8892c17fa776add3e55999dc67893
• Macro F1 score on reference / evaluation: 0.91 / 0.79
• Multiclass accuracy on reference / evaluation: 0.91 / 0.79
• Multiclass AUC on reference / evaluation: 1.00 / 1.00
• Macro Precision on reference / evaluation: 0.92 / 0.81
• Macro Recall on reference / evaluation: 0.91 / 0.79

This report was automatically generated by the scanning engine rime-0.21.0rc4.post195+git.2a88076b.d on 2023-01-07 17:28.