Security Attacks of Machine Learning (ML)

Have you wondered around what might it be like to have your Machine Learning (ML) models come under security attack? In other words, your machine learning models get hacked. Have you thoroughly considered how to check or monitor security attacks on your ML models? As a data scientist or machine learning researcher, it would be good to know some of the scenarios related to security or hacking attacks on ML models. In this post, you would find out about some of the following aspects related to security attacks on machine learning models.

Examples of Security Attacks on ML Models

Most of the time, it is the classification models which would come under the security attacks. The following are some of the examples:

• Classification models such as loan sanction models which could result in classifying the possible defaulter as good one thus leading to approval of loans which could later result in the business loss
• Classification models such as credit risk models which could be trained with adversary data sets to classify the business/buyer incorrectly to allow greater credit risk exposures.
• Classification models such as insurance which could be trained inappropriately to offer greater discounts to a section of users leading to business loss; Alternatively, models trained inappropriately to approve the insurance to those who are not qualified.

Different Type of Security Attacks

• Causative Attacks resulting in altering training data & related model: Based on exploratory attacks, the attacker could create appropriate input records pass through the system at regular intervals resulting into model either letting the bad records to sneak in or blocking good records to pass through. Alternatively, attackers could also hack into the system thereby altering the training data at large. The model when, later, gets trained with the training data, allows attackers to either compromise the systems integrity or availability as described in the above section.
• Integrity attacks compromising system’s integrity: With the model trained with attackers’ data allowing the bad inputs to pass through the system, the attacker could, on regular basis, compromise the systems’ integrity by having the system label bad input as good ones. This is as like system labeling the bad records as incorrectly label as negative which can also be called a false negative.
• Availability attacks compromising system’s availability: With the model trained with attackers’ data allowing the good inputs to get filtered through the system, the system would end up filtering out legitimate records false terming them as positive. This is similar to system labeling the good input record as positive which later turns out to be false positive.

Monitoring Security Attacks

• Review the training data at regular intervals to identify the adversary data sets lurking in the training data sets. This could be done with both product managers/business analyst and data scientist taking part in the data review.
• Review the test data and model predictions

• Perform random testing and review the outcomes/predictions. This could also be termed as security vulnerability testing. Product managers/business analysts should be involved in identifying the test datasets for security vulnerability testing.

In this post, we hope to introduce you about different aspects of security attacks/hacking of machine learning models. Primarily, the machine learning models could be said to be compromised if it fails to label the bad input data accurately due to attackers/hackers exploring attacks scenario and hacking the training data sets appropriate to alter the ML model performance. In the end of the day, it is still program write with the developer code, the quality, vulnerability free of the programming code remain essential, same as well for the program runtime against hacking attempt.

Feel free to contact E-SPIN for your specific operation or project requirement, from security of your programming code to pre-production quality assurance security assessment and code review, to post production routine security audit and check against hacking attempt.