SMS SPAM Filtering With Machine Learning Model Distilbert

Abstract

This work crops up and looks to address the fact that SMS spam detection involves transforming the language by latest technology such as transformers with BERT as its representative. Hijacked unsolicited messages pose serious threats to user privacy, network efficiency, and integrity in conversations. Traditional rule-based models and shallow machine learning techniques usually do not stand ground concerning their methods of capturing the continuously changing linguistic expressions of spam messages. Therefore, we have sought after the possibilities of understanding the contextual relationships by using BERT, the best model in natural language. A public dataset from Kaggle with spam messages and labelled ham messages was used in fine-tuning the model for training and testing. The data was processed and fed into the BERT model for fine-tuning in a binary classification setting. Performance metrics for the model were derived from the standard metrics such as accuracy, precision, recall, and F1 score. BERT is best poised to capture exceedingly subtle language features and detect spam messages accurately; thus, it proves to be a viable option for any real-time SMS filtering systems.