Natural Language Processing Assignment Help

Fake users send undesired messages, mails, posts to users to promote services or websites that not only affect legitimate users but also disrupt resource consumption.The increasing volume of unsolicited bulk e-mail (also known as spam) has generated a need for reliable anti-spam filters. Machine learning techniques now days are used to automatically filter the spam e-mail in a very successful rate.

 

Few more prominent  Machine learning algorithms that could be applied to such problems are listed below:

• Logistic Regression

• Naive Bayes

• Stochastic Gradient Descent

• K-Nearest Neighbours

• Decision Tree

• Random Forest

• Support Vector Machine

​

There are many more such methods or algorithms which come under the label Supervised Machine Learning algorithms and could be used for Spam detection problems. Machine learning field is a subfield from the broad field of artificial intelligence, this aims to make machines able to learn like humans. Learning here means understanding, observing and representing information about some statistical phenomenon. In unsupervised learning one tries to uncover hidden regularities (clusters) or to detect anomalies in the data like spam messages or network intrusion. In e-mail filtering tasks some features could be the bag of words or the subject line analysis. Thus, the input to e-mail classification task can be viewed as a two dimensional matrix, whose axes are the messages and the features. E-mail classification tasks are often divided into several sub-tasks. First, Data collection and representation are mostly problem specific, second, e-mail feature selection and feature reduction attempt to reduce the dimensionality for the remaining steps of the task. Finally, the e-mail classification phase of the process finds the actual mapping between training set and testing set.

 

Some of the most successful methodologies available today in the field are mentioned below :

​

Artificial Neural Networks classifier: An artificial neural network (ANN), also called simply a "Neural Network" (NN), is a computational model based on biological neural networks. It consists of an interconnected collection of artificial neurons. An artificial neural network is an adaptive system that changes its structure based on information that flows through the artificial network during a learning phase. The ANN is based on the principle of learning by example.

 

Support Vector Machines classifier: Support Vector Machines are based on the concept of decision planes that define decision boundaries. A decision plane is one that separates between a set of objects having different class memberships, the SVM modeling algorithm finds an optimal hyperplane with the maximal margin to separate two classes.

​

Artificial Immune System classifier: Biological immune System has been successful at protecting the human body against a vast variety of foreign pathogens. The role of the immune system is to protect our bodies from infectious agents such as viruses, bacteria. On the surface of these agents are antigens that allow the identification of the invading agents, thus provoking an immune response Recognition in the immune system is performed by lymphocytes. Each lymphocyte expresses receptor molecules of one particular shape on its surface called antibodies. An elaborate genetic mechanism involving combinatorial association of a number of gene segments underlies the construction of these receptors. The overall immune response involves three evolutionary methods: gene library, negative selection and clonal selection. In gene libraries, antibodies recognize antigens by the complementary properties that belong only to antigens. Thus, some knowledge of antigen properties is required to generate competent antibodies. Because of this evolutionary self-organization process, in spam management the gene libraries act as archives of information on how to detect commonly observed antigens. An important constraint that the immune has to satisfy is not to attack self cells. Negative selection eliminates inappropriate antibodies which bind to self. Clonal selection clones antibodies performing well. Thus, according to currently existing antigens, only the fittest antibodies survive. Similarly, instead of having the information about specific antigens, it organizes the fittest antibodies by interacting with the current antigens.

 

Rough sets classifier

​

The Rough set scheme follows following steps:

​

Step 1: With the incoming emails, first thing we need to do is to select the most appropriate attributes to use for classification. Then the input dataset is transformed into a decision system , which is then split into the training dataset and the testing dataset. A classifier will be induced from the training dataset and applied to the testing dataset to obtain performance estimation. For training dataset, do Step 2 and Step 3.

 

Step 2: Because the decision system has real values attributes, Boolean reasoning algorithms should be used to finish the discretization strategies.

 

Step 3: Genetic algorithms should be used to get the decision rules. Then For testing dataset, continue to Step 4.

 

Step 4: First, discretizes the testing dataset employing the same cuts computed from step 2. Then the rules generated in Step 3 are used to match every new object in the testing dataset to make a decision.

 

In case of spam detection the target variable contains the class labels which are listed below:

 

Spam - class: Spam is any kind of unwanted, unsolicited digital communication that gets sent out in bulk. And it's more than a nuisance. Spam today is a serious threat.

 

Non-Spam - class: This label represents all the non spam emails which are fine and can cause no threat.

 

Looking at these details, it comes under notice while looking at these algorithms is that all these algorithms are all supervised, this means there is always a target label available for us to train our model on.  Also when we look at details about the target class, they are only two classes. This shows that this problem could be represented as a binary classification problem. So what are binary classification problems? A short description of this is given below:

 

Binary Classification is one of the most common and frequently tackled problems in the machine learning domain. In it's simplest form the user tries to classify an entity into one of the two possible categories. For example, give the attributes of the fruits like weight, colour, peel texture, etc.