Support Vector Machine With Practical Implementation

Februari 15, 2023Januari 18, 2023 falma.u99 implementation, machine, practical, support, vector

In this chapter, we will discuss the Support Vector Machine Algorithm which is used for both classification and Regression problem too and its supervised machine learning algorithm.

This article spans three parts:

1. What is Support Vector Machine?
2. How Does Support Vector Machine work?
3. Practical implementation of SVM in Scikit Learn.

The support vector machine is a powerful algorithm in a supervised machine learning algorithm.

It is used both for classification and regression problems. However, it mostly used in classification problems.

In SVM we separate a data point into class attribute using hyperplane to separate our data.

In this algorithm, we lot each data item as a point in n-dimensional space(where n is the number of features you have) with the value of a particular coordinate. Then we perform classification by finding the hyperplane that differentiates the two classes very well

In short Support, Vector is simply the co-ordinate of individual observation. SVM is a frontier that best segregates the two-class by hyper-plane line as shown above figure.

A vanilla SVM is a type of linear separator. Suppose we want to split black circles from the white ones above by drawing a line. Notice that there is an infinite number of lines that will accomplish this task. SVMs, in particular, find the “maximum-margin” line — this is the line “in the middle”. Intuitively, this works well because it allows for noise and is most tolerant of mistakes on either side.

That’s all a (binary) SVM rea We draw a straight line through our data down the middle to separate it into two classes. But what if we don’t want a straight line but a curved one? We achieve this not by drawing curves, but by “lifting” the features we observe into higher dimensions. For example, if we can’t draw a line in the space (x1, x2) then we may try adding a third dimension, (x1, x2, x1 * x2). If we project the “line” (actually called a “hyperplane”) in this higher dimension down to our original dimension, it looks like a curve.

2.1: How SVM work in linear problems?
Classification can be viewed as the task of separating classes in feature space.

Here are two classes the red class and blue class and the task is that find the boundary between these two classes and make a separate by hyperplane line.

In Support Vector Machine we make an optimal line to separate these two classes.

Here we select three support vectors to start with.

They are S1, S2, and S3.

Here we will use vectors augmented with a 1 as a bias input and for clarity, we will differentiate these with an over-tilde.

This is

Now we need to find 3 parameters α1, α2 and α3 based on the following 3 linear equations:

After simplification we get

Simplifying the above 3 simultaneous equation we get

α1= -3.25 α2= -3.25 and α3=3.5

The hyperplane that discriminates the positive class from negative class is given by:

This is the expected decision surface of the LSVM.

2.2: How SVM work in Non-linear problem?
We look at the linear type at part 1 of our SVM algorithm. Here we look at an example like the one given below and find out how to carry out the classification.

Obviously, there is no clear separating hyperplane between the red class and the blue class.

Here we need to find a non-linear mapping function fi that can transform these data into a new feature space where a separating hyperplane can be found.

Let’s consider the following mapping function.

Now let us transform the blue and red class vectors using the non-linear mapping function fi

After applying the mapping function we transform all the points as shown in below fig.

Now our task is to find suitable support vectors to classify these two classes

Here we select the following 3 support vectors:

Here we will use vectors augmented with a 1 as a bias input and for clarity, we will differentiate these with an over-tilde.

That is

Now we need to find 3 parameters α1, α2 and α3 based on the following 3 linear equations:

After simplification we get

Simplifying the above 3 simultaneous equation we get

α1= 0.859 α2= 0.859 and α3=-1.4219

The hyperplane that discriminates the positive class from negative class is given by:

Substituting the values we get:

Our Vectors are augmented with a bias.

Hence we can equate the entry in as the hyperplane with an offset b.

Therefore the separating hyperplane equation

This is the expected decision surface of the non LSVM.

Note: If you want this article check out my academia.edu profile.

3.1: SVM for Classification approach
Data Description:

To demonstrate the RFMTC marketing model (a modified version of RFM), this study adopted the donor database of Blood Transfusion Service Center in Hsin-Chu City in Taiwan. The center passes its blood transfusion service bus to one university in Hsin-Chu City to gather blood donated about every three months. To build an FRMTC model, we selected 748 donors at random from the donor database. These 748 donor data, each one included R (Recency — months since last donation), F (Frequency — total number of donation), M (Monetary — total blood donated in c.c.), T (Time — months since first donation), and a binary variable representing whether he/she donated blood in March 2007 (1 stand for donating blood; 0 stands for not donating blood).

Part 1: Data Preprocessing:

1.1 Import the Libraries

In this step, we import three Libraries in Data Preprocessing part. A library is a tool that you can use to make a specific job. First of all, we import the numpy library used for multidimensional array then import the pandas library used to import the dataset and in last we import matplotlib library used for plotting the graph.

1.2 Import the dataset

In this step, we import the dataset to do that we use the pandas library. After import our dataset we define our Predictor and target attribute. we call ‘X’ predictor here and target attribute which we call ‘y’ here.

1.3 Split the dataset for test and train

In this step, we split our dataset into a test set and train set and an 80% dataset split for training and the remaining 20% for tests.

1.4 Feature Scaling

Feature Scaling is the most important part of data preprocessing. If we see our dataset then some attribute contains information in Numeric value some value very high and some are very low if we see the age and estimated salary. This will cause some issues in our machinery model to solve that problem we set all values on the same scale there are two methods to solve that problem first one is Normalize and Second is Standard Scaler.

Here we use standard Scaler import from Sklearn Library.

Part 2: Building the SVM model:

In this part, we model our model using Scikit Learn Library.

2.1 Import the Libraries

In this step, we are building our model to do this first we import a model from Scikit Learn Library.

2.2 Initialize our SVM model

In this step, we initialize our model and in the SVM classifier model take the linear kernel.

2.3 Fitting the Model

In this step, we fit the training data into our model X_train, y_train is our training data.

Part 3: Making the Prediction and accuracy result:

In this Part, we make a prediction of our test set dataset and visualizing the result using the matplotlib library.

3.1 Predict the test set Result

In this step, we predict our test set result.

3.2 Confusion Metric

In this step we make a confusion metric of our test set result to do that we import confusion matrix from sklearn.metrics then in confusion matrix, we pass two parameters first is y_test which is the actual test set result and second is y_pred which predicted result.

3.3 Accuracy Score

In this step, we calculate the accuracy score based on the actual test result and predict test results.

If you want dataset and code you also check my Github Profile.

3.2: SVM for Regression approach
Dataset Description:
The aids data frame has 570 rows and 6 columns. Although all cases of AIDS in England and Wales must be reported to the Communicable Disease Surveillance Centre, there is often a considerable delay between the time of diagnosis and the time that it is reported. In estimating the prevalence of AIDS, the account must be taken of the unknown number of cases that have been diagnosed but not reported. The data set here records the reported cases of AIDS diagnosed from July 1983 and until the end of 1992. The data are cross-classified by the date of diagnosis and the time delay in the reporting of the cases.

Part 1: Data Preprocessing:

1.1 Import the Libraries

1.2 Import the dataset

1.3 Split the dataset for test and train

In this step, we split our dataset into a test set and train set and an 80% dataset split for training and the remaining 20% for tests.

1.4 Feature Scaling

Here we use standard Scaler import from Sklearn Library.

Part 2: Building the SVM Regression model:

In this part, we model our model using Scikit Learn Library.

2.1 Import the Libraries

In this step, we are building our model to do this first we import a model from Scikit Learn Library.

2.2 Initialize our model

In this step, we initialize our model our decision tree regressor model. So here RBF is Gaussian kernel.

2.3 Fitting the Model

In this step, we fit the training data into our model X_train, y_train is our training data.

Part 3: Making the Prediction and Visualizing the result:

In this Part, we make a prediction of our test set dataset and visualizing the result using the matplotlib library.

3.1 Predict the test set Result

In this step, we predict our test set result.

3.2 Visualize our Result

In step, we make a visualizing of our test dataset.

If you want dataset and code you also check my Github Profile.

If you liked this article, be sure to click ❤ below to recommend it and if you have any questions, leave a comment and I will do my best to answer.

For being more aware of the world of machine learning, follow me. It’s the best way to find out when I write more articles like this.

You can also follow me on Github for code & dataset follow on Aacademia.edu for this article, Twitter and Email me directly or find me on LinkedIn. I’d love to hear from you.

That’s all folks, Have a nice day 🙂