Machine Learning

Support Vector Machine (SVM) algorithm can be used for solving following kinds of classification problems:

Binary classification: We can use SVM to create a binary classifier. In SVM, we can find the hyperplane to divide data into two distinct classes.

Multi-class classification: SVM algorithm works well to divide data into multiple classes. In this case, we have to determine multiple hyperplanes that separate the classes. In certain cases classes are not distinguished by a linear function. We can use kernel trick to transform low dimensional input space to higher-order dimensional space. E.g. we can use circle form to distinguish classes.

Support Vector Machine (SVM) algorithm has following pros and cons:

Pros:

• Clear: SVM provides a clear separation of data into classes. It works well and provides clear margin of separation between classes.



• High dimension: SVM is effective in classifying data in high dimensional space.



• Number of dimension: SVM can effectively classify data that has more number of dimensions than the number of samples.



• Memory: SVM uses a subset of training points in support vectors (decision function). Therefore it is an efficient algorithm from memory perspective.

 

• Cons:



• Low performance: SVM has low performance on a large data set, because it requires more training time to calculate and identify support vectors.



• Noise: SVM cannot perform well when classes are not distinct. High amount of noise or overlapping classes bring down the accuracy of SVM classification.



• Probability: We don’t get probability estimates with SVM.

In Machine learning, we can use multiple models and algorithms to get better performance in predictions. This approach of using multiple models is known as ensemble learning.

Ensemble learning is a form of supervised learning. We try multiple algorithms and combine their results to get the level of performance acceptable for our problem.

In general, we get better performance from ensemble learning when diverse models are used. Therefore, certain ensemble methods promote the use of diverse models in learning.

Some of the popular types of Ensemble learning methods are as follows:

Bayes optimal classifier

Bootstrap aggregating (bagging)

Boosting

Bayesian parameter averaging

Bayesian model combination

Bucket of models

Stacking

We use stacking as an ensemble learning method in machine learning. It is also known as stacked generalization.

It is a learning algorithm that combines the results of other algorithms to give a better predictive performance.

First we use different types of learning algorithms on original data for prediction. Then we create a combiner algorithm that provides final prediction by using the prediction results of algorithms used in the first step.

One variation of stacking is known as blending. It was used in a famous Netflix prediction competition.

In general, stacking gives better performance than Bayesian model averaging.