Estimator (Algorithm) Task Description Relevant studies
Linear regressions Regression The simplest form of regression models. Attempts to minimize the mean squared error (MSE) between actual target values and the target values estimated by fitting a linear equation to the training set. [114, 120, 121, 123]
k-nearest neighbors (KNN)
Classification Regression
A simple ML algorithm that stores k nearest neighbor samples in a dataset (k=1, 2, 3, …) based on the feature similarities.
[81, 108, 116]
Support vector machines (SVMs)
Classification Regression
The objective of SVM is to transform each data in an n-dimensional space (n: number of features) and separate data points into two categories in a manner that maximizes the width of the gap between the nearest observations. SVMs are divided into two main groups: Support vector regressors (SVRs) and support vector classifiers (SVCs).
[103, 104, 115, 116, 119]
Decision trees and random forest
Classification Regression Decision trees provide a tree-shaped structure, including nodes and branches. The algorithm makes decisions based on a hierarchy of if/else questions. Each node classifies the input depending on the question, and the branches are the final decisions representing the output. Random forest is an ensemble method that collects many randomly made decision trees. This algorithm aims to overcome the limitations of each individual decision tree and averages their results.
[106, 115, 116]
Gaussian process (GP)
Classification Regression
GP is a probabilistic non-parametric method that aims to make predictions and provide uncertainty information on the estimations based on Bayes’ rule. Gaussian process classification (GPC) and Gaussian process regression (GPR) are two main groups of GP.
[107]
Fuzzy logic (FL) Regression FL resembles the pattern of human reasoning for solving problems considering all available possibilities between Yes and No. [105]
Artificial neural networks (ANNs)
Classification Regression
ANNs are one of the most popular ML algorithms which are inspired by the human brain information process. ANNs are consist of interconnected neurons arranged in an input layer, a series of hidden layers, and an output layer. In these algorithms, each neuron makes decisions and gives it to other neurons in the next layer. The weighted sum of the inputs is calculated by an activation function. Then, the procedure tends to update the weights in order to minimize the prediction errors.
[99-102, 106, 115, 117, 122]