Accuracy in machine learning classification model (classifier) is a measure of how frequently each classification is correctly deemed positive or negative. Accuracy is calculated by the following mathematical formula accuracy = (true positives + true negatives) / (all estimated values) Use the following reference for some good visual examples of accuracy, precision and recall: https://www.evidentlyai.com/classification-metrics/accuracy-precision-recall.
Additive functions are functions which satisfy the condition f(x+y) = f(x) + f(y) for any x and y. Additive functions are linear functions.
The coefficient of determination (also depicted as R^2) is a calculation of the proportion of the variance in the dependent variable (noted as y) which is explained by a linear regression model. The value of the coefficient of determination can be in the 0..1 range. The coefficient of determination can be calculated by using the ... Read more
A convex function is a function which features a single global minimum, whereas a non-convex functions presents many local minima. A good analysis on the mathematical aspects and mathematical definition of convex and non-convex functions can be found at: https://rumn.medium.com/convex-vs-non-convex-functions-why-it-matters-in-optimization-for-machine-learning-39cd9427dfcc. Some examples of ML algorithms which have restrictions related to convex functions are the following: ... Read more
F1 score is the weighted average (harmonic mean) of precision and recall. The F1 score is calculated by the following formula: F1 score = (2 x Precision x Recall)/(Precision + Recall) The more the precision and recall metrics deviate from each other, the worse their harmonic mean (i.e. the F1 score).
Logit-transformation (aka as the odds ratio) is a logarithmic transformation which is used to transform proportional values into continues values in ML regression problems. The logit-transformation is defined by the following mathematical formula: logit(p)=log(p/(1-p)).
MAE in statistics and Machine Learning (ML) stands for the Mean Absolute Error. MAE is the average of the sum of the differences between the actual and predicted values in a dataset. In other words the MAE is the calculation of the the average of the residuals. MAE is expressed by the following mathematical formula. ... Read more
MSE in Machine Learning (ML) stands for Mean Squared Error and is an error calculation formula. MSE calculates the average value of the square power of the sum of differences between the original and predicted values in a dataset. It is similar to MAE, in that MSE is a calculation for the variance of residuals, ... Read more
In machine learning, normalization is a statistical technique by which the data in a dataset are transformed to have values in a normal (Gaussian) distribution. For each value x in the dataset, its corresponding normalized value x' is calculated in the value range [0,1] as follows. Alternatively, there can be a mean normalization, with normalized ... Read more
Machine learning can be summarized as learning a function (f) that maps input variables (X) to output variables (Y), such as Y = f(x). An algorithm learns this target mapping function from training data. The form of the function is unknown, so our job as machine learning practitioners is to evaluate different machine learning algorithms ... Read more
In statistics and machine learning, precision is a measure of how often the positives identified by a learning model are true positives. This is a division of true positives (based on the confusion matrix) by all estimated positives (=true positives + false positives). The precision metric is commonly used in conjunction with recall, to evaluate ... Read more
Recall (also known as sensitivity) is the ratio of true positives (based on the confusion matrix) by all positives (=true positives + false negatives). It is commonly used in conjunction with precision and it is needed when we must minimize false negatives. Recall can be considered the opposite metric of specificity. Recall is a measure ... Read more
In machine learning, regularization is a method by which the ML model cost/error function is changed, to include an extra variable called the regularization hyperparameter. There are two basic types of regularization: L1-norm (lasso regression) and L2-norm (ridge regression). The lasso regularization uses the L1 norm parameter. The lasso regularized cost function is calculated as ... Read more
The ReLU (rectified linear unit function) function is an ANN activation function which calculates a linear function of the inputs. If the result is positive, it outputs that result. If it is negative, it outputs 0. The mathematical formula for the ReLU function is f (x) = max(0, x). The graph of the ReLU function ... Read more
RMSE is simply the root of the MSE statistical metric. RMSE stands for Root Mean Squared Error. The RMSE is in the same metric scale as the observed parameters, same as the MAE metric. RMSE is a calculation for the standard deviation of residuals. Compared to RMSE, MSE is a calculation of the variance of ... Read more
In machine learning, standardization is a feature engineering technique by which the dataset features are re-scaled to achieve zero-mean value (μ=0) and unit standard deviation value (σ=1). Each x value in the dataset gets a corresponding x' standardized value, which is calculated as follows. , where μ is the x variable mean and σ is ... Read more
The tanh function, also known as the hyperbolic tangent function, is an activation function in artificial neural networks whose output values are constrained between the values of −1 and 1. The following screenshot provides a graph of function f(x)=tanh(x), as output from the Geogebra free online calculator.