On the principle of empirical risk minimization based on averaging aggregation functions

Abstract

An extended version of the principle of empirical risk minimization is proposed. It is based on the application of averaging aggregation functions, rather than arithmetic means, to compute empirical risk. This is justified if the distribution of losses has outliers or is substantially distorted, which results in that the risk estimate becomes biased from the very beginning. In this case, for optimizing parameters, a robust estimate of the mean risk should be used. Such estimates can be constructed by using averaging aggregation functions, which are the solutions of the problem of minimizing the function of penalty for deviation from the mean value. An iterative reweighting scheme for numerically solving the problem of empirical risk minimization is proposed. Illustrative examples of the construction of a robust procedure for estimating parameters in the linear regression problem and in the problem of linearly separating two classes based on the application of an averaging mean function, which replaces the α-quantile, are given.