On the choice of basic regression functions and machine learning

Abstract

As is known, the regression analysis task is widely used in machine learning problems, which allows to establish relationship between observed data and compactly store of information. Most often, a regression function is described by a linear combination of some of the selected functions fj (X), j = 1, . . . ,m, X 2 D ⊂ Rs. If the observed data contains a random error, then the regression function restored from the observed data contains a random error and a systematic error depending on the selected functions fj . The article indicates the possibility of optimal selection of functions fj in the sense of a given functional metric, if it is known that the true dependence is consistent with some functional equation. In some cases (regular grids, s ≤ 2), similar results can be obtained using the random process analysis method. The numerical examples given in this article illustrate much more opportunities for the task of constructing the regression function.