Where is the research of regularization method?

Regularization means that in linear algebraic theory, ill-posed problems are usually defined by a set of linear algebraic equations, which are usually derived from ill-posed inverse problems with a large number of conditions. A large condition number means that rounding error or other errors will seriously affect the result of the problem.

The general method to solve ill-posed problems is to approximate the solution of the original problem with a group of well-posed problems "adjacent" to the original problem. This method is called regularization method. How to establish an effective regularization method is an important content of ill-posed problems in the field of inverse problems. The usual regularization methods include Tikhonov regularization based on variational principle, various iterative methods and other improved methods. These methods are all effective methods to solve ill-posed problems, and have been widely used and deeply studied in various inverse problems.

Regularization: Regularization, a concept in algebraic geometry.

Generally speaking,

It represents the plane irreducible algebraic curve with some form of holomorphic parameters.

That is, for the irreducible algebraic curve C in PC 2, we find a compact Riemannian surface C* and a holomorphic mapping σ: c *→ PC 2, so that σ (c *) = C

The strict definition is as follows

Let C be an irreducible plane algebraic curve and S be a set of singularities of C. If there is a compact Riemannian surface C* and a holomorphic mapping σ: c *→ PC 2, then

(1) σ (c *) = c (2) σ (-1) (s) is a finite point set (3) σ: c * \ σ (- 1) (s) → c \ s is a one-to-one mapping.

Then (C*, σ) is called the regularization of C. When there is no confusion, C* can also be called the regularization of C.

Regularization is actually to separate curves with different tangents at the singularity of irreducible plane algebraic curves, thus eliminating this singularity.

Main problems to be solved

1. Regularization is to impose a constraint on the function that minimizes the empirical error, which can be interpreted as prior knowledge (regularizing parameters is equivalent to introducing prior distribution to parameters). Constraints play a guiding role. When optimizing the error function, we tend to choose the gradient reduction direction that meets the constraints, so that the final solution tends to conform to the prior knowledge (such as the general l- norm prior, which indicates that the original problem is easier and simpler, and such optimization tends to produce solutions with small parameter values, which generally corresponds to smooth solutions with sparse parameters).

2. At the same time, regularization solves the ill-posed problem, and the obtained solution exists and only depends on the data. Noise has a weak influence on ill-posed problems, and the solution will not be over-fitted. In addition, if the prior (regularization) is appropriate, the solution will tend to conform to the real solution (not over-fitting), even if there are almost no irrelevant samples in the training set.