Optimization

在computer science中，optimization是一类非常重要的问题。从wikipedia Mathematical optimization的内容可知，optimization是一个非常宏大的问题。作为software engineer，我们需要关注的是algorithm implementation。

wikipedia Mathematical optimization

Mathematical optimization (alternatively spelled optimisation) or mathematical programming is the selection of a best element (with regard to some criterion) from some set of available alternatives.[1] Optimization problems of sorts arise in all quantitative disciplines from computer science and engineering to operations research and economics, and the development of solution methods has been of interest in mathematics for centuries.[2]

In the simplest case, an optimization problem consists of maximizing or minimizing a real function by systematically choosing input values from within an allowed set and computing the value of the function. The generalization of optimization theory and techniques to other formulations constitutes a large area of applied mathematics. More generally, optimization includes finding "best available" values of some objective function given a defined domain (or input), including a variety of different types of objective functions and different types of domains.

Optimization problems

NOTE: 原文这一段，主要是参考维基百科 Optimization problem

Computational optimization techniques

NOTE: 原文这一段描述的是如何使用program来解决optimization问题，这是我们作为programmer需要重点学习的。

To solve problems, researchers may use algorithms that terminate in a finite number of steps, or iterative methods that converge to a solution (on some specified class of problems), or heuristics that may provide approximate solutions to some problems (although their iterates need not converge).

NOTE: 很多optimization，无法使用program求得最优值，而是求得一个近似值

Classification of optimization problem

以 wikipedia Optimization problem 中描述的分类方法为主，然后补充wikipedia Mathematical optimization （Optimization**:** Algorithms, methods, and heuristics ）中提供的各种更加具体的类别。

wikipedia Optimization problem

In mathematics and computer science, an optimization problem is the problem of finding the best solution from all feasible solutions.

Optimization problems can be divided into two categories depending on whether the variables are continuous or discrete.

• An optimization problem with discrete variables is known as a discrete optimization. In a discrete optimization problem, we are looking for an object such as an integer, permutation or graph from a countable set.
• A problem with continuous variables is known as a continuous optimization, in which an optimal value from a continuous function must be found. They can include constrained problems and multimodal problems.

NOTE: 在数学中，根据discrete、continuous来进行分类的方式普遍存在，比如：

Continuous optimization problem

The standard form of a continuous optimization problem is[1]

where

• $ f:\mathbb {R} ^{n}\to \mathbb {R} $ is the objective function to be minimized over the n-variable vector $ x $,
• $ g_{i}(x)\leq 0 $ are called inequality constraints
• $ h_{j}(x)=0 $ are called equality constraints, and
• $ m\geq 0 and p\geq 0 $.

If $ m $ and $ p $ equal 0, the problem is an unconstrained optimization problem. By convention, the standard form defines a minimization problem. A maximization problem can be treated by negating the objective function.

Combinatorial optimization problem

Main article: Combinatorial optimization

Formally, a combinatorial optimization problem $ A $ is a quadruple[citation needed] $ (I,f,m,g) $, where

• $ I $ is a set of instances;
• given an instance $ x\in I $, $ f(x) $ is the set of feasible solutions;
• given an instance $ x $ and a feasible solution $ y $ of $ x $, $ m(x,y) $ denotes the measure of $ y $, which is usually a positive real.
• $ g $ is the goal function, and is either $ \min $ or $ \max $.

The goal is then to find for some instance $ x $ an optimal solution, that is, a feasible solution $ y $ with $$ m(x,y)=g{\bigl {}m(x,y')\mid y'\in f(x){\bigr }} $$ For each combinatorial optimization problem, there is a corresponding decision problem that asks whether there is a feasible solution for some particular measure $ m_{0} $. For example, if there is a graph $ G $ which contains vertices $ u $ and $ v $, an optimization problem might be "find a path from $ u $ to $ v $ that uses the fewest edges". This problem might have an answer of, say, 4. A corresponding decision problem would be "is there a path from $ u $ to $ v $ that uses 10 or fewer edges?" This problem can be answered with a simple 'yes' or 'no'.

Summary

类别		子类别	方法论
Continuous optimization problem	- 维基百科 Continuous optimization	- Unconstrained nonlinear - Constrained nonlinear - Convex optimization	比较典型的有: - Iterative method
Combinatorial optimization problem	- 维基百科 Discrete optimization - 维基百科 Combinatorial optimization		比较典型的有: - relation-based algorithm model

方法论

一般，在实践中， 上述两种类别的optimization problem采用的是不同的计算方法。

一般要我们解决的optimization问题中往往只包含一个最值，如：

• All nearest smaller values的最值是nearest
• Maximum subarray problem的最值是largest

所以在求解最值问题的时候，一个非常重要的前提是要搞清楚它的最值是什么。

其次，我们往往是基于**比较**（打擂台）来求解最值，但是有有些是可以直接进行比较的，比如数值，但是有些是无法直接进行比较的，比如在All nearest smaller values中最值是nearest，除非记录每个元素的位置，否则只能够借助一个stack来实现nearest；