The Nelder-Mead method is an optimization algorithm from ’60s and can be used for optimizing multidimensional nonlinear unconstrained functions.
It is a direct method, meaning that it doesn’t use the function’s gradient during optimization. Furthermore, it is a pattern search method, meaning that it uses a pattern of points while searching for the extremum. More specifically, it uses a simplex of n+1 vertexes forming a geometrical pattern to explore the space problem, where n is the number of search space dimensions.
This was just some introduction. Enough with the algorithm logic. Let’s have some fun now!
Did you know that the algorithm sometimes is called also ‘Amoeba method’ ?
The reason is because the simplex geometrical pattern of the method loosely simulates an amoeba that searches for food (the extremum in our case). For example, an amoeba in 2D problems is formed by a triangle and in more dimensions can be more fine-grained. It is funny to see how it moves towards the extremum!
Here is an example of a 2D simplex amoeba evolving in search space to find the minimum of the Rosenbrock banana function.
Happy optimizations!
Efstathios Chatzikyriakidis 