Tutorials
#########
This section demonstrates how to use OMADS to solve optimization problems.

Toy problems
=======

The following subsections show how to use OMADS for solving selected 2D test functions.

2D-Rosenbrock
------

.. math::
   
   f(x_{1}, x_{2})=(1-x_{1})^{2}+100(x_{2}-x_{1}^{2})^{2}.

Rosenbrock function has curved contours, where its global minimum is in a parabolic banana-type valley. For some optimization algorithms, it is easy to find that region, but it is hard to find the global minimum as the function values in the valley do not palpably change.

The problem formulation can be introduced to OMADS as follows:

.. code-block:: python

   from OMADS import main
   from typing import Callable

   def rosen(x):
      x = np.asarray(x)
      y = [np.sum(100.0 * (x[1:] - x[:-1] ** 2.0) ** 2.0 + (1 - x[:-1]) ** 2.0,
            axis=0), [0]]
      return y

   x0 = [-2.0,-2.0]
   fun: Callable = rosen
   eval = {"blackbox": fun}
   param = {"baseline": x0,
               "lb": [-2.5, 1.],
               "ub": [2.5, 3.],
               "var_names": ["theta", "p"],
               "scaling": [5.0, 4],
               "post_dir": "./post"}
   options = {"seed": 0, "budget": 100000, "tol": 1e-12, "display": True}

   data = {"evaluator": eval, "param": param, "options": options}

   main(data)

The following summary is printed after running the previous snippet:

.. code-block::

    ---Run Summary---
   Run completed in 1.2768 seconds
   Random numbers generator's seed 0
   xmin = [0.9999930866631639, 0.9999860011687706]
   hmin = 1e-30
   fmin = 5.075969451541405e-11
   #bb_eval = 9997
   #iteration = 2500
   nb_success = 1286
   psize = 9.5367431640625e-07
   psize_success = 4.76837158203125e-07
   psize_max = 2.0

The following video illustrates the poll trajectory, frame size, and mesh coarseness during the optimization process.

.. video:: /Users/ahmedb/apps/code/Bay_dev/OMADS/docs/source/Figures/RB.mp4
   :width: 650
   :height: 400
   :autoplay:

2D-Alpine
------

.. math::
   
   f({\bf x})= \sum_{i=1}^{d} |x_{i} \sin(x_{i})+0.1x_{i}|.

The alpine function has non-smooth characteristics that might hinder some gradient-based algorithms from hitting the global minimum point.

The function has the following design space characteristics: non-smooth (non-continuous), non-convex, separable, differentiable, multimodal, non-random, and non-parametric.

The problem formulation can be introduced to OMADS as follows:

.. code-block:: python

   from OMADS import main
   from typing import Callable

   def alpine(x):
      y = [abs(x[0]*np.sin(x[0])+0.1*x[0])+abs(x[1]*np.sin(x[1])+0.1*x[1]), [0]]
      return y
   
   x0 = [5., 5.]
   fun: Callable = alpine
   eval = {"blackbox": fun}
   param = {"baseline": x0,
               "lb": [0., 0.],
               "ub": [10, 10.],
               "var_names": ["theta", "p"],
               "scaling": 10.0,
               "post_dir": "./post"}
   options = {"seed": 0, "budget": 100000, "tol": 1e-12, "display": True}

   data = {"evaluator": eval, "param": param, "options": options}

   main(data)

The following summary is printed after running the previous snippet:

.. code-block::

   ---Run Summary---
   Run completed in 0.0093 seconds
   Random numbers generator's seed 0
   xmin = [0.0, 0.0]
   hmin = 1e-30
   fmin = 0.0
   #bb_eval = 193
   #iteration = 48
   nb_success = 5
   psize = 9.094947017729282e-13
   psize_success = 8.0
   psize_max = 8.0

The following video illustrates the poll trajectory, frame size, and mesh coarseness during the optimization process.

.. video:: /Users/ahmedb/apps/code/Bay_dev/OMADS/docs/source/Figures/alpine.mp4
   :width: 650
   :height: 400
   :autoplay:

2D-Ackley 3
------

 Ackley is characterized by a noisy flat outer region and a large hole at the center. The function poses a risk for optimization algorithms, particularly hill climbing algorithms, to be trapped in one of its suboptimal solutions. Ackley-3 is the non-smooth version of Ackley
.. math::
   
   f(x_{1}, x_{2})= -200 \exp^{(-0.2 \sqrt{x^{2}_{1} + x^{2}_{2}})} + 5\exp^{(\cos(3x_{1})+\sin(3x_{2}))}.

The function has the following design space characteristics: non-continuous, non-convex, non-separable, differentiable, multimodal, non-random, non-parametric

The problem formulation can be introduced to OMADS as follows:

.. code-block:: python

   from OMADS import main
   from typing import Callable

   def Ackley3(x):
      return [-200*np.exp(-0.2*np.sqrt(x[0]**2+x[1]**2))+5*np.exp(np.cos(3*x[0])+np.sin(3*x[1])), [0]]
   
   x0 = [5., 5.]
   fun: Callable = Ackley3
   eval = {"blackbox": fun}
   param = {"baseline": x0,
               "lb": [-32., -32.],
               "ub": [32, 32.],
               "var_names": ["theta", "p"],
               "scaling": 64.0,
               "post_dir": "./post"}
   options = {"seed": 0, "budget": 100000, "tol": 1e-12, "display": True}

   data = {"evaluator": eval, "param": param, "options": options}

   main(data)

The following summary is printed after running the previous snippet:

.. code-block::

   ---Run Summary---
   Run completed in 0.0173 seconds
   Random numbers generator's seed 0
   xmin = [0.0, -0.006773456931114197]
   hmin = 1e-30
   fmin = -186.4112127112689
   #bb_eval = 265
   #iteration = 66
   nb_success = 13
   psize = 9.094947017729282e-13
   psize_success = 1.4901161193847656e-08


The following video illustrates the poll trajectory, frame size, and mesh coarseness during the optimization process.


.. video:: /Users/ahmedb/apps/code/Bay_dev/OMADS/docs/source/Figures/Ackley3.mp4
   :width: 650
   :height: 400
   :autoplay:

2D-Egg holder
-----

The Eggholder function is often used as a benchmark for optimization algorithms. Finding the single global minimum of this function is challenging due to the function's non-smoothness and design space characteristics which give it the egg-holder shape:

.. math::
   
   f(x_{1}, x_{2})= -(x_{2}+47)(\sin(\sqrt{|x_{2}+0.5x_{2}+47|}))-x\sin(\sqrt{|x_{1}-(x_{2}+47)|}).


The function has the following design space characteristics: non-continuous, non-convex, non-separable, differentiable, multimodal, non-random, non-parametric

The problem formulation can be introduced to OMADS as follows:

.. code-block:: python

   from OMADS import main
   from typing import Callable

   def eggHolder(individual):
      x = individual[0]
      y = individual[1]
      f = (-(y + 47.0) * np.sin(np.sqrt(abs(x/2.0 + (y + 47.0)))) - x * np.sin(np.sqrt(abs(x - (y + 47.0)))))
      return [f, [0]]

   x0 = [50., 280.]
   fun: Callable = eggHolder
   eval = {"blackbox": fun}
   param = {"baseline": x0,
               "lb": [-512., -512.],
               "ub": [512., 512.],
               "var_names": ["theta", "p"],
               "scaling": 1024.0,
               "post_dir": "./post"}
   options = {"seed": 0, "budget": 100000, "tol": 1e-12, "display": True, "psize_init": 512, "rich_direction": True}

   data = {"evaluator": eval, "param": param, "options": options}

   main(data)

The following summary is printed after running the previous snippet:

.. code-block::

   ---Run Summary---
   Run completed in 0.0210 seconds
   Random numbers generator's seed 0
   xmin = [512.0, 404.23180526795477]
   hmin = 1e-30
   fmin = -959.640662720851
   #bb_eval = 381
   #iteration = 95
   nb_success = 24
   psize = 9.094947017729282e-13
   psize_success = 5.820766091346741e-11
   psize_max = 512.0


The following video illustrates the poll trajectory, frame size, and mesh coarseness during the optimization process.


.. video:: /Users/ahmedb/apps/code/Bay_dev/OMADS/docs/source/Figures/EggHolder.mp4
   :width: 650
   :height: 400
   :autoplay:


Blackbox evaluation
==========

In this tutorial, we walk the reader through some useful options that help run blackbox executables efficiently. 

User parameters
-----

Some use cases require introducing constant parameters to the evaluator callable function. In this case, it is recommended to add those constants in the ``constants`` key, which is of type ``List``, under the ``evaluator`` dictionary. The parameters index in the constants list should follow the same indices used by the callable function. The callable function can have an extra open input arguments' field ``*argv``, but that pointer should be handled carefully within the callable function to avoid error evaluations. The first input argument is reserved for variables, and the second one is reserved for the ``constants`` list (if defined). The following example highlights how to define user parameters.

.. code-block:: python

   from OMADS import main
   from typing import Callable
   import numpy as np

   def rosen(x, *argv):
      x = np.asarray(x)
      p = argv[0]
      y = [np.sum(p[0] * (x[1:] - x[:-1] ** p[1]) ** p[1] + (1 - x[:-1]) ** p[1],
            axis=0), [0]]
      return y

   x0 = [-2.0,-2.0]
   fun: Callable = rosen
   eval = {"blackbox": fun, "constants": [100., 2.]}
   param = {"baseline": x0,
               "lb": [-2.5, 1.],
               "ub": [2.5, 3.],
               "var_names": ["theta", "p"],
               "scaling": [5.0, 4],
               "post_dir": "./post"}
   options = {"seed": 0, "budget": 100000, "tol": 1e-12, "display": True}

   data = {"evaluator": eval, "param": param, "options": options}

   main(data)


Serial mode
-----

Serial mode is the default running option for blackbox evaluation in ``OMADS``. If the ``parallel_mode`` boolean is set to ``False``, as shown below, the evaluator will execute the blackbox in serial mode, regardless of the number of processes defined ``np``.

.. code-block:: JASON

   "options":
    {
      "seed": 0,
      "budget": 10000,
      "tol": 1e-12,
      "psize_init": 1,
      "display": true,
      "opportunistic": false,
      "check_cache": true,
      "store_cache": true,
      "collect_y": false,
      "rich_direction": true,
      "precision": "high",
      "save_results": false,
      "save_coordinates": false,
      "save_all_best": false,
      "parallel_mode": false,
      "np": 4
    }

Parallel mode
-----

The latest version of ``OMADS``, ``OMADS-1.5.0``, supports parallel multiprocessing for poll points evaluation. This is a practical and recommended option when the blackbox is computationally expensive. To enable parallel execution, set the ``parallel_mode`` boolean to ``True`` and set the number of processes ``np`` key with a valid integer. The value of ``np`` should be less than or equal to the maximum number of cores on the local running CPU; otherwise, ``OMADS`` will set it to an appropriate value that fits well with the running CPU. The following is an example of the ``options`` dictionary for parallel execution.

.. code-block:: JASON

   "options":
    {
      "seed": 0,
      "budget": 10000,
      "tol": 1e-12,
      "psize_init": 1,
      "display": true,
      "opportunistic": false,
      "check_cache": true,
      "store_cache": true,
      "collect_y": false,
      "rich_direction": true,
      "precision": "high",
      "save_results": false,
      "save_coordinates": false,
      "save_all_best": false,
      "parallel_mode": true,
      "np": 4
    }



Results post-processing
========

Results are post-processed by saving the design history in a ``CSV`` file and/or saving the coordinates of the poll points in a ``JASON`` file which can be later used to visualize the spinner trajectory in 2D space.

Results file
----

Typically, results files are saved in the post directory defined in the ``post_dir`` key under the parameter dictionary as shown below:

.. code-block:: JASON

   "param":
    {
      "baseline": [-2.0,-2.0],
      "lb": [-2.5, -1],
      "ub": [2.5, 3],
      "var_names": ["x1", "x2"],
      "scaling": [5.0, 4],
      "post_dir": "./tests/bm/unconstrained/post"
    },

However, the ``save_results`` boolean key under the ``options`` dictionary should be set to ``True`` to populate the results to the ``CSV`` file. The ``CSV`` file is named ``MADS.csv``. The following table is an example to how the file looks like:

.. csv-table:: An example for the ``OMADS`` results table
   :file: /Users/ahmedb/apps/code/Bay_dev/OMADS/docs/source/table_1.csv
   :header-rows: 1
   

Coordinates ``JSON`` file
-----

This file holds the coordinates of all the evaluated poll points at each iteration. The file is saved in the following ``JSON`` format:

.. code-block:: JSON

   {
    "1": {
        "coord": [
            [
                -2.7618030000266476,
                -0.5
            ],
            [
                -0.5,
                -1.238196999973352
            ],
            [
                -1.2381969999733524,
                -3.5
            ],
            [
                -3.5,
                -2.761803000026648
            ]
        ],
        "iter": 1,
        "x_incumbent": [
            -2.0,
            -2.0
        ]
    },
    "2": {
        "coord": [
            [
                -2.288087006062905,
                -4.238196999973352
            ],
            [
                2.5,
                -3.026284006036258
            ],
            [
                1.2880870060629048,
                1.761803000026648
            ],
            [
                -3.5,
                0.549890006089554
            ]
        ],
        "iter": 2,
        "x_incumbent": [
            -0.5,
            -1.238196999973352
        ]
    },