The basic problem from YAML#

This tutorial runs the basic two-discipline problem of Getting started from its YAML definition and walks through what DMDO prints and writes.

The problem#

\[\begin{split}\begin{array}{ll} \min_{u,v,w} & u + v + a + b \\[2pt] \text{s.t.} & a = \log_{10} u + \log_{10} v + \log_{10} b, \qquad b = u^{-1} + w^{-1} + a^{-1}, \\ & w + b - 10 \le 0, \qquad 0 \le u, v, w, a, b \le 10. \end{array}\end{split}\]

Subproblem 1 owns \(v\), computes \(a\) and holds the objective; subproblem 2 owns \(w\), computes \(b\) and holds the constraint; both keep copies of \(u\), \(a\) and \(b\) (see Background for the partitioned formulation).

The analyses#

examples/basic_disciplines.py writes the two analyses as Discipline objects (inputs and outputs keyed by variable name) and the subproblem objectives as plain functions:

import numpy as np


class Analysis1:
  """a = log10(u) + log10(v) + log10(b)"""

  def compute(self, inputs):
    u, v, b = (max(float(inputs[n]), 1e-12) for n in ("u", "v", "b"))
    return {"a": np.log10(u) + np.log10(v) + np.log10(b)}


class Analysis2:
  """b = 1/u + 1/w + 1/a"""

  def compute(self, inputs):
    u, w, a = (max(float(inputs[n]), 1e-12) for n in ("u", "w", "a"))
    return {"b": 1.0 / u + 1.0 / w + 1.0 / a}


def opt1(x, y, c=None):
  """Subproblem 1 objective: u + v + b + a (no constraint)."""
  return [sum(x) + y[0], [0.]]


def opt2(x, y, c=None):
  """Subproblem 2: no objective, constraint w + b - 10 <= 0."""
  return [0., [x[1] + y[0] - 10.]]


A1 = Analysis1()
A2 = Analysis2()

# Names used in basic_problem.yaml. Over the HTTP service, give them as import paths ("basic_disciplines:A1", ...).
BLACKBOXES = {"A1": A1, "A2": A2, "opt1": opt1, "opt2": opt2}

The problem definition#

# The basic MDO problem
variables:
# vs:  [name, SP_index, Link, coupling type, lb, bl, ub, dim]
  v0:  ["u",     1,       2,          s, 0., 1., 10.,  1, 'REAL']
  v1:  ["v",     1,    None,          u, 0., 1., 10.,  1, 'REAL']
  v2:  ["a",     1,       2,         ff, 0., 1., 10.,  1, 'REAL']
  v3:  ["b",     1,       2,         fb, 0., 1., 10.,  1, 'REAL']
  v4:  ["u",     2,       1,          s, 0., 1., 10.,  1, 'REAL']
  v5:  ["w",     2,    None,          u, 0., 1., 10.,  1, 'REAL']
  v6:  ["a",     2,       1,         fb, 0., 1., 10.,  1, 'REAL']
  v7:  ["b",     2,       1,         ff, 0., 1., 10.,  1, 'REAL']

DA:
  DA1:
    index: 1
    inputs: [v0, v1, v3]
    outputs: [v2]
    blackbox: A1
    type: callable
    links: 2
    coupling_type: ff

  DA2:
    index: 2
    inputs: [v4, v5, v6]
    outputs: [v7]
    blackbox: A2
    type: callable
    links: 2
    coupling_type: ff

MDA:
  MDA1:
    index: 1
    nAnalyses: 1
    analyses: [1]
    variables: [v0, v1, v3]
    responses: [v2]

  MDA2:
    index: 2
    nAnalyses: 1
    analyses: [2]
    variables: [v4, v5, v6]
    responses: [v7]

coord:
  c1:
    index: 1
    type: ADMM
    beta: 1.3
    gamma: 0.5
    nsp: 2
    budget: 50
    index_of_master_SP: 1
    display: True
    scaling: 0.1
    mode: serial
    M_update_scheme: normal
    store_q_io: True

subproblem:
  sp1:
    nv: 3
    index: 1
    vars: [v0, v1, v3]
    resps: [v2]
    is_main: True
    MDA: 1
    coordinator: 1
    opt: opt1
    type: callable
    fmin_nop: inf
    solver: poll
    budget: 150
    display: False
    psize: 1.
    pupdate: Last
    freal: 2.625

  sp2:
    nv: 3
    index: 2
    vars: [v4, v5, v6]
    resps: [v7]
    is_main: False
    MDA: 2
    coordinator: 1
    opt: opt2
    type: callable
    fmin_nop: inf
    solver: poll
    budget: 150
    display: False
    psize: 1.
    pupdate: Last

USER:


MDO:
  architecture: IDF
  coordinator: 1
  subproblems: [1, 2]
  responses: [v2, v7]
  fmin: inf
  hmin: inf
  display: True
  inc_stop: 0.0000000001
  stop: "Iteration budget exhausted"
  tab_inc: []
  noprogress_stop: 100

The highlighted lines are the settings that matter most:

  • every variable is declared REAL. OMADS optimizes variables of any other type on an integer lattice, so real variables must be declared with this exact keyword;

  • the coordinator uses \(\beta = 1.3\) and \(\gamma = 0.5\) with the normal update scheme, and scales the inconsistencies by 0.1 (one tenth, which is \(1/(u - \ell)\) for these bounds);

  • both subproblems are solved by the OMADS poll step with 150 evaluations per outer iteration.

Running it#

from basic_disciplines import BLACKBOXES
from DMDO import Session

if __name__ == "__main__":
    session = Session.from_yaml("basic_problem.yaml", working_dir="_runs", blackboxes=BLACKBOXES)
    result = session.run()
    print(result.status, "|", result.stop_reason)
    print(result.iterations, result.objective, result.max_inconsistency, result.feasible)

With display: True (the MDO section), DMDO prints one line per outer iteration: the maximum inconsistency qmax, the objective of the master subproblem, the change of the design vector dx and the largest penalty weight, followed by the link with the largest inconsistency.

   0 || qmax: 0.09043601 ||  Obj: 3.00000000 ||   dx: 7.04896213 || max(w): 1.30000000
Highest inconsistency : b_2 to b_1
   1 || qmax: 0.04141597 ||  Obj: 4.86020038 ||   dx: 1.13437264 || max(w): 1.69000000
Highest inconsistency : a_2 to a_1
   2 || qmax: 0.02897034 ||  Obj: 4.90302218 ||   dx: 0.21088849 || max(w): 2.19700000
Highest inconsistency : a_2 to a_1
...
  48 || qmax: 0.00000050 ||  Obj: 5.58977652 ||   dx: 0.00000506 || max(w): 4843785982.75789833
Highest inconsistency : b_2 to b_1
  49 || qmax: 0.00000031 ||  Obj: 5.58977066 ||   dx: 0.00000507 || max(w): 6296921777.58526802
Highest inconsistency : a_2 to a_1
finished | Iteration budget exhausted
50 5.589770658112853 3.051476282101007e-07 True

After 50 outer iterations the two copies of every shared and coupling variable agree to \(3\times10^{-7}\):

>>> {k: round(float(v), 5) for k, v in result.variables.items()}
{'u_1': 1.54957, 'v_1': 0.91759, 'a_1': 0.56132, 'b_1': 2.56129,
 'u_2': 1.54957, 'w_2': 7.43805, 'a_2': 0.56132, 'b_2': 2.56129}

These values match, to every digit shown, the recorded regression run of this problem (basic_poll in Benchmarking), which builds the same problem with the Python API instead of YAML.

The max(w) column shows the penalty weights growing on the links that stall, from 1.3 to about \(6 imes10^{9}\); this growth is what eventually forces the copies together.

The output files#

The run writes its outputs to _runs/basic_problem_post:

_runs/basic_problem_post/
├── Coordination_history.out          one row per outer iteration
├── DMDO.log                          the log of the run
├── coord_chk.pkl                     checkpoint for resuming the run
└── coordination_summary_post/
    ├── SP_1/
    │   ├── SP_1_0.out ... SP_1_49.out   every evaluation of subproblem 1, per outer iteration
    │   └── OMADS.log
    └── SP_2/ ...

Coordination_history.out is a comma-separated table with, for each outer iteration, the time, the iteration number, the maximum inconsistency, the objective, the feasibility status, the change of the design, the largest penalty weight, the link with the largest inconsistency, the constraint violation of each subproblem and the value of every variable copy. It can be loaded with pandas.read_csv():

import pandas as pd

history = pd.read_csv("_runs/basic_problem_post/Coordination_history.out", skipinitialspace=True)
history.columns = history.columns.str.strip()
history.plot(x="Iteration #", y="Max. inconsistency", logy=True)

Outputs describes every file and column.