Customize a Routine Without Touching the Codebase#

This notebook walks through the supported pattern for adapting a built-in routine (here DCOPF) at runtime — adding parameters, services, variables, constraints, and extending the objective — using only public API on the System instance.

The notebook also demonstrates two safety properties of the codegen pipeline:

  1. Customizations stay per-instance. Mutations on sp2 do not leak into sp1 or sp3, even though all three share the same disk-cached pycode at ~/.ams/pycode/dcopf.py.

  2. You can see which path each item runs through. The routine.formulation_summary() API tells you exactly what’s in effect — codegen (fast AOT path) or eval (the eval-fallback helper, used for runtime customizations).

Architecture in one paragraph#

e_str is the authoring DSL. At first init() the codegen (ams.prep.generate_for_routine) compiles each routine’s e_str from a pristine source instance, never from the user’s sp, into named callables that live in ~/.ams/pycode/<routine>.py. The runtime then either wires those callables onto items (the codegen path) or, for items the user customized at runtime, falls back to a single eval-fallback helper (ams.opt._runtime_eval.eval_e_str) that resolves bare symbol names through RoutineNS. Both paths produce the same CVXPY object given the same e_str.

[1]:

import numpy as np
import ams

ams.config_logger(stream_level=20)  # show INFO so the per-init formulation line prints

sp1 — DCOPF, untouched, solve#

[2]:

sp1 = ams.load(ams.get_case('5bus/pjm5bus_demo.xlsx'),
                setup=True, no_output=True, default_config=True)
sp1.DCOPF.run(solver='CLARABEL')

Working directory: "/Users/jinningwang/work/ams/examples"
Parsing input file "/Users/jinningwang/work/ams/ams/cases/5bus/pjm5bus_demo.xlsx"...
Input file parsed in 0.1174 seconds.
Zero Line parameters detected, adjusted to default values: rate_b, rate_c.
All bus type are PQ, adjusted given load and generator connection status.
System set up in 0.0019 seconds.
Entering data check for <DCOPF>
 -> Data check passed
Building system matrices
<DCOPF> formulation: codegen=15/15
Parsing OModel for <DCOPF>
Evaluating OModel for <DCOPF>
Finalizing OModel for <DCOPF>
<DCOPF> initialized in 0.0416 seconds.
<DCOPF> solved as optimal in 0.0073 seconds, converged in 9 iterations with CLARABEL.

[2]:

True

[3]:

print('pg  =', sp1.DCOPF.pg.v)
print('obj =', float(sp1.DCOPF.obj.v))

pg  = [0.2        1.43998388 0.6        5.76001612 2.        ]
obj = 9.535953244734864

[4]:

sp1.DCOPF.formulation_summary()

<DCOPF> formulation summary (15 codegen / 0 eval / 0 manual / 0 pending)
  kind      name   source   e_str
  --------  -----  -------  ----------------------------------------
  expr      plf    codegen  Bf@aBus + Pfinj
  expr      pmaxe  codegen  cp.multiply(nctrle, pg0) + cp.multiply(ctrle, pmax)
  expr      pmine  codegen  cp.multiply(nctrle, pg0) + cp.multiply(ctrle, pmin)
  constr    pb     codegen  Bbus@aBus + Pbusinj + Cl@pd + Csh@gsh - Cg@pg == 0
  constr    sba    codegen  isb@aBus == 0
  constr    pglb   codegen  -pg + pmine <= 0
  constr    pgub   codegen  pg - pmaxe <= 0
  constr    plflb  codegen  -plf - cp.multiply(ul, rate_a) <= 0
  constr    plfub  codegen  plf - cp.multiply(ul, rate_a) <= 0
  constr    alflb  codegen  -CftT@aBus + amin <= 0
  constr    alfub  codegen  CftT@aBus - amax <= 0
  exprcalc  pi     codegen  pb.dual_variables[0]
  exprcalc  mu1    codegen  plflb.dual_variables[0]
  exprcalc  mu2    codegen  plfub.dual_variables[0]
  obj       obj    codegen  cp.sum(cp.multiply(c2, pg**2))+ cp.sum(cp.multiply(c1, pg))+

Note: every item shows codegen — the fast AOT path is in effect for the untouched routine. The INFO log line <DCOPF> formulation: codegen=15/15 confirms the same.

sp2 — DCOPF, customized, solve#

We extend the routine with an emission balance, an emission cap, and a small per-unit tax — none of which are in the source code. The customization is done by addRParam / addService / addVars / addConstrs and a direct obj.e_str += '...' append.

[5]:

sp2 = ams.load(ams.get_case('5bus/pjm5bus_demo.xlsx'),
                setup=True, no_output=True, default_config=True)
sp2.DCOPF.init()  # populate om so .get(...) works

stg_idx = sp2.DCOPF.pg.get_all_idxes()
pmax = sp2.DCOPF.get(src='pmax', attr='v', idx=stg_idx)

# Heterogeneous emission factor — proportional to 1/pmax (a stand-in
# for: small/peaker units are less efficient and emit more per p.u.).
ke = np.reciprocal(pmax)
# Per-unit tax — also 1/pmax.
ce = np.reciprocal(pmax)

sp2.DCOPF.addRParam(name='ke', tex_name='k_e', info='emission factor', v=ke)
sp2.DCOPF.addRParam(name='ce', tex_name='c_e', info='per-unit tax', v=ce)
# Cap chosen so the eub constraint actually binds. The baseline solution has
# sum(ke @ pg) ~ 1.53 — picking te = 1.0 forces ~35% redistribution.
sp2.DCOPF.addService(name='te', tex_name='t_e', info='emission cap', value=1.0)
sp2.DCOPF.addVars(name='eg', tex_name='e_g', info='gen emission',
                   unit='t', model='StaticGen')
# is_eq retired in v1.2.3 — embed the relational operator in e_str
# directly. LHS-zero discipline keeps `.v` reporting slack-from-zero
# (negative = respected, positive = violated) regardless of `<=`/`>=`
# direction (CVXPY canonicalizes both to `lhs - rhs <= 0` internally).
sp2.DCOPF.addConstrs(name='egb', info='emission balance',
                      e_str='eg - cp.multiply(ke, pg) == 0')
sp2.DCOPF.addConstrs(name='eub', info='emission cap',
                      e_str='cp.sum(eg) - te <= 0')
sp2.DCOPF.obj.e_str += '+ cp.sum(cp.multiply(ce, pg))'

sp2.DCOPF.run(solver='CLARABEL')

Working directory: "/Users/jinningwang/work/ams/examples"
Parsing input file "/Users/jinningwang/work/ams/ams/cases/5bus/pjm5bus_demo.xlsx"...
Input file parsed in 0.0797 seconds.
Zero Line parameters detected, adjusted to default values: rate_b, rate_c.
All bus type are PQ, adjusted given load and generator connection status.
System set up in 0.0019 seconds.
Entering data check for <DCOPF>
 -> Data check passed
Building system matrices
<DCOPF> formulation: codegen=15/15
Parsing OModel for <DCOPF>
Evaluating OModel for <DCOPF>
Finalizing OModel for <DCOPF>
<DCOPF> initialized in 0.0057 seconds.
Entering data check for <DCOPF>
 -> Data check passed
<DCOPF> formulation: codegen=14/17, eval(customized)=1, eval(added)=2
Parsing OModel for <DCOPF>
Evaluating OModel for <DCOPF>
Finalizing OModel for <DCOPF>
<DCOPF> initialized in 0.0057 seconds.
<DCOPF> solved as optimal in 0.0064 seconds, converged in 12 iterations with CLARABEL.

[5]:

True

[6]:

print('pg  =', sp2.DCOPF.pg.v)
print('obj =', float(sp2.DCOPF.obj.v))

pg  = [0.2        2.11651035 0.6        6.41765794 0.66583171]
obj = 11.297128566649729

[7]:

sp2.DCOPF.formulation_summary()

<DCOPF> formulation summary (14 codegen / 3 eval / 0 manual / 0 pending)
  kind      name   source   e_str
  --------  -----  -------  ----------------------------------------
  expr      plf    codegen  Bf@aBus + Pfinj
  expr      pmaxe  codegen  cp.multiply(nctrle, pg0) + cp.multiply(ctrle, pmax)
  expr      pmine  codegen  cp.multiply(nctrle, pg0) + cp.multiply(ctrle, pmin)
  constr    pb     codegen  Bbus@aBus + Pbusinj + Cl@pd + Csh@gsh - Cg@pg == 0
  constr    sba    codegen  isb@aBus == 0
  constr    pglb   codegen  -pg + pmine <= 0
  constr    pgub   codegen  pg - pmaxe <= 0
  constr    plflb  codegen  -plf - cp.multiply(ul, rate_a) <= 0
  constr    plfub  codegen  plf - cp.multiply(ul, rate_a) <= 0
  constr    alflb  codegen  -CftT@aBus + amin <= 0
  constr    alfub  codegen  CftT@aBus - amax <= 0
  constr    egb    eval     eg - cp.multiply(ke, pg) == 0
  constr    eub    eval     cp.sum(eg) - te <= 0
  exprcalc  pi     codegen  pb.dual_variables[0]
  exprcalc  mu1    codegen  plflb.dual_variables[0]
  exprcalc  mu2    codegen  plfub.dual_variables[0]
  obj       obj    eval     cp.sum(cp.multiply(c2, pg**2))+ cp.sum(cp.multiply(c1, pg))+

Three items are now eval (the eval-fallback helper):

  • egb and eub were added at runtime via addConstrs — the disk pycode doesn’t know about them, so they fall through.

  • obj was customized via obj.e_str += '...'. The descriptor mutex marked it _e_dirty, so _link_pycode skipped wiring its codegen callable.

Everything else stays on codegen.

sp3 — DCOPF, untouched, solve#

This is the contamination check: did sp2’s customization leak into the disk cache? If it did, sp3 would inherit sp2’s formulation.

[8]:

sp3 = ams.load(ams.get_case('5bus/pjm5bus_demo.xlsx'),
                setup=True, no_output=True, default_config=True)
sp3.DCOPF.run(solver='CLARABEL')

Working directory: "/Users/jinningwang/work/ams/examples"
Parsing input file "/Users/jinningwang/work/ams/ams/cases/5bus/pjm5bus_demo.xlsx"...
Input file parsed in 0.0315 seconds.
Zero Line parameters detected, adjusted to default values: rate_b, rate_c.
All bus type are PQ, adjusted given load and generator connection status.
System set up in 0.0019 seconds.
Entering data check for <DCOPF>
 -> Data check passed
Building system matrices
<DCOPF> formulation: codegen=15/15
Parsing OModel for <DCOPF>
Evaluating OModel for <DCOPF>
Finalizing OModel for <DCOPF>
<DCOPF> initialized in 0.0052 seconds.
<DCOPF> solved as optimal in 0.0050 seconds, converged in 9 iterations with CLARABEL.

[8]:

True

[9]:

print('pg  =', sp3.DCOPF.pg.v)
print('obj =', float(sp3.DCOPF.obj.v))

pg  = [0.2        1.43998388 0.6        5.76001612 2.        ]
obj = 9.535953244734864

[10]:

sp3.DCOPF.formulation_summary()

<DCOPF> formulation summary (15 codegen / 0 eval / 0 manual / 0 pending)
  kind      name   source   e_str
  --------  -----  -------  ----------------------------------------
  expr      plf    codegen  Bf@aBus + Pfinj
  expr      pmaxe  codegen  cp.multiply(nctrle, pg0) + cp.multiply(ctrle, pmax)
  expr      pmine  codegen  cp.multiply(nctrle, pg0) + cp.multiply(ctrle, pmin)
  constr    pb     codegen  Bbus@aBus + Pbusinj + Cl@pd + Csh@gsh - Cg@pg == 0
  constr    sba    codegen  isb@aBus == 0
  constr    pglb   codegen  -pg + pmine <= 0
  constr    pgub   codegen  pg - pmaxe <= 0
  constr    plflb  codegen  -plf - cp.multiply(ul, rate_a) <= 0
  constr    plfub  codegen  plf - cp.multiply(ul, rate_a) <= 0
  constr    alflb  codegen  -CftT@aBus + amin <= 0
  constr    alfub  codegen  CftT@aBus - amax <= 0
  exprcalc  pi     codegen  pb.dual_variables[0]
  exprcalc  mu1    codegen  plflb.dual_variables[0]
  exprcalc  mu2    codegen  plfub.dual_variables[0]
  obj       obj    codegen  cp.sum(cp.multiply(c2, pg**2))+ cp.sum(cp.multiply(c1, pg))+

Verify the two invariants#

[11]:

same_13 = np.allclose(sp1.DCOPF.pg.v, sp3.DCOPF.pg.v, atol=1e-8)
diff_12 = not np.allclose(sp1.DCOPF.pg.v, sp2.DCOPF.pg.v, atol=1e-6)
same_obj_13 = abs(float(sp1.DCOPF.obj.v) - float(sp3.DCOPF.obj.v)) < 1e-6

print(f'sp1.pg == sp3.pg ?  {"PASS" if same_13 else "FAIL"}')
print(f'sp1.pg != sp2.pg ?  {"PASS" if diff_12 else "FAIL"}')
print(f'sp1.obj == sp3.obj ? {"PASS" if same_obj_13 else "FAIL"}')

sp1.pg == sp3.pg ?  PASS
sp1.pg != sp2.pg ?  PASS
sp1.obj == sp3.obj ? PASS

How to know which path your customization is using#

At any point after init():

  • routine.formulation_summary() — full per-item table.

  • item.formulation_source — single-item check, returns one of 'codegen' | 'eval' | 'manual' | 'pending'.

  • The INFO log line <{routine}> formulation: codegen=X/Y, eval(...)=Z prints on every init().

If you customized something but its source still reads codegen, the customization didn’t take effect.

[12]:

sp2.DCOPF.obj.formulation_source

[12]:

'eval'

[13]:

sp2.DCOPF.constrs['egb'].formulation_source

[13]:

'eval'

[14]:

sp2.DCOPF.constrs['pb'].formulation_source

[14]:

'codegen'