SolverTraces.jl

Usage example

Using SolverTraces.jl is very easy:

N = 20 # Number of iterations

tol = Tolerance(1e-3)
tols = 10 .^ range(0, stop=-3, length=N)

trace = SolverTrace(N, CurrentStep(N),
                    ColumnSeparator(),
                    Performance(100), # Could be number of points processed each iteration
                    ColumnSeparator(),
                    tol)
print_header(trace)
for i = 1:N
    # Do work
    # ...
    # Set which tolerance was achieved in this iteration:
    tol.current = tols[i]

    SolverTraces.next!(trace)
end

which will give output similar to this:

Performance Tolerance [ 1/20] │ 970.904 kHz │ 1.00×10⁰ (10⁻³) [ 2/20] │ 436.679 kHz │ 6.95×10⁻¹ (10⁻³) [ 4/20] │ 672.164 kHz │ 3.36×10⁻¹ (10⁻³) [ 6/20] │ 892.722 kHz │ 1.62×10⁻¹ (10⁻³) [ 8/20] │ 1.074 MHz │ 7.85×10⁻² (10⁻³) [10/20] │ 1.168 MHz │ 3.79×10⁻² (10⁻³) [12/20] │ 1.207 MHz │ 1.83×10⁻² (10⁻³) [14/20] │ 1.303 MHz │ 8.86×10⁻³ (10⁻³) [16/20] │ 1.431 MHz │ 4.28×10⁻³ (10⁻³) [18/20] │ 1.561 MHz │ 2.07×10⁻³ (10⁻³) [20/20] │ 1.685 MHz │ 1.00×10⁻³ (10⁻³)

Optim.jl

It is similarly very easy to add colour to your Optim.jl runs:

f(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2
x0 = [0.0, 0.0]

max_iter = 100
tol = 1e-8

value = ScalarColumn(Inf, "Value")
tolerance = Tolerance(tol, "|∇|")

trace = SolverTrace(max_iter,
                    CurrentStep(max_iter), ColumnSeparator(), value, tolerance,
                    print_interval=1)
trace_callback = opt_state -> begin
    tolerance.current = opt_state.g_norm
    value.n = opt_state.value
    SolverTraces.next!(trace)
    false
end
options = Optim.Options(iterations=max_iter, g_tol=tol,
                        callback=trace_callback)
print_header(trace)
optimize(f, x0, options)

which will result in something similar to

Value |∇| [ 1/100] │ 9.507e-01 4.58×10⁻² (10⁻⁸) [ 2/100] │ 9.507e-01 2.02×10⁻² (10⁻⁸) [ 3/100] │ 9.507e-01 2.17×10⁻² (10⁻⁸) [ 4/100] │ 9.262e-01 5.24×10⁻² (10⁻⁸) [ 5/100] │ 8.292e-01 4.26×10⁻² (10⁻⁸) [ 6/100] │ 8.292e-01 4.27×10⁻² (10⁻⁸) [ 7/100] │ 8.139e-01 3.11×10⁻² (10⁻⁸) [ 8/100] │ 7.570e-01 3.22×10⁻² (10⁻⁸) [ 9/100] │ 7.383e-01 2.42×10⁻² (10⁻⁸) [ 10/100] │ 6.989e-01 2.43×10⁻² (10⁻⁸) [ 11/100] │ 6.800e-01 2.12×10⁻² (10⁻⁸) [ 12/100] │ 6.475e-01 1.81×10⁻² (10⁻⁸) [ 13/100] │ 6.377e-01 2.15×10⁻² (10⁻⁸) [ 14/100] │ 5.978e-01 1.65×10⁻² (10⁻⁸) [ 15/100] │ 5.978e-01 2.78×10⁻² (10⁻⁸) [ 16/100] │ 5.476e-01 2.43×10⁻² (10⁻⁸) [ 17/100] │ 5.449e-01 1.75×10⁻² (10⁻⁸) [ 18/100] │ 5.091e-01 1.47×10⁻² (10⁻⁸) [ 19/100] │ 5.091e-01 2.93×10⁻² (10⁻⁸) [ 20/100] │ 4.564e-01 2.39×10⁻² (10⁻⁸) [ 21/100] │ 4.564e-01 4.39×10⁻² (10⁻⁸) [ 22/100] │ 3.654e-01 3.91×10⁻² (10⁻⁸) [ 23/100] │ 3.654e-01 3.66×10⁻² (10⁻⁸) [ 24/100] │ 2.994e-01 2.79×10⁻² (10⁻⁸) [ 25/100] │ 2.994e-01 3.62×10⁻² (10⁻⁸) [ 26/100] │ 2.598e-01 2.98×10⁻² (10⁻⁸) [ 27/100] │ 2.266e-01 4.85×10⁻² (10⁻⁸) [ 28/100] │ 1.445e-01 3.58×10⁻² (10⁻⁸) [ 29/100] │ 1.445e-01 2.98×10⁻² (10⁻⁸) [ 30/100] │ 1.445e-01 2.45×10⁻² (10⁻⁸) [ 31/100] │ 9.866e-02 1.91×10⁻² (10⁻⁸) [ 32/100] │ 9.866e-02 2.64×10⁻² (10⁻⁸) [ 33/100] │ 6.483e-02 1.58×10⁻² (10⁻⁸) [ 34/100] │ 6.483e-02 1.84×10⁻² (10⁻⁸) [ 35/100] │ 5.485e-02 6.61×10⁻³ (10⁻⁸) [ 36/100] │ 4.880e-02 1.88×10⁻² (10⁻⁸) [ 37/100] │ 1.220e-02 1.49×10⁻² (10⁻⁸) [ 38/100] │ 1.220e-02 7.78×10⁻³ (10⁻⁸) [ 39/100] │ 1.220e-02 6.49×10⁻³ (10⁻⁸) [ 40/100] │ 5.976e-03 2.70×10⁻³ (10⁻⁸) [ 41/100] │ 5.976e-03 3.46×10⁻³ (10⁻⁸) [ 42/100] │ 2.624e-03 1.52×10⁻³ (10⁻⁸) [ 43/100] │ 2.624e-03 1.34×10⁻³ (10⁻⁸) [ 44/100] │ 2.624e-03 1.23×10⁻³ (10⁻⁸) [ 45/100] │ 3.583e-04 9.66×10⁻⁴ (10⁻⁸) [ 46/100] │ 3.583e-04 3.54×10⁻⁴ (10⁻⁸) [ 47/100] │ 4.827e-05 1.27×10⁻⁴ (10⁻⁸) [ 48/100] │ 4.827e-05 5.66×10⁻⁵ (10⁻⁸) [ 49/100] │ 4.827e-05 1.50×10⁻⁵ (10⁻⁸) [ 50/100] │ 3.590e-05 1.45×10⁻⁵ (10⁻⁸) [ 51/100] │ 1.332e-05 1.42×10⁻⁵ (10⁻⁸) [ 52/100] │ 1.565e-06 4.92×10⁻⁶ (10⁻⁸) [ 53/100] │ 1.565e-06 3.46×10⁻⁶ (10⁻⁸) [ 54/100] │ 1.565e-06 9.10×10⁻⁷ (10⁻⁸) [ 55/100] │ 1.565e-06 6.87×10⁻⁷ (10⁻⁸) [ 56/100] │ 4.966e-07 5.78×10⁻⁷ (10⁻⁸) [ 57/100] │ 2.273e-07 1.31×10⁻⁷ (10⁻⁸) [ 58/100] │ 2.120e-07 7.11×10⁻⁸ (10⁻⁸) [ 59/100] │ 6.942e-08 8.18×10⁻⁸ (10⁻⁸) [ 60/100] │ 1.876e-08 2.13×10⁻⁸ (10⁻⁸) [ 61/100] │ 1.876e-08 9.27×10⁻⁹ (10⁻⁸) * Status: success * Candidate solution Minimizer: [1.00e+00, 1.00e+00] Minimum: 3.525527e-09 * Found with Algorithm: Nelder-Mead Initial Point: [0.00e+00, 0.00e+00] * Convergence measures √(Σ(yᵢ-ȳ)²)/n ≤ 1.0e-08 * Work counters Seconds run: 0 (vs limit Inf) Iterations: 60 f(x) calls: 118

Reference

SolverTraces.SolverTraceType
SolverTrace(num_steps, i, print_interval, io, progress, columns, callbacks)

Represents a solver trace for num_steps iterations, i being the current one. Each time next! is called, i is incremented by one, and if a multiple of print_interval is reached, a line is printed in the trace. Optionally (default), a ProgressMeter.Progress is shown as well, providing progress information between solver trace printouts.

source
SolverTraces.SolverTraceType
SolverTrace(num_steps[, column=CurrentStep(num_steps), columns...;
                        io=stdout, num_printouts=10, progress_meter=true,
                        print_interval, kwargs...])

Construct a SolverTrace for num_steps iterations, with the columns that are to appear in the solver trace. The solver trace is printed to io, num_printouts lines will appear; alternatively, the print_interval can be specified directly. Below the solver trace, a progressbar is displayed, unless !progress_meter. kwargs... are passed on to ProgressMeter.Progress.

source
SolverTraces.next!Function
next!(fun, s::SolverTrace; kwargs...)

Signal to the SolverTrace s that the next iteration has been performed, increasing the internal counter s.i. If a multiple of s.print_interval is reached, a line in the solver trace is printed, formatting each and every trace column, the callback function fun is called, as well as all the registered callbacks.

kwargs are passed on to ProgressMeter.next! to allow e.g. showvalues functionality.

source