#!/usr/bin/python
#
# Usage: python optimizer_visualization_annotator.py path/to/the/doxygen/optimizertutorial.html
import re
import glob
import os
import sys
optimizerVisualizationFile = sys.argv[1]
with open(optimizerVisualizationFile) as f: content = f.read()
jsPattern = ''''''
jsAddition = '''
'''
visPattern = ""
visAddition = '''
This visualization allows us to see how many popular optimizers perform on different optimization problems. Select a problem to optimize, then select an optimizer and tune its parameters, and see the steps that the optimizer takes plotted in pink. Note that you can zoom in and rotate the drawing. Also, you can compare how different optimizers perform on a given problem in the second graph. As you try a given problem with more optimizers, the objective function vs. the number of iterations is plotted for each optimizer you have tried.
Step-size:0.01
Iterations: 4000
The defaults here are not necessarily good for the given problem, so it is suggested that the values used be tailored to the task at hand. (Use the mouse to zoom/drag the function.)
First moment coefficient:0.9
Second moment coefficient:0.999
Second moment coefficient:0.999
Second moment coefficient:0.999
As intuition says, system has higher probability of staying in the states with a smaller stepsize. As the stepsize goes up, imbalance becomes stronger. When the stepsize is close to zero, the system stays in the state(s) with the highest cost.
Evaluations:Dynamic
A plot of the cost reveals distinct properties for each optimizer with its own style of convergence.
Optimizer
'''
p = content.find(jsPattern)
if p > 0:
content = content[0:p+len(jsPattern)] + "\n" + jsAddition + "\n" + content[p+len(jsPattern):len(content)]
p = content.find(visPattern)
if p > 0:
content = content[0:p+len(visPattern)] + "\n" + visAddition + "\n" + content[p+len(visPattern):len(content)]
with open(optimizerVisualizationFile, "w") as f:
f.write(content)
