Testing Sensitivity of M-Estimator to Kernel Width
As we have noted though previous testing, the M-Estimator seems to be rather sensitive to the value taking by the user-defined kernel width. Additionally, the optimal kernel width seems to change for every datasets. Today we starting testing both of the comments made above.
To begin testing, several minor modifications were made to the GTSAM library and a simple python interface was developed to automate the data processing. All modifications have been pushed to a publicly available repo. This repository also houses a collection of pose-graphs collected from various sources.
Initial Testing
To limit the amount of time take for this initial testing, only the Huber and Tukey cost functions were utilized. Additionally, no false constraints were added to the pose-graph. Below is the simple python script and the generated results.
#!/usr/bin/env python
'''
Simple script to test the sensitivity of pose-graph optimization to
m-estimator kernel width.
'''
__author__ = 'ryan'
__email__ = "rwatso12@gmail.com"
import os, glob, subprocess, progressbar, argparse
import matplotlib.pyplot as plt
# Add command line interface
parser = argparse.ArgumentParser(description="Simple script to test the "
"sensitivity of pose-graph optimization to"
"m-estimator kernel width")
parser.add_argument('-i', '--inputFile', dest='input',
default='../../poseGraphs/manhattanOlson3500.g2o',
help="Define the input pose graph")
parser.add_argument('-o', '--outFile', dest='output',
help="Define the output file")
parser.add_argument('-s', '--script', dest='script',
default='./../../gtsam/build/examples/processG2O',
help="What's the GTSAM script used to process the graph")
parser.add_argument('-k', '--kernel', dest='kernel', default='huber',
help="define the kernel to be used")
parser.add_argument('--maxWidth', dest='maxWidth', default=10, type=int,
help="What's the maximum kernel width you would like to test?")
parser.add_argument('--kerInc', dest='kernelIncrement', default=0.1, type=float,
help="What's the kernel increment for testing sensivity?")
parser.add_argument('--saveGraph',action='store_true',
help="would you like to save the graph to the cur. dir.?")
args = parser.parse_args()
index = []
totalError = []
progress = progressbar.ProgressBar()
print('\n\n')
for k in progress(list(xrange(1, int(args.maxWidth/args.kernelIncrement)))):
kernelWidth = str(float(k)*args.kernelIncrement)
cmd = [args.script, '-i', args.input, '-k', args.kernel, '-w', kernelWidth]
proc1 = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = proc1.communicate()
index.append( float(kernelWidth) )
totalError.append( out )
plt.xkcd()
plt.plot(index, totalError, 'k',label=args.kernel, linewidth=3.0)
plt.ylabel('Final Graph Error')
plt.xlabel('Kernel Width')
plt.grid()
font = { 'size' : 22}
plt.rc('font', **font)
plt.legend()
plt.show(block = False)
if (args.saveGraph):
plt.savefig(args.kernel+".eps", format="eps", close=False, verbose=True)
if (args.output):
f=open(args.output,'w')
for line in zip(map(str,index),totalError):
f.write(' '.join(line)+'\n')
f.close()
Manhattan 3500
Fig 1 :: Sensitivity of M-Estimator on Manhattan3500 dataset
Manhattan 10000
Fig 2 :: Sensitivity of M-Estimator on City10000 dataset
CSAIL
Fig 3 :: Sensitivity of M-Estimator on CSAIL dataset
FR09
Fig 4 :: Sensitivity of M-Estimator on FR09 dataset
As can be seen from the four images provided above, the robustness of the M-Estimator is incredibly sensitive to both the data-set and the kernel width.
Next Steps
On Monday I will start by conducting a similar test when false constraints are added to the graph