The one thing that was disappointing in the results of the Python Port of my NN was the time it took to learn the training set. I wasn’t sure what the reason was, and had speculated that it might have something to do with not using the GPU. That turns out to be wrong.

As I’ve been getting ready to build the infrastructure for checkers, I’ve been looking at optimization, and after going down several rat holes to use CPU optimization and GPU optimization—all to no avail—it turns out that the real problem was in the particular implementation of fmin_cg that comes with SciPy.optimize. Both fmin_cg algorithms (SciPy and the hand-rolled one in the ML/O version) run 500 iterations. But the ML/O version calls my cost/gradient function about 760 times, whereas the SciPy version calls the cost function about 1,150 times. That ratio corresponds nicely with the differences in times.

I don’t know if there’s any value in the extra calls, but I suspect not since the accuracy of the two are indistinguishable from one another. If fmin_cg turns out to be something I use for my checkers, I’ll likely want to do some research into more efficient versions.