I just finished doing what I set out to do about a week ago. I took the my NN number-classifier and ported it over to Python. Here are my thoughts on the process and on Python itself.

My Approach

In case it’s not obvious, I didn’t take a rigorous, ground-up fundamentals approach to learning what I needed. To the extent I can say I have any “formal” knowledge of Python, it’s all knowledge that I’ve picked up by inference. For instance, I picked up by inference that blocks of code such as for loops or functions are delimited by a colon at the start and then explicit indentation continuing until the end of a block where returning to outdented lines of code indicates the end of the block. This is unlike almost every other language I’ve used, where blocks of code are always delimited by specific keywords (for/end, if/else/end) or punctuation( {} ). And indentation, while ubiquitously used by developers to create readable code, is ultimately optional. That’s just one example and because I didn’t learn that as part of some course, I’m not even certain I’m 100% correct on that. But there are many other examples, especially concerning array and matrix operations, where I figured out enough to do what I needed, but there is probably a good deal more to learn in order to optimize what I’ve written to perform as well as my Octave/Matlab NN. More on that later.

Setup

Getting Python up and running on my Mac was fairly straightforward after 20 minutes or so of Googling. In fact, it wasn’t necessary to install anything if I was willing to run the 2.x version of Python that was pre-installed on my laptop. I wanted to run the lates 3.x version though, so I installed that.

My Reactions

My thoughts on what I was exposed to as part of this project fall in two main areas. These are 1) the Python language itself, and 2) the particular libraries (NumPy mostly) that I used to do most of the data and model manipulation.

Python

I didn’t really have a clear idea of what to expect from the Python language when I started. Even among languages in which I’m not fluent, I have gotten a flavor for quite a few. Python, however, was completely virgin territory for me. That said, overall, I was pleasantly surprised by the ease of the experience. Unlike some other languages which, at least at first, have seemed inscrutable, Python is a fairly straightforward scripting language possessing a syntax which feels very natural. There were a few oddities to get used to, among them and most significantly, the colon/indentation syntax for blocks of code. For the most part, though, I took to it quickly.

Compared to Matlab/Octave, writing code is comparably easy and fast. The main area where Python suffers compared to Matlab/Octave is some of the overhead necessary to write modular code. For instance, in Octave, any function living in it’s own file separate from the main program is automatically available if that file is in the same directory as the program. In Python, any such functions need to be imported at the beginning of the main program. Furthermore, even if the main program imports all the functions, those functions need to be imported again if they’re called from outside functions.

Perhaps this is just one area where if I had spent more time refining my code or learned Python in a formal setting, I would know that I’m doing something the wrong way. If so, I’ll get there eventually. That said, I’m not necessarily bothered by that additional overhead. It didn’t take considerably more effort to add in all those extras, and in general, it’s the price one usually has to pay for working with a production-capable language.

NumPy

NumPy, on the other hand was a different story. There are probably a lot of details I could write about this, but at a high level, the way NumPy works with vectors and matrices feels very kludgy compared with MatLab/Octave.

For starters, while NumPy supports all the basic matrix operations that Matlab/Octave supports, it feels more like functionality pasted onto NumPy’s Array object. Arrays may be similar in many ways to matrices and vectors, but their inherent differences are what make the implementation feel like a hack. This is apparent in many areas, but I’ll just talk about two here.

Matlab/Octave uses a minimalistic, visual/spatial approach to creating and manipulating vectors and matrices. For instance, here’s how to create a 2×3 matrix:

m = [1 2 3; 4 5 6]

Playing back m we get this:

m =

1 2 3
4 5 6

Beautiful and elegant.

Here’s how to do the same thing in NumPy:

m = np.array([[1, 2, 3], [4, 5, 6]])

And playing it back, we get this:

array([[1, 2, 3],
       [4, 5, 6]])

Completely serviceable, but far less elegant.

And let’s look at adding another row and another column:

>> n = [9; 9]
n =

9
9

>> x = [m n]
x =

1 2 3 9
4 5 6 9

>> y = [9 9 9 9]
y =

9 9 9 9

>> x = [x; y]
x =

1 2 3 9
4 5 6 9
9 9 9 9

Again, visually clean and syntactically concise.

But here’s how we’d add the same row and column in NumPy:

>>> n = np.array([[9],[9]])

>>> n

array([[9],
       [9]])

>>> x = np.append(m, n, axis=1)

>>> x

array([[1, 2, 3, 9],
       [4, 5, 6, 9]])

>>> y = np.array([[9, 9, 9, 9]])

>>> y

array([[9, 9, 9, 9]])

>>> x = np.append(x, y, axis=0)

>>> x

array([[1, 2, 3, 9],
       [4, 5, 6, 9],
       [9, 9, 9, 9]])

Once again, perfectly adequate, but clunky and inelegant by comparison. And a lot more typing as well.

Finally, Matlab/Octave uses a very straightforward pattern for distinguishing between matrix operations and scalar and element wise operations of the same kind. Matrix operations are the primary operations with operators like * for matrix multiplication and / for matrix right division:

>> m * n %matrix multiplication of m and n

>> m / n %matrix right division of m and n

Adding a period in front of each makes each the scalar version.

>> m .* n %scalar or element wise multiplication of m and n

>> m ./ n %scalar or element wise right division of m and n

NumPy, on the other hand, uses the simple operators * and / for scalar operations, but resorts to verbose functional notation for matrix operations. Here’s matrix multiplication:

>>> np.dot(m, n) #matrix multiplication of m and n

Blechh. And from my experience, this sort of syntax and approach is how one does most involved operations in a NumPy environment.

Conclusions

In the grand scheme of things, these are not major problems. Had I never used Octave/Matlab, I would probably not even have raised these issues. Everything that NumPy does is right in line with the way most programming languages I’ve used do things. It’s more typing, but no big deal. In general, I’m pleased with how quickly and easily the transition from Matlab/Octave has worked.

However, looking forward, there’s another thing that I would like to cover, but not in this already-too-long post. The NN I ported over behaved comparably with the Matlab/Octave version in terms of accuracy on both training sets and validation sets. However, my vectorized implementation of back propagation runs only half as fast as it did in the original. It doesn’t matter for a test project such as this, but things like that are important for production applications, especially at a large scale. So there are a lot of questions about where the slowdown is with the Python version. That may well be the topic for my next post.