After working for nearly three years full-time I moved into a role that is primarily in Python. After roughly a year in a half I decided I’d try and write down some thoughts on the transition.

Background

The first programming language I ever learnt was Python 2 which I used to attempt to make games with a library called Pyglet. I also used Python for hacking together a few university projects and in some jobs for doing tasks mostly related to test automation, so I was no newcomer to Python. But the vast majority of the code I had written up to this point was probably in Go. I’d also used C++ in my first job and for most my university projects I’d used C/C++ or Java.

What I Expected

I went into the job having barely used Python 3 or not really keeping up with Python developments at all for quite a while. Going into the job I was a bit apprehensive about working on large Python projects. The primary concerns I had were around Python being dynamically typed making it hard to work with in large projects and performance. I also thought I would really miss Go’s concurrency primitives and fast compiler. Overall I was relatively negative about having to use Python full-time but that was a trade-off I was willing to take to try to break into a different industry.

How It Went

Type Annotations Are Crucial

I discovered that the code base was using Python’s type annotations heavily and not long into my tenure it became mandatory for new code to pass type checks run in CI with MyPy. This has made working with a relatively large Python codebase much easier than it would have been otherwise. As well as making the codebase much more readable, MyPy will from time to time actually catch mistakes and that is very useful. I like it when the computer is able to tell me I’ve some something silly, it helps you focus on big picture without getting bogged down in the details of every single line of code you’ve written.

Type annotations with MyPy is not a complete replacement for a statically typed language though for a few reasons:

• you may find the occasional bug/weird edge case with MyPy and that’s annoying
• someone can easily put a comment on a line of code that makes MyPy ignore it
• it’s incredibly slow

This means you can’t fearlessly refactor like you can with a statically compiled language with a quick compiler like Go and that’s frustrating. There’s always a chance you missed something due to an ignore and running MyPy takes a long time even on a few small files. Even with MyPy being very helpful I don’t quite trust any Python I write as much as I would have trusted some Go code.

Inheritance

I hate inheritance. In Go there is no inheritance and I think that is fantastic. Working with a codebase with a large tree of inheriting classes dotted over the codebase can be absolutely mind boggling to understand and near impossible to refactor. This is because different method may be implemented at different points in the tree of classes and some overridden here or there. It means you need to jump around a lot to get a full picture and it’s very frustrating in my experience.

Go only had a concept of a interface a.k.a. a pure abstract class. This forces the code into having a more well thought through design in my opinion, as well as making it much more readable. I try my best to stick to the same methodology in my Python code. But to my dismay I have once or twice given into the temptation to use inheritance from a non-purely abstract class, probably because it was the easiest thing to do at the time.

In Python we even have the lovely feature that is multiple inheritance. Which means of course you can end up with the dreaded (Diamond Problem)[https://en.wikipedia.org/wiki/Multiple_inheritance#The_diamond_problem]. Go thankfully prevents this from happening at compile time. Python uses al algorithm to determine the “Method Resolution Order” for each method of a class. I don’t have a clue how it works and I doubt 99.9% of people writing Python do either, I think I’d prefer a compile time failure.

Slow

Python’s performance is pretty terrible as everyone knows. This of course doesn’t matter too much until it does. This slowness may result in making software operationally more complicated in order to distribute work to get speed ups. This in turn will likely make the system harder to maintain and less reliable. Another area where the slowness of Python can be felt quite hard is in running tests. A sort of TDD-like dev cycle can be slowed significantly by waiting to run tests. Of course a similar wall will be hit in any compiled language if the compiler is slow and or the project is very large. Though in Go I’ve never had a problem with compile times.

Too Much Magic

I massively dislike anything in code that looks like magic. I like to understand what is going on so that when things don’t work as they should I know what to do. But in Python I find that developers (including myself) as tempted to make difficult to understand solutions in an attempt to make a generic solution or to reduce the amount of code that needs to be written. There are some other things I also strongly dislike that can appear as “magic”, one of the main things being operator overloading.

Too Many Ways To Do Anything

In Go there is pretty much one way to do anything and one well known pattern for doing most things. This made reading code very easy. If you know Go you can drop into any project using Go on Github and have a read around and actually start to understand what is going on. In Python this is absolutely not the case. There are four different ways to format strings in Python for example. Any time I google how to do something I land in a Stack Overflow post with a large number of answers. Sometimes, if you’re lucky, the poster will also include some mini benchmark numbers of the different approaches. I’ve noticed that when a good samaritan posts some mini benchmarks the different solutions often have quite different performance, for example simply (concatenating strings)[https://stackoverflow.com/questions/1316887/what-is-the-most-efficient-string-concatenation-method-in-python].

Terrible Concurrency

In Go concurrency is easy, in Python it’s a complete joke. All you need to do is launch some Goroutines. To communicate between Goroutines you can use channels. There are built in helpers such as wait groups.

Example:

var wg sync.WaitGroup
for i := 1; i <= 5; i++ {
    wg.Add(1)

    i := i

    go func() {
        defer wg.Done()
        worker(i)
    }()
}

wg.Wait()

Similar variations of this pattern occur naturally in code that does a lot of calling out to other services and I found myself writing code like this almost every day in Go. In Python doing something similar is much less elegant. If I don’t want to spin up threads I could use async to achieve something similar.

async def f():
    await asyncio.sleep(0.1)
    print('done')

async def main():
    pending = []
    for i in range(5):
        pending.append(f())

    group = asyncio.gather(*pending, return_exceptions=True)
    await group

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

Immediately, you can see that the word async sticks out in the above example. Functions need to be explicitly marked as async and therefore it can’t just be called as a normal function anymore. This is in complete contrast to Go where any function can be simply called with the go keyword and it’s spun off into a green thread. This means any function in go can be called both with or without the keyword and this will cause the function to be called serially or concurrently respectively. Another great feature of Go is that event loop for scheduling the Goroutines is built into the language, the main function is actually a Goroutine itself so every Go program is ‘async’ by default and you’re always in the ‘async’ world. Where as in Python, async is an afterthought tacked onto the language and you need to tippy toe between the sync and aync worlds. As a result you need to manually invoke the event loop in Python as we see above.

What I Do Like

Quick/Interactive Development

The ability to just spin up an interpreter, write some code and see the output quickly can be very useful. An extension of this is the Jupyter ecosystem. Jupyter notebooks are a very nice way to play around with data and do experiments. Plotting libraries and data libraries allow for quickly writing code to analyse some data. I can very quickly write code that pulls some data and does some comparisons and that’s very useful in my job. Having an interpreter also makes the debugging experience much more interactive. I routinely find myself writing code into the interpreter at a breakpoint to help diagnose a problem.

Massive Ecosystem

Python being as popular as it is, a large ecosystem has grown around it in terms of libraries. You can often find a decent library to do most tasks. Libraries such as Django are excellent at what they do. Python has become incredibly important in the Data and Data Science space.

Conclusion

There are many more negative points here that positive. In the end I think I much prefer Go and working with statically typed compiled languages. I do appreciate that I can write Python much more quickly than I could some Go, Rust or C++ but that comes with a big cost in my opinion.