Uwe’s Blog

My writing about data engineering, opensource development, general programming and thoughts about engineering culture.

  • PyFlame: profiling running Python processes

    Identifying performance bottlenecks in long-running processes often involves careful instrumentation ahead or guessing where the root of the problem may be. A very welcome set of tools are the ones that help you diagnose problems of live systems without modifying them. One important tool I recently came across is the pyflame profiler.

  • Use Numba to work with Apache Arrow in pure Python

    Apache Arrow is an in-memory memory format for columnar data. In more “plain” English, it is a standard on how to store DataFrames/tables in memory, independent of the programming language. One of its most prominent uses is for the @pandas_udf decorator in Apache Spark to move data quickly between Scala and Python/pandas.

  • AHL Python Hackathon April 2018

    Three weeks ago MAN AHL organised an opensource hackathon at their London office. As part of the Hackathon people should contribute to one of the PyData artifacts they regularly use. To support them in making their first contribution, AHL also coordinated that several core committers of opensource projects were present at the event. I joined in as the representative...

  • Use Media Keys to control Tomahawk in Awesome WM

    Nowadays for controling a mediaplayer the MPRIS specification exists, sadly this interface seems unsupported by awesome. One solution would be to add some lines to the configuration of xbindkeys and to start it in the background. But as awesome already can handle global keybindings adding these lines to your .config/awesome/rc.lua will transmit the actions of...

  • Songride Beta available for testing

    About 3 years ago I made a little mashup using Ruby, Google Charts API and the Last.fm API and a bit of magic that listed you from where your favorite artist were coming from. As an example this was the chart output for my favorite artists:

  • redis and hiredis added to Gentoo Overlay

    While I was writing some more code for songride I felt that I should have the latest redis version installed. After running emerge '>=dev-db/redis-2.4.17' I was confronted with the following error message:

  • ktoblzcheck bindings for Node.JS

    Checking the correctness of a combination of bank identification number (BLZ) and account number is a complicated task in Germany. There are about 140 different methods how this combination is checked depending on the bank from which the combination originates. A library which solves this problem is ktoblzcheck. At the moment you could use it simply as a C++...