The Binary Toolbox (for JavaScript)

    The first time I tried my luck in parsing binary files within the browser must have been the Cloudtracker2 project, my (slightly out of date) try to make a good Protracker player/Editor for the web (It sort of lives on in the Halfplayer project if you’re interested). Parsing binary files in the browser is actually no longer a problem, but I thought it might be a fun exercise to write down some notes on what I’ve come across in one of my current projects, which involves intensive binary data munging on a much bigger scale than what I have tried so far.

    So, here’s the toolboxes contents:

    Additionally, we’re going to talk about file drag and drop, creating object URLs and other things.

    The Rails Architecture Fallacy

    Someone has been wrong on the internet. I hate it when that happens. And so I started to write a comment and then I thought to my self, hey, this is great blog post material, why should I waste it on someone who is wrong on the internet. So here we go. It reads like a comment on someone who was wrong on the internet at times, because that’s exactly what it is :)

    Today, I’ve stumbled across a blog post, via RubyFlow, which is boldly named Ruby on Pains by Facundo Spagnuolo. It is a melange of falsly applied pure OOD wisdom and (I can only assume) juvenile arrogance (Do I sould like an old fart already? I’m in my forties now, I have to sound like an old fart now), that made me a little angry and made me want to reply. Not sure this is a winning move, but I think my reply does contain some parts that bear repeating, so here we go.

    The other side of static vs. dynamic typing systems

    I’m currently working on a small web application that has to do a fair amount of munging binary data in the front end (meaning: JavaScript). One of the things it needs to do is inspect data packets, unpack them (from a simple 7/8 bit encoding scheme invented in the 80’s) and checksum them. The checksumming is done with a standard CRC32 algorithm. It took me a few hours to find a JavaScript library that uses the same polynomial as the counterpart of the app uses (which, luckily, is a widely used one, for example zlib uses it) and was usable within my Ember/Rails setup.

    I tested around a bit and had a setup that worked, until I started testing with bigger packets and suddenly, the checksums wouldn’t match anymore. As it turns out the fact that the library worked in the first place was by chance: It returns a signed 32 bit integer and my test setup in the beginning simply produced a checksum that didn’t have the sign bit set. In parallel, I verified the results with two tools: The Ruby zlib bindings (part of the stdlib) and the crc32 command line tool that comes with OS X. Both return unsigned integers.

    My 2015 in Review

    I’ve recently taken a look at my blog archive and it seems I never actually did a year-in-review blog post which was kinda surprising to me. 2015 was a year that turned out completely different from what I thought it would, so I thought, this year, it might we worthwhile to reflect on that a little. Also, I’m now slowly becoming somewhat of a senior (with my 40th birthday approaching way faster than I appreciate) and with seniority comes the privilege of sharing your thoughts whether others want to hear it or not. Haha.

    Anyway. Let’s start with a simple subject

    Having fun with pixels and lua

    The following text is somewhat like the long version of a lightning talk I gave at the most excellent Eurucamp. As I used a lot of animated GIFs in Keynote, it’s kinda hard for me to publish a working version of this presentation in any form. So this article must do.

    In her keynote at Eurucamp, Joanne talked about awe and how to combine coding with our passions.

    For me, one of the ways to get that extra kick is dabble in game programming. I’m bad at it, though. The reason is, I guess that I always lose myself in complexity and then all the fun and awe is lost. This is partly because I tend to over-complect both my thinking and my creations, but also partly because most tools do not keep you from doing that.

    A few weeks ago, around the time the amazing SoCoded happened, I stumbled over a thing called Pico-8. It is a so-called fantasy console, a sort of virtual machine or emulator of an imaginary console from the 80’s, designed and built by Joseph “zep” White from lexaloffle games.

    Pico-8 is wonderfully quirky, but it also has some very interesting technical limits, artificially but deliberately chosen by its creator.

    The Birth of a Radical

    Most people who would consider themselves “radical” in a specific sense usually didn’t become “radicals” overnight. It’s a slow process that can take years. And one day you find yourself wondering “how did this happen?”

    I’ve been wondering about this for quite some time now. Somehow, between 2010, when I started cycling to work every day, and today, I’ve become a radical. Not in the sense you might imagine if you’re a die-hard motorist, though: I drive conciously, often very asserting, and certainly a bit cheekily, but I try very hard to do this within the boundaries of our “road code” which has the very poetic name “StVO” or “Straßenverkehrsordnung”. The problems actually start right here: In contrast to the many many motorists I run into conflict with every day, I know the relevant parts of the road code very well. It’s a very common phenomenon as a cyclist in Hamburg to be yelled at for basically doing exactly what the law wants you to do. Or to be yelled at for telling a motorist that she or he just violated the road code in a very dangerous and reckless manner.

    Service-Wüste Kabel Deutschland

    18:50 - Ich komme nach Hause. Das Internet hat offensichtlich gerade noch ausgereicht um meine Hues anzuknippsen, kaum sitze ich am Rechner, ist das Internet weg. Die Outbound-Connection-Lampe am Kabeldeutschland-Router blinkt.

    18:55 - Nach durchbooten des O2-Routers (Ja, ich habe zwei Internet-Anschlüsse, und ich weiss auch warum) habe ich zumindest wieder 8 MBit von 116 bezahlten MBit zur Verfügung (16 über O2, 100 über Kabeldeutschland).

    18:57 - Ich fange an, mich durch die Hilfe-Seiten von Kabeldeutschland zu klicken. Seit der Vodafone-Übernahme ist da viel kaputt gegangen, aber siehe da: Ich habe den richtigen Knopf gefunden:

    Super cheap VPS providers

    I am usually not a cheapskate. I love to pay decently for a service or a thing worth paying for. With servers, things are a little different. First of all, the price span between super cheap and super expensive is usually really huge, while it’s often times totally unclear what the difference in service might be. Services like Digital Ocean are able to charge a healthy surplus by providing a superior “experience”, while their machines are, for the most part, beefy, but not exactly outstandingly so. I am writing this being a (happy) Digital Ocean customer, by the way.

    So, a few weeks back, I was doing some research (Read: I googled a lot) on VPS hosters with a little twist: I tried to find European or even German services, with the clear goal of finding the cheapest ones. This didn’t become some sort of shootout, because that would have meant that I needed to deploy compareable things to to it, so don’t expect something like that here. I’m going to name a few names, but not to especially endorse them but to make my report a little more concrete.

    Via google, I stumbled upon some sort of comparing site, webhostlist.de (unfortunately in German only). They listed a lot of machines way below the 5 EUR mark, which actually caught me by surprise. I remember I rented my first, very small VPS back in 2009 or so and it was a tiny, tiny machine for nearly 10 EUR a month. So this made me curious. I further trimmed down the list to services with more or equal to 1 GB of memory, because I think this is currently a pretty good limit for hosting small things.

    I instantly bought three services, all of them high on the webhostlist.de list and as soon as the accounts rolled in, I started to deploy things to it. So far, my list looks like this:

    • An iRedMail Mailserver, which is my testbed for self hosting email again (most likely worth an own article)
    • A server that runs graphite, as a central hub for metrics coming in from other servers
    • An application server for an upcoming little thing

    Decentralize ALL THE THINGS!

    This text is a modified version of my manuscript for my talk “Decentralize ALL THE THINGS” I gave at Eurucamp 2014. I sometimes write manuscripts for mostly non technical talks. In this case, it made a lot of sense, because it was also a good way to reason about the talk with my great Mentor, Frank Webber. After Eurucamp I thought this was a good way to kick off my long overdue article series about Decentralization.

    There’s also the accompanying slide deck you could take a look at.

    So, how was your year?

    Of course I didn’t mean personally. Professionally, if you’re not totally ignoring the rest of the world, 2013 and 2014 seem to be quite shitty years for working in IT. After all the Snowden Revelations, the shitty, inappropriate government reactions and then all the security holes in critical pieces of infrastructure (most of you will at least remember Heartbleed, of course) and the countless account breaches (Remember the Adobe breach?), I definitely considered leaving our field completely, with a strong urge to do something simple as gardening (And thus completely ignoring the devastating problems gardening is currently facing, with a fast and steady decline of bee populations, the general issues of pollution and the strong bias of regulators for multinational corporations).

    So, this is why I’ve turned a rather large part of my attention to this idea called decentralization. Because I believe that we need to change our thinking on what we’re doing here. All of us. First of all me. So this is a work in progress. I’m new to this, so please, if you’ve been preaching decentralization since, like, ever, bear with me for a a few paragraphs. I’m hopeful that I can actually shed light on this from some interesting, non common angles.

    Redesign, Schmedesign

    After reading @adactio’s great post on indie web building blocks, I wanted to do something. I started by adding a few rel=”me”’s to the page and then I stumbled over some weird styling issues (a time tag that I declared as display:block globally for some reason) and one thing lead to another and suddenly I found myself in this weird downward spiral of yak shaving (or sharpening the yak shaving scissors. Well, actually I had to wetten the sharpening stone first but there was a hole in the bucket. Long story.)

    Visiting Belgrade

    I don’t remember the exact date and I can’t find the needed data quickly (even Wikipedia fails me), but somewhere around the 24th of March of 1999, I was standing in the lecture hall at my university and opened up our bi-annual students assembly (I was part of the students council at the time) with a few, probably very incoherent words about what just started to happen in the Kosovo region. For the first time since the end of the second world war, german military forces were allowed to shoot enemy combatants on foreign ground. The german air forces were part of what was called OAF (Operation Allied Forces), an operation against what at that time was called Federal Republic Of Yugoslavia, which consisted of Serbia and Montenegro.

    I’m not interested in discussing the details of this conflict and to be honest, I had to read up on it on Wikipedia because I seem to have forgotten most of the details. But I do remember that for me as a german citizen, it felt like the end of an era: Post-WWII Germany crossed a line I personally hoped would never again be crossed. Under a left/green government no less.

    Going Full Encryption

    One of the main outcomes of the whole NSA thing for me is that we actually need to step up our game. And with we, I, above all, mean me. And what exactly means stepping up our game?

    If you haven’t, read Private By Default by one of our wise men, Tim Bray. It makes a few good points, why it makes sense to even encrypt traffic that seems harmless.

    So, setting this up on my personal web server (which is actually a cheap root server that by now mostly hosts static web pages) was relatively easy. There are some weird pitfalls though.

    A public service announcement

    I have three computers I use on and off. On all of them, I have a relatively long blog post draft lying around I haven’t gotten around to finish and publish.

    That’s not a good sign.

    Over the course of the last weeks and festivities, I gave that some thought. And I believe I have found an answer, although I am not very happy with it.

    2013 was the year when, at least for me, the last pieces of that already pretty much ruined facade that covered our society and political order (our so-called democracy) finally crumbled to dust.

    It’s actually less the revelations by Snowden (and, me visiting the Chaos Communication Congress for the first time, had my fair share of new revelations and other depressing news there) than the way our society, the politicians, the press, but also the general public reacting to them that finally cemented my belief that we’re indeed now in a post-democratic state (As in state machine, not as in state prison). It’s not that I think that all is lost, but I think that the way out of this crisis will either be extremely long and painful, or extremly violent.

    Thing is: I have no idea what to do now. I have a few vague ideas on how we could fix society on a grand scale, but none of them is especially applicable in the current situation. I am just a software developer. A quite thoughtful software developer with strong political opinions, but one who now needs to take a step back and listen, read and make up his worried and confused mind on what would be a meaningful contribution to the future of our society.

    And in the meantime, I will try to just be a software developer, at least here, in this blog. As much as I feel the pressure to comment on the current turmoil, I found out over the last months that I simply can’t, at least not in a coherent, useful way. I’m afraid “F**K ALL OF THIS S**T” is hardly a useful contribution to the discussion, as much as it properly reflects my current feelings.

    I hope that re-enables me to write on this blog. It will be, for the time being, more technical, more shallow, perhaps, but hopefully it will be revived in 2014 this way.

    Here’s to hoping that from here on, things can only get better (My pessimistic winter soul disagrees, but eff it).

    The Problem with end-to-end encryption

    A friend of mine, Hendrik Mans, wrote a pretty good article (in german) about the whole PRISM etc. dilemma. His gist: We should stop to act in this case as if we are actually able to tell whats going on there. For the most part, we simply don’t know. Because we’re are not tech savvy enough to understand the technologies involved (which may or may not be true for you) but also because, for the most part, this stuff is still happening largely in secret and although Edward Snowden gave us some ideas of what’s happening, we still don’t know much about how that’s happening.

    He also rants about the “solutionists” crying “you only need to use end-to-end encryption” and that’s a point I actually want to elaborate on a bit.

    First of all, what do we mean with end-to-end encryption (e2ee) . For the sake of the argument, let’s assume it means something like PGP (or GPG for that matter) and everyone uses securely stored keys, the encryption is asymmetric and the keysize is big enough (which are a lot of assumtions already if you think about it) and so we can be relatively sure that this encryption is safe. Please note that I am not going to explain all of this to you. Fortunately, there are a lot of places on the web where you can get good information on this.

    This is a great tool if your name is Alice and you want to send Bob a message which should stay secret between you two.

    That is to say, if the secrecy of the message is what you actually care about.

    Hamburg Deine Radwege Teil 2

    Ich hatte neulich eine erst etwas unfreundliche, aber dann doch entspannte und lehrreiche Konversation mit einem Stadteil-Polizisten, der mich vollkommen zurecht darauf hinwies, dass ich auf dem Fahrradweg in der falschen Richtung, bzw. auf der falschen Seite unterwegs sei. Im gegensatz zu 100% aller Radfahrer, die der Beamte danach noch angehalten hat, wusste ich wenigstens, das ich im Unrecht war.

    Aber darum soll es hier nicht gehen. Einer der Aha-Momente in unserer kurzen Unterhaltung war, dass in Hamburg momentan eine kleine Revolution stattfindet, von der niemand etwas mitbekommt.

    Dazu muss ich etwas ausholen - Es gibt nämlich Radwege und Radwege. Genauer gesagt: Benutzunsgpflichte Radwege und Nicht benutzungspflichtige Radwege. Schon gewusst?

    Der Unterschied ist eigentlich ganz einfach zu erkennen: Benutzungspflichtige Radwege sind mit einem Schild (Wer auf StVO-Beamten-Sprachverirrungen steht, dem sei der Wikipediaartikel ans Herz gelegt) mit dem typischen Fahrrad auf blauem Grund versehen.

    Nicht benutzungspflichtige Radwege hingegen sehen nur nach Radweg aus, haben aber keine Beschilderung. Und wie der Name es sagt: Man muss diese Radwege nicht benutzen. Man darf es aber.

    Get the book while it is hot!

    So, finally, about at least two months after my original schedule, I started selling my book.

    The Single Page App Jumpstart

    It’s been a tough decision. Not because I didn’t think the book was ready (I always planned to release it as soon as I thought it had enough material to be interesting to people), but because I, after Hendrik asked me if I knew Leanpub, basically threw 20-30 hours of work into the gutter and went for them instead of building the book on my own.

    I did know Leanpub, of course. I even checked it out back in the days. But somehow I ended up trying to do all of this stuff on my own. Which was, as I see it now, a bad decision. From now on, every minute I invest into this project will be about the text and the example code and not about how to fix some weird code highlighter bug in Apple iBooks.

    So, thanks Hendrik, for poking me in exactly the right moment (After I spent a full weekend on readying my own infrastructure and publishing chain).

    It took me another day or so to convert the whole book to the slightly different markdown dialect Leanpub uses. Their publishing workflow is just insanely simple and brilliant - Thank you Peter and Scott for this great great platform.

    And now I advise you to Buy The Book!

    JavaScript Styleguides

    As an author of a soon-to-be-published book on JavaScript applications, I had to come up with a coding style to use in the examples I have in the book. So I wrote this article to explain my way of thinking about JavaScript coding styles.

    Having said that, the JavaScript community if there’s such a thing, has a few elements that leave me baffled every time. One of them is the battle over coding styles.

    One particularly curious example, that also often gets cited, is the npm styleguide. So to explain why my JavaScript coding style looks as old fashioned and C-Like, let’s look at this extreme example and why I don’t follow it.

    Premature Abstractions Illustrated

    A lot of the discussion in the Ruby community are revolving around abstractions and when to use/do them and when not to. This is a struggle every developer faces every day, on various levels of, wait for it, abstractions.

    It is also something the JavaScript community discusses very often, hotly, with very mixed results. If you look at, for example, the discussions around backbone.js and all of those add ons like Marionette, you can easily see that this is a topic that can lead to hot blood and unclear results.

    Funnily, I recently fell into the “premature abstractions” trap pretty badly myself. I can’t share details in code, because it’s a yet to be released client project, but the story goes a little something like this (drop the bass):

    Your Design Principles

    A quick show of hands, how many of you have seen Dave Thomas’ closing keynote at the Scottish Ruby Conf?

    For me, this keynote was one of the most liberating moments of 2012. Why? Because in essence, he was telling me that I wasn’t crazy. He was telling my that my gut feelings of what I thought was wrong with the rails community were at least shared by someone who I hold in high regard.

    (By the way. If you have never read “The Pragmatic Programmer”, please do. Really.)

    Here’s the contents of the slide where Dave made me jump with joy:

    “Good” code is not the only code

    Dave said:

    I think there’s this increasing tendency in the ruby community to feel that somehow we’re the guardians of “good” programming taste. Right? Everything has to be done right. Everything has to be done using patterns. Everything has to be done using factories and injection and all this other kind of bullshit that I tried to escape when I left the Java world 1999. And what happens? You’re bringing it back to me. STOP!

    Yes, you could say that this resonated with me.

    Der Sündenfall der Deutschen Presse

    Lasst uns bitte aufhören über das Leistungsschutzrecht zu reden. Also ich meine Inhaltlich. Das haben genug Menschen bereits getan und alle die sich auskennen und nicht befangen sind kommen zu dem Schluss dass das Quatsch ist. It’s understood.

    Der große weiße Elefant im Raum ist, und darüber wurde zwar auch reichlich berichtet (wenn auch nicht in “der Presse”), aber vielen ist hier die Dimension glaube ich nicht wirklich klar: Das LSR stellt für die Deutsche Presse (und das gilt dieses Mal leider dann nicht mehr nur für Springer) einen echten Sündenfall dar.