Erfahrungen mit Docker

:: Docker

Ein Bekannter bat mich, einmal von meinen Erfahrungen mit Docker zu berichten. Also hier als Blog-Post dazu. Ich werde nicht beschreiben, was genau Docker ist und wie es funktioniert. Hierzu gibt es mittlerweile schon eine Reihe von guten Quellen (bspw. auf der Docker-Homepage, in der c’t war in Heft 17 (Seiten 146–151) auch eine ganz gute Einführung enthalten).

Aktenführung beim Beitragsservice

:: Allerlei, Behörden

Im Jahr 2012 habe ich mich rechtmäßig (was allerdings auch etwas Überzeugungsarbeit kostete) von der damaligen GEZ abgemeldet und die neue Beitragsnummer mitgeteilt, über die meine damalige Rundfunkgebühr verbucht werden sollte. Der GEZ war meine alte Beitragsnummer, die neue Beitragsnummer sowie meine Anschrift bekannt. Soweit so gut, besondere Veränderungen sind in der Zwischenzeit nicht eingetreten.

Mind the storage driver for Ubuntu cloud images (on Azure)

:: Docker, IT

A few days ago I wanted to build Firefox OS’ newest release for a friend. Because I did not wanted these GB of code, binaries etc. on my notebook I fired up an Ubuntu image on Microsoft Azure. I feared that at a certain point in the build process I may had to download everything to my local machine and therefore I installed Docker via a simple

sudo apt-get install docker.io

Then I started the build process as laid out on Mozilla’s Developer Network. But, during downloading the source code (that’s about 12 GB of Git repositories from Mozilla and Android), I got a “no more space left on device”. That was strange: I had a 100 GB volume attached to the VM and enough space and inodes left. After some searching I asked on the IRC channel and got a good hint: “What’s your storage driver?”

Well, I thought that it’s AUFS; I wanted to add “as usual” because AUFS was available on my notebook from the beginning. But a docker.io info gave me:

$ sudo docker.io info
Containers: 0
Images: 0
Storage Driver: devicemapper
 Pool Name: docker-8:1-131188-pool
 Data file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata file: /var/lib/docker/devicemapper/devicemapper/metadata
 Data Space Used: 291.5 Mb
 Data Space Total: 102400.0 Mb
 Metadata Space Used: 0.7 Mb
 Metadata Space Total: 2048.0 Mb
Execution Driver: native-0.1
Kernel Version: 3.13.0-29-generic
WARNING: No swap limit support

I then learned that somehow the DeviceMapper driver only allows a certain amount of diffs and I reached that amount with my build process. (Maybe it’s possible to relax that restriction but I do not know how.)

I learned as well that the Ubuntu cloud image that is provided by Microsoft Azure doesn’t have AUFS support. Therefore Docker uses the DeviceMapper storage driver instead. After I installed the AUFS support I could export the images, change the storage driver and import the images again.

It would be nice seeing the Docker documentation being more detailed on those storage drivers.

(Update 2014–10–23) Thanks to this blog post from Iron.io I found some documentation of the devicemapper storage driver. It is located in the Repository.

DateTime conversion can be tricky

:: JavaScript, Common Lisp, IT

I wrote a small Lisp application and a JavaScript client gets some data from that application. All time stamps are returned as “Lisp” time stamps, i.e. an integer with seconds where zero equals Jan 01 1900.

In the JS client the time stamp is then converted to JS time stamps, i.e. millisconds where zero equals Jan 01 1970.

When testing the application I noticed that sometimes the displayed date is one day behind. For example in the data base I have Jan 05 1980 but in JavaScript I get a Jan 04 1980. But some other dates worked: A time stamp Jan 05 1970 was correctly converted to Jan 05 1970.

I had a look into the JavaScript code and found:

convA = function(ts) {
  tmp = new Date(ts*1000);
  tmp.setFullYear(tmp.getFullYear() - 70);
  return tmp.getTime();
}

It’s likely the developer thought: “Well, it’s millisecond instead of second. Therefore I multiply by 1,000. But then I am 70 years in the future and I have to substract 70 years and everything will be ok.”

After thinking a while I came to the conclusion: Of course not!

The developer made the assumption that there are as many leap years between 1900 and 1970 as between ts and ts+70. Obviously that assumption does not hold for all time stamps. And therefore sometimes the resulting JavaScript date is one day behind.

So a better solution would be to substract all seconds between 1900 and 1970 from ts, multiply by 1,000 and treat this as a JavaScript time stamp. Perhaps best would be to do the conversion in the Lisp process and only deliver a JavaScript-like time stamp.

I learned something about symbols and packages

:: Common Lisp, IT

I am using Common Lisp for developing a web application. Several days ago a new part of this application didn’t worked as supposed and I spent a considerable large amount of time in finding the bug. It was a very simple problem with symbols where I mixed something up.

In the application the web server somewhen gets some JSON data from the browser. It is then converted to Lisp object using the CL-JSON package. This package converts JSON objects to a-lists and converts the member keys to symbols (see CL-JSON’s documentation. I then wanted to look something up in that a-list and failed.

I wrote a small test case to show the effect and explain what went wrong.

(ql:quickload '("hunchentoot" "cl-who"))
;; direct loading via ql only for demonstration purposes, normally I
;; would use a asdf:defsystem for that.

(in-package :cl-user)

(defpackage :my-app (:use :cl))

(in-package :my-app)

(defparameter *my-a-list* 
  '((foo . 100)
    (bar . 200)))   ;; in the real application this a-list is
		    ;; generated by a JSON-to-lisp conversion by
		    ;; CL-JSON; in CL-JSON the object member keys are
		    ;; converted to symbols.

(defun get-value (key)
  "Returns the value with KEY from *MY-A-LIST*."
  (cdr (assoc (intern (string-upcase key)) *my-a-list*)))

(hunchentoot:define-easy-handler (web-get-value :uri "/get-value") (id)
  (cl-who:with-html-output-to-string (*standard-output* nil :prologue t)
    (:p (cl-who:fmt "Value of ~a is: ~a" id (get-value id)))))

(defun start ()
  (hunchentoot:start (make-instance 'hunchentoot:easy-acceptor :port 4242)))

So on the REPL everything looks fine: MY-APP> (get-value "foo") 100 MY-APP> (get-value "bar") 200 MY-APP>

But when I used my web browser to give me these results as well I got something strange. For example here are some results when using curl: ~> curl http://localhost:4242/get-value?id=foo <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <p>Value of foo is: NIL</p>

I was puzzled: The value is NIL?

After some debugging I found out that the easy handler from Hunchentoot runs with *package* set to COMMON-LISP-USER (and not to MY-APP as I implicitly assumed). That means that assoc looked up COMMON-LISP-USER::FOO in the a-list where the keys are MY-APP::FOO and MY-APP::BAR. And this test fails. Therefore NIL is returned which is correct.

So I rewrote the get-value function to: (defun get-value (key) "Returns the value with KEY from *MY-A-LIST*." (cdr (assoc (intern (string-upcase key) (find-package :my-app)) *my-a-list*))) Now the symbols are interned in the same package and everything went well: ~> curl http://localhost:4242/get-value?id=foo <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <p>Value of foo ist: 100</p> ~> curl http://localhost:4242/get-value?id=bar <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <p>Value of bar ist: 200</p>

Therefore I was reminded to think about packages when interning symbols. A good guide to symbols and packages could be found in this document: The Complete Idiot’s Guide to Common Lisp Packages.

Unicode support for Octopress

:: Docker, Octopress, IT

Well, it seems Octopress/Jekyll would like to have a locale set for UTF–8 support. I followed this (text in German) hint and now my Dockerfile looks like this:

# dockerfile for octopress

FROM ubuntu:14.04
MAINTAINER krrrcks <krrrcks@krrrcks.net>
ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update; \
  apt-get -q -y upgrade
RUN /usr/sbin/locale-gen en_US.UTF-8; \
  update-locale LANG=en_US.UTF-8
RUN apt-get -q -y install git curl; \
  apt-get clean
RUN git clone git://github.com/imathis/octopress.git /opt/octopress
RUN curl -L https://get.rvm.io | bash -s stable --ruby
ENV HOME /root
RUN echo "export LC_ALL=en_US.UTF-8" >> /root/.bashrc
RUN echo "export LANG=en_US.UTF-8" >> /root/.bashrc
RUN echo "source /usr/local/rvm/scripts/rvm" >> /root/.bashrc; 
RUN /bin/bash -l -c "source /usr/local/rvm/scripts/rvm; \
  rvm install 1.9.3; \
  rvm use 1.9.3; \
  rvm rubygems latest; \
  cd /opt/octopress; \
  gem install bundler; \
  bundle install; \
  rake install" 
RUN echo "rvm use 1.9.3" >> /root/.bashrc

WORKDIR /opt/octopress
EXPOSE 4000
CMD ["/bin/bash"] 

After playing around with Docker and Octopress I put the whole /opt/octopress folder on my host machine and then restarted the image with the -v flag. Therefore I can edit the files on my host machine with my favorite editor and use the container only for producing the HTML files, for preview and for publishing.

The rake preview is a neat feature because the server always looks for changed files and produces the HTML files on the fly. That means I can edit the files in my editor and could see the resulting pages in a browser nearly the same time.

Relaunch fast fertig

:: Homepage, Octopress

So, nach einigem Gefummel mit Ruby und Octopress und den Weiterleitungsmöglichkeiten meines Hosters ist die Sache wohl bald so am laufen, wie ich mir das vorstelle. Es fehlt noch ein wenig das Eindeutschen der Octopress-Texte und vielleicht so eine Kategorien-Liste rechts.

Ansonsten habe ich, um der “Gem-Hölle” zu entgehen, das Octopress in einen Docker-Container gepackt; hier ist ein Link zu dem verwendeten Dockerfile. Ich habe es dann auch so gemacht, dass ich das /opt/octopress-Verzeichnis auf den Host-Rechner verschoben habe und in den Container als Volume hineinpacke. So kann ich auf dem Host-Rechner ganz bequem mit dem Emacs die Files editieren und verwende den Container nur noch für die Erstellung, das “Preview” und für das Hochladen.

Die “preview”-Funktion ist übrigens ein wirklich nettes Features: Die kann man derart nutzen, dass ein Webserver den aktuellen Stand lokal ausliefert. Dabei beobachtet der Dienst, ob sich Dateien ändern und erstellt unmittelbar wieder die HTML-Seiten. So kann man bequem im Editor das Markdown bearbeiten und im Browser daneben das fertige Ergebnis beurteilen.

Ein Relaunch

:: Homepage, Octopress

Ich denke, mir geht es wie vielen: Nach einer gewissen Zeit, in der das bisherige Blog, die bisherige Web-Seite nicht gepflegt wird, steht irgendwann ein Neubeginn an, dann gleich mit einem “Relaunch” und vielen neuen und guten Vorsätzen.

Ich bin zwar nach wie vor ein großer Freund und Nutzer von Org-Mode, aber für das Führen eines Blogs erschien mir der Ablauf doch immer recht umständlich. Ich habe mir daher einmal Octopress angeschaut (vor allem, da die Blog-Funktion von Org-Mode, die ich nutzte, auch mit Jekyll arbeitete) und finde das eigentlich ganz hübsch. Einzig so ein wenig “eindeutschen” sollte ich es noch. Außerdem ist das Umleiten von der bisherigen Domain hierher noch ein wenig kaputt; mein bisheriger Hoster lässt mich das nicht so schön mit CNAME-Einträgen gestalten, wie das eigentlich gedacht ist.

Nun also ein Relaunch, aber ohne große Vorsätze. Es wird hier also nur selten und nur wenig zu lesen geben.

My Dockerfile for setting up Octopress

:: Docker, Octopress, IT

After my trouble with installing all the dependencies for Octopress I came up with the following Dockerfile for Docker. This follows the instructions from the Octopress homepage and uses RVM for managing the ruby dependencies.

# dockerfile for octopress

FROM ubuntu:14.04
MAINTAINER krrrcks <krrrcks@krrrcks.net>
ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update; \
  apt-get -q -y upgrade
RUN apt-get -q -y install git curl; \
  apt-get clean
RUN git clone git://github.com/imathis/octopress.git /opt/octopress
RUN curl -L https://get.rvm.io | bash -s stable --ruby
ENV HOME /root
RUN echo "source /usr/local/rvm/scripts/rvm" >> /root/.bashrc; 
RUN /bin/bash -l -c "source /usr/local/rvm/scripts/rvm; \
  rvm install 1.9.3; \
  rvm use 1.9.3; \
  rvm rubygems latest; \
  cd /opt/octopress; \
  gem install bundler; \
  bundle install; \
  rake install" 
RUN echo "rvm use 1.9.3" >> /root/.bashrc

WORKDIR /opt/octopress
EXPOSE 4000
CMD ["/bin/bash"]