[ about | machines | pics | projects | repos | weblog ]

[ ~derf ]

Projects

App::Raps2 v0.53
Password safe, take two
App::Slackeria v0.12
Project status overview
Travel-Routing-DE-VRR v2.02
Interface to the VRR online itinerary service

News

Travel-Status-DE-VRR-1.02.tar.gz (signature)

  • Patch by Roland Hieber: respect proxy environment variables
2013-03-16 00:00

icli v0.44

icli-0.44.tar.gz (signature)

  • Ignore module field in the icinga status file
  • Add -o / --overview option to display a 'tactical overview'-style host and service listing
2013-03-05 10:25

Backups and Monitoring

(tldr: Beware of pipes with set -e. And write more checks.)

At the Chaosdorf, we have an automated weekly backup of all servers and other hosts. The script uses set -e right at the start and reports its success with send_nsca just before quitting. A freshness threshold is used to produce an alert if a backup run does not report in time.

This sounds like nothing can go wrong without being noticed. However, there is a problem: backup_external uses pipes. And in a pipe, only the return value of the last command is actually evaluated:

descent ~ > ( set -e; false | true; echo foo )
foo

So, if something along the way (e.g. tar or gpg) has a problem, the script will happily run along and report its success at the end. Which will result in something like this:

flux ~ > sudo ls -l /chaosdorf/backups/09 | fgrep feedback
-rw-r--r-- 1 chaosdorf chaosdorf    0 Mar  4 00:03 feedback.chaosdorf.dn42_etc.tar.xz.gpg
-rw-r--r-- 1 chaosdorf chaosdorf  24K Mar  4 00:03 feedback.chaosdorf.dn42_packages
-rw-r--r-- 1 chaosdorf chaosdorf    0 Mar  4 00:03 feedback.chaosdorf.dn42_root.tar.xz.gpg
-rw-r--r-- 1 chaosdorf chaosdorf    0 Mar  4 00:03 feedback.chaosdorf.dn42_usr_local.tar.xz.gpg
-rw-r--r-- 1 chaosdorf chaosdorf    0 Mar  4 00:03 feedback.chaosdorf.dn42_var_local.tar.xz.gpg
-rw-r--r-- 1 chaosdorf chaosdorf    0 Mar  4 00:03 feedback.chaosdorf.dn42_var_log.tar.xz.gpg

In this case, it was likely GPG refusing to work on a readonly filesystem (it's an embedded host running on an SD card, so making it readonly makes sense).

The good thing about this is: The failed backups are all empty files, and finding empty files is as easy as running find -size 0. So now we have a second check on the receiving host to alert me whenever an obviously failed backup is transferred.

So:

  • Never, ever trust a single check
  • If you have the disk space, keep more than just the most recent three backups (I actually did this right)

Travel-Status-DE-VRR-1.01.tar.gz (signature)

  • Properly parse "tram canceled" information (EVAG, maybe also VRR)
2012-08-28 00:00

icli v0.43

icli-0.43.tar.gz (signature)

  • Add missing documentation for -a / --acknowledge
  • Add -m / --match option to select on plugin output
2012-08-19 16:05

Semantic Mediawiki Examples

It's been two months since we installed Semantic Mediawiki and Semantic Forms in the Chaosdorf Wiki, and I've got to say it's pretty nice.

The basic idea of semantic mediawiki is: Pages are not just there, but can have Properties (like a language, hostname, preview image etc.), and based on these properties one can create queries, such as selecting all pages in Category:Projects which have a description and a preview image.

This article is not meant as an Introduction to Semantic Mediawiki (see the link for that), just some real-world examples of what's possible. I tried to sort the examples by ascending complexity.

Eliminating Category: pages

wiki/Foo certainly looks nicer than wiki/Category:Foo. It's already possible to set a #REDIRECT from the former to the latter, but that still leaves the long page title as default. Using the query

{{#ask: [[Category:Foo]]
 | format = ul
}}

you get a listing very similar to a category page. If the page containing the query is in the category itself, it will even be marked bold.

Example: https://wiki.chaosdorf.de/History

Listing pages with a certain property (e.g. items in a certain room)

Assume you have a page for every room or location of your whatever (Hackerspace in our case), and also a page for every item inside. If you have a property describing the item's location (which is very easy with Semantic Forms, it even provides autocompletion of existing locations), there's a simple way to have every location page list its contents.

Each item page specifies its location as [[Has location::Someplace]], the query is

{{#ask: [[Has location::{{PAGENAME}}]]
 | format = ul
 | intro=Beinhaltet:
}}

The intro text is only displayed if there is something to list, so if the location has no items, nothing will be displayed.

Example: https://wiki.chaosdorf.de/Serverraum (the query is hidden inside https://wiki.chaosdorf.de/Template:Location)

Pretty-listing pages with certain properties (e.g. preview images)

With projects or resources in a wiki, it'd be very nice to have more than a simple list of page names, like a table with preview images and short descriptions. It is of course possible to create one manually, but mainting it quickly becomes a pain or is simply forgotten. Also, duplicate information sucks.

So: Create a Mediawiki Template and a Form for project/resource pages, which contain (with properties) its name, description, preview image (if available), etc. Semantic Mediawiki allows formatting its query results with templates, so it's not hard to get a reasonably nice list of projects with images.

Also, Semantic Forms allows uploading the preview image from inside the Form, so when creating a project/resource page one does not even need to open Special:Upload in another tab and copy-paste filenames.

The query in this case is

{{#ask: [[Category:Projects]] [[Has image::+]] [[Duplicate::false]]
| format = template
| template = PreviewSMW
| outrotemplate = PreviewEnd
| link = none
| ?Has description
| ?Has image
}}

So it formats each result with Template:PreviewSMW, does not link to the pages by itself (the template will do it), and also passes the properties Has description and Has image of each page to the template. Example: https://wiki.chaosdorf.de/Projects

Library system with lend book / return book buttons

This is not what Semantic Mediawiki is meant for, but a fun experiment. Create a page for each book you have (in our case in the custom Book namespace). The most important part here is the lent by property, which corresponds to the lent argument of Template:Book and Form:Book and, if set, is the user who has currently lent it.

A list of all books can be obtained in the standard query fashion:

{{#ask: [[Book:+]]
 | format = table
 | ?Has author = Autor
 | ?Has ISBN = ISBN
 | ?lent by = ausgeliehen?
}}

Thanks to the #autoedit function, it's possible to add lend / return buttons to each book page (via Template:Book, which we already have anyways to fill in the properties and have a book form). First, let's have some markup:

{{#if:{{{lent|}}}|
{{#autoedit:form=book
 | target=Book:{{PAGENAME}}
 | link type=button
 | link text=zurückgeben
 | Book[lent]=
 | Book[lent at]=now
 | namespace=Book
 | reload
}}
|
{{#autoedit:form=book
 | target=Book:{{PAGENAME}}
 | link type=button
 | link text=ausleihen
 | Book[lent]={{#USERNAME:}}
 | Book[lent at]=now
 | namespace=Book
 | reload
}}
}}

The {{#if: condition | yes | no }} part comes from Parserfunctions, which is not part of Semantic Mediawiki / Semantic Forms, but also nice to have. The two buttons are mutually exclusive, so it makes sure only the correct one is shown.

target=Book:{{PAGENAME}} tells autoedit to edit the current page, and reload makes sure a wiki user sees the effect of their actions. (otherwise the page would be changed, but the user would still see the its old version).

The interesting parts are Book[lent] and Book[lent at]. Right in the first line, we tell autoedit to use the "Book" form, and in these two places we access parameters of that form. The first one works like using {{Book | ... | lent = someuser}}.

In Book[lent at]=now, the "now" becomes the YYYY/MM/DD representation of today. This works because in Form:Book, the lent at parameter declared to be a date. (If lent at corresponded to a property of type date, this would not even be neccessary).

2012-06-01 00:00

icli v0.42

icli-0.42.tar.gz (signature)

  • Add -U / --as-contact option to only operate on services visible to a certain contact
  • New dependency List::MoreUtils
2012-04-15 00:00

icli v0.41

icli-0.41.tar.gz (signature)

  • Add icli host/service as icli -h host -s service shortcut
  • Add -u / --force-recheck option (Patch by Hugh Brown)

Travel-Routing-DE-VRR-2.02.tar.gz (signature)

  • Patch by Roland Hieber: respect proxy environment variables

First, there's an excellent parallel port howto on epanorama.net.

What I'd like to add: You can't just control LEDs or relay coils / transistors with the parallel port, it's also pretty easy to talk to microcontrollers (an ATTiny 2313 in my case).

interfacing

Basically, you need one or two parallel port pins as inputs to the AVR, and then come up with some sort of protocol.

one-wire

No clock signal, so we rely on proper timing. The most primitive solution is to toggle the pin n times and have the AVR increment a counter on every toggle, then read the counter value after a fixed time without counter increment has passed. Excruciatingly slow, but dead simple to implement both on the computer and the AVR.

two-wire

One pin for data, one for the clock (e.g. set data, then toggle clock, set an AVR interrupt on rising edge on clock to read data pin). I didn't try it yet.

more?

So far, the computer talks to the AVR but not vice versa. Two-way communication shouldn't be hard, but I didn't try it yet. If I do, I may write a new entry.

hardware

To be on the safe side, I decided to completely isolate the parallel port from the rest of the circuit.

components used:

  • KB817 opto-coupler. forward voltage ~1.2V, collector-emitter voltage 35V/6V
  • 3.3K resistor between opto-coupler and parallel port, since the parallel port provides 2.4 to 5V
  • BC338-40 transistor to make sure the opto-coupler output is registered by the microcontroller

The rest is usual stuff. The parallel port circuit is located in the bottom left of the schematic.

software

microcontroller

Assuming the transistor (optocoupler) is connected to INT0

  • set an interrupt on INT0 rising edge: increment counter and reset timeout
  • set a timer interrupt: check timeout, if zero handle counter as command

code snippet:

ISR(INT0_vect)
{
    cli();
    command++;
    cmd_wait = 6;
}

ISR(TIMER1_COMPA_vect)
{
    if (cmd_wait)
        cmd_wait--;
    else if (command) {
        run_command();
        command = 0;
    }
}

the whole file is available on github: main.c

computer

I'm using the parapin library, see pgctl.c on github. Note that timing is important here, so I'm running the code at the lowest possible niceness.

That's it. An example project is available as derf/pgctl on github.

The whole point of this post is: interfacing with the parallel port is easy. It doesn't have a future, but if you still have one, you can use it for quite a lot of things.

Travel-Routing-DE-VRR-2.01.tar.gz (signature)

  • Fix bug in departure_stop_and_platform where the arrival_stop was used in case of unknown platforms
  • Document -E / --extended-info in efa(1)

(because it's way more awesome than using just a Wiimote)

For this to work, you will need an up-to-date cwiid installation, the most important part is commit b54bd05.

Next, check out neverball svn and apply tilt wii.patch.

Build it (make ENABLE_TILT=wii), then run ./neverball and press the Balance Board's red sync button.

If everything works out, the Power LED should turn on. Now you can use the mouse / keyboard to select the level to play, and then control the level itself using the Balance Board.

There's a (really low-quality) Video (also on Youtube) available. I hope to have a higher resolution and less laggy one by saturday.

For the record, the complete patched tilt wii.c is also available.

Update:
I've got a new Video! (also on Youtube). As you can see, controlling neverball this way is pretty hard :)

With the most recent Patch, you can control the level with the balance board and the camera and menus with the wiimote. Usage: Start neverball, connect wiimote, wait 'til the leds stop blinking, then connect balance board. There are some timing issues right now, so if it crashes while connecting the devices, just try again.

Travel-Routing-DE-VRR-2.00.tar.gz (signature)

  • Now uses the efa.vrr.de XML interface
  • properly decode UTF-8 strings

Travel::Routing::DE::VRR::Route

  • new accessors: duration, ticket_type, fare_adult, fare_child and vehicle_time

Travel::Routing::DE::VRR::Route::Part

  • arrival_stop and departure_stop no longer include the platform, use arrival_stop_and_platform / departure_stop_and_platform instead
  • New accessors: arrival_date, arrival_platform, arrival_sdate, arrival_stime, arrival_stop_and_platform, delay, departure_date, departure_platform, departure_sdate, departure_stime, departure_stop_and_platform, via
  • ->extra no longer contains information about delays, use ->delay instead

efa

  • New -E / --extended-info option to display travel fares

Travel-Status-DE-DeutscheBahn-1.01.tar.gz (signature)

  • Detect ambiguous input errors (available via errstr)
  • Add language switch for additional information (german/english)

Travel::Status::DE::DeutscheBahn::Result

  • Result: Add line (== train) and date accessors

db-ris

  • Add -l / --lang option

Travel-Status-DE-VRR-1.00.tar.gz (signature)

  • switch to efa.vrr.de XML interface
  • properly decode UTF-8 strings
  • Switch from WWW::Mechanize to LWP::UserAgent
  • Add ->lines and Travel::Status::DE::VRR::Line describing all lines serving the given stop

Travel::Status::DE::VRR::Result

  • Add countdown, date, delay, lineref, platform_db, sched_date, sched_time and type accessors

efa-m

  • Add -L / --linelist option
2011-11-02 00:00

App::Hashl v1.00

App-Hashl-1.00.tar.gz (signature)

  • Add "hashl list regex" to filter listed files by name
  • Do not fail when encountering zero-length files

Travel-Status-DE-VRR-0.02.tar.gz (signature)

efa-m

  • Add --platform / --line options to filter output
  • Align times properly if departure list contains a daychange
  • Make --date and --time actually work

Travel-Routing-DE-VRR-1.06.tar.gz (signature)

  • Fix dependencies. We claimed to use WWW::Mechanize but in fact only need LWP::UserAgent

Travel-Status-DE-DeutscheBahn-1.00.tar.gz (signature)

db-ris

  • Add --full-route option

Travel::Status::DE::DeutscheBahn

  • Fix bug when receiving no platform numbers from the DB RIS

Travel::Status::DE::DeutscheBahn::Result

  • Add route_timetable accessor containing station names and their corresponding arrival times