Claasloader

My IT blog (and other stuff)

Enable PS/2 Keyboard in Ubuntu 12.04

| Comments

Installing Ubuntu 12.04 on one of our servers, it turned out that the PS/2 keyboard didn’t work. Luckily, only a few steps were required to make it work - so if you’ve got the same problem, you might give it a try:

1
2
3
4
5
6
7
8
$ sudo vi /etc/default/grub

# Change 
#   GRUB_CMDLINE_LINUX=""
# to
#   GRUB_CMDLINE_LINUX="i8042.nopnp"

$ sudo update-grub

And restart.

Bounce Mail Handling

| Comments

If you have an application which sends out e-mails, you would sooner or later be facing the problem that some of your e-mail addresses become out of date. It would be nice if your system could detect this automatically and take invalid addresses out of the database.

Unfortunately, there’s no standard method for that kind of problem. One of the difficulties is to detect a bounce e-mail. Different e-mail servers each create very different types of bounce backs, so it’s not trivial to distinguish a bounce e-mail from a regular e-mail.

Given the fact that a lot of people should face this, there are surprisingly few software libraries around. My research lead to

To be honest, I haven’t tried any of these. The two inactive projects are no good choices for obvious reasons, and I neither want to use a commercial product if I can avoid it. So what’s left is Mail::DeliveryStatus::BounceParser, but well - we’re not a Perl shop and actually there’s a slightly simpler solution.

A Simple Procmail Recipe

Procmail is a mail delivery agent (MDA), i.e., on your e-mail server it delivers mail to your local mailboxes. If you run a Linux e-mail server, it’s easy to integrate it; just Google, there are tons of tutorials. It includes a DSL for filtering out e-mails (actually it’s been around much longer than the term “DSL” is being hyped). A filtering rule is called recipe in Procmail. We use the following recipe to detect bounce mails:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
BOUNCE="((^Subject:( )*delivery status notification)|\
(^Subject:( )*delivery failure)|\
(^Subject:( )*delivery notification: delivery)|\
(^Subject:( )*failure (notice|delivery))|\
(^Subject:( )*invalid (e)?(-)?mail account)|\
(^Subject:( )*mail could not be delivered)|\
(^Subject:( )*mail delivery fail(ed|ure))|\
(^Subject:( )*(e)?mail delivery error)|\
(^Subject:( )*mail system error)|\
(^Subject:( )*message not delivered)|\
(^Subject:( )*message rejected)|\
(^Subject:( )*nondeliverable (e)?mail)|\
(^Subject:( )*non( )?delivery report)|\
(^Subject:( )*notice: mail delivery status)|\
(^Subject:( )*returned (e)?mail)|\
(^Subject:( )*returnmail)|\
(^Subject:( )*undeliverable mail)|\
(^Subject:( )*undeliverable message)|\
(^Subject:( )*undeliverable:)|\
(^Subject:( )*undelivered mail)|\
(^Subject:( )*warning: could not send message)|\
(^Subject:( )*warning:( )?message .* delayed)|\
(^Subject:( )*message status - undeliverable)|\
(^Subject:( )*email delivery error)|\
(^From:.*nondelivery@)|\
(^From:.*postmaster)|\
(^From:.*mail delivery (sub)?system)|\
(^From:( )*(e)?(-)?mail administrator)|\
(^From:.*mailer-daemon)|\
(^From:.*fetchmail-daemon)|\
(^From:.*admin@))"

:0:
* $ ${BOUNCE}
/var/mail/mybounce

Of course it’s not perfect, you won’t get a 100% detection rate. But it’s a simple and pragmatic approach which - at least for us - worked pretty well so far.

Dimensional (Volumetric) Weight Calculations in Ruby and Excel

| Comments

When you are writing software for logistics or trading companies, sooner or later you might have to deal with the concept of dimensional weight (sometimes called ‘volumetric weight’).

What Is Dimensional (Volumetric) Weight?

The first time I faced the term of dimensional weight, I was quite confused. So let me explain the background a little bit.

Simply spoken, dimensional weight is an additional number used when you ship a box via one of the big carriers like UPS, FedEx or DHL. The basic formula is

dimensional weight (kg) = box volume (cm³) / 5000

(There are variants of this formula, but that’s not important for the moment. The logic behind is always the same.)

Remembering your physics course back in high school, you see that units are not the same on the left and the right side of this equation. Weight is measured in kg, volume is measure in cm³, so isn’t this just a mistake? Well, what you have to accept is that ‘dimensional weight’ has no physical meaning. Rather it is a number used by logistics companies to simplify their cost calculations.

When - let’s say - UPS ship a box from A to B, they probably use a truck for transportation. The truck has a maximum loading weight, which is maybe 1000 kg. So the heavier your box is, the more capacity of the truck is used for your box. As a simplification, we might assume that cost is proportional to the weight. (For 1 kg you may pay 5$, for 2kg 10$.)

But there’s another limiting factor in the truck: the volume. You can only put up to maybe 10 m³ of stuff into the truck. So the second thing UPS needs to consider is how volumetric your box gets. If your box is very light but also very big, this somehow needs to go into the cost calculation. Dimensional weight is a concept to make this calculation easier. You can put it like this:

For a light and big box, dimensional weight is the weight of that tiny and heavy box which creates the same cost of transportation.

Implementation Details

At least UPS claims that dimensional weight is always rounded up to the next whole kilogram. So even for 1.01 kg, they may charge you 2 kg.

Sample Code

Note: You should check proper rounding method for your carrier and your country. The implementations below are for UPS.

Ruby

1
2
3
4
5
6
7
8
9
10
11
12
13
  def dimensional_weight(dims)
    # dims = [cm, cm, cm]
    return 0 if dims.size != 3
    round_ups(dims.inject(:*) / 5000.0)
  end

  # Round to the next 0.5 or 1 kg
  def round_ups(flt)
    x = flt - flt.floor
    return flt if x.zero?
    return flt.floor + 0.5 if x <= 0.5
    return flt.floor + 1.0
  end

Excel

1
2
3
4
5
6
|   | A      | B      | C      | D      | E      | F        | G                                 |
| 1 | Number | Weight | Size 1 | Size 2 | Size 3 | Subtotal | Dimensional                       |
|   | of     |        |        |        |        | Weight   | Weight                            |
|   | Boxes  | (kg)   | (cm)   | (cm)   | (cm)   | (kg)     | (kg)                              |
|---+--------+--------+--------+--------+--------+----------+-----------------------------------|
| 2 | 3      | 11.00  | 40     | 41     | 42     | 33.00    | = CEILING(C2*D2*E2/5000,0.5,1)*A2 |

Install YouTrack 4.2 on Ubuntu 12.04

| Comments

These are some short instructions how to install Jetbrain’s YouTrack on Ubuntu 12.04. Disclaimer: This setup is more for playing around and trying the software, for a productive environment with lots of users you would probably need some more optimizations.

Tested with Ubuntu 12.04.02 64 and YouTrack 4.2.2.

Download

On the download page, select “Java EE Container”. Clicking on Download will give you a .war file (the .jar file would not be suitable for the setup I’m going to describe here).

Preparation

I created a dedicated VM for YouTrack. During Ubuntu’s setup wizard, you can select ‘Tomcat Server’. This would install all needed packages (except the YouTrack package itself). However, if you are using an existing machine, all you need is to

1
$ sudo aptitude install tomcat6

This would install all necessary packages.

Configure Tomcat

Your Tomcat needs a little tweaking. First, you need to setup the admin user. Add these lines to /etc/tomcat6/tomcat-users.xml:

/etc/tomcat6/tomcat-users.xml
1
2
<role rolename="manager-gui"/>
<user username="tomcat" password="s3cret" roles="manager-gui"/>

As described in the installation manual, you need to increase the JVM heap size to at least 512MB. I first tried to run YouTrack on a VM with 512MB total RAM and heap size of 128MB, but YouTrack wouldn’t start. Running the server with 2GB RAM and 768MB heap size works fine. So open /etc/default/tomcat6 and edit the line

/etc/default/tomcat6
1
JAVA_OPTS="-Djava.awt.headless=true -Xmx128m -XX:+UseConcMarkSweepGC"

to increase your heap size (i.e., change the 128 to at least 512).

Finally, restart Tomcat

1
$ sudo /etc/init.d/tomcat6 restart

Open http://yourserver:8080 to see if it works.

Configure YouTrack

There’s one little problem we need to fix before deploying YouTrack. YouTrack comes with a built in database which is by default located in {user.home}. $user is the tomcat6 user and tomcat6’s home directory is /usr/share/tomcat6. Unfortunately, /usr/share/tomcat6 is not writable for user tomcat6, so YouTrack could not create its datastore and deployment would fail with an error message like

1
ExodusException: Exodus 1.0.0: Failed to create directory: /usr/share/tomcat6/teamsysdata

There are a few solutions for this problem, I personally chose to tell YouTrack to use a different path. To achieve this, we first have to create a writable directory for YouTrack’s datastore:

1
2
$ sudo mkdir /opt/tomcat6
$ sudo chown tomcat6:tomcat6 /opt/tomcat6/

and then configure YouTrack to use this path:

  • Open youtrack-4.2.2.war with Ubuntu’s default zip program (double click)
  • From inside the archive, open WEB-INF/web.xml
  • Replace ${user.home}/teamsysdata by /opt/tomcat6/teamsysdata and ${user.home}/teamsysdata-backup by /opt/tomcat6/teamsysdata-backup
  • Save the file (confirm to update the .war archive)

Deploy

Now we’re ready to deploy YouTrack. Open the Tomcat Manager on http://yourserver:8080/manager/html, use the ‘Select WAR file to upload’ form, and click Deploy.

YouTrack should now be available under http://yourserver:8080/youtrack-4.2.2

Ubuntu 11.04 64 on Thinkpad T410(i)

| Comments

My seven years old Asus D3000 getting a bit slow, I decided it was time for a new laptop. So what to choose? Basically, I was thinking about three options:

  • MacBook Pro
  • Dell Vostro V13
  • Lenovo Thinkpad R or T series

What I don’t like about the Mac is the keyboard. One thing I want to do with the laptop is programming, but German Mac keyboards are missing keys for [, ], { and } - characters which are quite useless for “ordinary” users, but important when you are programming. (Of course you can type these characters on a Mac too - but they are not printed on the keys and AFAIK you need to type three keys at the same time, something like Alt + Shift + 8).

The Vostro V13 looks pretty good, it has a non glossy display, an aluminium body and is very thin. But in the end a 13 inch display is too small for me. So it would be the Thinkpad.

I used a Thinkpad R500 at work for a few years and really enjoyed it. What I like about the Thinkpads:

  • Fantastic keyboard
  • “Classic” design
  • Generally good reputation for Linux support

Which Thinkpad to choose? I compared a lot of models, not really caring about whether they shipped with a Core2, i3, i5 or whatever, but what does matter to me is LCD resolution. Sadly, most of the R models I found (and even some of the cheaper T models) came with 1366x768 - which is much too low from my point of view. Basically, I picked the cheapest one which could give me more than 1366x768 - a T410i with 1440x900.

First Impression

Back in the days 2004, when I installed SuSE 8.2 on my Asus D3000, Linux support for laptops was … you know, somehow problematic. I even had to compile the video driver on my own. Not mentioning the things I never get running, like external VGA output or Standby mode. (Nethertheless I was quite happy with Linux on this laptop, in the end basic things were working and I used it every day for my master thesis.)

So now, installing Ubuntu 11.04 (64bit version) on the brand new Thinkpad T410i, things are so much easier! Almost everything works out of the box. (Well … almost everything.)

Details

  • Touchpad, TrackPoint, scrolling, volume control keys: Works
  • WLAN, Bluetooth: Works
  • Video: Works (during installation, you are asked whether to automatically install 3rd party drivers; I answered yes, and it installed the official nVidia drivers)
  • Keyboard Illumination: Works
  • Standby, Hibernate: Works
  • Webcam: Works (tested with Skype)
  • Watching DVD: Works
  • Microphone Mute key: Doesn’t work
  • Display Brightness keys (Fn + Home/End): Works after fixing xorg.conf:
1
2
3
4
5
Section "Device"
    Identifier "Default Device"
    Option "NoLogo" "True"
    Option "RegistryDwords" "EnableBrightnessControl=1"
EndSection

External Monitor Setup (VGA)

This gave me the most headaches, and I’m still not happy with it: The default Ubuntu control System | Administration | Multiple Monitors is useless in my case. It doesn’t show the external monitor. There’s a special control coming from nVidia: System | Administration | NVIDIA X Server Settings | X Server Display Configuration. With this tool, I couldn’t manage to disable the laptop LCD and use the external monitor as the one and only display. However it works to run the laptop in “TwinView” mode, i.e. extending the display to the external monitor. You can even set up the external monitor as the primary display, but this requires some tweaking like changing the position of the notification bubbles. Still, changing the monitor setup (say you want to disconnect from the external monitor to carry the laptop to somewhere), requires a X server restart which basically means you have to logout and login again. This is really a pain. In Windows, you just press Fn+F7, change the monitor setup, and all your applications and windows keep open.

Ubuntu Note

Please note that Ubuntu 11.04 ships with a kernel which has a higher power consumption. So you might consider to use 10.10 instead. (Maybe the standard Multiple Monitor control is working in this version, too.) I haven’t tried that yet.

Resources

Thinkpad T Series vs R Series

I couldn’t find a good description about the difference between the R and the T series, so I list this here for those of you who have to decide between R and T. Note: I can only compare a R500 with a T410i which is not totally fair since these models are not of the same generation.

  • T models are a little bit thinner and lighter
  • The T body is more robust, it is somehow specially hardened
  • The surface of the T is totally non reflecting while the R surface is a little reflecting (it is not a glossy surface however). On the T surface you see fingerprints easily. On the R surface, you don’t see them and it’s also easier to clean. On the other hand, the R surface looks somehow cheaper
  • In general, the T’s are more powerful regarding CPUs etc., but this highly depends on the concrete model

Find Bad Oracle Identifiers (ORA-00972)

| Comments

Today I had to struggle with some Oracle errors like

1
ORA-00972: identifier is too long

(Oracle identifiers can have a length up to 30 chars only.) Unfortunately, Oracle doesn’t tell you which identifier is too long. For me following Ruby snippet did the trick. Just copy your SQL into a file view.sql and run this one-liner:

1
IO.read("view.sql").split(/[\.\(\s]/).each { |c| puts c if c.length > 30 }