bash quicksand 1: whitespaces in variable assignment

bash is a very powerful scripting language. However, there is a learning curve that frustrates many.

In a series of bash quicksand blog entries, I hope to post some warning signs about what to avoid.
I want to write them down because after one gets used to bash syntax, one tends to forget about the initial hurdles. Hopefully, no late comers will be trapped in bash quicksand.

Let's begin.

Say you want to assign the text string myvalue to some variable myvariable. What is more simple than that?

$ myvariable = myvalue
bash: myvariable: command not found
$

Oops. And you vowed that it worked before.

What transpired was that bash attempts to execute the command myvariable with the arguments = and myvalue.

Proper syntax requires no whitespace before and after the equal sign:

$ myvariable=myvalue
$ echo $myvariable
myvalue
$

Trick grep not to report itself in a process search

How often have you searched for a process, using ps and grep?

$ ps -ef |grep emacs
peter     7107     1  1 09:10 ?        00:00:08 /usr/bin/emacs-snapshot-gtk
peter     7377  7050  0 09:19 pts/0    00:00:00 grep emacs

It always reports one more process than you want to see ... namely the grep process itself.

In the example above, the process 7377 is the grep process itself. What you really want is the 7107 emacs process.

While this is really harmless (albeit annoying), it can be a real pain if you put this in a script. In that case, you have to parse out the grep process itself.

You can trick grep not to report itself by enclosing a character in the search string in square brackets:

$ ps -ef |grep emac[s]
peter     7107     1  1 09:10 ?        00:00:10 /usr/bin/emacs-snapshot-gtk

Square brackets in bash are character matching patterns. emac[s] will only match the string emacs.

How to check the exit status code

When a command finishes execution, it returns an exit code. The exit code is not displayed on the screen by default. To examine the exit code, you need to examine a special variable, "$?"

Say, you are searching for a string in a text file.

$ grep x1y2z3 somefile.txt
$

The standard output of the command returns null, which is a pretty good indication that the string cannot be found in the file.

But what if you embed the grep command in a script? How can you tell if the string is found or not?

Checking the exit code will tell you. Let's first try it out interactively.

$ grep x1y2z3 somefile.txt
$ echo $?
1

Note that in bash, the exit status is 0 if the command succeeded, and 1 if failed. For grep, 0 means that the string was found, and 1 (or higher), otherwise.

To check the exit status in a script, you may use the following pattern:

somecommand  argument1 argument2
RETVAL=$?
[ $RETVAL -eq 0 ] && echo Success
[ $RETVAL -ne 0 ] && echo Failure

Using sed to extract lines in a text file

If you write bash scripts a lot, you are bound to run into a situation where you want to extract some lines from a file. Yesterday, I needed to extract the first line of a file, say named somefile.txt.

$ cat somefile.txt
Line 1
Line 2
Line 3
Line 4

This specific task can be easily done with this:

$ head -1 somefile.txt
Line 1

For a more complicated task, like extract the second to third lines of a file. head is inadequate.

So, let's try extracting lines using sed: the stream editor.

My first attempt uses the p sed command (for print):

$ sed 1p somefile.txt
Line 1
Line 1
Line 2
Line 3
Line 4

Note that it prints the whole file, with the first line printed twice. Why? The default output behavior is to print every line of the input file stream. The explicit 1p command just tells it to print the first line .... again.

To fix it, you need to suppress the default output (using -n), making explicit prints the only way to print to default output.

$ sed -n 1p somefile.txt
Line 1

Alternatively, you can tell sed to delete all but the first line.

$ sed '1!d' somefile.txt
Line 1

'1!d' means if a line is not(!) the first line, delete.

Note that the single quotes are necessary. Otherwise, the !d will bring back the last command you executed that starts with the letter d.


To extract a range of lines, say lines 2 to 4, you can execute either of the following:

• $ sed -n 2,4p somefile.txt
  
• $ sed '2,4!d' somefile.txt
  

Note that the comma specifies a range (from the line before the comma to the line after). What if the lines you want to extract are not in sequence, say lines 1 to 2, and line 4?

$ sed -n -e 1,2p -e 4p somefile.txt
Line 1
Line 2
Line 4

If you know some different ways to extract lines in a file, please share with us by filling out a comment. P.S. Related articles from this blog:

• Use sed or perl to extract every nth line in a text file.
  
• Use awk to extract lines.
  
• Extract columns and fields from text files.
  
• Useful sed tricks to customize configuration files.
  

It is about Time ... a process

Sometimes, it is easy to overlook the simple Linux commands. Take the command time, for example.

It simply times how long a command takes to run, and gives you 3 statistics:

1. Elapsed real time (in seconds).
   
2. Total number of CPU-seconds that the command spent in user mode.
   
3. Total number of CPU-seconds that the command spent in kernel mode.
   

I use time a lot to benchmark network performance, e.g.,

$ time scp some-file  peter@192.168.22.104:/home/peter/some/location/
real    0m17.742s
user    0m0.364s
sys     0m0.476s

You can be creative with time, and run it like this:

$ time cat 

What does it do? It starts a timer, and stops when you enter Control D (to terminate the input stream to the cat command). It is a quick timer.

Remap Caps Lock key for virtual console windows

My last blog entry explains how to use xmodmap to remap the Caps Lock key to the Escape key in X. That takes care of the keyboard mapping when you are in X. What about when you are in a virtual console window? You need to follow the steps below. Make sure that you sudo root before you execute the following commands.

1. Find out the keycode of the key that you want remapped.
   
   Execute the showkey command as root in a virtual consolde:
   

   $ showkey
   kb mode was UNICODE
   
   press any key (program terminates after 10s of last keypress)...
   0x9c

   
   
   Hit the Caps Lock key, wait 10 seconds (default timeout), and the showkey command will exit on its own.
   

   $ showkey
   kb mode was UNICODE
   
   press any key (program terminates after 10s of last keypress)...
   0x9c
   0x3a 
   0xba

   
   
   The keycode for the Caps Lock key is 0x3a in hex, or 58 in decimal.
   


2. Find out the symbolic name (key symbol) of the key that you want to map to.
   You can list all the supported symbolic names by dumpkeys -l and grep for esc:
   

   $ dumpkeys -l |grep -i esc 
   0x001b Escape
   0x081b Meta_Escape

   
   


3. Remap the keycode 58 to the Escape key symbol.
   

   $ (echo `dumpkeys |grep -i keymaps`;  \
      echo keycode 58 = Escape)          \
      | loadkeys -
   

   
   Thanks to cjwatson who pointed me to prepending the keymaps statement from dumpkeys. The keymaps statement is a shorthand notation defining what key modifiers you are defining with the key. See man keymaps(5) for more info.

To make the new key mapping permanent, you need to put the loadkeys command in a bootup script.

For my Debian Etch system, I put the
(echo `dumpkeys |grep -i keymaps`; echo keycode 58 = Escape) |loadkeys - command in /etc/rc.local.

Remap useless Caps Lock key in X

Caps Lock is in my opinion one of the most useless keys on a keyboard. Unless you own one of those keyboards specifically designed for Linux, your keyboard most likely has a Caps Lock key located in a prime real estate area: right above the Shift key.

How do we remap the Caps Lock key to something more useful, say the Esc key? Why the Esc key? See my earlier article.

First, specify the new key mapping in the file ~/.Xmodmap:

$ cat >> ~/.Xmodmap
remove Lock = Caps_Lock
keysym Caps_Lock = Escape

Note that the file ~/.Xmodmap may not pre-exist in your distro. The above command will create the file.

Next, execute those new mappings by:

$ xmodmap ~/.Xmodmap

Test it out by opening a command window in X. Type in some words on the command line. Press the new Esc key(formerly Caps Lock), and then the key b. This should move the cursor back a word.

The above will do the key remapping for your X-Window environment. The key mapping is not changed for your console windows. We will save that topic for another day.