Useful sed tricks to customize configuration files

A typical software installation goes like this. You install the software using apt-get install or yum install.
Then, you manually edit the software's configuration file in order to satisfy your requirements. If you have to repeat the install on multiple machines, this quickly becomes tedious.

Instead of manually editing the file, I run a text manipulation command such as sed or awk to make the required changes. Then, I script the procedure by inserting the commands in a bash script file.

The scripting of configuration changes serves multiple purposes:

• It is a permanent record of the configuration changes.

• It is readily repeatable on the same or a different machine.

Below, I illustrate 2 sed tricks to make configuration changes to the Apache webserver. The target configuration file is /etc/apache2/apache2.conf.

Before you make any change, please first backup the original configuration file.


$ sudo cp /etc/apache2/apache2.conf /etc/apache2/apache2.conf.orig


Replacing first occurrence of a string

The default apache2.conf file contains the following line:


Timeout 300


Below is the sed command to change the first occurrence of Timeout in the file to 100.


$ sudo sed -i "0,/^Timeout\s/ s/^Timeout\s\+[0-9]\+/Timeout 100/" /etc/apache2/apache2.conf


The -i parameter tells sed to edit the file in place - that is, directly in apache2.conf.

0,/^Timeout\s/ specifies the range of lines over which the sed command is to be executed. In this example, the starting line is the first line (line 0). The finishing line is the line returned by a search for the word Timeout which appears at the beginning of a line (^) and followed by a whitespace (\s).

The line range parameter limits the change to only the first occurrence of Timeout in the file. If you leave out the line range, each occurrence of Timeout in the file will be modified. In many scenarios, leaving it out is OK because the parameter occurs only once in the configuration file.

For some configuration files, a parameter can occur multiples times, in different sections. Next, I illustrate how to limit the change to within a particular section in the configuration file.

Replacing a string within a target section

The MaxClients parameter occurs in 3 sections within the apache2.conf file:

• mpm_prefork_module

• mpm_worker_module

• mpm_event_module

I want to change the MaxClients parameter within the mpm_prefork_module only.

The default mpm_prefork_module is like this:


<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 150
MaxRequestsPerChild 0
</IfModule>


Note that a section is delimited by the opening <IfModule> and closing </IfModule> statements.

The following sed command modifies the value of MaxClients to 18 within the mpm_prefork_module section.

$ sudo sed -i "/<IfModule mpm_prefork_module>/,\@</IfModule>@ s/MaxClients\s\+[0-9]\+/MaxClients 18/" /etc/apache2/apache2.conf


The line range is defined by the /<IfModule ... >/,\@</IfModule>@ clause in the above statement. The opening line in the line range is specified by a search for the <IfModule ... > pattern. The closing line is specified by the search pattern \@</IfModule>@.

An explanation of the closing line pattern is warranted. The slash (/) character is part of the search pattern for the closing line (</IfModule>). However, the slash is also the default delimiter for sed. Therefore, we must use a different delimiter (@) for the closing-line search pattern. Note that the first @ is escaped (\@).

The s/MaxClients.../MaxClients 18/ clause changes the value of MaxClients to 18.

Conclusion

The above are examples of how you can use sed to script common scenarios of changing configuration files. You can achieve the same result using other tools such as awk or perl. Please use the comment system to let us know your own examples.

If you are interested to learn more about sed, please read my earlier posts on the tool:

• Extract lines in a text file.

• Extract every nth line.

• Indent lines.

How to add and delete a user from a Linux group

Being in the right Linux group expedites many system administration tasks, and will save you time. For example, the Apache web server logs have the following file ownership and permissions.

# ls -al /var/log/apache2/*.log
-rw-r----- 1 root adm 882984 May 11 12:14 /var/log/apache2/access.log
-rw-r----- 1 root adm 418 May 11 01:55 /var/log/apache2/error.log
...

To read the Apache logs, you need root permissions. However, there is a shortcut that does not require you to run sudo. Note that adm - the admin group for Debian-based systems - is the group owner of the log files. So, if you become a member of adm, you don't need to sudo to read the log files.

To add peter to the adm group, execute any of the following commands:

• $ sudo usermod -aG adm peter

• $ sudo gpasswd -a peter adm

To verify that peter is now a member of the adm group, execute any of the following commands:

• $ id -nG peter
peter adm www-data

  You may be tempted, as I was, to not specify peter in the above command. Don't skip the parameter. Without the user parameter, you won't see the effect of the change in group membership - unless you log out and log back in. If you are running X, it means you have to log out of X, not just opening a new command shell window within the same X session.

• $ groups peter
peter : peter adm www-data

  Again, specify peter in the command. Otherwise, you must log out and then log back in before executing the command.

• $ grep adm /etc/group
adm:x:4:peter

If you have made a mistake, and now want to remove peter from the adm group, run any of the following commands:

• $ sudo gpasswd -d peter adm
Removing user peter from group adm

• $ sudo deluser peter adm
Removing user `peter' from group `adm' ...
Done.

Besides the adm group, you should consider adding yourself to the www-data group. The Apache web server runs under the www-data user account on Debian systems. As a member of the www-data group, you can more easily modify web server files and directories.

An earlier post on group membership appears here.