While automating things, you might come across the need to replace a block of lines within a file – like replacing a section with IP addresses you want to block in an .htaccess file. Automating this task means replacing this bunch of lines without starting a file editor. Linux ships with a lot of very powerful tools to manipulate file content which makes it possible to automate such things.
Lets start with describing the starting condition for our example before digging into the details of replacing a section in a file. In the following example I will use an .htaccess file where IP addresses from users that tried to attack a password form are blocked. When the list of IPs is regenerated, the complete list in the .htaccess file should be replaced with the new list. As .htaccess does not allow including another file with the “deny from” lines, it needs to be placed inside the .htaccess file itself. With that in mind the .htaccess might look like this.
Order allow,deny Allow from all # BLOCK ATTACK IPs - START deny from 195.88.203.179 deny from 197.242.159.253 deny from 187.45.241.241 deny from 46.235.47.79 deny from 78.108.84.111 # BLOCK ATTACK IPs - END deny from 10.0.0.0/24 allow from 192.168.0.0/24 php_value memory_limit 128M # Block the include-only files. RewriteEngine On RewriteBase /
As you can see in this example, the list of blocked IP addresses is surrounded by 2 comment lines to identify the start and the end of the block. This is the block of lines to replace. The “deny from ” shown outside of the block should stay untouched while replacing the lines between the comments with a new list of IP addresses.
Using .htaccess to block the IP address instead of iptables does sound strange. Going into details about this might exceed the scope of this article. In short, there might be the need because some VPS (Virtual Private Server) providers do not support iptables inside the VPS using paravirtualization.
When the list of IP addresses changes, the old list needs to be replaced with the new one. The following shows the new list that should be placed in the .htaccess file.
deny from 213.151.42.186 deny from 195.88.203.179
The following one-line sed command will do exactly that. It will replace the old list from the .htaccess with the new one while creating a backup of the file before modifying it.
$ sed -e '/# BLOCK ATTACK IPs - START/,/# BLOCK ATTACK IPs - END/{/^# BLOCK ATTACK IPs/!d}' \
-e '/# BLOCK ATTACK IPs - START.*$/r /path/to/new/ip_address_list' \
--in-place=.bak /var/www/html/.htaccess
To understand what that command line is doing, I will explain it in sections. The command line shown above is divided into 3 lines for better reading. The “\” (backslash) at the end of the line tells the shell that this command is not yet over and continues in the next line. It is therefore not essential and the complete command could be placed on one line.
The command is executing sed with 2 commands defined by the “-e” parameter. The first command contains 2 main components.
$ sed -e '/# BLOCK ATTACK IPs - START/,/# BLOCK ATTACK IPs - END/{/^# BLOCK ATTACK IPs/!d}' \ -e '/# BLOCK ATTACK IPs - START.*$/r /path/to/new/ip_address_list' \ --in-place=.bak /var/www/html/.htaccess
The highlighted section of the first command shows two matches separated by a comma “,”. These match to the comment line surrounding the list to be replaced. This part selects from the starting line “# BLOCK ATTACK IPs – START” everything up to the line “# BLOCK ATTACK IPs – END”. This includes the two ‘marker’ comment lines.
$ sed -e '/# BLOCK ATTACK IPs - START/,/# BLOCK ATTACK IPs - END/{/^# BLOCK ATTACK IPs/!d}' \ -e '/# BLOCK ATTACK IPs - START.*$/r /path/to/new/ip_address_list' \ --in-place=.bak /var/www/html/.htaccess
With the following part, highlighted above, the previous selected lines are taken and filtered by the match. Every line not matching this regular expression. The function “!d” instructs sed to delete every non-matching line. The result of this command is that all lines between the comment lines are deleted.
$ sed -e '/# BLOCK ATTACK IPs - START/,/# BLOCK ATTACK IPs - END/{/^# BLOCK ATTACK IPs/!d}' \ -e '/# BLOCK ATTACK IPs - START.*$/r /path/to/new/ip_address_list' \ --in-place=.bak /var/www/html/.htaccess
With the second command, the new list will be added to the the file. This command starts again with a match for the starting comment line followed by the function “r”. The function “r” instructs sed to include the content of the file before it processes any more lines. The result of this is that the content of the file defined, directly after the “r” function, is copied directly after the starting comment line.
$ sed -e '/# BLOCK ATTACK IPs - START/,/# BLOCK ATTACK IPs - END/{/^# BLOCK ATTACK IPs/!d}' \
-e '/# BLOCK ATTACK IPs - START.*$/r /path/to/new/ip_address_list' \
--in-place=.bak /var/www/html/.htaccess
Using the “–in-place=” parameter allows the file specified afterwards (example /var/www/html/.htaccess) to be modified directly. The defined file name suffix “.bak” will be used before any modification is made to create a backup copy of the file. This way you have a backup in case something is not working as expected.
Read more of my posts on my blog at http://blog.tinned-software.net/.
