Not every command you might execute only runs for a couple of seconds. When you have a long running command a couple of problems can come with it. When the ssh connection is not stable, the command might be interrupted, but does the command handle such interruptions correctly? I want to show here that you have multiple options to make sure the command you are executing is not affected by an ssh disconnect or interruption.
Why is a command affected by a ssh disconnect?
When you login to a server using ssh, all commands you execute on the server are linked to the login session you opened when you logged in. When the ssh connection is lost for any reason, like an unstable network connection, the session is closed. With the session all of the commands that are currently executing are terminated as well.
Control an already running job
Long running commands can be managed and controlled in the shell. In the following I will show how to do this using the bash(1) shell. For the example I will use sleep(1) to simulate a command that takes a long time to finish.
$ sleep 1800
This command will run for 30 minutes before it completes. To not block the shell while such a command is running you can hit the key combination control-z which will suspend it.
^Z [1]+ Stopped sleep 1800 $
The output shows that the so called “job” is suspended. The prompt is now accessible again, where you can enter other commands. As an example, the command “bg” can be used which will move the job into background.
$ bg %1 [1]+ sleep 1800 & $
As the job is now running in the background, the shell is still free to be used for other things. The list of all the jobs can be shown as follows.
$ jobs [1]+ Running sleep 1800 &
In this case there is only one job in this shell session. As such the “%1” parameter in “bg” as well as in the following “fg” command can be omitted. With more then one job, this parameter is required to identify the job to control.
$ fg %1 sleep 1800
The “fg” command as shown above will get the job from the background back into the foreground.
With the job running in foreground, the job will be terminated if the ssh session gets disconnected. When the job is set to run in the background, the job is still attached to the session but gets disconnected from it when the ssh session is disconnected. In this case the job continues to run.
The first example output shows the process and its relation to the bash process which represents the shell the user is logged in to.
$ ps af | grep sleep -B 3 4095 pts/0 S+ 0:00 -bash 4116 pts/0 S 0:00 \_ sleep 1800
The below shows the job still running after the bash session being closed. The job continues to runn without any relation to a shell session.
$ ps af | grep sleep -B 3 4116 pts/0 S 0:00 sleep 1800
While this is preventing the job from being terminated when the session is closed, it does not give you control over it after you have reconnected. You can see the job running and terminate the job using kill(1) but you can not interact with it or see the jobs output any more.
Using screen
Using a utility called “screen” will allow you to interact with a job after reconnecting to the server again. The utility screen will open a seperate session in which the job is then started. The user has the ability to disconnect from such a session and reconnect to it. The job continues to run and the job’s output is stored in the screen session. When attached again to the screen session, the output can be seen as well and the user can again interact with the job.
$ screen -S session-name
The above command will start a new named sesson. If you do not want to name the session you can ommit the “-S session-name” parameter.
The screen session will present you with a prompt, like your usual shell. Inside the screen session any command can be executed as usual. To detach from the screen session without interrupting the running job, use the keyboard short-cut “control-a” (this is the escape sort-cut used by screen) followed by “d“. The screen session will continue to exist but the user is returned to the shell they were using before the screen session.
$ screen -S session-name [detached] $ screen -ls There is a screen on: 5997.session-name (Detached) 1 Socket in /var/run/screen/S-root.
Executing screen with the “-ls” parameter will show the list of all open screen sessions. The screen sessions opened with a name will show their name in the list. To reconnect to a screen session, use the following command while specifying the name or the ID of the session.
screen -r session-name
If the running job produced a lot of output, screen provides the possibility to scroll in the output. To scroll, use the short-cut “control+z” followed by “Esc“. This will switch to the “Copy mode”. This is indicated with a notification similar to this.
Copy mode - Column 11 Line 21(+100) (99,21)
With the copy mode enabled, the Page-Up and Page-Down keys can be used to scroll the output up and down. To end the copy mode, hit “Esc” again.
To detach and logout from a screen-session the following short-cut is used. Hit “control+z” followed by “D” (capital “D”) twice. This will end the screen-session.
$ screen [power detached] Screen session of username ended.
Alternative to the “control+z D D” shortcut, a screen-session can also be ended by typing “exit” into the screen-session while no job is running. The output in this case looks a bit different but the result is the same.
$ screen -S session-name [screen is terminating]
The screen utility is a mighty tool with many more features like splitting the screen, multiple windows and much more. To read about all the features of screen read the sreen(1) manpage.
Read more of my posts on my blog at http://blog.tinned-software.net/.
