Tuesday, November 6, 2007

Keeping Command History across Multiple Sessions

The bash shell maintains a history of the commands you entered. You can re-execute a command by recalling it from the history, without having to re-type it.

The command history is stored in a file, specified by an environment variable.
$ echo $HISTFILE
/home/peter/.bash_history

The maximum number of commands that it will save depends on the value of this environment variable:
$ echo $HISTSIZE
500

Life is simple if we operate on a single shell session at any given time. If you have 2 simultaneous sessions, you may be surprised that the history does not have the commands you expect.

The default behavior is that the command history is saved ONLY on session exit. Moreover, the existing contents in the history are overwritten by the new contents.

Assuming you already have a session running, opening a second session will not have the latest commands from the first session. This is because the first shell hasn’t exited yet. Ultimately, the contents of the history file depends on which session exits last, hence overwriting the commands from the session that finishes earlier.

This can become quite confusing.

The remedy is simple. Change the history behavior as follows:

  1. Append commands to the history file, rather than overwrite it.
    $ shopt -s histappend

  2. Save each command right after it has been executed, not at the end of the session.
    $ PROMPT_COMMAND='history -a'


Insert the 2 statements into ~/.bashrc
shopt -s histappend
PROMPT_COMMAND='history -a'

Note: If the PROMPT_COMMAND is already initialized, then you probably want to concatenate the history -a part to what is already there.

To determine if PROMPT_COMMAND has a value:
echo $PROMPT_COMMAND

To concatenate:
PROMPT_COMMAND="history -a;$PROMPT_COMMAND"

16 comments:

kuriharu said...

I don't have 'shopt' as a command. I couldn't find it under apt-cache search, either.

Peter Leung said...

shopt is a built-in bash command.

Which shell are you using?

Anonymous said...

Hi,
Using bash in a virtual console, as I type the second command, it complains :
bash: PROMPT_COMMAND: line 3: syntax error near unexpected token `;'
bash: PROMPT_COMMAND: line 3: `;history -a'
(line 3 'cos I echoed $HISTSIZE)
?

Peter Leung said...

I suspect that $PROMPT_COMMAND was null in the first place. Therefore, did not like it when you concatenate the new stuff.

To correct the problem:

1. First, clear the $PROMPT_COMMAND
unset PROMPT_COMMAND

2. Then set it
PROMPT_COMMAND='history -a'

I will make it clear in my blog post.

Sorry for the inconvenience

mae said...

ahhh thanks for that last bit, I was wondering today why this wasn't working and it was because of that

Alexey Torkhov said...

Any way to do it in Midnight Commander?

Unknown said...

I followed the instructions, and I see that my prompts from all sessions are being properly appended to $HISTFILE. However, I think that my 'history' command is not taking the recent $HISTFILE entries into account from the other sessions.

For example, things start out synchronized when I open the two sessions:

(open session1)
(open session2)

[session1]: history
498 ls
499 exit
500 exit
501 history

[session2]: history
498 ls
499 exit
500 exit
501 history

But if I run some commands in one session, they don't appear in the 'history' of the other session:

[session1]: echo 1
[session2]: echo 2

[session1]: history
500 exit
501 history
502 echo 1
503 history

[session2]: history
500 exit
501 history
502 echo 2
503 history

How do I make 'history' read the latest contents of $HISTFILE?

Unknown said...

Your solution ignores the setting HISTIGNORE, so I see a lot of duplicates ( which I used to avoid via HISTIGNORE )

I now have the following in my .bashrc

_archive_history ()
{
history -a
}

trap _archive_history EXIT

If I now exit my xterm's only the changes from those sessions are recorded

Jesus Carretero said...

Dave E is very right. PETER LEUNG doesn't keep command history across multiple sessions due to the problem described by Dave E.

Good try. Thanks anyway.

Steve said...

@Jesus Carretero - Dave E is correct but only *because* he didn't read the blog accurately.

History is written to on *session exit*. He is looking for real time history write as each command is written.

Peter Leung's post is not a 'good try' it is an accurate statement of how to do exactly what he says.

dc46and2 said...

Thanks for the useful post. The issue Dave E pointed out seems to be due to the fact that each bash session reads the history file at shell initialization and thereafter maintains its own copy of history in memory. The 'history -a' command will update the history file after every command, but it doesn't affect the in-memory history of other running shells.

You can force the current shell to reread the history file with 'history -r'. Run that manually when you want to use or view history from other sessions, or use

PROMPT_COMMAND="history -a; history -r; $PROMPT_COMMAND"

Remember the PROMPT_COMMAND is only executed after a command, so you will have to execute a command or just hit enter to pull in the history generated by other shells since the last command in the current shell.

meathead9999 said...

The PROMPT_COMMAND posted by dc46and2 *almost* gave me what I needed, but I found each time I did a command then issued a 'history' command it kept adding all 1000+ entries into my current term history. Modifying the 'history -r' to 'history -n' did it for me. I also want to point out that the spaces between the semicolon and the command that follows is important. I was getting errors after each command I entered until I added spaces. THANKS dc46and2! I FINALLY have what I wanted!!!

Anonymous said...

i have something cleaning up my history every logon, no idea what.

Anonymous said...

very handy, thanks a lot.

Simon said...

Hi,
I ended up solving this problem by writing a script called hiset, it basically just automates the
history -a
export HISTFILE="$1"
history -r $HISTFILE
But provides functionality to list view and search all existing history files along with tab completion.
I have written about it on my blog: http://simotek.net/tech/hiset-multiple-history-files-and-sessions-for-bash-and-other-tinkering/
and the source is avalible https://github.com/simotek/scripts-config

Anonymous said...

Thanks for sharing.