bash-stdops – A collection of Bash helper scripts that facilitate operations

Bash shell: Interactive Menu to Insert any String from the Tmux Scrollback Buffer Into the Shell Prompt

Imagine you’re working in a tmux session, navigating logs, editing config files, or running commands, and you suddenly need to reuse a path, a variable name, or a keyword that appeared earlier on the tmux scrollback buffer. Instead of scrolling back or retyping it manually, you press Ctrl-n, type a few fuzzy letters, press enter, and the desired string from the tmux scrollback is instantly inserted at your cursor.

The tmux scrollback buffer is the internal history of terminal output that tmux maintains for each pane. It consists of the lines that have scrolled off the visible screen but are still accessible for review or processing.

This article presents a Bash function that scans the tmux scrollback buffer, lets you interactively select strings using fzf, and inserts the chosen string directly into the command line prompt. It offers a fast, context-aware mechanism for inline string insertion, allowing you to work efficiently without breaking your flow.

Requirements

Implementation using Bash functions and readline

Add the following code snippet to your ~/.bashrc to bind Ctrl-n for selecting strings from the tmux scrollback buffer using fzf, and inserting the selected string directly into the shell prompt:

#!/usr/bin/env bash
# Author: James Cherti
# License: MIT
# URL: https://www.jamescherti.com/tmux-autocomplete-fzf-fuzzy-insertion-scrollback/

__tmux_fzf_autocomplete__() {
  # Capture the last 100,000 lines from the tmux scrollback buffer, reverse
  # order, and extract strings
  tmux capture-pane -pS -100000 \
    |
    # Split input on spaces and newlines, remove duplicates while preserving
    # order, and keep only strings longer than 4 characters
    awk 'BEGIN { RS = "[ \t\n]" } length($0) > 4 && !seen[$0]++' \
    |
    # Invoke fzf for case-insensitive exact fuzzy matching, with results shown
    # in reverse order
    fzf --no-sort --exact +i --tac
}

__tmux_fzf_autocomplete_inline__() {
  local selected
  selected="$(__tmux_fzf_autocomplete__)"

  local before
  before="${READLINE_LINE:0:$READLINE_POINT}"

  local after
  after="${READLINE_LINE:$READLINE_POINT}"
  
  READLINE_LINE="${before}${selected}${after}"
  READLINE_POINT=$((READLINE_POINT + ${#selected}))
}

# Pressing Ctrl-n autocompletes from the Tmux scrollback buffer
bind -x '"\C-n": "__tmux_fzf_autocomplete_inline__"'Code language: Bash (bash)

Key components:

tmux capture-pane: Retrieves the scrollback content from the active tmux pane.
awk: Processes tokens, removes duplicates, and filters out strings shorter than 5 characters to produce a concise list.
fzf: Offers an interactive fuzzy interface for filtering and selecting the desired token. The --tac flag reverses the line order, prioritizing the most recent content.
bind: Inserts the selected token inline at the Bash prompt using a readline binding, activated by Ctrl-n. This binding does not interfere with typical command-line editing, preserves the existing prompt content, and updates the cursor position correctly.

Conclusion

The code snippet in this article enhances the Bash shell by turning the tmux scrollback buffer into a live source of contextual string insertion. Within tmux workflows, it removes the need to scroll through output, improving both speed and accuracy during command-line tasks.

Related Links

This function is included in the author’s .bashrc, which is available in the jc-dotfiles repository.

jc-dotfiles – A collection of configuration files for UNIX/Linux systems

The jc-dotfiles repository houses James Cherti’s dotfiles and configuration scripts:

Shell Configuration (.bashrc, .profile, and .bash_profile): Optimized Bash shell settings for efficient command execution and interactive sessions.
Terminal Multiplexer (.tmux.conf): Configuration for Tmux, enhancing terminal session management and productivity.
Readline configuration (.inputrc): Inputrc configuration that also allows using Alt-h, Alt-j, Alt-k, and Alt-l as a way to move the cursor.
Other: .gitconfig, ranger, .fdignore, .wcalcrc, mpv, picom, feh, and various scripts and configuration files for managing system settings, aliases, and more.

Here are additional dotfiles and configuration files maintained by the same author:

jc-dotfiles @GitHub: A collection of UNIX/Linux configuration files. You can either install them directly or use them as inspiration your own dotfiles.
bash-stdops @GitHub: A collection of Bash helper shell scripts.
jc-gnome-settings: GNOME customizations that can be applied programmatically.
jc-firefox-settings @GitHub: Provides the user.js file, which holds settings to customize the Firefox web browser to enhance the user experience and security.
jc-gentoo-portage @GitHub: Provides configuration files for customizing Gentoo Linux Portage, including package management, USE flags, and system-wide settings.
jc-xfce-settings: GNOME customizations that can be applied programmatically.
watch-xfce-xfconf: A command-line tool that can be used to configure XFCE 4 programmatically using the xfconf-query commands displayed when XFCE 4 settings are modified.

Installation

Here’s how to install James Cherti’s dotfiles:

Clone the Repository:

git clone https://github.com/jamescherti/jc-dotfiles

Navigate to the jc-dotfiles directory:
```
cd jc-dotfiles
```
Install:
```
./install.sh
```

Usage

.bashrc

Tmux/fzf auto complete: Pressing Ctrl-n calls a custom Bash autocomplete function that captures the current tmux scrollback buffer, extracts unique word-like tokens, and presents them via fzf for interactive fuzzy selection. The selected word is then inserted inline at the current cursor position using a readline binding.
The .bashrc file can be extended by adding configurations to ~/.bashrc.local.
The o alias calls a function that provides a cross-platform way to open files or URLs using the appropriate command for the system. This function opens files or URLs using the appropriate command (xdg-open on Linux, open on macOS, and start on Windows). If more than 7 arguments are passed, the user is prompted for confirmation before proceeding. Example usage:
```
o file1.jpg file2.png file3.jpeg
```

Customizations in .bashrc to add to ~/.profile.local:

# Use trash-rm as a safer alternative to rm by moving files to the trash instead
# of deleting them permanently.
#
# JC_TRASH_CLI=1 replaces the standard 'rm' command with a wrapper function
# that:
# - Provides a detailed summary of all specified files and directories,
#   including total size and file count.
# - Prompts the user for confirmation before proceeding with the deletion.
# - Moves files to the trash using 'trash-put' instead of permanently deleting
#   them with 'rm'.
# - Reports the current size of the trash in megabytes after each deletion.
# - Optionally wraps 'trash-empty' with an interactive prompt before purging the
#   trash.
#
# This setup is only activated for non-root users when 'trash-put' is available
# and 'JC_TRASH_CLI' is set to a non-zero value.
#
JC_TRASH_CLI=1

# Enable Emacs integration for vterm and EAT, configuring shell-side support for
# features such as prompt tracking and message passing
JC_EMACS_INTEGRATION=1  # Default: 0

# Display the current Git branch in the shell prompt (PS1)
JC_PS1_GIT_BRANCH=1  # Default: 0

# Display the count of unread mails in the shell prompt (PS1)
JC_PS1_MAILDIR=1  # Default: 0

# Directory containing the mail (e.g., to "$HOME/Mail")
JC_PS1_MAILDIR_PATH="$HOME/Mail"

License

Distributed under terms of the MIT license.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

bash-stdops – A collection of useful Bash Shell Scripts

The jamescherti/bash-stdops project is a collection of helpful Bash scripts that simplify various operations, including file searching, text replacement, and content modification.

I use these scripts in conjunction with text editors like Emacs and Vim to automate tasks, including managing Tmux sessions, replacing text across a Git repository, securely copying and pasting from the clipboard by prompting the user before executing commands in Tmux, fix permissions, among other operations.

The bash-stdops Script Collection Overview

Files, Paths, and Strings

walk: Recursively lists files from the specified directory.
walk-run: Executes a command on all files.
sre: Replaces occurrences of a specified string or regular expression pattern, with support for case-insensitive matching and regular expressions.
git-sre: Executes sre at the root of a Git repository to replace text within files.
path-tr, path-uppercase, path-lowercase: Processes a file path to convert the filename to uppercase or lowercase.
autoperm: Sets appropriate permissions for files or directories (e.g., 755 for directories).
path-is: Prints the path and exits with status 0 if the file is binary or text.

Git

git-dcommit: Automates the process of adding, reviewing, and committing changes in Git.
git-squash: Squashes new Git commits between the current branch and a specified branch.

SSH

esa: Manages the SSH agent, including starting, stopping, adding keys, and executing commands.
sshwait: Waits for an SSH server to become available on a specified host.

Tmux

tmux-cbpaste: Pastes clipboard content into the current tmux window with user confirmation.
tmux-run: Executes a command in a new tmux window. If inside tmux, it adds a new window to the current session; otherwise, it creates a window in the first available tmux session.
tmux-session: Attaches to an existing tmux session or creates a new one if it doesn’t exist.

X11/Wayland

xocrshot: Captures a screenshot, performs OCR on it, displays the extracted text, and copies it to the clipboard.

Misc

haide: Uses AIDE to monitor the file integrity of the user’s home directory.
cbcopy, cbpaste, cbwatch: Manages clipboard content by copying, pasting, or monitoring for changes.
outonerror: Redirects the command output to stderr only if the command fails.
over: Displays a notification once a command completes execution.
largs: Executes a command for each line of input from stdin, replacing {} with the line.

Conclusion

The jamescherti/bash-stdops scripts provide a variety of tasks, including file manipulation, Git management, and SSH automation, improving efficiency and usability.

Making ‘cron’ notify the user about a failed command by redirecting its output to stderr only when it fails (non-zero exit code)

Monitoring the success or failure of cron jobs can be challenging, especially when multiple jobs cause cron to send emails even when they don’t fail. A more effective approach to handling cron job errors is to use a Bash script that directs the output to stderr only if the job fails, causing cron to notify the user about the error only when the script fails.

Script overview

The script provided below ensures that any command passed to it will redirect its output to a temporary file. If the command fails (i.e., returns a non-zero exit code), the script sends the content of the temporary file to stderr, which causes cron to notify the user.

Here is the complete script to be installed in /usr/local/bin/outonerror:

#!/usr/bin/env bash
# Description:
# Redirect the command's output to stderr only if the command fails (non-zero
# exit code). No output is shown when the command succeeds.
#
# Author: James Cherti
# License: MIT
# URL: https://www.jamescherti.com/cron-email-output-failed-commands-only/

set -euf -o pipefail

if [[ "$#" -eq 0 ]]; then
  echo "Usage: $0 <command> [args...]" >&2
  exit 1
fi

cleanup() {
  if [[ "$OUTPUT_TMP" != "" ]] && [[ -f "$OUTPUT_TMP" ]]; then
    rm -f "$OUTPUT_TMP"
  fi

  OUTPUT_TMP=""
}

OUTPUT_TMP=$(mktemp --suffix=.outonerror)
trap 'cleanup' INT TERM EXIT

ERRNO=0
"$@" >"$OUTPUT_TMP" 2>&1 || ERRNO="$?"
if [[ "$ERRNO" -ne 0 ]]; then
  cat "$OUTPUT_TMP" >&2
  echo "$0: '$*' exited with status $ERRNO" >&2
fi

cleanup
exit "$ERRNO"
Code language: Bash (bash)

To use this script with a cron job, save it as /usr/local/bin/outonerror and make it executable by runnning:

chmod +x /usr/local/bin/outonerrorCode language: plaintext (plaintext)

Integration with Cron

Cron sends an email by default whenever a job produces output, whether standard output or error output. This is typically configured through the MAILTO environment variable in the crontab file. If MAILTO is not set, cron sends emails to the user account under which the cron job runs. Cron will only be able to send emails if the mail transfer agent (e.g., Postfix, Exim, Sendmail) is configured properly.

Here is how to schedule the cron job to use the outonerror script:

MAILTO="your-email@example.com"
* * * * * /usr/local/bin/outonerror your_command_hereCode language: plaintext (plaintext)

With this setup, the cron job will execute your_command_here and only send an email if it fails, thanks to cron’s default behavior of emailing stderr output to the user.

Conclusion

This script is a simple yet effective solution for improving cron jobs by ensuring that the user is notified only when something goes wrong. It reduces unnecessary notifications for successful executions and provides clear error messages when failures occur.

Arch Linux: Preserving the kernel modules of the currently running kernel during and after an upgrade

One potential issue when upgrading the Arch Linux kernel is that the modules of the currently running kernel may be deleted. This can lead to a number of problems, including unexpected behavior, system crashes, or the inability to mount certain file systems (e.g. the kernel fails to mount a vfat file system due to the unavailability of the vfat kernel module).

The Arch Linux package linux-keep-modules (also available on AUR: linux-keep-modules @AUR), written by James Cherti, provides a solution to ensure that the modules of the currently running Linux kernel remain available until the operating system is restarted. Additionally, after a system restart, the script automatically removes any unnecessary kernel modules that might have been left behind by previous upgrades (e.g. the kernel modules that are not owned by any Arch Linux package and are not required by the currently running kernel).

The linux-keep-modules package keeps your system running smoothly and maintains stability even during major Linux kernel upgrades.

Make and install the linux-keep-modules package

Clone the repository and change the current directory to ‘archlinux-linux-keep-modules/’:

$ git clone https://github.com/jamescherti/archlinux-linux-keep-modules.git
$ cd archlinux-linux-keep-modules/Code language: plaintext (plaintext)

Use makepkg to make linux-keep-modules package:

$ makepkg -fCode language: plaintext (plaintext)

Install the linux-keep-modules package:

$ sudo pacman -U linux-keep-modules-*-any.pkg.tar.*Code language: plaintext (plaintext)

Finally, enable the cleanup-linux-modules service:

$ sudo systemctl enable cleanup-linux-modulesCode language: plaintext (plaintext)

(The cleanup-linux-modules service will delete the Linux kernel modules that are not owned by any a package at boot time)

The linux-keep-modules Arch Linux package offers a solution to preserve kernel modules during and after upgrades, ensuring that the necessary modules for the currently running kernel remain present in the system even after the kernel is upgraded. This solution keeps your system running smoothly and maintains stability even during major upgrades.

Links related to the pacman package linux-keep-modules

How to make Vim edit/diff files from outside of Vim? (e.g. from a shell like Bash, Zsh, Fish..)

The Vim editor offers the ability to connect to a Vim server and make it perform various tasks from outside of Vim. The command-line tools vim-client-edit, vim-client-diff and the vim_client Python module, written by James Cherti, can be used to easily find and connect to a Vim server and make it perform the following tasks:

Edit files or directories in new tabs (The command-line tool vim-client-edit),
Diff/Compare up to eight files (The command-line tool vim-client-diff),
Evaluate expressions and return their result (The Python module vim_client),
Send commands and expressions to Vim (The Python module vim_client).

The command-line tools vim-client-edit and vim-client-diff are especially useful when a quick edit or comparison needs to be performed on a file from outside of Vim (e.g. from a shell like Bash, Zsh, Fish, etc.).

Additionally, the vim_client Python module allows running expressions on a Vim server and retrieving their output, which can be useful for automating tasks or scripting. For example, you can use vim-client to run a search and replace operation on a file or directory, or to perform a complex diff operation between two files.

Overall, vim-client is a powerful tool for interacting with Vim from the vim-client-edit and vim-client-diff command-line tools. The vim_client Python module can also be used to run and retrieve the output of Vim expressions, which can help automate various tasks.

Please star vim-client on GitHub to support the project!

Requirements

To use vim-client, you will need to have Vim and Python installed on your system.

Installation

The vim-client package can be installed with pip:

$ sudo pip install vim-clientCode language: Bash (bash)

Execute Vim server

The Vim editor must be started with the option “–servername”, which enables the Vim server feature that allows clients to connect and send commands to Vim:

$ vim --servername SERVERNAMECode language: plaintext (plaintext)

Make Vim server edit multiple files in tabs

Editing a list of files in new tabs:

$ vim-client-edit file1 file2 file3

Make Vim server diff files (like vimdiff)

Comparing/diff up to eight files:

$ vim-client-diff file1 file2

Useful ~/.bashrc aliases:

Adding the following aliases to ~/.bashrc is recommended as it makes it easy to execute the command-line tools vim-client-edit and vim-client-diff:

alias gvim=vim-client-edit
alias vim=vim-client-edit
alias vi=vim-client-edit
alias vimdiff=vim-client-diff

Links related to vim-client

Bash shell: Perform tab-completion for aliases (bash-completion)

# Author: James Cherti
# License: MIT
# Requirements: bash-completion
# Description: Perform tab-completion for aliases in Bash (bash-completion).
# URL: https://www.jamescherti.com/bash-shell-perform-tab-completion-for-aliases/
#
# Add the following function to ~/.bashrc :

alias_completion() {
  local func_name='alias_completion'
  if [[ $# -lt 2 ]]; then
    echo "Usage: $func_name <cmd> <alias> <alias2> <...>" >&2
    return 1
  fi

  local cmd; cmd="$1"
  shift

  # Load the completion
  if ! type _completion_loader >/dev/null 2>&1; then
    echo "Error: $func_name: '_completion_loader' was not found." >&2
    return 1
  fi

  _completion_loader "$cmd"

  if ! complete -p "$cmd" >/dev/null; then
    echo "Error: $func_name: 'complete -p $cmd' failed." >&2
    return 1
  fi

  # Add aliases
  local alias
  for alias in "$@"; do
    complete_cmd=$(complete -p "$cmd" 2>/dev/null | sed -e 's/[[:space:]][^[:space:]]\+$//')
    complete_cmd="${complete_cmd} $alias"

    if ! ( echo "$complete_cmd" | grep -P '^\s*complete\s' >/dev/null 2>&1 ); then
      echo "Error: $func_name: alias '$alias': '$complete_cmd' is an invalid command." >&2
      return 1
    fi

    eval "$complete_cmd"
  done

  return 0
}Code language: Bash (bash)

Examples of aliases:

alias s='ssh'
alias_completion ssh s

alias g='git'
alias_completion git gCode language: Bash (bash)

Bash shell: A better version of the default bash built-in command “cd”

#!/usr/bin/env bash
# Author: James Cherti
# License: MIT
# URL: https://www.jamescherti.com/shell-bash-replacement-bash-cd-change-directory/
#
# Description:
# 1. 'cd path/to/file' will change the directory to 'path/to'
#    (the parent directory of 'file').
#
# 2. 'cd path/to/dir with spaces' will change the directory to
#    "path/to/dir with spaces".
#
# 3. 'cd file:///home/user' will change the directory to "/home/user".
# 
# 4. You can switch to the previous directory with 'popd' or 'cd -'.
#
# Add the following function and alias to ~/.bashrc :
#

_better_cd() {
  # Previous directory ('cd -')
  if [[ $# -eq 1 ]] && [[ $1 = '-' ]]; then
    popd >/dev/null || return 1
    return 0
  fi

  # Join paths
  local path
  if [[ $# -eq 0 ]]; then
    path="$HOME"
  else
    path=$(echo "$1" | sed -e 's/^file:\/\///')
    shift

    local item
    for item in "$@"; do
      path="${path} ${item}"
    done
  fi

  # Checks
  local errno=0
  if [[ -f "$path" ]]; then
    path=$(dirname "$path")
  fi

  if ! [[ -d "$path" ]]; then
    echo "$(basename "$0"):" "cd: $path: No such file or directory" >&2
    return 1
  fi

  # Change the directory
  local oldcwd; oldcwd=$(pwd)
  pushd . >/dev/null || return 1
  builtin cd "$path" >/dev/null 2>&1 || errno=1
  if [[ $oldcwd = "$(pwd)" ]]; then
    popd >/dev/null || return 1
  fi

  return "$errno"
}

alias cd=_better_cdCode language: Bash (bash)

Table of Contents

Install bash-stdops scripts

System-wide installation

Alternative installation: Install in your home directory

Install dependencies

Install dependencies on Debian/Ubuntu based systems

Install dependencies on RedHat/CentOS/Fedora based systems

Install dependencies on Gentoo based systems

Install dependencies on Arch Linux based systems

Scripts

Script category: tmux

Script: tmux-cbpaste

Script: tmux-run

Script: tmux-session

Script category: files, paths, and strings

Script: walk

Script: walk-run

Script: sre

Script: git-sre

Scripts: path-tr, path-uppercase, path-lowercase

Script: autoperm

Script: path-is

Script category: git

git-dcommit

git-squash

git-finder

git-finder-exec

git-ourstheir

git-sync-upstream

Script category: ssh

Script: esa

Script: sshwait

X11/Wayland scripts

xocrshot

Script category: Misc

Script: haide

Scripts: cbcopy, cbpaste

Script: outonerror

Script: over

Script: largs

License

Links

Requirements

Implementation using Bash functions and readline

Conclusion

Installation

Usage

.bashrc

License

Links

The bash-stdops Script Collection Overview

Files, Paths, and Strings

Git

SSH

Tmux

X11/Wayland

Misc

Conclusion

Script overview

Integration with Cron

Conclusion

Make and install the linux-keep-modules package

Links related to the pacman package linux-keep-modules

Requirements

Installation

Execute Vim server

Make Vim server edit multiple files in tabs

Make Vim server diff files (like vimdiff)

Useful ~/.bashrc aliases:

Links related to vim-client