Category Archives: Python

Python: How to Clear Stdin Before Using the input() Function

Reply

In Python, the input() function is commonly used to capture input from the user. However, issues may arise when unwanted or unexpected data is present in the stdin, leading to incorrect or incomplete results when calling input().

A reliable approach to address this issue is to clear the stdin buffer before invoking input(). This article discusses why discarding the stdin buffer is necessary and how to implement it correctly to ensure accurate and clean input handling on both POSIX systems (such as Unix, Linux, and macOS) and Windows.

Why does stdin need to be cleared?

When working with user input, especially in interactive command-line programs, it is important to ensure that the data in stdin is fresh and relevant. Several scenarios can lead to unwanted data lingering in stdin, such as:

  1. Stale Input from Previous Operations: If the user has previously entered data, but the program did not consume it fully (for instance, if the user presses “Enter” before the program reads the input), it remains in the stdin buffer. This can cause the next call to input() to read this leftover data instead of waiting for new input.
  2. Automated or Unexpected Input: In automated or scripted environments, programs may send data to stdin, which could interfere with interactive user input. Similarly, input that wasn’t expected may get in the way of user interaction.

Clearing stdin before invoking input() ensures that any previous data in the stdin buffer is discarded, giving the command-line Python program a clean slate to properly receive and process the user’s input.

How to Clear stdin Properly in Python

On Unix-like systems (such as Linux and macOS), one effective method to discard the input buffer is by using the termios module. The termios.tcflush() function discards any unread data from the input buffer.

On Windows, the msvcrt module can be used to achieve a similar effect by reading and discarding characters from the buffer until it is empty.

The following code snippet demonstrates how to clear stdin before calling input() to ensure only fresh input is received, for both Unix-like systems and Windows:

# Author: James Cherti
# License: MIT
# URL: https://www.jamescherti.com/python-flushing-stdin-before-using-input-function/

import os
import sys

def clear_stdin():
    """Clear any pending input from the standard input buffer.

    This function ensures that no stale or unintended data remains in stdin
    before reading user input interactively. On Windows, it uses the msvcrt
    module to discard characters from the input buffer. On POSIX-compliant
    systems (e.g., Linux, macOS...), it uses select to check for available
    input without blocking and either discards the data by reading or flushes
    it using termios.tcflush if stdin is a terminal.
    """
    try:
        if os.name == "nt":
            import msvcrt  # pylint: disable=import-outside-toplevel

            # For Windows systems, Check if there is any pending input in the
            # buffer Discard characters one at a time until the buffer is empty.
            while msvcrt.kbhit():
                msvcrt.getch()
        elif os.name == "posix":
            import select  # pylint: disable=import-outside-toplevel

            # For Unix-like systems, check if there's any pending input in
            # stdin without blocking.
            stdin, _, _ = select.select([sys.stdin], [], [], 0)
            if stdin:
                if sys.stdin.isatty():
                    # pylint: disable=import-outside-toplevel
                    from termios import TCIFLUSH, tcflush

                    # Flush the input buffer
                    tcflush(sys.stdin.fileno(), TCIFLUSH)
                else:
                    # Read and discard input (in chunks).
                    while sys.stdin.read(1024):
                        pass
    except ImportError:
        passCode language: Python (python)

Conclusion

Clearing stdin before calling input() helps ensure that a Python program processes only the intended user input. This can be useful in interactive command-line applications, where residual data in the input buffer may otherwise lead to unexpected behavior.

Python: Extract variables/values from source code comments

Reply

The source code in this article can be used to extract variables and values from source code comments. The code is written in Python and uses a combination of regular expressions and Python’s built-in string functions to extract specific information from source code comments.

#!/usr/bin/env python
# Author: James Cherti
# License: MIT
# URL: https://www.jamescherti.com/python-extract-variables-values-from-source-code-comments/
"""Extract variables/values from source code comments."""

import re
from typing import Dict


def get_variables_from_comments(
        source_code_content: str,
        comment_pattern: str = r'[^\w\s]+') -> Dict[str, list]:
    """Extract variables/values from source code comments.

    Source code example:
        #!/usr/bin/env python
        # This is a simple comment.
        print("Hello world")
        #
        # myvar: value 1
        # myvar: value 2
        # myvar: value 3
        # AnotherVar: value 1

    Here is how to extract the variables and their values from the
    source code above:
    >>> get_variables_from_comments(source_code_content)
    {'AnotherVar': ['value 1'], 'myvar': ['value 1', 'value 2', 'value 3']}

    """
    source_code_lines = source_code_content.splitlines()

    result: Dict[str, list] = {}
    for line in source_code_lines:
        re_str = (r'^\s*' +
                  comment_pattern +
                  r'\s*([\w\d]+)\s*:\s*(.*)\s*$')
        match_result = re.search(re_str, line)
        if match_result:
            var_name = match_result.group(1)
            var_value = match_result.group(2)

            if var_name not in result:
                result[var_name] = []

            result[var_name].append(var_value)

    return result


def main():
    """Try the method 'get_variables_from_comments()'."""

    source_code = (
        "#!/usr/bin/env python\n"
        "# This is a simple comment.\n"
        "print(\"Hello world\")\n"
        "#\n"
        "# myvar: value 1\n"
        "# myvar: value 2\n"
        "#\n"
        "# myvar: value 3\n"
        "# AnotherVar: value 1\n"
    )

    __import__('pprint').pprint(get_variables_from_comments(
        source_code_content=source_code,
        comment_pattern=re.escape('#'))
     )


if __name__ == '__main__':
    main()Code language: Python (python)

Python: Tab completion against a list of strings (readline)

Reply 
#!/usr/bin/env python
# License: MIT
# Author: James Cherti
# URL: https://www.jamescherti.com/python-tab-completion-readline-against-list/
"""Tab completion against a list of strings (readline)"""

import readline
from typing import Any, List, Union


class ReadlineCompleter:
    """A readline completer."""

    def __init__(self, options: List[str]):
        """Store the options = ['word1', 'word2']."""
        readline.set_completer_delims('')
        self.options = options
        self.matches: List[str] = []

    def complete(self, _, state):
        """Complete a readline sentence."""
        if state == 0:
            origline = readline.get_line_buffer()
            begin = readline.get_begidx()
            end = readline.get_endidx()
            being_completed = origline[begin:end]
            words = origline.split()

            if not words:
                self.matches = self.options[:]
            else:
                try:
                    if begin == 0:
                        matches = self.options[:]  # First word
                    else:
                        first = words[0]  # Later word
                        matches = self.options[first]

                    if being_completed:
                        # Match options with portion of input
                        # being completed
                        self.matches = [w for w in matches
                                        if w.startswith(being_completed)]
                    else:
                        # Matching empty string so use all candidates
                        self.matches = matches
                except (KeyError, IndexError):
                    self.matches = []

        try:
            return self.matches[state]
        except IndexError:
            return None


def input_completion(prompt: Any,
                     list_options: Union[None, List[str]] = None):
    """Read a string from standard input and complete against 'list_options'.

    The trailing newline is stripped. The prompt string is printed to
    standard output without a trailing newline before reading input.

    If the user hits EOF (*nix: Ctrl-D, Windows: Ctrl-Z+Return), raise
    EOFError. On *nix systems, readline is used if available.

    """
    readline.parse_and_bind('tab: complete')
    if list_options is None:
        list_options = []

    save_completer = readline.get_completer()
    try:
        readline.set_completer(
            ReadlineCompleter(list_options).complete
        )
        return input(prompt)
    finally:
        readline.set_completer(save_completer)


def main():
    """Try input_completion()."""
    list_options = ["yes", "no", "cancel"]
    value = input_completion("Proceed (press the Tab key)? ",
                             list_options=list_options)
    print("Value:", value)


if __name__ == "__main__":
    main()Code language: Python (python)

A tool to Execute a Command in a new Tmux Window

Reply

The Python script tmux-run.py allows executing a command in a new tmux window. A tmux window is similar to a tab in other software.

If the script is executed from within a tmux session, it creates a tmux window in the same tmux session. However, if the script is executed from outside of a tmux session, it creates a new tmux window in the first available tmux session.

(Requirement: libtmux)

The Python script: tmux-run.py

#!/usr/bin/env python
# License: MIT
# Author: James Cherti
# URL: https://www.jamescherti.com/python-script-run-command-new-tmux-window/
"""Execute a command in a new tmux window.

This script allows executing a command in a new tmux window (a tmux window is
similar to a tab in other software).

- If it is executed from within a tmux session, it creates a tmux window
in the same tmux session.
- However, if the script is executed from outside of a tmux
session, it creates a new tmux window in the first available tmux session.

"""

import os
import shlex
import shutil
import sys

import libtmux


SCRIPT_NAME = os.path.basename(sys.argv[0])


def parse_args():
    if len(sys.argv) < 2:
        print(f"Usage: {SCRIPT_NAME} <command> [args...]",
              file=sys.stderr)
        sys.exit(1)

    args = sys.argv[1:]
    args[0] = shutil.which(args[0])
    if args[0] is None:
        print(f"{SCRIPT_NAME}: no {args[0]} in "
              f"({os.environ.get('PATH', '')})", file=sys.stderr)
        sys.exit(1)

    return args


def get_tmux_session():
    tmux_server = libtmux.Server()
    if not tmux_server.sessions:
        print(f"{SCRIPT_NAME}: the tmux session was not found",
              file=sys.stderr)
        sys.exit(1)

    tmux_session_id = os.environ["TMUX"].split(",")[-1]
    if tmux_session_id:
        try:
            return tmux_server.sessions.get(id=f"${tmux_session_id}")
        except Exception:  # pylint: disable=broad-except
            pass

    return tmux_server.sessions[0]


def run_in_tmux_window():
    try:
        command_args = parse_args()
        tmux_session = get_tmux_session()
        command_str = shlex.join(command_args)
        tmux_session.new_window(attach=True, window_shell=command_str)
    except libtmux.exc.LibTmuxException as err:
        print(f"Error: {err}.", file=sys.stderr)
        sys.exit(1)


if __name__ == '__main__':
    run_in_tmux_window()


Code language: Python (python)

Python: Read the shebang line of a script

Reply 
#!/usr/bin/env python
# Author: James Cherti
# License: MIT
# URL: https://www.jamescherti.com/python-read-the-shebang-line-of-a-script/
"""Read the shebang line of a script."""

import sys
import os
import shlex
from pathlib import Path
from typing import Union


class ShebangError(Exception):
    """Error with the method read_shebang()."""


def read_shebang(script_path: Union[Path, str]) -> list:
    """Return the shebang line of a file.

    >>> shebang("file.sh")
    ['/usr/bin/env', 'bash']

    """
    with open(script_path, "rb") as fhandler:
        line = fhandler.readline().strip().decode()

    if len(line) > 2 and line[0:2] == '#!':
        shebang_split = shlex.split(line[2:].strip())
        if not Path(shebang_split[0]).is_file():
            raise ShebangError(f"the shebang '{shebang_split}' does not exist")

        if not os.access(shebang_split[0], os.X_OK):
            raise ShebangError(f"the shebang '{shebang_split}' is not "
                               "executable")

        return shebang_split

    raise ShebangError("the shebang line was not found")

    
if __name__ == "__main__":
    try:
        print(read_shebang(sys.argv[1]))
    except IndexError:
        print(f"Usage: {sys.argv[0]} <file>", file=sys.stderr)
        sys.exit(1)Code language: Python (python)

Python: Calculate the size of a directory and its sub-directories

Reply 
#!/usr/bin/env python
# Author: James Cherti
# License: MIT
# URL: https://www.jamescherti.com/python-calculate-the-size-of-a-directory-and-its-sub-directories/
"""Calculate the size of a directory and its sub-directories."""

import os
from pathlib import Path
from typing import Union

def get_size(path: Union[Path, str], include_dirs_size=True) -> int:
    """Return the size of a file or a directory in bytes."""
    path = Path(path)
    size = 0

    if path.is_dir():
        list_paths = path.glob("**/*")
    elif path.is_file():
        list_paths = [path]  # type: ignore
    else:
        list_paths = []  # type: ignore

    for cur_path in list_paths:
        if not include_dirs_size and cur_path.is_dir():
            continue

        if not cur_path.is_symlink():
            size += cur_path.stat().st_size

    return sizeCode language: Python (python)