Handle State Library

Location

  • config/handle_state.sh

Purpose

This library provides two core capabilities for Bash libraries:

  • Persisting local variable state from one function to another (typically initialization to cleanup) via a generated snippet that can be eval’d.

  • As its state persistence functions output code on stdout, the library allows provides the means to display messages to stdout from within initialization functions using a logging FIFO and background reader process.

Dependencies

This library depends on the Command Guard Library (config/command_guard.sh) for secure execution of external commands. The dependency is automatically resolved when the library is sourced.

Quick Start

Source the file once, then use hs_persist_state in the init function and eval the state in cleanup. For cleaner code, assign to a variable instead of capturing stdout.

# Source once in the main script of your library
source "$(dirname "$0")/config/handle_state.sh"

init_function() {
    # Direct output to stdout would mess up the state snippet, so use hs_echo if needed
    hs_echo "Initializing..."
    # Define some opaque library resources
    local temp_file="/tmp/some_temp_file"
    local resource_id="resource_123"
    hs_persist_state "$@" temp_file resource_id
}

cleanup() {
    local state="$1"
    local temp_file resource_id
    eval "$state"
    rm -f "$temp_file"
    echo "Cleaned up resource: $resource_id"
}

# State is assigned to the variable, no stdout capture needed
local my_state
init_function -S my_state
# Your main script logic here
cleanup "$my_state"

For backward compatibility, the old stdout capture method still works:

# Capture the opaque state snippet emitted on stdout
state=$(init_function)
cleanup "$state"

Public API

hs_setup_output_to_stdout

Sets up a FIFO and background reader process that forwards log lines from subshells to the main script output. The function initializes internal state, creates hs_cleanup_output, and defines hs_echo for use inside subshells.

  • Behavior: no-op if logging is already set up (detected via hs_get_pid_of_subshell).

  • Side effects: creates a FIFO via a temporary file, opens it on a file descriptor, and removes the FIFO path while the descriptor remains open.

    Warning

    This library allocates a new, unused file descriptor for internal job operations. While this descriptor will not interfere with any file descriptors already in use, please note that file descriptors are a global resource. The library will break down if downstream code attempts to use or manipulate its private file descriptor.

hs_cleanup_output

Defined dynamically by hs_setup_output_to_stdout. Sends the kill token to the FIFO, waits for the background reader to exit, and redefines itself as a no-op while resetting hs_echo to a plain echo.

hs_echo

Defined dynamically by hs_setup_output_to_stdout. Writes messages to the FIFO so they appear in the main script stdout even when stdout of a subshell is being captured.

  • Usage: hs_echo “message”

  • Notes: preserves Bash echo argument concatenation behavior.

hs_persist_state

Emits Bash code that restores specified local variables in a receiving scope. The emitted snippet only assigns values if the target variable is declared local in the receiving scope and is still empty.

The function accepts optional -s or -S arguments:

  • -s <state>: Treats <state> as an existing state snippet to append to.

  • -S <var>: Assigns the state (appended to any existing content in <var>) to the variable named <var> instead of printing to stdout.

These options can be used together; when both are provided, <var> is used for output and must be empty or uninitialized.

This allows avoiding stdout output for opaque data when assigning to a variable, while maintaining backward compatibility for appending to state strings or state vars.

Warning

When using -S with a variable name, the function will eval the current contents of that variable during collision checking. Callers must ensure the variable contains only safe, trusted Bash code or is empty/unset to avoid execution of harmful code.

Code protections ensure that prior state is only evaluated when necessary for collision checking, and only if the variable is not empty. Corrupted state code is detected when it calls undefined commands during this evaluation, and when the evaluation takes more than one second (to prevent hangs).

Libraries are encouraged to provide the same -s and -S options to their initialization functions to allow callers to chain state snippets together.

When appending to an existing state snippet, the function checks for name collisions and refuses to overwrite existing variables. Some library combinations can be incompatible with the chaining approach because they use overlapping variable names. The alternate solution is to keep and eval separate state snippets for each library.

  • Usage: hs_persist_state [-s <state> | -S <var>] var1 var2 …

  • Output: a string of Bash code intended to be eval’d by the caller (when not assigning to variable).

  • Errors: - Refuses to take into account more than one prior state. - Detects an invalid variable name passed to option -S. - Refuses to persist reserved names __var_name, __existing_state, __output_state_var and __output. - Rejects collisions when a variable already exists in the provided prior state. - Detects some the most severe forms of corrupted prior state code (hangs or undefined commands).

  • Guarantees: - Errors out or succeeds in an atomic manner; no partial state is emitted on error.

Warning

The function cannot currently properly capture arrays, namerefs, associative arrays nor functions. Only scalar string variables are supported.

hs_read_persisted_state

This function is a simple wrapper around echo. There is no way in Bash to set variables in the caller scope without eval`ing code, so this function is only provided for symmetry with `hs_persist_state. Its output must be eval’d by the caller to restore state anyway and the effect is identical to eval “$state”. Code readability is slightly improved by using this function.

  • Usage: eval “$(hs_read_persisted_state “$state”)”

hs_get_pid_of_subshell

Parses the hs_cleanup_output function definition to extract the background reader PID. This is used to detect whether logging setup has already occurred.

Error Codes

  • HS_ERR_RESERVED_VAR_NAME=1: a reserved variable name was passed to hs_persist_state.

  • HS_ERR_VAR_NAME_COLLISION=2: the requested variable was already defined in the state string when persisting.

Behavior Details

Logging FIFO

When sourced, the library calls hs_setup_output_to_stdout automatically. It spawns a background reader that:

  • reads lines from the FIFO with a timeout loop,

  • echoes them to stdout,

  • exits after a magic kill token or an idle timeout,

  • closes the FIFO descriptor before exiting.

State Persistence

hs_persist_state captures caller-local variables by name, embeds their values in a guarded assignment snippet, and prints that snippet. The guards ensure that only local variables are populated in the receiving scope and that non-empty locals are not overwritten.

Supported Variables

hs_persist_state reliably preserves local scalar variables (strings or numbers) that are defined in the calling scope and re-declared as local in the receiving scope.

Known Limitations (Tracked)

The following behaviors are tracked in GitHub and should be considered when using this library:

  • Unknown variable names are silently ignored instead of erroring: Issue #1.

  • Function names are silently ignored instead of erroring: Issue #2.

  • Indexed arrays only preserve the first element (marked major): Issue #3.

  • Associative arrays are silently ignored: Issue #4.

  • Namerefs are persisted as scalars (indirection is lost): Issue #5.

Workarounds

  • Associative arrays can be represented as two indexed arrays (keys and values).

  • Indexed arrays can be represented as a string with encoding.

  • Other complex constructs can sometimes be replaced by scalar strings or rebuilt from scalars using custom logic.

Caveats

  • Always declare target variables local before eval’ing the state snippet; otherwise assignments are skipped to avoid leaking globals.

  • The library uses eval internally; treat state strings as trusted input.

  • Call hs_cleanup_output when you are done to stop the background reader.

Source Listing

  1#!/bin/bash
  2# File: config/handle_state.sh
  3# Description: Helper functions to carry state information between initialization and cleanup functions.
  4# Author: Jean-Marc Le Peuvédic (https://calcool.ai)
  5
  6# Sentinel
  7[[ -z ${__HANDLE_STATE_SH_INCLUDED:-} ]] && __HANDLE_STATE_SH_INCLUDED=1 || return 0
  8
  9# Source command guard for secure external command usage
 10# shellcheck source=config/command_guard.sh
 11# shellcheck disable=SC1091
 12source "${BASH_SOURCE%/*}/command_guard.sh"
 13
 14# Library usage:
 15#   In an initialization function, call hs_persist_state with the names of local variables
 16#   that need to be preserved for later use in a cleanup function.
 17# Example:
 18#   init_function() {
 19#       local temp_file="/tmp/some_temp_file"
 20#       local resource_id="resource_123"
 21#       hs_persist_state temp_file resource_id
 22#       exit 0
 23#   }
 24#   cleanup() {
 25#       local temp_file
 26#       local resource_id
 27#       eval "$1"  # Recreate local variables from the state string
 28#       # Now temp_file and resource_id are available for cleanup operations
 29#       rm -f "$temp_file"
 30#       echo "Cleaned up resource: $resource_id"
 31#   }
 32#
 33# Upper level usage: state=$(init_function)
 34#                    cleanup "$state"
 35
 36guard mkfifo rm timeout
 37
 38# --- logging from $(command) using a FIFO ---------------------------------------
 39# This section sets up a FIFO and a background reader process to allow functions
 40# to log messages to the main script's stdout/stderr even when they are called
 41# from subshells (e.g., inside `$(...)` command substitutions). Functions can
 42# use `hs_echo "message"` to send messages to the main script's output.
 43
 44# Function: 
 45#   _hs_set_up_logging
 46# Description:
 47#   Sets up a FIFO and background reader process for logging.
 48#   The background reader must start before any I/O redirection occurs. This
 49#   is why it is called at the end of this file, so that when this file is sourced,
 50#   the logging is set up.
 51# Usage:
 52#   Do not call directly; called automatically when this file is sourced.
 53#   When done with the library, call `hs_cleanup_output` to terminate the background reader
 54#   or else a 5 seconds idle timeout will occur.
 55hs_setup_output_to_stdout() {
 56    # Test if already set up
 57    if hs_get_pid_of_subshell >/dev/null 2>&1; then
 58        echo "[WARN] hs_setup_output_to_stdout: already set up; skipping." >&2
 59        return 0
 60    fi
 61    # Create a FIFO using a proper temporary file and file descriptor 3 
 62    fifo_file=$(mktemp -u)
 63    mkfifo "$fifo_file"
 64    # Redirect fd 3 into the FIFO
 65    exec {_hs_fifo_fd}<> "$fifo_file"
 66    # Make file disappear immediately. The FIFO remains accessible via fd 3.
 67    rm "$fifo_file"
 68    # Kill token
 69    _hs_fifo_kill_token="hs_kill_${$}_$RANDOM_$RANDOM"
 70    _hs_fifo_idle_limit=5  # seconds before self-termination
 71
 72    # Run a background task that reads from the FIFO and displays messages
 73    (
 74        idle=0
 75
 76        while true; do
 77            line=''
 78
 79            if IFS= read -t 1 -r line <&"${_hs_fifo_fd}"; then
 80                # Received a line; reset idle counter
 81                idle=0
 82                # Self-terminate on the exact magic token
 83                if [ "${line:-}" = "${_hs_fifo_kill_token}" ]; then
 84                    break
 85                fi
 86                echo "$line"
 87            else
 88                # read timed out or encountered EOF/error - increment idle counter
 89                idle=$((idle + 1))
 90                if [ "$idle" -ge "$_hs_fifo_idle_limit" ]; then
 91                    break
 92                fi
 93                # continue to wait
 94            fi
 95        done
 96        # Dismantle FIFO: close fd
 97        exec {_hs_fifo_fd}>&-
 98    ) & _hs_fifo_reader_pid=$!
 99 
100    # Function:
101    #   hs_cleanup_output, or redefined globally with the kill token embedded.
102    # Description:
103    #   Sends the magic kill token to the logging FIFO to terminate the background reader.
104    #   Waits for the background reader to exit and redefines itself to a no-op.
105    #   Redefines hs_echo to a simple echo.
106    # Parameters:
107    #   None
108    printf -v _hs_qtoken '%q' "$_hs_fifo_kill_token"
109    eval "hs_cleanup_output() {
110        if hs_echo $_hs_qtoken ; then
111            wait $_hs_fifo_reader_pid 2>/dev/null
112            hs_cleanup_output() { :; }
113            hs_echo() { echo \"\$*\" ; }
114        fi
115        return 0
116    }"
117        
118    # Function:
119    #   hs_echo
120    # Description:
121    #   Writes messages to the logging FIFO for display in the main script's stdout.
122    #   Specifically designed to work inside subshells called via `$(...)`.
123    #   Mimic Bash echo argument concatenation behavior.
124    #   Here IFS acts on "$*" expansion to insert spaces between arguments.
125    # Arguments:
126    #   $* - echo options -neE or message parts to echo
127    eval "hs_echo() {
128        IFS=\" \" echo \"\$*\" >&\"${_hs_fifo_fd}\"
129    }"
130}
131
132# --- Public error codes --------------------------------------------------------
133readonly HS_ERR_RESERVED_VAR_NAME=1
134readonly HS_ERR_VAR_NAME_COLLISION=2
135readonly HS_ERR_MULTIPLE_STATE_INPUTS=3
136readonly HS_ERR_CORRUPT_STATE=4
137readonly HS_ERR_INVALID_VAR_NAME=5
138
139# --- hs_persist_state ----------------------------------------------------------
140# Function:
141#   hs_persist_state
142# Description:
143#   Emits a bash code snippet that, when eval'd in the receiving scope,
144#   will recreate the specified local variables with their current values.
145#   The emitted code checks if the variable is declared `local` in the receiving
146#   scope before assigning to it, to avoid polluting global scope.
147#   If the variable already exists and is non-empty in the receiving scope,
148#   an error message is printed and the assignment is skipped.
149# Arguments:
150#   -s <state> - optional; if provided, appends the emitted code to variable
151#                definitions found in <state> (bash code snippet).
152#   $@ - names of local variables to persist.
153# Usage examples:
154#   # direct eval
155#   state=$(hs_persist_state var1 var2)
156#   cleanup() {
157#       local var1 var2
158#       eval "$1"
159#       # vars are available here
160#   }
161hs_persist_state() {
162    local __existing_state=""
163    local __output_state_var=""
164    while [ $# -gt 0 ]; do
165        case "$1" in
166            -s)
167                shift
168                __existing_state="$1"
169                shift
170                ;;
171            -S)
172                shift
173                __output_state_var="$1"
174                # Check that __output_state_var is a valid variable name
175                if ! [[ "$__output_state_var" =~ ^[a-zA-Z_][a-zA-Z0-9_]*$ ]]; then
176                    echo "[ERROR] hs_persist_state: invalid variable name '$__output_state_var' for -S option." >&2
177                    return "$HS_ERR_INVALID_VAR_NAME"
178                fi
179                ;;
180            *)
181                break
182                ;;
183        esac
184    done
185    # If __output_stat_var is set, we have to check if __existing_state is set too,
186    # and enforce that ${!__output_state_var} is uninitialized or empty to ensure
187    # that we are not dealing with multiple prior state strings.
188    if [ -n "$__output_state_var" ] && [ -n "$__existing_state" ]; then
189        echo "[ERROR] hs_persist_state: cannot pass prior state using both -s and -S options simultaneously." >&2
190        return "$HS_ERR_MULTIPLE_STATE_INPUTS"
191    fi
192    # If __output_state_var is set, retrieve prior state from it
193    if [ -n "${__output_state_var}" ]; then
194        __existing_state="${!__output_state_var}"
195    fi
196    # Initialize output state string
197    local __output=""
198    if [ -n "$__existing_state" ]; then
199        __output="$__existing_state"
200    fi
201    local __var_name
202    for __var_name in "$@"; do
203        # Check that the value of __var_name is neither "__var_name" nor "__existing_state"
204        if [ "$__var_name" = "__var_name" ] || [ "$__var_name" = "__existing_state" ] || [ "$__var_name" = "__output_state_var" ] || [ "$__var_name" = "__output" ]; then
205            echo "[ERROR] hs_persist_state: refusing to persist reserved variable name '$__var_name'." >&2  
206            return "$HS_ERR_RESERVED_VAR_NAME"
207        fi
208        # Detect name collisions if __existing_state is provided
209        if [ -n "$__existing_state" ]; then
210            # In a time-constrained subshell, declare "$__var_name" as local to capture its value
211            # and attempt to restore it from "$__existing_state".
212            timeout --preserve-status -k 2 1 "$0" --noprofile -elc "
213                command_not_found_handle() {
214                    echo \"[ERROR] hs_persist_state: command '\$1' not found.\" >&2
215                    exit 127
216                }
217                test_collision() {
218                    local \"$__var_name\"
219                    eval \"$__existing_state\" >/dev/null
220                    # Check if the variable pointed to by __var_name has been initialized
221                    if ! [ -z \"\${${__var_name}+x}\" ]; then
222                        echo \"[ERROR] hs_persist_state: variable '$__var_name' is already defined in the state, with value '\${${__var_name}}'.\" >&2
223                        exit 1
224                    fi
225                }
226                test_collision
227            " 
228            local status=$?
229            if [ $status -eq 124 ] || [ $status -eq 127 ] || [ $status -eq 137 ] || [ $status -eq 143 ]; then
230                # Status code snippet timed out: 124 (timeout), 137 (killed), 127 (command not found), 143 (sigterm)
231                echo "[ERROR] hs_persist_state: prior state is corrupted." >&2
232                return $((HS_ERR_CORRUPT_STATE))
233            elif [ $status -eq 1 ]; then
234                return $((HS_ERR_VAR_NAME_COLLISION))
235            elif [ $status -ne 0 ]; then
236                echo "[ERROR] hs_persist_state: internal error while checking for variable name collision for '$__var_name'." >&2
237                return $((HS_ERR_CORRUPT_STATE))
238            fi
239        fi
240        # Check if the variable exists in the caller (local or global). We avoid
241        # using `local -p` here because that only inspects locals of this
242        # function, not the caller's scope. If the variable exists, capture its
243        # value and emit a guarded assignment that will only set it in the
244        # receiving scope if that scope has declared it `local`.
245        if [ "${!__var_name+x}" ]; then
246            # Get the value of the variable
247            local var_value
248            var_value="${!__var_name}" || eval "var_value=\"\${$__var_name}\"" || eval "var_value=\"\$$__var_name\""
249            # Emit a snippet that, when eval'd in the receiving scope, will
250            # restore the existing, empty local variables from the saved state.
251            __snippet=$(printf "
252if local -p %s >/dev/null 2>&1; then
253  if [ -n \"\${%s+x}\" ] && [ -n \"\${%1s}\" ]; then
254    printf \"[ERROR] local %1s already defined; refusing to overwrite\\n\" >&2
255    return 1
256  else
257    %s=%q
258  fi
259fi
260" "$__var_name" "$__var_name" "$__var_name" "$__var_name" "$__var_name" "$var_value")
261            __output="${__output}${__snippet}"
262        fi
263    done
264    if [ -n "$__output_state_var" ]; then
265        eval "$__output_state_var=\"\$__output\""
266    else
267        printf '%s\n' "$__output"
268    fi
269}
270
271# --- hs_read_persisted_state --------------------------------------------------------
272# Function: 
273#   hs_read_persisted_state
274# Description: 
275#   Emits the state string produced by `hs_persist_state` without
276#   evaluating it. This avoids executing the state inside this function's scope
277#   (which would prevent assignments to `local` variables declared in the
278#   calling function). Callers should `eval "$(hs_read_persisted_state "$state")"`
279#   or simply `eval "$state"` to recreate variables in the caller scope.
280#   Can be called several times to extract distinct variables.
281#   "$state" is a bash code snippet that assigns values to existing local and empty
282#   variables in the current scope.
283# Arguments:
284#   $1 - state string produced by `hs_persist_state`
285# Usage examples:
286#   # direct eval
287#   cleanup() {
288#       local temp_file resource_id
289#       eval "$1"
290#       # vars are available here
291#   }
292#
293#   # helper wrapper form (prints state; caller evals it in its own scope)
294#   cleanup() {
295#       local state="$1"
296#       local temp_file resource_id
297#       eval "$(hs_read_persisted_state \"$state\")"
298#   }
299hs_read_persisted_state() {
300    local state_string="$1"
301    printf '%s' "$state_string"
302}
303
304# --- Utility functions --------------------------------------------------------
305# Function:
306#    hs_get_pid_of_subshell
307# Description:
308#    Returns the PID of the current subshell that works in conjunction with hs_echo to
309#    ensure output is properly captured and redirected to whatever stdout was when the
310#    library was sourced.
311# Usage:
312#    pid=$(hs_get_pid_of_subshell)
313# Return status:
314#    0 - Success
315#    1 - Internal error: hs_cleanup_output not defined or doesn't have the expected format.
316hs_get_pid_of_subshell() {
317    # Extract the PID of the background reader process from the function definition
318    local func_def
319    func_def=$(declare -f hs_cleanup_output)
320    local pid
321    pid=${func_def##*wait }
322    # The above string substitution will just return $func_det without an error if "wait " is not found.
323    if [ "$pid" = "$func_def" ]; then
324        echo "hs_cleanup_output function not found or has unexpected format" >&2
325        return 1
326    fi
327    pid=${pid%%[^0-9]*}
328    printf '%s' "$pid"
329}
330
331# Initialize logging when the script is sourced
332hs_setup_output_to_stdout
333
334# Note: Remember to call hs_cleanup at the end of your main script to clean up resources.