Remote Run Library

Location

  • config/remote_run.sh

Purpose

This library provides a rr_run entry point which executes a local shell script on a remote host via SSH without writing any file to the remote filesystem. The main script and every library it sources are kept on the local machine; the remote side receives file content on demand through a private TCP channel established by SSH port-forwarding.

The library is designed for scripts that are already validated locally and use source as their composition mechanism (e.g. handle_state.sh, command_guard.sh).

Dependencies

Dependency

Side

Notes

Bash ≥ 4.3

Both

Nameref (local -n) in handle_state.sh; auto-assigned FDs (exec {var}<>file) in remote_run.sh

OpenSSH client

Local

ssh, -T, -R required

OpenSSH server

Remote

sshd must be running and reachable

nc

Local

OpenBSD nc (-lU socket); other variants are not supported

base64

Remote

Standard on all major Linux distributions

All local-side dependencies are checked when the library is sourced via cg_guard. If any required tool is absent the library fails to load, emits a diagnostic, and returns RR_ERR_DEPENDENCY_MISSING.

Architecture Overview

rr_run establishes a ControlMaster connection and multiplexes two operations over it. Each concurrent rr_resolve call adds a third channel for the duration of its file transfer, then closes it at EOF:

Bootstrap channel (SSH stdin via -T) — delivers the bootstrap shell fragment to the remote bash. The fragment is piped via process substitution; the write end closes when the generator exits, signalling EOF to remote bash stdin after the last bootstrap command. No PTY is allocated; ssh -T is always used.

Protocol channel (TCP via SSH -R) — carries the file-serving protocol after the bootstrap is running. Each rr_run call starts a nc listener on an ephemeral Unix-domain socket. SSH forwards an auto-allocated remote TCP port to that socket. The remote bash opens that port directly using Bash’s built-in /dev/tcp/localhost/<port> path with exec {fd}<>/dev/tcp/…; no SSH client and no additional tool is required on the remote side. All subsequent GET / RESOLVE / OK / ERR messages travel over that single bidirectional file descriptor.

local bootstrap pipe ──► SSH -T stdin ──► remote bash stdin (bootstrap)

local nc server ◄── SSH -R tunnel ◄── remote bash fd (protocol)
  (GET / RESOLVE / OK / ERR)

The remote bootstrap, delivered via SSH stdin, performs the following steps before any user code runs:

  1. Allocates the protocol file descriptor dynamically (exec {fd}<>).

  2. Generates and evaluates the source override with the fd value inscribed literally, so that source works correctly regardless of what the user script does to shell variables.

  3. Propagates the local shell flags ($-) from the rr_run call site so that set -x, set -e, set -u, set -o pipefail etc. are active on the remote if and only if they were active locally. Note that the remote script may subsequently change these flags at any time, exactly as it would when run locally. PS4 is configured to display real source paths rather than internal wrapper names when tracing is active.

  4. Runs the user script inside the double wrapper (see below).

Quick Start

# Source the library once
source "$(dirname "$0")/config/remote_run.sh"

# Run a local script on a remote host (no prior initialisation needed)
rr_run user@host deploy.sh --env production

# Capture default options once, then run in parallel on multiple hosts
local state
rr_init -S state --ssh-opt "-i ~/.ssh/deploy_key" --allow /opt/shared/libs
for host in "${servers[@]}"; do
    rr_run -S state "root@$host" deploy.sh --env production &
done
wait

Inside deploy.sh or any script it sources, source works exactly as locally:

#!/usr/bin/env bash
source config/handle_state.sh     # fetched from local machine on demand
source config/command_guard.sh    # nested source — also fetched locally

guard curl jq

init_deployment() {
    local endpoint="$1"
    # ... deployment logic ...
}

init_deployment "$1"

Public API

rr_init

rr_init [-S <var>] [--allow <path>] [--ssh-opt <opt>]

Optional. Captures default options into a handle_state vector by name. Calling rr_run without a prior rr_init is valid; built-in defaults are used.

-S <var>

Name of the caller-owned variable that will receive the updated state vector. Declare local <var> before calling rr_init.

--allow <path>

Default whitelist entry. May be repeated.

--ssh-opt <opt>

Default SSH option. May be repeated.

rr_run

rr_run [-S <var>] [--allow <path>] [--ssh-opt <opt>] [--] <user@host> <script.sh> [args...]

Executes <script.sh> on <user@host> via SSH. All source calls inside the script resolve against the local filesystem. Each call allocates its own fd and nc instance; multiple rr_run calls may run in parallel against different hosts.

-S <var>

State vector produced by rr_init. Options in the vector are used as defaults and may be overridden by per-call --allow / --ssh-opt.

--allow <path>

Permit access to <path> in addition to the whitelist. May be repeated. Relative paths are resolved against the local working directory at invocation time.

--ssh-opt <opt>

Pass an extra option to every ssh invocation. May be repeated:

rr_run --ssh-opt "-p 2222" \
       --ssh-opt "-i ~/.ssh/deploy_key" \
       --ssh-opt "-o StrictHostKeyChecking=no" \
       user@host deploy.sh
<user@host>

SSH target. Any format accepted by ssh is valid.

<script.sh>

Path to the local script to execute, or a /dev/fd/N path produced by rr_resolve for relay scenarios.

[args...]

Positional arguments forwarded to the script as $1, $2, …

Exit code

Returns the exit code of the remote script, or one of the named error constants defined in the Error Codes section below:

Constant

Condition

RR_ERR_UNKNOWN_ARGUMENT

Unrecognised option in the option loop

RR_ERR_MISSING_ARGUMENT

<user@host> or <script> absent

RR_ERR_PATH_RESOLUTION_FAILED

realpath failed on an --allow arg

RR_ERR_SCRIPT_NOT_FOUND

Script file not found or not readable

RR_ERR_SSH_CONNECT_FAILED

ControlMaster connection failed

RR_ERR_PORT_FORWARD_FAILED

-O forward port allocation failed

HS error codes 1–12 may also be returned when -S state-read operations propagate a failure via || return $?.

rr_resolve

rr_resolve [-S <state>] <file>

Makes a local file available as a readable file descriptor and prints the corresponding /dev/fd/N path. The behaviour depends on which machine the function runs on:

On the originating machine (A): no-op; returns the file path unchanged.

On a relay machine (B, running a script fetched from A): sends a RESOLVE <path> request on the protocol channel back to A. A opens a dedicated nc listener on a new ephemeral port and establishes a second SSH -R tunnel to B for that port. A signals readiness with RESOLVE_OK <port> only after the tunnel is active (synchronised via ControlMaster -O forward). B allocates a new fd dynamically (exec {fd}<>) and returns /dev/fd/$fd. The dedicated nc exits naturally at EOF; no file is written to B.

Recursive relay (A → B → C): B’s rr_resolve triggers a RESOLVE toward A; A opens a tunnel through B to C. Each level uses its own dynamically allocated fds; no name collisions occur.

Intended use in a relay script running on B:

#!/usr/bin/env bash
source config/remote_run.sh          # fetched from A via the implicit source override
rr_run "root@C" "$(rr_resolve deploy.sh)" --env production

rr_cleanup

rr_cleanup [-S <var>]

Removes rr-managed variables from the named shared state object so the same state variable can be reused safely by a later rr_init call. Reserved for future ControlMaster teardown as well.

Remote Script Categories

Two categories of scripts may be executed via rr_run:

Category 1 — remote-run-aware scripts

The script explicitly sources config/remote_run.sh and uses the rr_* API. It may call rr_run itself for nested remote execution, using rr_resolve to transfer scripts from the originating machine without writing to the relay filesystem.

Category 2 — unaware scripts

The script has no knowledge of remote_run. It uses source as normal; the implicit source override intercepts these calls transparently. No explicit rr_* calls are made.

Double Wrapper

Every file fetched via the source override is executed through a double wrapper:

Outer wrapper (persistent for the duration of the source call):

  1. Reads the entire file content from the protocol fd into a local variable; the dedicated nc sees EOF and terminates cleanly.

  2. Calls the inner wrapper.

  3. Executes cleanup after the inner wrapper returns — whether by normal completion or by return.

Outer wrapper local variables are protected from eval inside the inner wrapper by Bash scope rules. handle_state is not used on the remote side.

Inner wrapper (one per sourced file):

  • Executes eval "$content" as a single block, correctly handling multi-line constructs (function bodies, if/fi, while/done) and reproducing the parse-then-execute semantics of source.

Note: exit destroys the entire shell stack immediately; the outer wrapper cannot intercept it. Documented as an accepted limitation.

source Semantics

Feature

Status

Notes

return in sourced file

Faithful

Outer wrapper recovers

$1, $@, $#, shift

Faithful

Args forwarded to wrapper

Variables defined in sourced file

Faithful

Same shell; no subshell

Functions defined in sourced file

Faithful

Same shell; no subshell

Nested source

Faithful

Recursive; same mechanism

local at file top level

Diverges

Becomes valid (see below)

BASH_SOURCE[0] accuracy

Approx.

May show eval context

FUNCNAME / BASH_LINENO accuracy

Approx.

Wrapper frame may be visible

Shell flags (set -e, set -x, …)

Faithful

Propagated from call site

Limitations

  • No PTY is allocated. rr_run always uses ssh -T. Remote commands that require a controlling terminal (sudo password prompts, read -s, readline programs) will fail or behave unexpectedly. Use key-based sudo NOPASSWD or pre-configure secrets before calling rr_run.

  • ``.`` (dot) is not overridden. Using . instead of source will attempt to read a file from the remote filesystem and will fail if the file is absent there.

  • Local CWD is fixed at invocation time. All paths given to source are resolved relative to the local working directory at the time rr_run is called, not relative to the currently executing script.

  • ``local`` at the top level of a sourced file becomes valid. Under the wrapper function it is silently accepted. This difference is benign for libraries that do not rely on this error.

  • ``BASH_SOURCE[0]`` may not show the expected path. A parallel map is maintained for diagnostic use (visible in PS4 output when set -x is active).

  • ``exit`` cannot be intercepted. A call to exit in the remote script or any sourced file terminates the entire remote shell immediately.

  • File descriptors opened with hard-coded numbers after library initialisation may collide. See File Descriptor Allocation Rule below.

  • Protocol channel limited to ~48 KB per file. Files larger than the OS socket buffer size require chunked transfers, which are not implemented. Realistic deployment scripts are well under this threshold.

File Descriptor Allocation Rule

All libraries in this project allocate file descriptors dynamically using the Bash auto-assignment form exec {var}<>. A library cannot protect itself against application code that later opens a hard-coded fd number that happens to coincide with a library-held fd.

Applications must either:

  • Use dynamic fd allocation themselves (exec {var}<>), or

  • Open all hard-coded fd numbers before sourcing any library from this project.

This rule applies to handle_state.sh, remote_run.sh, and all future libraries that hold open file descriptors.

Security Model

The local file server enforces a whitelist of allowed paths:

  • The default whitelist contains the directory of <script.sh>.

  • Additional entries are added with --allow (in rr_init or rr_run).

  • All requested paths are normalised (realpath -m) before checking.

  • Requests for paths outside the whitelist receive an ERR response; the remote source call fails with exit code 1 and an error message on stderr.

  • Path traversal sequences (..) are handled by normalisation and do not bypass the whitelist.

The system is designed for executing already-approved local scripts, not for sandboxing hostile code.

Error Codes

All public functions return 0 on success and a named constant on failure. The constants are readonly globals available to callers after the library is sourced. Values 1–12 are reserved for handle_state.sh error codes that may propagate via || return $?; no RR_ERR_* constant uses those values unless the semantics are identical and confusion is impossible.

  • RR_ERR_MISSING_STATE_VAR=7: -S was required but not supplied. Aligned with HS_ERR_STATE_VAR_UNINITIALIZED.

  • RR_ERR_MISSING_ARGUMENT=8: a required positional argument is absent. Aligned with HS_ERR_MISSING_ARGUMENT and CG_ERR_MISSING_ARGUMENT.

  • RR_ERR_UNKNOWN_ARGUMENT=9: an unrecognised token appeared in the option-parsing loop. Not limited to dash-prefixed tokens. Aligned with CG_ERR_SYNTAX_ERROR (pending rename to CG_ERR_UNKNOWN_ARGUMENT).

  • RR_ERR_PATH_RESOLUTION_FAILED=13: realpath -m failed on an --allow path argument.

  • RR_ERR_SCRIPT_NOT_FOUND=14: the <script.sh> argument is not a readable file.

  • RR_ERR_SSH_CONNECT_FAILED=15: the ControlMaster SSH connection to the target host could not be established.

  • RR_ERR_PORT_FORWARD_FAILED=16: SSH -O forward failed to allocate a remote TCP port for the protocol channel.

  • RR_ERR_FETCH_FAILED=17: the remote bootstrap received an ERR response to a GET request. Set by _rr_outer_wrapper on the remote.

  • RR_ERR_RESOLVE_FAILED=18: the remote bootstrap received a non-RESOLVE_OK response. Set by _rr_do_resolve on the remote.

  • RR_ERR_DEPENDENCY_MISSING=19: a required tool (nc, ssh, base64, realpath, mktemp, dirname, cat, or sleep) could not be resolved by guard at source time. The library failed to load entirely; no rr_* functions are available.

Change History

PR

Summary

#TBD

add Error Codes section; fix nc dependency note; fix exit-code table