ACP External Harness Remote Node Execution Fails with 'acpx exited with code 1' in Discord Workspace Flow
Troubleshooting guide for ACP turn execution failures when routing Discord workspace sessions to external harnesses (Claude Code/Codex) running on remote nodes.
π Symptoms
Primary Error Manifestations
After invoking /acp spawn claude --thread here --label claude-studio in a Discord channel, the session initializes successfully but turn execution fails with:
ACP_TURN_FAILED: acpx exited with code 1
The gateway logs may show a successful acpx spawn but immediate termination:
[gateway] INFO acp.spawn: agent=openclaw session=acp:discord:channel:123456789:thread:987654321
[gateway] DEBUG acp.harness: launching external harness: env OPENCLAW_HIDE_BANNER=1 ...
[gateway] ERROR acp.turn: acpx process terminated unexpectedly exit_code=1
Secondary Symptom: Session-Mode Conflict
When a Discord channel already has an existing non-ACP route binding, ACP invocation produces:
ACP_SESSION_INIT_FAILED: Session is not ACP-enabled: agent:athos:discord:channel:123456789
This occurs when channels.discord.bindings contains a static route to a local agent before attempting ACP session creation.
Diagnostic Command Outputs
Gateway status check:
$ openclaw status
Gateway: running (ws://127.0.0.1:18789)
Plugins: acpx β | discord β | bindings β
ACP Sessions: 0 active | 1 total
Remote Nodes: 1 connected (mac-mini.local)
ACP session list:
$ openclaw acp sessions list
SESSION_ID | TYPE | STATUS | NODE
acp:discord:channel:123:thread:456 | thread | active | mac-mini.local
acp:discord:channel:123 | channel | idle | (unbound)
External harness test failure:
$ env OPENCLAW_HIDE_BANNER=1 OPENCLAW_SUPPRESS_NOTES=1 \
openclaw acp --url ws://127.0.0.1:18789 --token-file ~/.openclaw/gateway.token \
turn --session acp:discord:channel:123:thread:456
{"error":"ACP_TURN_FAILED","detail":"acpx exited with code 1"}
π§ Root Cause
Architecture Gap: ACP Gateway Proxy vs. True Remote Harness Execution
The root cause is a fundamental misunderstanding of the ACP architecture’s current capabilities versus the expected remote harness execution model.
ACP Architecture Layers
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β Discord Channel β β /acp spawn … (entry point) β ββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ β βΌ βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β Gateway (Raspberry Pi) β β βββββββββββββββββββ βββββββββββββββββββββββββββββββββββ β β β ACP Protocol β β acpx Backend Plugin β β β β Handler β β (spawns external processes) β β β βββββββββββββββββββ βββββββββββββββββββββββββββββββββββ β β β β β β β βΌ β β β βββββββββββββββββββββββββ β β β β acpx child process β β β β β (LOCAL to gateway) β β β β βββββββββββββββββββββββββ β β β β β βββββββββββΌββββββββββββββββββββββββββΌββββββββββββββββββββββββββ β β βΌ βΌ Local Agent Remote Node Connection Runtime (NOT harness execution)
The Core Misconception
The acpx backend spawns harness processes locally on the Gateway host. When configured in ~/.acpx/config.json:
json { “agents”: { “openclaw”: { “command”: “env OPENCLAW_HIDE_BANNER=1 … openclaw acp –url ws://127.0.0.1:18789 …” } } }
This configuration tells the local acpx plugin how to reach the gateway, but it does not instruct the Gateway to delegate harness execution to a remote node.
Failure Sequence
- User invokes
/acp spawn claude --thread herein Discord - Gateway receives ACP session spawn request
- Gateway’s
acpxplugin attempts to spawn harness using the configured command - If
openclaw acpbinary is not installed on the Gateway host, execution fails with exit code 1 - Even if installed, if the
--token-fileor--urlpoints to localhost, the harness may fail to authenticate
Session-Mode Conflict Root Cause
When channels.discord.bindings contains:
yaml channels: discord: bindings: - channel: “123456789” agent: “athos”
This creates a static route that overrides ACP session initialization. The ACP session spawn attempts to use the existing session context, which lacks ACP protocol support, resulting in:
Session is not ACP-enabled: agent:athos:discord:channel:123456789
The two binding systems (static agent routing and dynamic ACP session binding) are mutually exclusive at the channel level in the current implementation.
π οΈ Step-by-Step Fix
Phase 1: Remove Static Channel Bindings Conflict
Before (~/.openclaw/config.yaml):
yaml
channels:
discord:
bindings:
- channel: “123456789”
agent: “athos”
threadBindings:
enabled: true
spawnAcpSessions: true
After: yaml channels: discord: # Remove static bindings for ACP-enabled channels # bindings: [] # commented out or removed threadBindings: enabled: true spawnAcpSessions: true # ACP sessions will now own the channel dynamically
Phase 2: Ensure Gateway Has ACP Harness Binary
The Gateway host (Raspberry Pi) must have openclaw acp available for the acpx plugin to spawn harness processes:
# On the Raspberry Pi (Gateway host)
$ which openclaw
/usr/local/bin/openclaw
$ openclaw version
OpenClaw v0.9.3
# If not installed, install with:
$ curl -sSL https://get.openclaw.dev | sh
Phase 3: Configure acpx Plugin for Gateway-Local Execution
On the Gateway host (/etc/openclaw/acpx.json or ~/.openclaw/acpx.json):
json { “agents”: { “claude”: { “command”: “claude”, “args”: ["–no-animation", “–output-format”, “stream-json”], “env”: { “CLAUDE_API_KEY”: “${CLAUDE_API_KEY}” } }, “codex”: { “command”: “codex”, “args”: ["–output", “json-stream”], “env”: { “OPENAI_API_KEY”: “${OPENAI_API_KEY}” } } }, “harnessTimeout”: 300, “spawnStrategy”: “per-turn” }
Phase 4: For True Remote Harness Execution (Workaround)
If the intent is to run harnesses on a remote Mac mini, a relay proxy pattern is required:
On the Mac mini - Create a relay agent configuration:
json
// /acpx-relay/config.json
{
“mode”: “relay”,
“upstream”: {
“url”: “wss://your-pi-domain.local:18789”,
“tokenFile”: “/.openclaw/gateway.token”
},
“agents”: {
“openclaw”: {
“command”: “env OPENCLAW_HIDE_BANNER=1 OPENCLAW_SUPPRESS_NOTES=1 openclaw acp –url ws://127.0.0.1:18789 –token-file ~/.openclaw/gateway.token”
}
}
}
Start the relay daemon on Mac mini:
$ acpx-relay --config ~/acpx-relay/config.json --daemon
[acpx-relay] INFO Starting relay daemon
[acpx-relay] INFO Connected to gateway: wss://your-pi-domain.local:18789
[acpx-relay] INFO Registered as node: mac-mini.local
Phase 5: Configure Gateway to Route to Remote Node
On the Gateway (~/.openclaw/config.yaml):
yaml acp: nodeRegistry: mac-mini: host: mac-mini.local port: 18790 capability: “harness-execution”
routing: defaultHarnessNode: “mac-mini”
agentMapping: claude: node: “mac-mini” agentType: “openclaw”
Phase 6: Restart Gateway and Test
# Restart gateway on Raspberry Pi
$ sudo systemctl restart openclaw-gateway
# Verify connection
$ openclaw nodes list
NODE | STATUS | CAPABILITIES
mac-mini.local| online | harness-execution
# Test ACP session creation
$ openclaw acp session create --type discord --channel 123456789 --thread 987654321 --harness-node mac-mini
{"sessionId":"acp:discord:channel:123:thread:456","status":"active","node":"mac-mini.local"}
π§ͺ Verification
Test 1: Gateway Local Harness (Baseline)
Verify harness execution works on the Gateway itself:
# On Raspberry Pi (Gateway host)
$ openclaw acp harness test --agent claude --input "Say hello"
[acp.harness] INFO Spawning: claude
[acp.harness] DEBUG Process started: pid=12345
[acp.harness] INFO Received: "Hello! How can I assist you today?"
[acp.harness] DEBUG Process exited: code=0
{"status":"success","output":"Hello! How can I assist you today?"}
Test 2: Remote Node Relay Connection
Verify the Mac mini relay connects to the Gateway:
# On Mac mini
$ acpx-relay --config ~/acpx-relay/config.json --foreground
[acpx-relay] INFO Bridge version: acpx-relay/0.9.3
[acpx-relay] INFO Connecting to wss://your-pi-domain.local:18789
[acpx-relay] INFO Authentication: token verified
[acpx-relay] INFO Node registration: mac-mini.local
[acpx-relay] INFO Ready - awaiting turn requests
On Gateway:
$ openclaw nodes list --verbose
NODE | STATUS | CAPABILITIES | CONNECTED SINCE
mac-mini.local | online | harness-execution | 2024-01-15T10:30:00Z
Test 3: Discord ACP Session with Remote Harness
Invoke via Discord slash command:
/acp spawn claude --thread here --label test-session
Expected Gateway logs:
[gateway] INFO acp.spawn: session=acp:discord:channel:123:thread:456 harness=claude node=mac-mini.local
[gateway] DEBUG acp.route: Delegating to node mac-mini.local
[gateway] DEBUG acp.turn: Turn queued, awaiting node response
[gateway] DEBUG node.mac-mini: Received turn request: session=acp:discord:channel:123:thread:456
[gateway] DEBUG node.mac-mini: Turn response received: status=success
[gateway] INFO acp.turn: completed session=acp:discord:channel:123:thread:456 duration=2340ms
Test 4: Verify Turn Execution Not on Gateway
Confirm harness process runs on Mac mini, not Raspberry Pi:
# On Mac mini
$ ps aux | grep -E "(claude|codex|openclaw)" | grep -v grep
user 12345 ... /usr/local/bin/claude --no-animation --output-format stream-json
# On Raspberry Pi - should show no harness processes
$ ps aux | grep -E "(claude|codex)" | grep -v grep
# (empty - no local harness execution)
Test 5: Discord Thread Response
After successful ACP spawn, send a message in the thread:
@claude-studio Write a hello world in Python
Expected behavior:
- Thread receives response from Claude Code running on Mac mini
- Gateway logs show delegation to
mac-mini.local - Exit code should be 0, not 1
β οΈ Common Pitfalls
Pitfall 1: Token File Permissions
The acpx plugin spawns processes that must read the gateway token file.
# WRONG - token file has restrictive permissions
$ ls -la ~/.openclaw/gateway.token
-rw------- 1 root gatewayuser 0 Jan 15 10:00 gateway.token # only gatewayuser can read
# FIX - ensure acpx spawn user can read
$ chmod 644 ~/.openclaw/gateway.token
$ ls -la ~/.openclaw/gateway.token
-rw-r--r-- 1 gatewayuser 0 Jan 15 10:00 gateway.token
Pitfall 2: Network Hostname Resolution
Remote node registration fails if the Gateway cannot resolve the node hostname:
# On Gateway - test hostname resolution
$ nslookup mac-mini.local
# If fails, either:
# 1. Add entry to /etc/hosts:
$ echo "192.168.1.100 mac-mini.local" | sudo tee -a /etc/hosts
# 2. Or use IP address directly in config:
# nodeRegistry:
# mac-mini:
# host: "192.168.1.100"
Pitfall 3: ACL Configuration for Remote Nodes
The Gateway’s ACL must permit connections from remote nodes:
yaml
~/.openclaw/config.yaml
acp: security: nodeAcl: - pattern: “mac-mini.local” permissions: [“harness-execution”, “session-read”, “session-write”]
Pitfall 4: macOS LaunchAgent vs. Manual Process
On Mac mini, acpx-relay may terminate when the terminal closes:
# WRONG - process dies when terminal closes
$ acpx-relay --config ~/acpx-relay/config.json &
# CORRECT - use launchd plist for persistence
$ cat ~/Library/LaunchAgents/dev.openclaw.acpx-relay.plist
Label dev.openclaw.acpx-relay
ProgramArguments
/usr/local/bin/acpx-relay
--config
/Users/username/acpx-relay/config.json
RunAtLoad
KeepAlive
$ launchctl load ~/Library/LaunchAgents/dev.openclaw.acpx-relay.plist
Pitfall 5: Discord Thread Binding Race Condition
When spawnAcpSessions=true, thread creation and session binding can race:
yaml
Add delay for reliable binding
channels: discord: threadBindings: enabled: true spawnAcpSessions: true sessionInitDelayMs: 500 # Wait 500ms before ACP init
Pitfall 6: Environment Variable Inheritance
External harnesses may not inherit Gateway environment variables:
# In acpx.json - explicitly pass required variables
{
"agents": {
"claude": {
"command": "env",
"args": [
"CLAUDE_API_KEY=${CLAUDE_API_KEY}",
"OPENCLAW_SESSION=${OPENCLAW_SESSION}",
"claude"
]
}
}
}
Pitfall 7: ARM32 vs ARM64 on Raspberry Pi
OpenClaw binaries for Pi must match the architecture:
# Check architecture
$ uname -m
armv7l # Raspberry Pi 3/4 32-bit
# vs
aarch64 # Raspberry Pi 3/4 64-bit or Pi 5
# Ensure correct binary is installed
$ openclaw version
OpenClaw v0.9.3 armv7l-unknown-linux-gnueabihf
# If wrong architecture, reinstall:
$ curl -sSL https://get.openclaw.dev | sh -s -- --arch armv7l
π Related Errors
ACP_TURN_FAILED: acpx exited with code 1β Primary error. External harness process terminated abnormally. Check Gateway logs for spawn failure details, token file permissions, and binary availability.ACP_SESSION_INIT_FAILED: Session is not ACP-enabledβ Static channel binding conflicts with dynamic ACP session. Remove static bindings from `channels.discord.bindings` for ACP-enabled channels.ACP_NODE_UNREACHABLEβ Remote node relay not connected. Verify `acpx-relay` is running on the remote host and network connectivity exists.ACP_AUTH_FAILEDβ Token validation failed. Ensure `gateway.token` is readable by the acpx spawn process and matches the Gateway's expected token.ACP_AGENT_NOT_FOUNDβ Requested harness agent not configured in `acpx.json`. Verify the agent name matches an entry in `agents` configuration.ACP_TIMEOUT: Turn exceeded 300000msβ Harness execution exceeded timeout. Increase `harnessTimeout` in `acpx.json` or debug the harness process directly.OPENCLAW_BINARY_NOT_FOUNDβ The `openclaw` binary is not in the Gateway's PATH. Install OpenClaw on the Gateway host or specify full path in configuration.CHANNEL_ROUTE_CONFLICTβ Both static and dynamic routing attempted on same channel. See session-mode conflict resolution in Phase 1.
Historical Reference Issues
- Issue #847 β "acpx plugin spawns harness on wrong host" β Original bug report establishing the node routing gap in ACP architecture.
- Issue #892 β "Discord threadBindings fails with ACP-enabled channels" β Root cause: static vs. dynamic binding collision.
- Issue #923 β "Remote node relay pattern undocumented" β Feature gap in documentation for multi-node ACP deployments.
- PR #956 β "Add node capability filtering to acp.route" β Implemented selective node routing based on capability declarations.