Client Commands

Client sends commands to ARC which cause an action in the component. They consist of a $type field identifying the command by name and optionally parameter fields specific to the command. The first field must be $type.

To send a command, call the method SendCommand(string) on GameObject ClientAPI, putting the json as the string parameter.

API

Playback

Type Parameters Example

QueueSpeech

provider - Name of provider (Microsoft, A2X, ElevenLabs).

hash - Base64 hash of the speech item this packet is a part of.

isFirst, isLast - True when the packet is first or last in a speech.

sampleRate - Number of audio samples per second (e.g. 16000 for 16kHz).

bytes - Array of PCM bytes encoded in a base64 string.

frameRate - Frames per second of the blendshape animation data provided if present, otherwise 0.

blendShapeWeights - Two-dimensional array of blendshape values. The shape is [frameCount][blendshapesPerFrame] and each value is 0-255.

visemes - Array of timed visemes. Each is an object with an id (index inside the provider’s viseme list) and offset (time offset form the start of the speech in milliseconds).

 
{
  "$type": "QueueSpeech",
  "provider": "ElevenLabs",
  "hash": "urbnk3490mndfsko32987",
  "isFirst": true,
  "isLast": false,
  "frameRate": 0,
  "bytes": "",
  "blendShapeWeights": [],
  "visemeType": "default",
  "visemes": []
}

SetTranscript

text - Contents of ASR.

isInterim - True if transcript is interim.

 
{
  "$type": "SetTranscript",
  "text": "recognized speech",
  "isInterim": false
}

SetState

emotions - Map of emotions and their normalized weights.

reflectiveStates - Map of reflective states and their normalized weights.

signals - Names of signals to send (UNUSED).

 
{
  "emotions" : {
    "awkwardness" : 0.3,
    "concentration" : 0.4
  },
  "rStates" : {
    "hostility" : 0.2,
    "dominance" : 0.5
  },
  "signals" : [ "speechStarted", "surprise" ]
}

ControlPlayback

mode - One of Pause, Resume, Stop.

 
{
  "$type": "ControlPlayback",
  "mode": "Pause"
}

CompleteTurn

-

 
{
  "$type": "CompleteTurn"
}

Settings

Type Parameters Example

SetVisual

avatar - Avatar identifier.

environment – Environment identifier.

camera (optional):

position [x,y,z] (meters)

rotation [pitch,yaw,roll] (deg)

FOV (deg)

sensorShift [x,y] (normalized)

viewport [x,y,w,h] (normalized)

 
{
  "$type": "SetVisual",
  "avatar": "eli",
  "environment": "nycwall",
  "camera": {
    "position": [0, 1.47, -1.5],
    "rotation": [0, 0, 0],
    "FOV": 35,
    "sensorShift": [0, 0],
    "viewport": [0, 0, 1, 1]
  }
}

SetOrb

shaderProperties - map of property names and values.

Optionally also:

smoothTime - transition time in seconds, set for any properties changed in the same command.

visible - true or false to show or hide the orb.

 
{
	"$type": "SetOrb",
    "smoothTime": 1.2,
	"shaderProperties": {
		"_BackgroundBrightness": 0.8,
		"_C3_Color1": [1.0, 0.5, 0.2]
	}
}

SetConfig

One or more from:

quality - Name of one of the available graphics tiers returned from GetInfo.

resolutionScale - Scale for internal rendering resolution in range \$\[0.1, 2]\$ (1.0 = native resolution).

bundleUrl - Base URL for loading asset bundles.

mediaUrl - Base URL for loading media files.

branch - Branch of assets to download.

hdr.enabled - Enable/disable HDR rendering.

hdr.paperWhite - Paper‑white luminance in nits.

hdr.exposure - Exposure compensation in EV.

hdr.contrast - Additional contrast adjustment (0 = neutral).

frameRate - Target frame rate (-1 → Unity default (30 on mobile devices)).

audioBufferMs - Audio streamer buffer size in ms.

 
{
  "$type": "SetConfig",
  "quality": "High",
  "resolutionScale": 0.75,
  "bundleUrl": "https://example.com/assets",
  "mediaUrl": "https://example.com/media",
  "branch": "prod",
  "hdr": {
    "enabled": true,
    "paperWhite": 300,
    "exposure": 0.5,
    "contrast": 10
  },
  "frameRate" : 60,
  "audioBufferMs" : 300
}

Developer commands

Type Parameters Example

GetInfo

info - One of Version, Performance, Quality.

 
{
  "$type": "GetInfo",
  "info": "Version"
}

SetLogging

One or more from:

severity - Minimal severity to always log.

categories - Array of categories to log.

Automatically switches logging to use client events instead of console (more).

 
{
  "$type": "SetLogging",
  "severity": "Warning",
  "categories": [
    "AudioStreamer",
    "ClientAPI"
  ]
}

SetDebug

statistics - true or false to enable or disable statistics.

 
{
  "$type": "SetDebug",
  "statistics": false
}

ClearCache

-

 
{
  "$type": "ClearCache"
}

Behavior