5.2.2 Screen Control

The screen control interface provides full display control capabilities, including emoji playback, video playback, and more. It enables developers to implement a wide range of visual interaction features for the robot.

Key Features

• Playback of multiple preset emoji expressions

• Custom video playback

• Playlist and loop playback

• Playback priority control

Emoji Playback Status Topic

Topic Name	Data Type	Description	QoS	Frequency
/face_ui_proxy/status	FaceEmojiStatus	Emoji playback status	BEST_EFFORT+TRANSIENT_LOCAL	Event-driven + 1 Hz overlaid during playback (see below)

Frequency: This topic has two overlapping push modes: ① pushed immediately on state transitions (event-driven, including playback item switches); ② during emoji and video playback, running(2) progress is additionally pushed at 1 Hz. Both modes are active simultaneously and are not mutually exclusive.

• FaceEmojiStatus ros2-msg @ face_ui/FaceEmojiStatus.msg

  # Emoji Status Info
  
  MessageHeader header             # Message header
  string e_path                    # Path of the emoji file
  string[] e_path_list             # List of emoji paths in the current sequence
  uint8 e_id                       # Emoji ID
  uint8 mode                       # Playback mode (1 (EMOTION_MODE_ONCE once), 2 (EMOTION_MODE_LOOP loop))
  int32 priority                   # Priority
  uint8 status                     # Current status (0 (STATUS_IDLE idle), 1 (STATUS_START start), 2 (STATUS_RUNNING running), 3 (STATUS_FINISHED finished), 4 (STATUS_STOPPED stopped))
  float64 time_to_end_ms           # Remaining time of the current item (unit: ms)
  uint8 display_type               # Active display layer (0 (idle), 1 (video), 2 (svga), 3 (rgb), 4 (image), 5 (qr), 6 (text))
  string trace_id                  # RPC call ID that triggered this status change
  int32 error_code                 # Not enabled
  

Note

Field Usage Notes:

• status: The current firmware pushes only running(2) during emoji/video playback and transitions; stopped(4) is not triggered in normal use; 0-idle, 1-start, and 3-completed are not yet used. Subscribers should handle all values for forward compatibility.

• time_to_end_ms: Remaining time of the current item (duration - position). The .msg comment states “if looping, the remaining loop time is included”, but the actual implementation only returns the remaining time of the current item, not including subsequent loop times.

• display_type: Normal SDK usage only sees video(1) (triggered by PlayEmoji/PlayVideo), others are triggered through non-SDK services.

Emoji Playback Service

Service Name	Data Type	Description
/aimdk_5Fmsgs/srv/PlayEmoji	PlayEmoji	Play emoji
/aimdk_5Fmsgs/srv/PlayEmojiGroup	PlayEmojiGroup	Play emoji group list

• PlayEmoji ros2-srv @ face_ui/srv/PlayEmoji.srv

  # Play Emoji
  # Service: /aimdk_5Fmsgs/srv/PlayEmoji
  
  # Request
  CommonRequest header  # Request header
  
  uint8 emotion_id  # Emoji ID
  uint8 mode  # Playback mode enum (1 (EMOTION_MODE_ONCE once), 2 (EMOTION_MODE_LOOP loop))
  int32 priority  # Playback priority
  
  ---
  
  # Response
  CommonResponse header  # Response header (not enabled; see note below)
  
  bool success  # Whether the command succeeded
  string message
  

Note

Response field notes: The header field is not enabled; header.status.value is always UNKNOWN(0), and header.code is always 0. Read the outer success and message fields directly and ignore header.

emotion_id Emoji Mapping Table:

Emoji ID	Emoji name	Description
1	Blink	Basic blink action
10	Calm - eye variation 1	Eye variation for calm state
11	Calm - eye variation 2	Eye variation for calm state
20	Calm - game	Game-state emoji
30-33	Calm - cute	Cute expression series
40	Close eyes	Close-eye action
50	Open eyes	Open-eye action
60	Bored	Bored expression
70	Abnormal	Abnormal state
80	Sleeping	Sleeping state
90	Happy	Happy expression
100-101	Extra happy / ecstatic	Extremely happy expression
110	Sad	Sad expression
120	Sympathy	Sympathetic expression
130	Confused	Confused expression
140	Shocked	Shocked expression
150	Acting cute	Cute/affectionate expression
160	Serious	Serious expression
170	Thinking	Thinking expression
180	Angry	Angry expression
190	Extra angry	Extremely angry expression
200	Adoration	Adoring expression
210	Extra adoring	Extremely adoring expression
220	Charging	Charging state

priority Priority Mechanism Explanation:

• This priority mechanism applies to all emoji and video playback interfaces in this section.

• If the new request’s priority is not lower than the current one, it overrides the current request.

• If the new request’s priority is lower, it is ignored.

• A preempted playback item does not push a stopped(4) status; check changes in e_path/e_id/trace_id to detect preemption.

• System modules (e.g. task_manager) trigger expression/video playback with layered priorities by fault severity (first-level over-temperature = 8; severe faults such as damping-fall / upper-limb-disable = 10). A request with a priority no lower than the currently effective value overrides it; to ensure preemption by system modules is avoided, use a priority value greater than 10.

Attention

Avoiding display control conflicts:

During runtime, the task_manager module automatically invokes the expression/video playback interfaces on boot, fault diagnosis, site tours, and other scenarios, and the interaction_slave module also invokes the expression playback interface via an internal channel on charging state changes; both may override the user’s playback settings. There are two ways to avoid being preempted:

• Set a high priority: call the playback interfaces with a priority value greater than 10 (system modules use up to 10 for fault scenarios) to block system-module auto-control. This needs no module shutdown, but every call must specify it, and it overrides the fault-expression alerts triggered by system modules (e.g. fall protection, over-temperature warnings), so the user can no longer perceive those faults via the screen.

• Stop system modules: run the following command on the Motion Control Computing Unit (PC1, 10.0.1.40) to stop task_manager (interaction_slave is similar, located on the Interaction Computing Unit (PC3, 10.0.1.42)), but this also affects navigation, site tours, and other functions:

aima em stop-app task_manager

Even with the above measures taken, the /face_ui_proxy/status topic may still push state changes not initiated by the user (from other system modules or App remote control). Use the trace_id field to distinguish the source of a state change.

• PlayEmojiGroup ros2-srv @ face_ui/srv/PlayEmojiGroup.srv

  # Play emoji group
  # Service: /aimdk_5Fmsgs/srv/PlayEmojiGroup
  
  # Request
  CommonRequest header             # Request header
  uint8[] emotion_ids              # Emoji ID list
  uint8 mode                       # Playback mode enum (1 (EMOTION_MODE_ONCE once), 2 (EMOTION_MODE_LOOP loop))
  int32 priority                   # Playback priority
  
  ---
  
  # Response
  CommonResponse header            # Response header (not enabled; see PlayEmoji)
  bool success                     # Whether playback succeeded
  string message                   # Response message
  

  See the emoji reference table for the emoji id list, and above for the priority mechanism.

Video Playback Service

Before using this feature, read the notes on task_manager automatic control in the Priority Mechanism to avoid display control conflicts.

Service Name	Data Type	Description
/aimdk_5Fmsgs/srv/PlayVideo	PlayVideo	Play video
/aimdk_5Fmsgs/srv/PlayVideoGroup	PlayVideoGroup	Play a list of videos

• PlayVideo ros2-srv @ face_ui/srv/PlayVideo.srv

  # Play Video
  # Service: /aimdk_5Fmsgs/srv/PlayVideo
  
  # Request
  CommonRequest header             # Request header
  string video_path                # Absolute path of video file (must be on the interaction compute unit and readable by all)
  uint8 mode                       # Playback mode (1 (EMOTION_MODE_ONCE once), 2 (EMOTION_MODE_LOOP loop))
  int32 priority                   # Playback priority
  
  ---
  
  # Response
  CommonResponse header            # Response header (not enabled; see PlayEmoji)
  bool success                     # Whether playback succeeded
  string message                   # Response message
  

  Notes:

  • By default, video playback does not include audio.

  • Audio and video files must use absolute paths.

  • Audio and video files must be stored on the interaction compute unit (PC3, 10.0.1.42), not the development compute unit (PC2).

  • Audio and video files (and all parent directories up to root) must be readable by all users(new subdirectory under /var/tmp/ is recommended)

  • See above for the priority mechanism.

• PlayVideoGroup ros2-srv @ face_ui/srv/PlayVideoGroup.srv

  # Play Video Group
  # Service: /aimdk_5Fmsgs/srv/PlayVideoGroup
  
  # Request
  CommonRequest header             # Request header
  string[] video_path_list         # Absolute path list of video files (must be on the interaction compute unit and readable by all)
  uint8 mode                       # Playback mode (1 (EMOTION_MODE_ONCE once), 2 (EMOTION_MODE_LOOP loop))
  int32 priority                   # Playback priority
  
  ---
  
  # Response
  CommonResponse header            # Response header (not enabled; see PlayEmoji)
  bool success                     # Whether playback succeeded
  string message                   # Response message
  

  Notes:

  • See the PlayVideo notes.

Programming Examples

For detailed code examples and explanations, refer to:

• C++ Examples:

  • Emoji Control

  • Play Video

• Python Examples:

  • Emoji Control

  • Play Video

Safety Notes

Warning

Display Control Limitations

• Emoji/video playback involves display resources; mind priority conflicts (see Priority Mechanism).

• Video playback requires correct and accessible file paths.

Caution

As standard ROS DO NOT handle cross-host service (request-response) well, please refer to SDK examples to use open interfaces in a robust way (with protection mechanisms e.g. exception safety and retransmission)

While the robot is in Stable Standing Mode or Locomotion Mode, DO NOT launch ROS nodes in rapid bulk (no more than 2 nodes per second is recommended), as a large number of nodes joining DDS discovery within a short period causes communication congestion and degrades motion control real-time performance, which may cause the robot to lose balance and fall

Note

Best Practices

• Use appropriate playback modes to avoid unnecessary looping.

• Subscribe to the /face_ui_proxy/status topic to monitor playback state and re-apply settings after a preemption.

• Mind the video file path and permission requirements.