GDK Interaction Interface Documentation (C++)¶

Overview¶

The interaction module provides voice interaction, display control, audio playback, and other functions for the G02 robot. Through the C++ interface, developers can easily implement voice control, TTS playback, audio/video playback, display control, and other functions for the robot, suitable for various scenarios such as voice interaction, multimedia presentation, and human-robot interaction.

Notes¶

The interaction interface relies on the cross-subnet communication module. Therefore, if you use this type of interface in an isolated environment (such as a docker container), you need to ensure the container can modify the host machine's network configuration. (Add the --privileged parameter when starting the container.)

Interface Description¶

Interaction Class¶

This class encapsulates the main functional interfaces for robot interaction.

1. `SetLanguage()`¶

Function: Set the voice language
Parameters:

Parameter	Type	Description
`language`	`const Language&`	Language type. `Language::kLanguageChinese` indicates Chinese, `Language::kLanguageEnglish` indicates English

Return value: GDKRes, the operation result status code. Returns GDKRes::kSuccess on success

Language enum values:

Enum value	Description
`Language::kLanguageChinese`	Chinese
`Language::kLanguageEnglish`	English

enum class Language {
  kLanguageChinese = 0,
  kLanguageEnglish = 1,
};

Example:

#include "gdk/gdk.h"
#include <iostream>
#include <chrono>
#include <thread>

int main()
{
    // Initialize the GDK system
    if (agibot::gdk::GDKInit() != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "GDK initialization failed" << std::endl;
        return -1;
    }
    std::cout << "GDK initialized successfully" << std::endl;

    agibot::gdk::Interaction interaction;
    std::cout << "Interaction init" << std::endl;
    std::this_thread::sleep_for(std::chrono::seconds(1));

    // Set the language to Chinese
    if (interaction.SetLanguage(agibot::gdk::Language::kLanguageChinese) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to set language" << std::endl;
    } else {
        std::cout << "Language set successfully" << std::endl;
    }

    // Release GDK system resources
    if (agibot::gdk::GDKRelease() != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "GDK release failed" << std::endl;
        return -1;
    }
    std::cout << "GDK released successfully" << std::endl;

    return 0;
}

2. `SetCallMode()`¶

Function: Set call mode
Parameters: | Parameter | Type | Description | | :--- | :--- | :--- | | is_on | const bool& | true enables call mode, false disables call mode |
Return value: GDKRes, the operation result status code. Returns GDKRes::kSuccess on success
Notes:
After enabling call mode, the device automatically enters the wakeup state; after disabling call mode, it automatically exits the wakeup state — no wakeup word or end word is needed
In call mode, you can capture the user's voice input and retrieve the corresponding recognized text through the interface
In call mode, you can call GetAsrText() or register a callback via RegisterCallback() to obtain the user's recognized voice text
Example:

#include "gdk/gdk.h"
#include <iostream>
#include <chrono>
#include <thread>

int main()
{
    if (agibot::gdk::GDKInit() != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "GDK initialization failed" << std::endl;
        return -1;
    }
    std::cout << "GDK initialized successfully" << std::endl;

    agibot::gdk::Interaction interaction;
    std::this_thread::sleep_for(std::chrono::seconds(1));

    // Enable call mode
    bool is_on = true;
    if (interaction.SetCallMode(is_on) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to enable call mode" << std::endl;
    } else {
        std::cout << "Call mode enabled successfully" << std::endl;
    }

    is_on = false;
    if (interaction.SetCallMode(is_on) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to disable call mode" << std::endl;
    } else {
        std::cout << "Call mode disabled successfully" << std::endl;
    }

    agibot::gdk::GDKRelease();
    return 0;
}

3. `SetVolume()`¶

Function: Set volume
Parameters:

Parameter	Type	Description
`volume`	`const int32_t&`	Volume value, typically in the range 0-500

Return value: GDKRes, the operation result status code. Returns GDKRes::kSuccess on success
Example:

#include "gdk/gdk.h"
#include <iostream>
#include <chrono>
#include <thread>

int main()
{
    if (agibot::gdk::GDKInit() != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "GDK initialization failed" << std::endl;
        return -1;
    }
    std::cout << "GDK initialized successfully" << std::endl;

    agibot::gdk::Interaction interaction;
    std::this_thread::sleep_for(std::chrono::seconds(1));

    // Set volume to 50
    int32_t volume = 50;
    if (interaction.SetVolume(volume) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to set volume" << std::endl;
    } else {
        std::cout << "Volume set successfully" << std::endl;
    }

    agibot::gdk::GDKRelease();
    return 0;
}

4. `SetWakeupSwitch()`¶

Function: Set the wakeup switch
Parameters:

Parameter	Type	Description
`is_on`	`const bool&`	`true` enables wakeup, `false` disables wakeup

Return value: GDKRes, the operation result status code. Returns GDKRes::kSuccess on success
Example:

#include "gdk/gdk.h"
#include <iostream>
#include <chrono>
#include <thread>

int main()
{
    if (agibot::gdk::GDKInit() != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "GDK initialization failed" << std::endl;
        return -1;
    }
    std::cout << "GDK initialized successfully" << std::endl;

    agibot::gdk::Interaction interaction;
    std::this_thread::sleep_for(std::chrono::seconds(1));

    // Enable the wakeup function
    bool is_on = true;
    if (interaction.SetWakeupSwitch(is_on) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to set wakeup switch" << std::endl;
    } else {
        std::cout << "Wakeup function enabled successfully" << std::endl;
    }

    agibot::gdk::GDKRelease();
    return 0;
}

5. `SetAudioSwitch()`¶

Function: Set the audio switch
Parameters:

Parameter	Type	Description
`is_on`	`const bool&`	`true` enables audio, `false` disables audio

Return value: GDKRes, the operation result status code. Returns GDKRes::kSuccess on success
Example:

#include "gdk/gdk.h"
#include <iostream>
#include <chrono>
#include <thread>

int main()
{
    if (agibot::gdk::GDKInit() != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "GDK initialization failed" << std::endl;
        return -1;
    }
    std::cout << "GDK initialized successfully" << std::endl;

    agibot::gdk::Interaction interaction;
    std::this_thread::sleep_for(std::chrono::seconds(1));

    // Enable the audio function
    bool is_on = true;
    if (interaction.SetAudioSwitch(is_on) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to set audio switch" << std::endl;
    } else {
        std::cout << "Audio function enabled successfully" << std::endl;
    }

    agibot::gdk::GDKRelease();
    return 0;
}

6. `SetDisplaySwitch()`¶

Function: Set the display switch
Parameters:

Parameter	Type	Description
`is_on`	`const bool&`	`true` enables display, `false` disables display

Return value: GDKRes, the operation result status code. Returns GDKRes::kSuccess on success
Example:

#include "gdk/gdk.h"
#include <iostream>
#include <chrono>
#include <thread>

int main()
{
    if (agibot::gdk::GDKInit() != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "GDK initialization failed" << std::endl;
        return -1;
    }
    std::cout << "GDK initialized successfully" << std::endl;

    agibot::gdk::Interaction interaction;
    std::this_thread::sleep_for(std::chrono::seconds(1));

    // Enable the display function
    bool is_on = true;
    if (interaction.SetDisplaySwitch(is_on) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to set display switch" << std::endl;
    } else {
        std::cout << "Display function enabled successfully" << std::endl;
    }

    agibot::gdk::GDKRelease();
    return 0;
}

7. `PlayTts()`¶

Function: Play TTS (text-to-speech)
Parameters:

Parameter	Type	Description
`text`	`const std::string&`	The text content to play

Return value: GDKRes, the operation result status code. Returns GDKRes::kSuccess on success
Example:

#include "gdk/gdk.h"
#include <iostream>
#include <chrono>
#include <thread>

int main()
{
    if (agibot::gdk::GDKInit() != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "GDK initialization failed" << std::endl;
        return -1;
    }
    std::cout << "GDK initialized successfully" << std::endl;

    agibot::gdk::Interaction interaction;
    std::this_thread::sleep_for(std::chrono::seconds(1));

    // Play TTS
    std::string text = "Hello, I am Genie G2";
    if (interaction.PlayTts(text) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to play TTS" << std::endl;
    } else {
        std::cout << "TTS played successfully" << std::endl;
        std::this_thread::sleep_for(std::chrono::seconds(3));  // Wait for playback to complete
    }

    agibot::gdk::GDKRelease();
    return 0;
}

8. `PlayAudio()`¶

Function: Play an audio file
Parameters:

Parameter	Type	Description
`audio_path`	`const std::string&`	Audio file path

Return value: GDKRes, the operation result status code. Returns GDKRes::kSuccess on success
Example:

#include "gdk/gdk.h"
#include <iostream>
#include <chrono>
#include <thread>

int main()
{
    if (agibot::gdk::GDKInit() != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "GDK initialization failed" << std::endl;
        return -1;
    }
    std::cout << "GDK initialized successfully" << std::endl;

    agibot::gdk::Interaction interaction;
    std::this_thread::sleep_for(std::chrono::seconds(1));

    // Play an audio file
    std::string audio_path = "/path/to/audio.wav";
    if (interaction.PlayAudio(audio_path) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to play audio" << std::endl;
    } else {
        std::cout << "Audio played successfully" << std::endl;
        std::this_thread::sleep_for(std::chrono::seconds(5));  // Wait for playback to complete
    }

    agibot::gdk::GDKRelease();
    return 0;
}

9. `PlayVideo()`¶

Function: Play a video file
Parameters:

Parameter	Type	Description
`video_path`	`const std::string&`	Video file path
`loop_count`	`const int32_t&`	Number of loop plays; -1 means infinite loop

Return value: GDKRes, the operation result status code. Returns GDKRes::kSuccess on success
Example:

#include "gdk/gdk.h"
#include <iostream>
#include <chrono>
#include <thread>

int main()
{
    if (agibot::gdk::GDKInit() != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "GDK initialization failed" << std::endl;
        return -1;
    }
    std::cout << "GDK initialized successfully" << std::endl;

    agibot::gdk::Interaction interaction;
    std::this_thread::sleep_for(std::chrono::seconds(1));

    // Play a video file, looping once
    std::string video_path = "/path/to/video.mp4";
    int32_t loop_count = 1;
    if (interaction.PlayVideo(video_path, loop_count) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to play video" << std::endl;
    } else {
        std::cout << "Video played successfully" << std::endl;
        std::this_thread::sleep_for(std::chrono::seconds(10));  // Wait for playback to complete
    }

    // Loop playback indefinitely
    loop_count = -1;
    if (interaction.PlayVideo(video_path, loop_count) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to play video" << std::endl;
    } else {
        std::cout << "Video started looping" << std::endl;
    }

    agibot::gdk::GDKRelease();
    return 0;
}

10. `GetFuncStatus()`¶

Function: Get the voice function status
Parameters:

Parameter	Type	Description
`func_status`	`VoiceFuncStatus&`	Output parameter, the voice function status object

Return value: GDKRes, the operation result status code. Returns GDKRes::kSuccess on success; the func_status parameter contains the function status information

VoiceFuncStatus Object Details¶

The VoiceFuncStatus struct contains the following members:

Member	Type	Description	Unit
`func_status`	`uint32_t`	Function status	None
`wakeup_status`	`uint32_t`	Wakeup status	None
`requester`	`std::string`	Requester	String
`wakeup_enabled`	`bool`	Whether the wakeup function is enabled	Boolean
`display_enabled`	`bool`	Whether the display function is enabled	Boolean
`audio_enabled`	`bool`	Whether the audio function is enabled	Boolean
`en_settings`	`VoiceSettings`	English voice settings	Voice settings
`cn_settings`	`VoiceSettings`	Chinese voice settings	Voice settings
`timestamp`	`uint64_t`	Timestamp	Nanoseconds

func_status: | Value | Description | | :--- | :--- | | 0 | Idle | | 1 | Call mode | | 2 | Voice wakeup + free Q&A | | 3 | Voice wakeup + ASR | | 4 | Voice announcement | | 5 | Multi-turn voice interaction | | 6 | Audio playback | | 9 | Exception |

wakeup_status: | Value | Description | | :--- | :--- | | 0 | Not woken up | | 1 | Listening after wakeup | | 2 | Thinking after wakeup | | 4 | Announcing after wakeup |

struct VoiceFuncStatus {
  uint32_t func_status{0};
  uint32_t wakeup_status{0};
  std::string requester{};
  bool wakeup_enabled{false};
  bool display_enabled{false};
  bool audio_enabled{false};
  VoiceSettings en_settings{};
  VoiceSettings cn_settings{};
  uint64_t timestamp{0};
};

VoiceSettings struct:

Member	Type	Description	Unit
`volume`	`uint32_t`	Volume	None
`speech_rate`	`float`	Speech rate	None
`voice_tone`	`std::string`	Voice tone	String
`is_curr_setting`	`bool`	Whether this is the current setting	Boolean

struct VoiceSettings {
  uint32_t volume{0};
  float speech_rate{0.0};
  std::string voice_tone{};
  bool is_curr_setting{false};
};

Example:

#include "gdk/gdk.h"
#include <iostream>
#include <chrono>
#include <thread>

int main()
{
    if (agibot::gdk::GDKInit() != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "GDK initialization failed" << std::endl;
        return -1;
    }
    std::cout << "GDK initialized successfully" << std::endl;

    agibot::gdk::Interaction interaction;
    std::this_thread::sleep_for(std::chrono::seconds(1));

    agibot::gdk::VoiceFuncStatus func_status;
    if (interaction.GetFuncStatus(func_status) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to get function status" << std::endl;
    } else {
        std::cout << "Function status information:" << std::endl;
        std::cout << "  Function status: " << func_status.func_status << std::endl;
        std::cout << "  Wakeup status: " << func_status.wakeup_status << std::endl;
        std::cout << "  Requester: " << func_status.requester << std::endl;
        std::cout << "  Wakeup function enabled: " << (func_status.wakeup_enabled ? "Yes" : "No") << std::endl;
        std::cout << "  Display function enabled: " << (func_status.display_enabled ? "Yes" : "No") << std::endl;
        std::cout << "  Audio function enabled: " << (func_status.audio_enabled ? "Yes" : "No") << std::endl;
        std::cout << "  Chinese settings - Volume: " << func_status.cn_settings.volume
                  << ", Speech rate: " << func_status.cn_settings.speech_rate
                  << ", Voice tone: " << func_status.cn_settings.voice_tone << std::endl;
        std::cout << "  English settings - Volume: " << func_status.en_settings.volume
                  << ", Speech rate: " << func_status.en_settings.speech_rate
                  << ", Voice tone: " << func_status.en_settings.voice_tone << std::endl;
        std::cout << "  Timestamp: " << func_status.timestamp << std::endl;
    }

    agibot::gdk::GDKRelease();
    return 0;
}

10. `GetAsrText()`¶

Function: Get ASR (Automatic Speech Recognition) text
Parameters:

Parameter	Type	Description
`asr_text`	`std::string&`	Output parameter, the recognized text content

Return value: GDKRes, the operation result status code. Returns GDKRes::kSuccess on success; the asr_text parameter contains the recognized text
Example:

#include "gdk/gdk.h"
#include <iostream>
#include <chrono>
#include <thread>

int main()
{
    if (agibot::gdk::GDKInit() != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "GDK initialization failed" << std::endl;
        return -1;
    }
    std::cout << "GDK initialized successfully" << std::endl;

    agibot::gdk::Interaction interaction;
    std::this_thread::sleep_for(std::chrono::seconds(1));

    std::string asr_text;
    if (interaction.GetAsrText(asr_text) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to get ASR text" << std::endl;
    } else {
        std::cout << "Recognized text: " << asr_text << std::endl;
    }

    agibot::gdk::GDKRelease();
    return 0;
}

11. `RegisterCallback()`¶

Function: Register a callback function
Parameters:

Parameter	Type	Description
`type`	`const std::string&`	Callback type
`callback`	`std::function<void(const std::any&)>`	Callback function

Return value: GDKRes, the operation result status code. Returns GDKRes::kSuccess on success
Currently supported callback types:
get_asr_text: The callback function's parameter type is const std::string&. In the wakeup state, when new voice input is recognized, the callback function is invoked with the recognized text content
Other function callbacks are not yet supported

12. `UnregisterCallback()`¶

Function: Unregister a callback function
Parameters:

Parameter	Type	Description
`type`	`const std::string&`	Callback type

Return value: GDKRes, the operation result status code. Returns GDKRes::kSuccess on success
Currently supported callback types:
get_asr_text: Unregisters the callback function. After unregistering, in the wakeup state, the callback function will no longer be invoked when new voice input is recognized
Other function callbacks are not yet supported
Example:

#include "gdk/gdk.h"
#include <iostream>
#include <chrono>
#include <thread>
#include <atomic>

std::atomic<bool> g_running{true};

void signal_handler(int signum) {
  std::cout << "Received interrupt signal" << std::endl;
  g_running = false;
}

int main()
{
    signal(SIGINT, signal_handler);
    signal(SIGTERM, signal_handler);

    if (agibot::gdk::GDKInit() != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "GDK initialization failed" << std::endl;
        return -1;
    }
    std::cout << "GDK initialized successfully" << std::endl;

    agibot::gdk::Interaction interaction;
    std::this_thread::sleep_for(std::chrono::seconds(1));

    // Enable call mode
    if (interaction.SetCallMode(true) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to enable call mode" << std::endl;
    } else {
        std::cout << "Call mode enabled successfully" << std::endl;
    }

    std::string asr_text;
    if (interaction.RegisterCallback("get_asr_text", [&asr_text](const std::any& data) {
        asr_text = std::any_cast<const std::string&>(data);
        std::cout << "ASR text: " << asr_text << std::endl;
    }) != agibot::gdk::GDKRes::kSuccess) {
        std::cout << "Failed to register callback" << std::endl;
    } else {
        std::cout << "Callback registered successfully" << std::endl;
    }

    while (g_running) {
        std::this_thread::sleep_for(std::chrono::seconds(1));
    }

    interaction.SetCallMode(false);
    interaction.UnregisterCallback("get_asr_text");
    agibot::gdk::GDKRelease();
    return 0;
}

Usage Notes¶

GDK initialization: Before using Interaction functions, you must first call agibot::gdk::GDKInit() to initialize the GDK system
GDK release: Before the program ends, you must call agibot::gdk::GDKRelease() to release GDK system resources
Initialization wait: After creating an Interaction object, it is recommended to wait 1 second to ensure the DDS connection is established
Return value check: All interfaces return a GDKRes status code; check the return value promptly to ensure the operation succeeded
File path: When playing audio and video, make sure the file path is correct and the file exists
Language setting: Setting the language affects the TTS language; set it according to your actual needs
Volume range: When setting the volume, keep it within a reasonable range, avoiding values that are too high or too low
Loop playback: When looping video playback, use -1 to indicate infinite loop, and be sure to stop it in time
Status query: Periodically query the function status to ensure all functions are working properly
ASR text: When retrieving ASR text, make sure the speech recognition function is enabled
String parameters: When passing string parameters, ensure the string is valid and not empty
Error handling: Always check the GDKRes return value to ensure the operation succeeded
Call mode: In call mode, you can capture the user's voice input and retrieve the corresponding recognized text through the interface
Registering a callback function: In call mode, you can call GetAsrText() or register a callback via RegisterCallback() to obtain the user's recognized voice text

Application Scenarios¶

Voice interaction: Implement the robot's speech recognition and speech synthesis functions
Multimedia presentation: Play audio and video content for information display
Human-robot interaction: Interact with humans through voice and display
Status monitoring: Monitor voice function status to ensure the system is running properly
Multilingual support: Support switching between Chinese and English to adapt to different scenario needs
Volume control: Dynamically adjust volume to adapt to different environments
Function switches: Flexibly control wakeup, audio, display, and other function switches

GDK Interaction Interface Documentation (C++)¶

Overview¶

Notes¶

Interface Description¶

Interaction Class¶

1. SetLanguage()¶

2. SetCallMode()¶

3. SetVolume()¶

4. SetWakeupSwitch()¶

5. SetAudioSwitch()¶

6. SetDisplaySwitch()¶

7. PlayTts()¶

8. PlayAudio()¶

9. PlayVideo()¶

10. GetFuncStatus()¶