MarsCat Development manual

• Connect Display screen, mouse and keyboard.

• Connect wifi, Remote connection such as ssh, vnc.

  Note: Account is pi, Passward is raspberry

• MarsCat's boot-up service needs to be stopped before development

  cd ~/marsai
  ./tools/stop-systemd-services.sh
  ./tools/disable-systemd-services.sh
  

• If you're about to give up, restart the MarsCat's service

  cd ~/marsai
  ./tools/enable-systemd-services.sh
  ./tools/start-systemd-services.sh
  

Download API

cd ~
git clone https://github.com/elephantrobotics/marscatAPI.git
cd marscatAPI

Eyedisplay

The eyedisplay is used to display pictures in the eyes of MarsCat.

Functions

reset_eye_background

• Prototype: reset_eye_background(cls)

• Description: Set eye background color.

• Parameters

  • cls : [Blue, Green, Purple, Yellow].

blink

• Prototype: blink()

• Description: Display blink animation.

display_eye_squint

• Prototype: display_eye_squint()

• Description: Display squint animation.

display_eye_wakeUp

• Prototype: display_eye_wakeUp()

• Description: Display wake-up animation.

display_eye_close

• Prototype: display_eye_close()

• Description: Display close eyes animation.

display_eye_sleepy

• Prototype: display_eye_sleepy()

• Description: Display sleepy animation.

get_eye_lid

• Prototype: get_eye_lid(number)

• Description: Convert eye lid images to RGBA images.

• Parameters

  • number : eye lid number 1 ~ 14 31 ~ 40.

get_eye_ball

• Prototype: get_eye_ball(number)

• Description: Convert eye ball images to RGBA images.

• Parameters

  • number : eye ball number 1 ~ 10.

display_eye

• Prototype: display_eye(ball_num, lid_num, ball_type, dx, dy)

• Description: Display specified eye animation.

• Parameters

  • ball_num : Eyeball size 1 ~ 10
  • lid_num : Eyelid size 1 ~ 14, 31 ~ 40
  • ball_type : [eye_ball, ball,face,flip,heat,hug,teaser,voice,low_power,charging,dizzy, bowlorhungry, sleep,high_temperature, hw_error]
  • dx,dy : Coordinates of eyeball, (0, 0) means in the middle

Instructions

Display eye animations

import eye.eyedisplay
import time

color = ['Blue', 'Green', 'Purple', 'Yellow']

eyes = eye.eyedisplay.EyeDisplay()

for c in color:
    eyes.reset_eye_background(c)
    time.sleep(1)

    eyes.blink()
    time.sleep(3)

    eyes.display_eye_squint()
    time.sleep(3)

    eyes.display_eye_wakeUp()
    time.sleep(3)

    eyes.display_eye_close()
    time.sleep(3)

    eyes.display_eye_sleepy()
    time.sleep(3)

    eyes.get_eye_lid(2)
    time.sleep(1)

    eyes.get_eye_ball(2)
    time.sleep(1)

    eyes.display_eye(ball_type ='charging')
    time.sleep(3)

Movement

The movement is used to control and get data from MarsCat.

Functions

set_walk

• Prototype: set_walk(step, speed)

• Description: Walk movement.

• Parameters

  • step(int) : Number of steps to walk.
  • speed(float:0.1~1.0) : Speed of walking

set_run

• Prototype: set_run(step, speed)

• Description: Run movement.

• Parameters

  • step(int) : Number of steps to walk.
  • speed(float:0.1~1.0) : Speed of walking

turn

• Prototype: turn(direction, step, speed)

• Description: Turn movement.

• Parameters

  • direction(str:'left'or'right') : Direction of turning
  • step(int) : Number of steps to walk.
  • speed(float:0.1~1.0) : Speed of walking

backward

• Prototype: backward(step, speed)

• Description: Backward movement.

• Parameters

  • step(int) : Number of steps to walk.
  • speed(float:0.1~1.0) : Speed of walking

set_head_angle

• Prototype: set_head_angle(joint_no, angle, speed)

• Description: Set head angle.

• Parameters

  • joint_no(int) : 1 - up and down, 2 - left and right
  • angle(int) : 1 - [4, 28], 2 - [-20, 20]
  • speed(float:0.1~1.0) : Speed of moving.

get_head_angle

• Prototype: get_head_angle(joint_no)

• Description: Get head angle.

• Parameters

  • joint_no(int) : 1 - up and down, 2 - left and right

• Return

  • Joint_no angle.

set_leg_angle

• Prototype: set_walk(leg_no, joint_no, angle, speed)

• Description: Walk movement.

• Parameters

  • leg_no(int) : 1 ~ 4
  • joint_no(int): 1 ~ 3

  • angle(int) :

    | leg | joint | min_angle | max_angle | | :----: | :----: |:----: |:----: | |leg 1 |joint 1 | -20 | 20| |leg 1 |joint 2 | -45 | 70| |leg 1| joint 3 | 5 | 75| |leg 2 |joint 1 | -20 | 20| |leg 2 |joint 2 | -45 | 70| |leg 2| joint 3 | 5 | 75| |leg 3 |joint 1 | -20 | 20| |leg 3 |joint 2 | -70 | 30| |leg 3 |joint 3 | -100 | -15| |leg 4| joint 1 | -20 | 20| |leg 4 |joint 2 | -70 | 30| |leg 4 |joint 3 | -100 | -15|

  • speed(float:0.1~1.0) : Speed of moving.

get_leg_angle

• Prototype: get_leg_angle(leg_no, joint_no)

• Description: Get leg angle.

• Parameters

  • log_no(int) : 1 ~ 4
  • joint_no(int): 1 ~ 3

• Return

  • Joint angle.

get_gyro

• Prototype: get_gyro(mode)

• Description: Get gyro data.

• Parameters

  • mode(int) : 0/1-RX/RY 3/4/5-AX/AY/AZ

• Return

  • gyro angle/ acceraltion.

get_tof

• Prototype: get_tof()

• Description: Get tof data.

• Return

  • tof_data(float).

get_battery

• Prototype: get_battery(mode)

• Description: Get battery value.

• Parameters

  • mode(int) : 0 - get battery voltage , 1 - get battery percentage

• Return

  • battery_value(float).

set_servos_power

• Prototype: set_servos_power(servo_no, servo_state)

• Description: Electrify or energize servo.

• Parameters

  • servo_no(int) : 0 - all servo , 1~16 - single servo
  • servo_state : 0 - power off, 1 - power on, 2 - foucs (servos can only be focused one by one)

Instructions

Use cases

import move.movement
import time

mv = move.movement.MoveMent()

# Basic Move

mv.set_walk(8, 0.7)

mv.set_run(8, 0.7)

mv.turn('left', 10, 0.7)
mv.turn('right', 10, 0.7)

mv.backward(10, 0.7)

# Basic Control

for h in range(1, 3):
    if h == 1:
        for a in range(4, 28):
            mv.set_head_angle(h, a, 0)
    else:
        for a in range(-20, 20):
            mv.set_head_angle(h, a, 0)

for h in range(1, 3):
    print(mv.get_head_angle(h))

for l in range(1, 5):
    for j in range(1, 4):
        mv.set_leg_angle(l, j, 0, 0)

for l in range(1, 5):
    for j in range(1, 4):
        print(mv.get_leg_angle(l, j))

# Basic Mode

# 0 / 1 - RX / RY 3 / 4 / 5 - AX / AY / AZ
GR = [0, 1, 3, 4, 5]
for g in GR:
    mv.get_gyro(g)

mv.get_tof()

mv.get_battery(0)
mv.get_battery(1)

mv.set_servos_power(0, 1)
mv.set_servos_power(0, 0)
mv.set_servos_power(0, 2)

Touch

The touch is to get signals from the sensors inside MarsCat.

Functions

get_touch

• Prototype: get_touch()

• Description: Get signals from the sensors.

• Return

  • GPIO.input() -> {value}: Used to display value change of the IO

Instructions

Detect touch

import sensor.touch
import time

tc = sensor.touch.Touch()

"""
If all sensors aren't triggered, it will return [0, 0, 0, 0, 0, 0, 0] 
If head sensors are triggered, it will return [1, 1, 0, 0, 0, 0, 0] 
If jaw sensors aren triggered, it will return [0, 0, 1, 0, 0, 0, 0] 
If back sensors are triggered, it will return [0, 0, 0, 1, 1, 1, 1] 
"""

while True:
    tc.get_touch()
    time.sleep(0.1)

Sound

The sound is to control the loudspeaker inside MarsCat to play different audios.

Functions

meow

• Prototype: meow()

• Description: Play meow sounds.

purr

• Prototype: purr()

• Description: Play purr sounds.

purr

• Prototype: purr()

• Description: Play purr sounds.

angry

• Prototype: angry()

• Description: Play angry sounds.

eat

• Prototype: eat()

• Description: Play eat sounds.

unhappy

• Prototype: unhappy()

• Description: Play unhappy sounds.

meow_draw_attention

• Prototype: meow_draw_attention()

• Description: Play meow sounds.

play_sound

• Prototype: play_sound(sound_name)

• Description: Play specified sound.

• Parameters

  • sound_name : Name of the audio.

Instructions

Play sounds

import sound.catsound
import time

s = sound.catsound.CatSound()

s.meow()
time.sleep(2)

s.purr()
time.sleep(6)

s.angry()
time.sleep(4)

s.eat()
time.sleep(9)

s.unhappy()
time.sleep(1)

s.meow_draw_attention()
time.sleep(1)

"""
If you want to add a new sound file, add it to the "Sounds" folder.
One sound file corresponds to three files with different names.
like this:
sounds/new_file-quite.mp3
sounds/new_file-medium.mp3
sounds/new_file-loud.mp3

add new function:
def new_sound(self, async_play=True):
self.play_sound('new_file.mp3')
"""

s.play_sound('meow-normal-2s-loud.wav')

Vision

The vision is for using the camera on the nose to detect fall people, detect objects, detect faces or checking moving objects.

Functions

process_frame

• Prototype: process_frame(frame)

• Description: Parse video frames.

• Parameters

  • frme : video frame.

• Return

  • corners: Used to identify the specified QR code.
  • ids: Used to identify the specified QR code.
  • frame2: previous frame.
  • contours: Used to detect specified moving objects.
  • circles: Used to detect the specified blue ball.
  • faces: for detecting faces.
  • gray: for detecting faces.
  • status: for detecting falls.

if_has_fall

• Prototype: if_has_fall(status)

• Description: Detect if a person has fallen.

• Parameters

  • status : Return value from process_frame function.

• Return

  • {"type":"fall_people","data":True}: Falling person detected
  • None: no one fell.

if_has_face

• Prototype: if_has_face(faces, gray, frame)

• Description: Detect faces.

• Parameters

  • faces : Return value from process_frame function.
  • gray: Return value from process_frame function.
  • frame : Video frame.

• Return

  • {"type":"human","data":face_data}: face detected.
  • None : face not detected.

if_has_ball

• Prototype: if_has_ball(circles)

• Description: Detect the specified blue ball.

• Parameters

  • circles : Return value from process_frame function.

• Return

  • {"type":"ball","data":data}: blue ball detected.
  • None : blue ball not detected.

if_has_moving_obj

• Prototype: if_has_moving_obj(contours)

• Description: Detect specified toys.

• Parameters

  • contours : Return value from process_frame function.

• Return

  • {"type":"obj","data":data}: The specified toy was detected.
  • None : The specified toy was not detected.

if_has_qrcode

• Prototype: if_has_qrcode(corners, ids)

• Description: Detect specified toys.

• Parameters

  • corners : Return value from process_frame function.
  • ids : Return value from process_frame function.

• Return

  • {"type":"qrcode","data":data}: The specified QR code was detected.
  • None : The specified QR code was not detected.

detect_fall_people

• Prototype: detect_fall_people(frame)

• Description: Detect people falling.

• Parameters

  • frame : video frame.

• Return

  • True: A fall is detected.
  • False : No fall detected.

get_face_ball_obj_qrcode

• Prototype: get_face_ball_obj_qrcode()

• Description: Integrated visual detection functions, including face, QR code, toys, fall detection.

• Return

  • dictionary data: detected.
  • None : Nothing detected.

Instructions

Integrated detection

import vision.vision_base

vs = vision.vision_base.Vision()
while True:
    res = vs.get_face_ball_obj_qrcode()

Detect faces

import cv2
import vision.vision_base

vs = vision.vision_base.Vision()
cap = cv2.VideoCapture(0)
cap.set(3, vs.cam_width)
cap.set(4, vs.cam_height)

while True:
    frame = cap.read()[1]
    corners, ids, frame2, contours, circles, faces, gray, status = vs.process_frame(frame)
    result = vs.if_has_face(faces, gray, frame)
    print(result)
    k = cv2.waitKey(1) & 0xff
    if k == 27:
        cap.release()
        break

Voice Recognition

Library - pocketSphinx

You can download CMU pocket sphinx from here

Support recognition of three languages: English, Chinese

• Supported directives are stored in corpus/corpus.txt

speak_config

• Prototype: speak_config(language)

• Description: Initialize the speech recognition language library.

• Parameters

  • language : language to be recognized (Englishor Chinese).
• Return
  • voice decoder.

load_word

• Prototype: load_word()

• Description: Get Supported Instructions.

• Return

  • All supported commands.

get_speak_queue

Prototype: get_speak_queue()

• Description: construct a queue.

• Return

  • queue.

get_commands

Prototype: get_commands(speak_queues)

• Description: Get the value in the queue.

• Parameters

  • speak_queues : queue.

• Return

  • value: recognized words.
  • None: The specified command is not recognized

speak_monitor

• Prototype: speak_monitor(speak_queue, decoder, command)

• Description: speech recognition function.

• Parameters

  • speak_queue : queueused to store recognized words.
  • decoder: voice decoder.
  • command : All supported commands.

Instructions

import threading
import voice.voice_base

vc = voice.voice_base.Voice()
command = vc.load_word()
decoder = vc.speak_config("English")

speak_queue = vc.get_speak_queue()

p = threading.Thread(target=vc.speak_monitor,
                        args=(speak_queue, decoder, command), daemon=True)
p.start()

while True:
    commands = vc.get_commands(speak_queue)
    if commands:
        print(commands)