Kinect screen with Brainiac

Please note this screen is no longer available, and has been replaced in newer RoboThespians and in SociBot with the simplified Sensors screen. These pages are retained for reference only.

Overview

Please note this screen is no longer available, and has been replaced in newer RoboThespians and in SociBot with the simplified Sensors screen. These pages are retained for reference only.

The depth sensor is used by RoboThespian to detect people, track and focus on an individual he is interacting with, and detect and mimic the body positions of people in his field of view. An early video of RoboThespian mimicking body movements can be seen here - http://www.youtube.com/watch?v=VGt_W6hwnoo

The 'Auto' or 'Kinect Screen' (name varies depending on software version) allows control of the features accessible through the Kinect sensor. Please note that these are beta features and development and improvements are ongoing.

At the top of the screen there are two views as seen by the Kinect sensor:

• The left view shows the image from the Kinect's colour camera
• The right view shows the image from the Kinect's infra red camera, used to help calculate depth or distance from the Kinect of objects in its field of view

The 3 buttons at the left and right edge of the screen will apply labels to anyone detected by the Kinect

• Show Label = label anyone in view of Kinect with a number in the order that they appear
• Show Status = show what Kinect is doing with a person, for example, 'Looking for pose' or 'Calibrating'
• Show Skeleton = shows what position Kinect thinks a person's skeleton is in. This is a simplified skeleton.

(Click on the picture for a larger image)

Brainiac speech interaction

Brainiac is a feature (not currently in development) that offers interactivity through talking with the robot. Please see Brainiac

Follow Person Feature

RoboThespian is able to follow people in view of the Kinect. He will move his eyes and head to look at them.

This feature along with more complex behaviours is now achieved using Tritium Control Functions on 2015 and onwards model RoboThespians or SociBots running on Tritium

On older robots running on IOServe the following functions were achieved in the Touchscreen Interface software. This information is kept for reference:

Click on 'Follow Person' (also called 'Track Player' or 'Follow Player' in some versions) - the button should change colour to green.

In 'Auto Detect' mode RoboThespian will follow the first person to come into his field of view automatically. To stop RoboThespian following people automatically click on the 'Auto-Detect' button - it will change to blue. Click it again to turn this feature back on - it will turn green again.

You can manually select the person RoboThespian should follow by clicking on the number next to the person in either of the camera views at the top of the screen.

Simplified 'Auto' screen

The latest version of RoboThespian has a simplified Auto screen with on-screen instructions:

Troubleshooting

RoboThespian is looking to the side / above / below the person he is following.

Check that the Kinect is correctly positioned. See Positioning_Kinect

If there are no views at the top of the screen...

If the two Kinect views are missing. First check that the Kinect is plugged in to the touchscreen PC and connected to the mains. Then press the 'Restart Kinect' button to restart the Kinect drivers.

If the camera views have frozen

Occasionally the views from the Kinect can freeze. Should this happen press the 'Reset Camera' button.

If people do not appear in the right hand (infra-red / depth) view

Increase the depth of view by using the 'Kinect Maximum User Range - 1000 to 6000' slider at the bottom of the screen