SociBot v1 Kinect Depth Sensor Setup

From Engineered Arts Wiki
Revision as of 11:17, 6 May 2016 by Marcus (talk | contribs) (Use)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Please note this is no longer necessary. The Kinect has been replaced in newer RoboThespians and in SociBot with an onboard depth sensor (Asus Xtion) already connected to the robot's onboard PC. No setup required. These pages are retained for reference only.

Setup

Please note this is no longer necessary. The Kinect has been replaced in newer RoboThespians and in SociBot with an onboard depth sensor (Asus Xtion) already connected to the robot's onboard PC. No setup required. These pages are retained for reference only.

Kinect autoscreen connecting USB.jpg

Kinect

Connect the usb from the kinect to a usb port on the back of the touch screen pc.

The Kinect will also need to be plugged into a mains outlet, use one of the ones on the power strip located on the bottom shelf of the kiosk.

Asus

Connect the usb from the Asus to a usb port on the back of the touch screen PC.

Use

The sensor (Kinect or Asus) is used by SociBot to track people and focus on them during interaction. It is also used for detecting facial expressions to mimic and for identifying facial features to be used to determine the approximate age of the individual.

The Kinect sensor has an approximate field of view of 90 degrees.

The Asus sensor has an approximate field of view of 80 degrees.

Kinect autoscreen position.jpg

The 'Auto' or 'Kinect Screen' (name varies depending on software version) allows control of the features accessible through the Kinect sensor. Please note that these are beta features and development and improvements are ongoing.

At the top of the screen there are two views as seen by the sensor:

  • The left view shows the image from the sensor's colour camera with some information about each visible persons facial expressions and their approximate age. It is flipped along the vertical axis.
  • The right view shows the image from the sensor's infra red camera, used to help calculate depth or distance from the Kinect of objects in its field of view. It also displays a skeleton on all detected people.

The buttons located underneath the RGB and depth images offer some options.

  • Copy People=copy the detected main persons facial expression
  • Reset Camera=Resets the cameras segmentation and labelling. Useful if any issues are encountered.

Socibot kinect screen.jpg

Troubleshooting

If there are no views at the top of the screen...

If the two Sensor views are missing. First check that the sensor is plugged in to the touchscreen PC and, if its a Kinect, connected to the mains. Then press the 'Reset Camera' button to restart the sensor drivers.

If the camera views have frozen

Occasionally the views from the Kinect can freeze. Should this happen press the 'Reset Camera' button.

If people do not appear in the right hand (infra-red / depth) view

Contact an Engineered Arts Technician.

We are continuing to develop these features and can teach SociBot to respond in different ways to different questions / names / keywords etc. If you have any special requests please let us know.