Virtual Robot Interface & Environment

Virtual Robot > Interface & Environment

The Virtual Robot Interface

Introduction

Virtual Robot provides a simple to use web based tool for content creation on EA robots like RoboThespian and Socibot.

This page introduces the Virtual Robot interface and working environment. Start here if you are not familiar with Virtual RoboThespian.

Links at the bottom of the page show you how to create a basic animation, transfer it to your robot and link to more advanced animation topics.

All EA robots are shipped with a basic Virtual Robot account enabled you can access it here, login required.

You may want to open your Virtual Robot in another window while you read these pages.

If you don't know your Virtual Robot login details contact Engineered Arts Ltd

A note on Technical requirements

Virtual RoboThespian is designed to be used on any platform. That said, there are some hardware configurations which can affect performance; if you are having problems please check this list and try to ensure whenever possible that your system meets specifications.

Devices: Virtual RoboThespian runs best in a personal PC or laptop. Many mobile devices do not support HTML5, and therefore will not correctly render the virtual environment, nor will all the pose editing tools be available on a phone or tablet. If using a laptop, we recommend connecting a mouse, as it is difficult to position the robot with precision on a touchpad, and some touchpads do not have scrolling functionality (which is required to zoom the display).

Browser: Virtual RoboThespian will only work in HTML5-enabled browsers. For best results, we recommend using Chrome, with FireFox as second choice. Internet Explorer does not fully support WebGL and requires 3rd party plugins; we do not endorse the use of Virtual RoboThespian with IE at this time.

Hardware/Drivers: Any 3D Graphics card should be sufficient to run Virtual RoboThespian seamlessly on your computer, however WebGL performance will depend on the integration of your browser with the chosen OS. If you find you are having problems with the display (a list of known issues is included in the Troubleshooting section), we recommend checking the following link to see if your combination of OS, browser and device driver fully supports WebGL: https://www.khronos.org/webgl/wiki/BlacklistsAndWhitelists

Interface and Environment

The Virtual Robot interface is divided into several sub areas

The Virtual Robot Interface

Menu Bar - file controls, QR Code generator and login details.
Staging area - control and pose RoboThespian using your mouse.
Library - comes packed with pre-installed content for you to drag and drop into the timeline.
Timeline - arrange poses, audio components and speech to create your own robot performance.
Inspector - real-time readouts of actuator data and view key frame data.

Here we provide a detailed guide to how these various elements combine to create an intuitive interface for animating a robot.

Menu Bar

File & Login options are located in a menu bar, at the top of the browser window.

Menu Bar

File buttons - New blank timeline, Save the current performance, Copy the current performance, Delete the current performance.
File List drop down menu - Type in here to give a name to new performances, or use your mouse to select an existing performance from the drop-down menu
File History drop down menu - This will list all previous saves to the current file selection.
QR code button - Click to generate a QR code for your specific performance. If you store the image, you can use it to trigger your performance by holding it in the visual field of a real robot.

Staging area

This is the main area where you can view the robot, watch a sequence play out, or create new poses. Click and drag anywhere in the staging area except on the robot to change the camera angle. Use the mouse scroll wheel (or equivalent laptop function) to zoom in and out. You can create poses by moving the robot parts (limbs, fingers, head, etc) to the desired position and pressing 'add pose'. The top menu contains some useful options for changing the eye animations and body LED colours. For a detailed guide to pose editing, see posing the robot, below. Double clicking on the robot torso or legs will open the Routines tab in the library. Double clicking on the head will bring up the Head sequences tab, on an arm will bring up the Pose tab, and on the jaw will open the Speech tab.

The Staging Area

Mirror Eyes Button - Toggle this on/off to control eyes individually or with mirroring function.
Eye Graphics Drop Down Menu - Select animatable or static eye graphics from the list.
Light Colour Selector - Click on the robots Head, Torso, Arms and Legs (they will highlight blue), then click the drop down button.
Colour Cursor - Move this point around to select you desired colour for the selected body part.
Slider - Adjusts the light brightness for the selected body part.
Robot - Click on him to move and select for light colours.

Library

The Library

The library window allows you to access stored data to use in timeline sequences. When dropped into the timeline, the length of the audio or motion clip will be reflected in the component length as displayed in the timeline. Currently grouped into the following sublibraries:

Pose

Useful static poses and short gestures (eg beckon).

Head

Head poses and movements.

Eyes

Eye positions and expressions, and a blink function.

Colour

As with the cheeks, some useful colours to be used in the body/arms/pelvis LEDs.

Cheeks

Some useful precomposed cheek colours for the LEDs in the face.

Routines

Longer routines created for real RoboThespian installations, which are entertaining or useful in a general setting.

Audio

Stored audio segments. These can be sound snippets, music, a recorded speech segment - anything you like.

To upload a new audio file. Open folder containing sound file, drag and drop into timeline.

Speech

Text to speech input. Key below:

The Speech Tab

Language
Voice selection (number of voices available depends upon language).
Voice style - some voices (especially US English) have optional speech modulations, eg mood, age, character.
Text input - enter your desired speech here, and press 'Add Speech' at the bottom right to insert into the timeline.
Volume - adjust the volume of the tts component.
Speed - adjust the speed.
Shaping - how much inflection is given to the tts component.
Lypsync Gain - Control the range of motion for the jaw
Append & Add at Marker - Add tts component to the speech track.
Expand & Contract the Library Area.

Like the pre-composed elements, tts segments will reflect their true length when dropped into the timeline, however it may take a few moments for the engine to calculate the length.

Timeline

The timeline lets you view, play and edit complete routines for the Virtual (and real!) RoboThespian.

The Timeline Menu
The Timeline Tracks
The Animation Toolbox

Animation Toolbox - Open/Close the floating animation toolbox
Copy and Paste - Select an item to copy, and paste at current marker position
Green Timeline Marker & Marker Time display
Undo / Redo (Unlimited - currently only works for toolbox actions)
Item Mode / Frames Mode (Current mode highlighted in orange. Each mode offers different ways to edit)
Play/Pause & Restart your performance. Restart plays from the beginning
Reset - Resets the virtual robot to a default start pose
Minimise Timeline area
Scrollbar - Drag to navigate the timeline (Dragging the ends will compress or extend the timeline)
Current time in seconds of green marker position
Track Type Names
Individual Track Show/Hide Toggles
Motion, Audio, Speech, and Lipsync Tracks (Item edit mode with keys toggled to SHOW, audio and TTS showing file names)
Motion, Audio, Speech, and Lipsync Tracks (Item edit mode with keys toggled to HIDE, audio and TTS showing wave forms and linear interpolation between keys)
Motion Track (with Keyframe Edit Mode #5 toggled on)
The Animation Toolbox (individual buttons described in Animation Toolbox )

Drag Library elements into the timeline area where you wish to insert them. You can drag elements to reorder them (although note that routines containing both sound and movement have these inextricably linked). Audio files such as your own mp3 files can be dragged directly into the timeline area. Audio files can also be added from the audio tab in the library. Text to Speech (tts) items are created at the markers current position or appended to the last one in the timeline if it exists. A lipsync data item is automatically created and is linked to the tts items. If a tts item is deleted, the linked lipsync data item will remain and will also need to be deleted if not required.

Animation Toolbox

The animation toolbox provides a selection of tools (upper row) and body part filters (lower row). its a floating toolbox, so can be moved around the browser and placed anywhere you like. Tools are ordered from Left to Right. Keyboard short cuts for each button display when hovering the mouse over the button.

The Animation Toolbox

Upper Row Buttons

Add Key Pose - Adds current robot pose at current marker position based on filter selection.
Interpolation Type - Chose between Stepped/Linear/Spline interpolation types.
Apply Interpolation - Apply the selected interpolation to the currently selected timeline item.
Remove Key Pose - Removes the key pose at current marker position based on filter selection.
Remove All Key Poses - Removes all keys poses in selected timeline item based on filter selection.
Add single Input - Create an input to trigger external devices, such as a sequence on another robot.
Slice - Slice selected item at current marker position.
Weld - Weld selected item at current marker position.

Lower Row Buttons

Select All/None
Eyes
Head
Body
Right Arm
Left Arm
Hands
Lights
Play Inputs

The body part filters are toggle buttons. If you wish to only add or remove keys for the body, then only toggle the body selection, and so on. The filters are useful for creating animation in passes or for isolating problems with movement.

Item Selection

Items on the timeline (motion data, audio & TTS clips) can be selected by LMB clicking on them. Whilst an item is selected, holding LMB and dragging left and right, allows an item to be moved. Multiple items can be selected and manipulated by LMB clicking to select, holding CTRL to add to selection, and holding CTRL + SHIFT to add range in a single track.

Interpolation Types

Individual frames are distinguished by a fine coloured grating within the timeline element. They are not necessarily evenly spaced, as they correspond to event times rather than clock cycles or animation frames. The type of interpolation between them is represented by either a blank space for stepped and a faded but constant grating for linear.

Stepped Interpolation

Stepped interpolation (blank space between key poses) means the key pose value remains unchanged until the marker plays the next value along the timeline. This will result in a jumpy playback effect but serves well when planning out key poses.

Linear Intrerpolation

Linear interpolation (faded grated colours) means the values from one key pose to the next is averaged out over the time between them. The result is an smooth movement between key poses with an equal spacing between each frame.

Linear interpolation updates automatically when adding key poses between existing keys poses. You can also change the interpolation type, by selecting the element you wish to change, then selecting the interpolation type you require from the animation toolbox drop down list, and clicking the interpolate button next to the drop down list.

Inspector

The right slideable window gives a real-time list of all hardware outputs.

Values for the elements can be directly modified here.

Either left click the mouse on the value to change, keep holding left click and move the mouse up to increase, down to decrease.

Or, double click on the value to change and type in the desired value. A red value indicates that it is outside the range of movement and will not be accepted.

Posing the robot

Most robot elements can be controlled with the mouse alone. The following is a list of parts, their controllable aspects or axes, and how to move each one.

Robot Part	Controllable feature	How to edit
Eyes	Iris position	Mouse: left click anywhere in the eyeball and hold the button down to drag the eyes to an appropriate position.
Eyes	Pupil size	Mouse+ctrl: left click on the iris or pupil while holding down the ctrl key on your keyboard. Dragging the mouse will increase or decrease pupil size.
Eyes	Graphics	A drop-down menu on the upper left of the pose screen lists graphical display options for the eyes. Be warned that not all of these will animate or have editable features.
Eyelids	Vertical position	Mouse: left click on the upper or lower eyelid and drag to position.
Eyelids	Tilt	Mouse+ctrl: left click on the upper or lower eyelid while holding down the ctrl key on your keyboard. Dragging the mouse will tilt the robot's eyelids up or down.
Eyes + Eyelids	Simultaneous editing	To the left of the dropdown graphics menu, there is a 'mirror eyes' toggle. This is on by default, and means that edits to one eye will be reflected in the other. To change an individual eye or eyelid, without affecting the other, toggle this option off.
Head	Yaw and pitch position	Mouse: left click and drag with the mouse to control yaw and pitch of the head. Lateral (x) mouse position controls yaw, vertical (y) controls the pitch.
Head	Roll position	Mouse+ctrl: left click on the head while holding down the ctrl key on your keyboard to edit the head rotation about the roll axis only. The other axes will be disabled when ctrl is held.
Head	Jaw	Mouse: Left click and drag the mouse to open or close the robot's jaw. This is a binary mode (open/closed) without interim positioning.
Head	Face LEDs	Drop-down colour menu/brightness slider: Click briefly on the head (it should flash blue to indicate it is selected). Using the drop down colour map on the upper menu bar, and the brightness slider tool, you can control the LED output for the robot cheeks.
Upper arm/shoulder	Yaw and pitch position	Mouse: left click on the upper arm or shoulder joint and drag to rotate the arm yaw or pitch. Lateral (x) mouse position controls the yaw (abduction/adduction of the arm), vertical (y) mouse position controls the pitch (raising or lowering the joint). For best and most precise results, use the shoulder joint for pitch control and the centre of the joint (around the bicep) for yaw control. Note that controlling yaw from the shoulder can cause overlap difficulties with body roll angle.
Upper arm/shoulder	Roll position	Mouse+ctrl: left click on the upper arm while holding down the ctrl key to control arm roll (pronation/supination).
Upper arm	Arm LEDs	Drop-down colour menu/brightness slider: Click briefly on the arm (it should flash blue to indicate it is selected). Using the drop down colour map on the upper menu bar, and the brightness slider tool, you can control the LED output for the arms separately.
Elbow	Pitch position	Mouse: left click on the forearm and drag along the vertical axis to control elbow pitch. Note that the elbow has only one true degree of freedom, however you can control the roll axis of the wrist (see Wrist/Roll) and the yaw axis of the shoulder from the forearm.
Wrist	Roll position	Mouse+ctrl: left click on the forearm or hand to control the roll axis of the wrist/hand.
Hand	Pitch position	Mouse: left click on the hand to control wrist pitch. Note that there is no yaw degree of freedom in the wrist on this RoboThespian model - lateral and vertical mouse movement both control wrist pitch only. Depending on the overall body pose, it may be more advantageous to switch between vertical or lateral motion to achieve the full sensitivity and precision.
Hand	Finger flexion	Mouse: Click on a finger to flex or extend it. Each finger has only one degree of freedom with two possible positions (fully extended or fully curved).
Torso	Roll and pitch position	Mouse: left clicking on the torso and dragging the mouse will change the pitch and roll of the robot torso. Lateral (x) mouse position controls the roll, vertical (y) mouse position controls the pitch. Note that these controls are sensitive to the yaw position of the upper body and for best results, rotate the camera frame so the robot is facing you.
Torso	Yaw position	Mouse+ctrl: left click on the torso while holding down the ctrl key to control yaw rotation of the upper body.
Torso	Body LEDs	Drop-down colour menu/brightness slider: Click briefly on the torso (it should flash blue to indicate it is selected). Using the drop down colour map on the upper menu bar, and the brightness slider tool, you can control the LED output for the torso.
Legs	Pelvis LEDs	Drop-down colour menu/brightness slider: Click briefly on the legs (they should flash blue to indicate they are selected). Using the drop down colour map on the upper menu bar, and the brightness slider tool, you can control the LED output for the pelvis.

Creating a simple animation

Now that you are familiar with the Virtual Robot Interface and Environment, let's start a simple performance.

Please see Virtual Robot Creating a Simple Performance

Transferring animation to your robot

Once you have created a new performance on Virtual Robot, you will want to see it working on a real robot. It's easy to transfer to your robot, even if the robot is in a different part of the world.

Please see Virtual Robot Transferring Content