The character can now repeat the player’s movements. With a webcam switched on it reproduces the position of the arms, the tilt of the head and the facial expression, and players nearby can see it. There is one important condition: the video from the camera is not sent to the server, and access to this feature is granted by an administrator.
Where the recognition happens
Pose and face recognition run locally, in the browser itself, on a separate background thread (worker). Body landmarks are determined from the frame, they are converted into joint angles, and those angles are applied to the character in real time.
What is sent to other players
Only numbers travel over the network, that is joint angles. Not a single frame and not a single image. The server, as in the case of voice chat, does not look inside the data: it checks who sent the packet, whether the player has permission and whether the data is arriving too often, after which it copies the array on to everyone within 26 metres.
The radius is larger than for voice, where it is 20 metres, because this is about being seen rather than being heard: the isometric camera shows about 11 metres in half of the screen, so everyone who appears in frame has to move.
A permission that leaves no trace
Controlling the character with a camera is a permission an administrator grants to an individual player, just like the right to speak in voice chat. If the permission has not been granted, the microphone and camera buttons are not shown at all: not greyed out and not locked, but absent, together with their dividers and their line in the settings.
There is one exception: a browser that technically does not support tracking keeps the button and states the reason. That is information about the device, not about the account.
The face and the hands
The character’s mouth consists of three parts joined at the corners, and it darkens as it opens. At isometric distance three centimetres of travel produce one pixel, so an open mouth reads through contrast rather than through shape.
The fingers learned to fan out and became longer, because with short fingers every gesture had the same silhouette.
