designmatcha
All projects

Hackathon · Robotics × HCI2026

Clippy Deskbot

Digital updates. A physical presence. A desk robot inspired by Microsoft’s Clippy that turns Teams, Outlook, Calendar and Figma notifications into glances, nods and a few spoken words, so you know something happened without leaving your work.

Role
Interaction designer and co-creator, with Vishruth
Focus
Eye animation, gesture language and notification behaviour
Built with
Raspberry Pi 5, two servos, a 128 × 64 OLED, camera, speaker
Thanks
Aatman and the Microsoft Garage
The Clippy Deskbot prototype on a desk beside a laptop: a cream paperclip frame around a black head with cyan eyes on a small display, on a rotating black base
The working prototype on a desk

01The problem

A notification pulls us away from the task we were doing.

Teams, Outlook, Calendar and Figma all announce themselves the same way: a banner, an icon or a sound. Each one asks us to switch windows just to find out whether it matters.

Our hypothesis: a small, recognizable physical cue could help people notice an update without immediately leaving their work. The question to test was whether people can read the cue accurately and get back to their task sooner.

Clippy shouldn’t behave like a computer with a face. It should behave like a tiny desk companion that happens to understand your computer.

North star

02The idea

Turn every digital event into one coordinated, readable response.

Clippy sits beside the screen and answers events through three channels, each with one job. Software sequences them; the robot isn’t inferring anyone’s emotions.

  • Eyes

    Emotion

    Expressions and app-specific pupils on a small cyan display show the source, the urgency and the robot’s state.

  • Movement

    Attention

    Two servos turn and tilt the body. A slow turn means “something happened”; a quick one means “now”.

  • Voice

    Information

    A short spoken line adds detail only when the event deserves more than a glance.

Every interaction follows the same four beats, so people learn the pattern once:

  1. Attention. Something happened.
  2. Expression. Here’s how I feel about it.
  3. Information. Here’s what’s happening.
  4. Resolution. Back to idle.
Feature poster: Clippy Deskbot on a desk with six features listed, eyes with personality, recognizable app cues, a voice of its own, turns, nods and glances, quiet on your terms, and face-following vision
The feature overview from our science fair poster

03Eye language

Expressive eyes, not another screen.

The 128 × 64 one-bit display can’t show an interface, and it shouldn’t. We treated it as a tiny animation canvas: independent lids, brows and gaze, 29 classic eye animations plus a one-eye wink, and app pupils that grow and shrink with the beat.

EyesMeaning
BlinkIdle behaviour and transitions
Double blinkAcknowledgement or confirmation
Look left / rightAttention, searching, curiosity
Look upThinking or processing
WideSurprise, an important notification
HappySuccess, thanks
SadAn error or disappointment
AsymmetricUncertainty, a misunderstood request
App pupilsMail, Teams or Figma, recognizable at a glance
SleepDo Not Disturb or long inactivity
Six one-bit eye frames from the simulator: Mail, Teams and Figma app pupils, and happy, wink and sleep expressions
Actual simulator framebuffers: app pupils, happy, wink and sleep

04Gesture language

Motion should always mean something.

The pan servo turns the body left and right; the tilt servo nods. Only one axis moves at a time, and Clippy always faces the person before it speaks.

GestureMeaningTypical use
Slow turn toward youAttentionA normal notification
Quick turnUrgencyA meeting starting now
Small left-right movementThinkingAn API lookup or confusion
Small nodYes, acknowledgedSuccess or thanks
Small shakeNo, failedAn error or rejected action
Curious turnInterestA Figma comment
Return to centreResolutionThe end of every interaction

Behaviours are composed from a small vocabulary of reusable primitives rather than one-off animations. A Teams message, for example, is QUICK_TURN + ALERT + NOTIFICATION_SOFT + VOICE + CENTER.

Three close-ups of the robot’s head during a quiet-mode nod: down at 95 degrees, up at 88.6 degrees and settled at 90 degrees, with notes on the pan and tilt channels
Two axes, one readable gesture: a captured quiet-mode nod

05Attention rules

Not every update deserves the same reaction.

Every event passes through the same pipeline: detect, prioritize, orient, animate, sound, speak, then return to idle. Priority decides how far along that pipeline it gets.

PriorityExamplesBehaviour
P0 SilentBackground sync, minor Figma updatesEyes only. No sound, speech or movement.
P1 AmbientNew email, a Figma commentA small movement and an eye animation
P2 AttentionA Teams message, a meeting soonTurn toward you, a sound and a short line
P3 UrgentA meeting starting nowA fast turn, wide eyes and voice. Can interrupt P0 and P1.
  1. Queue

    Events wait in priority order, first in, first out at the same priority. An urgent alert waits behind a conversation already in progress.

  2. Aggregate

    Repeated waiting events combine, so seven Teams messages become one line: “You’ve got seven new Teams messages.”

  3. Filter

    Source controls and Do Not Disturb are checked first. Under DND only urgent P3 events get through.

You decide the interruption level

Voice can be off, important only or everything. Movement can be off, subtle or full. Intensity can be quiet, normal or expressive, and each notification source can be switched off.

Four priority levels: P0 silent, P1 ambient, P2 attention and P3 urgent, with queue, aggregate and filter rules
Four levels of attention, with a queue that knows what can wait

06Camera and trust

Aware, not watchful.

A camera lets Clippy turn toward the person who is talking to it, or toward something they hold up and ask about. It uses face detection to find where a face is, not who it is.

  1. Slow, damped tracking

    A dead zone ignores small offsets and direction changes need confirming, so the robot doesn’t twitch. When the face is lost it returns to centre.

  2. Looking is always visible

    When the camera is processing, the eyes change and a scan animation plays. Clippy never looks silently.

  3. Presence, not identity

    Clippy can wake when you sit down and hold notifications while you’re away, then summarize them when you’re back.

  4. Off means off

    Do Not Disturb and Camera Off suppress visual reactions and camera processing entirely.

The camera module wired to a Raspberry Pi beside a terminal log of the pan tracker, showing face detection, pan angles and thermal safeguards pausing tracking
The pan tracker running on the Pi, with thermal safeguards

07Building it

Design the behaviour before moving hardware.

We built an interaction studio first: a 3D simulator where a reaction is composed step by step. Each step sets an expression, gaze, app pupils, a gesture, a hold, a sound and a voice line, and the whole sequence previews instantly. No hardware is needed to design a behaviour.

The interaction studio: an event library, a 3D view of Clippy on a desk, an OLED monitor and a step editor for a Teams message sequence
The interaction studio: compose, preview and export a reaction

The same sequences then play on the robot, either over an authenticated connection to the Pi or as an offline package with prepared speech and display frames. Speech is generated locally with Piper, so no cloud service is needed.

CAD model of the robot: the paperclip frame, the head with display and camera, and the base
The CAD model
Assembly: the camera and OLED being mounted in the head, and the base with the Raspberry Pi, servo driver and a servo
Assembling the head and the base
PartRole
Raspberry Pi 5Runs the behaviour and playback software
SSD1306 OLED128 × 64 one-bit eyes and app signals
PCA9685 servo driverChannel 0 pans, channel 1 tilts
Camera Module 3Feeds the face-tracking script
MAX98357A amplifierSpeaker playback
INMP441 microphoneWired, with live voice input still to come

Safety is part of the behaviour

An early undervoltage incident shaped the design. Servos run on their own regulated supply, only one axis moves at a time, power to the servos is released once they settle, and a playback has to be explicitly armed. Connecting never moves a servo by itself.

08A workday with Clippy

24 authored steps, 18 spoken lines, about 86 seconds.

To show the system end to end, we scripted a workday with John, a designer. Clippy helps him get back into his design, flags quiet mail with envelope pupils, answers “what does my day look like?”, surfaces an important Teams message, warns him about his Figma AI allowance, goes quiet on request, and finally tells a joke, winks and goes to sleep.

The full workday story, played in the 3D studio with subtitles

Open to new work

Have a complex product that needs to feel obvious?

I turn tangled enterprise and AI workflows into experiences people trust, from research to shipped UX.

Let’s collab
Next project Microsoft Places