Microduck Studio books · Book 1 of 4 · Beginner

Meet the Microduck

How a little robot duck senses, thinks, moves and learns. Ten lessons, each with a pop-up to play.

Who it is for: readers 7 and up, with a grown-up. There is no code, and you do not need a robot: paper Duck Cards, Microduck Studio and Kith are enough.

A body, senses and a brain
Do this, then that
Say it shorter
When it falls
Why walking is hard
Learning by trying
A practice world
Pretend is not perfect
Kind company
Your duck, your idea

How a lesson works: read the left page and make a prediction. Then press the steps on the right page, one at a time, and find out if you were right. Each lesson ends with things to try for real, notes for grown-ups, and where every number comes from.

Lesson 1 · The robot

A body, senses and a brain

The Microduck is a little robot duck, a little shorter than a school ruler is long.

It has a body: two legs to walk on, a long neck, and a head with a beak.

It has senses. A sensor inside its body feels which way is down, a bit like the feeling in your tummy on a swing. And every motor can feel how far it has turned.

It has a brain: a program on a tiny computer inside it. The brain reads the senses, decides what to do, and tells the motors to move. Then it does it all again. And again.

Sense, think, move. Then do it again.

Predict

How many times does the duck sense, think and move in one second?

50 times Just once 5 times

Check your prediction

50 times every second. That is faster than you can blink, so the duck can feel a wobble and fix it before it turns into a fall.

The pop-up

Body. The paper duck stands up: two legs, a body, a neck, and a head with a beak.
Senses. The sensor in its body lights up and points down. Then every motor glows, one after another: each one can feel how far it has turned.
Brain. The brain lights up, and dots carry what the senses felt into it.
Move. The brain sends dots out to the motors, and the duck takes a step.
Again. Sense, think, move, round and round, at the real speed: 50 times every second. Watch the counter.

A paper Microduck: its senses, its brain and its motors, running its loop at the real robot's speed.

Big idea: a robot is a loop. It senses, thinks and moves, then does it all again.

Try it for real

Be the loop. One person is the senses and calls out "tipping left!" or "tipping right!". One is the brain and decides. One is the motors and moves a toy's legs. How many loops can you do in 10 seconds? The duck does 500.

No robot needed to see one move: Microduck Studio drives a duck in physics on an iPhone, iPad or Mac.

For parents and teachers

Sense, think, act is one of the oldest ideas in robotics, and everything later in these books has its shape, so it is worth getting the loop solid here. The duck has 15 motors: 14 move its legs, neck and head, and those are the ones its walking brain drives, while the other one opens its beak.

The paper duck in the pictures is our own drawing: it has the robot's joints, but it is not Pollen Robotics' design.

Something in this lesson unclear, or wrong? Tell us about it.

Where these numbers come from

50 a second: the robot's code runs "a 50 Hz control loop driving fifteen servos" (Pollen Robotics' microduck README), and its walking brains "are trained here at 50 Hz" (microduck_rl README).
14 and 15 motors: "Joints exclude the mouth throughout" (robotd design), and the training code's "Joint layout (14 servos)".
500 is 50 times 10.

Lesson 2 · Duck Cards

Do this, then that

A robot does what it is told, in the order it is told. Duck Cards are one way to tell it. Each card is one thing the duck can do.

Lay cards in a row and read them from left to right. That row is a program: a plan the duck follows, one card at a time.

On the paper board, one Walk forward card moves the duck one square. One Turn card turns it a quarter turn, on the spot.

One Walk card is one square. One Turn card is a quarter turn.

Predict

The duck is in the bottom left corner, facing up. The pond is one square up and two squares to the right. How many Walk and Turn cards get it there, at the fewest?

3 cards 4 cards 6 cards

Check your prediction

4 cards: Walk forward, Turn right, Walk forward, Walk forward. One card to go up, one to turn, and two to go across. A turn is a card too.

The pop-up

The board. A board of 3 by 3 squares, the duck in the bottom left corner facing up, and a pond.
The cards. Four cards land in a row: Walk forward, Turn right, Walk forward, Walk forward.
Run it. The duck does the cards in order, one at a time: up a square, a quarter turn, then across.
Splash. It ends on the pond, 4 cards later.

The paper board and its rule: one Walk card is one square, one Turn card a quarter turn.

Big idea: a program is a row of steps, done in order. Change the order and the duck goes somewhere else.

Try it for real

Print the starter deck and play Be the duck: one person is the duck and does exactly what the cards say, and nothing else.

Then play the pack Get to the pond, six challenges on the same small board. Tape the squares on the floor so the person playing the duck can stand in one.

For parents and teachers

Let the child build the whole row before anything moves. A wrong row is the best part of the lesson: ask "where did the duck go that you did not mean?", and that question is debugging.

Turn cards trip almost everyone at first, because a turn moves the duck nowhere. Standing up and turning on the spot settles it.

Older players can take the same idea to the talent show in Pollen's Arena, where a duck's act is a list of timed commands: plan it as a row of cards such as Sit or stand and Wheee, one beat at a time, with each Look card under the card it changes. Race in Pollen's Arena shows how.

Something in this lesson unclear, or wrong? Tell us about it.

Where these numbers come from

The board rule, "One Walk card moves the duck one square. One Turn card is a quarter turn, where it stands" (Duck Card packs).
The board of 3 by 3 squares is the size of Get to the pond.
4 cards: counted by playing the row on the board.

Lesson 3 · Loops

Say it shorter

Some programs do the same thing again and again. To walk round a square, the duck walks forward twice and turns, then does that again, and again, and again.

Written out card by card, that is a long row of 12 cards. Long rows are hard to read and easy to get wrong.

A Repeat card says it shorter. Write on it how many times. Put the cards to repeat after it, and close them with End repeat. Everything between the two is a loop.

A loop does the cards inside it again and again.

Predict

How many cards does the short row need, counting the Repeat and End repeat cards?

3 cards 5 cards 12 cards

Check your prediction

5 cards: Repeat 4 times, Walk forward, Walk forward, Turn left, End repeat. The duck walks the very same square with 7 fewer cards.

The pop-up

The long row. Twelve cards in a line: Walk forward, Walk forward, Turn left, four times over.
Run it. The duck walks all the way round the square, one card at a time.
Fold it. The cards that repeat slide together. A Repeat card goes in front, and End repeat goes behind.
Run it again. The duck walks the same square. The counter shows each time round the loop.

The same square walked twice: once from the long row, once from the short one.

Big idea: a loop says "again" for you. Same moves, fewer cards, fewer places for a mistake to hide.

Try it for real

Do the deck's lesson Say it shorter: write a dance that does a few cards again and again, then rewrite it with Repeat and End repeat and count the cards you saved.

The pack Loop the course is scored by fewest cards, so loops pay off there.

For parents and teachers

Ask what changes if the dance repeats once more. In the long row it is three more cards, but in the loop it is one number, and that difference is why programmers love loops.

A loop also keeps a bug in one place. Put a wrong card inside it and the duck repeats the mistake four times, which makes the mistake easy to spot, and there is still only one card to fix.

Something in this lesson unclear, or wrong? Tell us about it.

Where these numbers come from

12 cards: three cards for each of the four sides. 5 cards: counted in the short row. 7 fewer is 12 take away 5.
The square, 2 squares on a side, fits the board of 4 by 4 squares drawn here, the size used by the pack Duck maze.

Lesson 4 · Events

When it falls

Walking robots fall. A good program says what to do when that happens.

A "When it falls" card starts a second row. That row waits. It does nothing at all until the duck falls over.

The real Microduck already knows how to get up. Knock it over and it stands itself up again.

A When row waits for something to happen. Then it runs.

Predict

The duck is halfway through its first row when a ball knocks it over. Once it is back on its feet, what does it do?

It finishes its first row first. It starts its first row over again. It does its "When it falls" row.

Check your prediction

It does its "When it falls" row. Once the duck is back on its feet, the second row runs. The rest of the first row does not run. The second row was waiting the whole time, and a fall is the thing it was waiting for.

The pop-up

Two rows. The first row walks. The second row starts with "When it falls", then Alarm and Walk back.
Walk. The duck starts its first row. The second row waits.
Bump. A ball rolls in and knocks the duck over.
Up again. The duck stands itself back up.
Alarm. Now the waiting row runs: the duck sounds the alarm and steps back.

A row that waits for a fall. The getting-up is the duck's own; the second row is ours.

Big idea: a program can wait for something to happen, and already know what to do when it does.

Try it for real

Play the deck's lesson When it falls. The person playing the duck sits down when you say "fall", gets up, and does the second row.

Then argue about it: should the duck do the same thing after every fall?

For parents and teachers

Programmers call a When row an event handler, and a phone is full of them: one runs when a message arrives, another when the screen is tapped.

Getting up after a fall is not a card, though. It is a skill the real robot learned: Pollen Robotics trained one brain that walks and recovers from falls, and another that stands up from lying face down, face up or sitting.

Something in this lesson unclear, or wrong? Tell us about it.

Where these numbers come from

No numbers in this lesson. "Knock it over and it stands itself up" is from Pollen Robotics' microduck README; the trained skills are the VelStand and StandUp tasks in the microduck_rl README.

Lesson 5 · Balance

Why walking is hard

Stand on one foot. Feel the wobble? You are balancing.

Every body has a middle point, where all its weight seems to sit. It is called the center of mass. To stay up, that point has to stay above your feet.

The duck's head is heavy: more than a third of the whole duck. When its head swings, its middle point moves, and the duck has to keep it over its feet.

Walking is even harder. A walking body moves its middle point ahead of its feet on purpose, starts to fall, and catches itself with the next step. Over and over.

Keep your middle point over your feet.

Predict

The duck leans forward more and more. When does it tip over?

As soon as it starts to lean forward. When its middle point passes its toes. When its heavy head goes past its toes.

Check your prediction

When its middle point passes its toes. Leaning is fine while the middle point is still over the feet. Past the toes, nothing is holding it up.

The pop-up

Middle point. A dot appears where the duck's weight sits, with a string hanging straight down from it.
Feet. The ground under its feet glows. That is where the string has to land.
Lean. The duck leans slowly forward, and the string slides toward its toes.
Tip. The string passes the toes, and the duck tips over.
Catch it. Up again. This time, as it leans, it steps forward and lands its foot under the string. That is a step.

Drag the lean slider to tip the duck yourself. A step is a fall that gets caught.

Big idea: walking is falling forward and catching yourself, again and again.

Try it for real

Hang a weight on a string from a toy's middle: a key on a shoelace works. Lean the toy until the string passes the edge of its feet, and it falls.

Then walk across the room very slowly. Notice the moment you start to fall forward, and your foot arriving to catch you.

For parents and teachers

The ground the feet cover is called the support area, and to stand still, the middle point has to stay above it. Walking is harder, because the duck spends part of every step on one foot.

The heavy head matters more than it looks. Its trainers found that the head has to swing while the duck walks: when they punished every little head wobble, standing still scored higher than walking, so the duck learned to stand still.

Something in this lesson unclear, or wrong? Tell us about it.

Where these numbers come from

More than a third: the training code says the head is "280 g" and "38% of robot mass" (the walking task's settings).
The duck that stood still: the same file, "Standing still scored higher, so it stood still."

Lesson 6 · Learning

Learning by trying

How did you learn to ride a bike? Nobody could tell you exactly what to do with every muscle. You tried, you wobbled, and slowly your body worked it out.

Robots can learn like that too. It is called reinforcement learning. The robot tries something. When it goes well, it gets points, called a reward. After many, many tries, it does more of whatever earned points.

Nobody tells it how. It finds out.

Try. Get points for good tries. Do more of what worked.

Predict

A duck in a little maze gets a point each time it reaches the pond. At first it wanders about. After many tries, what does it do?

It finds the short way to the pond. It walks its first lucky path again. It wanders less, but still by luck.

Check your prediction

It finds the short way to the pond. Its first lucky path wandered, but every try that reached the pond left a little trail of "this was good". After enough tries, the trail leads straight there.

The pop-up

Try. The duck wanders the maze, bumping into walls, until by luck it finds the pond and gets a point.
Remember. The squares it crossed just before the pond glow: good places to be.
Many tries. Try after try, the glow spreads back toward the start.
Learned. Arrows appear on the squares, and the duck walks the short way to the pond.

A toy maze. The real duck learns to walk the same way, with far more tries and far more to learn.

Big idea: points for good tries, and lots of practice, turn wandering into knowing the way.

Try it for real

Play hot and cold. One person hides a toy. The seeker moves one step at a time, and the others say only "warmer" or "colder". That is a reward, and the seeker is learning from it.

Then play it with a rule: "warmer" only when the seeker finds the toy. It takes much longer. Rewards that come early help learning.

For parents and teachers

This maze learns with the simplest kind of reinforcement learning, where every square keeps a number for how good it is to stand there. The real duck cannot keep a number for every position of every joint, because there are far too many of them, so it learns a small network instead, and Book 2 opens that network up.

The points in the picture are made up for the toy; the real duck's points are in Book 2, lesson 5.

Something in this lesson unclear, or wrong? Tell us about it.

Where these numbers come from

No real numbers here: the maze is a toy, and its points are ours.

Lesson 7 · Simulation

A practice world

Learning by trying takes an enormous number of tries, and an enormous number of falls. A real duck would wear out.

So the Microduck practises in a pretend world inside a computer, called a simulation. There, falling does not hurt anything, and the pretend duck can be reset in a blink.

Better still, the computer can run 4,096 pretend ducks.

A fast computer can teach the pretend ducks a walk that works in about 1 to 2 hours.

Practise in pretend first, with lots of ducks at once.

Predict

Why practise with 4,096 ducks instead of just one?

So each duck learns a walk of its own. So the others can copy the best duck. So one brain learns from all of them.

Check your prediction

So one brain learns from all of them. All 4,096 ducks try at the same time. Every one of them tries, falls and learns, and all of them share one brain. That brain learns from all their tries at once, so it gets its practice thousands of times faster.

The pop-up

One duck. A pretend duck tries to walk, wobbles and falls. Reset, and it tries again.
Copies. The page fills with pretend ducks: 4,096 of them.
All at once. They all try at the same moment. Some fall, some get a few steps.
Gather. Everything they tried flows into the one brain they share.
Learned. Practice after practice, the whole crowd walks.

Real numbers: 4,096 pretend ducks, and about 1 to 2 hours on a graphics chip for a walk that works.

Big idea: many pretend ducks practising at once learn fast, and nobody gets hurt.

Try it for real

Studio's Learn tab is nine short lessons, from driving the duck to how it is trained, each with a button that opens the real screen.

A grown-up with a computer that has an NVIDIA graphics card can start this exact practice with Pollen Robotics' microduck_rl. Book 2, lesson 6 shows how.

For parents and teachers

The pretend world is MuJoCo, a physics engine that works out how bodies push, fall and slide. What makes 4,096 ducks affordable is a graphics chip, which is built to do thousands of small sums at once, like a class where every child does one sum at the same moment instead of one person doing every sum in turn.

The words "a walk that works" are chosen with care, because 1 to 2 hours buys a usable walk, not the best one.

Something in this lesson unclear, or wrong? Tell us about it.

Where these numbers come from

4,096 ducks and 1 to 2 hours: "~1-2 h for a usable gait at 4096 envs" (microduck_rl README). An env is one pretend duck in its pretend world.
Nine lessons in Learn: Studio's docs.

Lesson 8 · Sim to real

Pretend is not perfect

The pretend world is close to the real one, but it is not the same. A real floor can be more slippery. A real battery gets tired. Every real motor is a tiny bit different from the next.

So while the ducks practise, the pretend world is shaken up on purpose. Some ducks get slippery floors and some get grippy ones. Every duck gets a little push now and then. A duck that learned on all of them is ready for a floor it has never met.

Practise on many different pretend worlds, so the real one is just one more.

Predict

A duck that practised in the pretend world walks a row of 4 Walk cards on a floor it has never met. Each card is meant to go one square. Where does it end up?

Near the fourth square, a little off. Right on the fourth square, as in practice. It falls over before the second square.

Check your prediction

Near the fourth square, a little off. Even after all that practice, a real duck slowly wanders off its path. Each step goes a little wrong, and the error grows the longer it walks. The longer the row, the farther off it ends. For now, a Walk card on a real duck is timed: it walks for two seconds, and it does not land exactly on a square.

The pop-up

Pretend. In the world it practised in, the duck walks 4 squares and lands exactly on the last one.
New floor. On a floor it never practised on, each step goes a little wrong, and the errors add up as it walks.
Shake it up. In practice, the floors change: slippery, grippy, a push from the side.
Ready. A duck that practised on every kind of floor copes better with a new one.

A toy picture of drift: the size of each miss here is made up. Real misses depend on the floor, the duck and its battery.

Big idea: a robot that practised in many slightly wrong worlds copes with the real one.

Try it for real

Walk across the room with your eyes closed, counting 4 steps, and mark where you stop. Do it 5 times. Your marks spread out, just like the duck's.

Tape the paper board on the floor in squares of 30 cm, so the person playing the duck can stand in one.

For parents and teachers

Shaking up the pretend world is called domain randomization. Each pretend duck keeps its own floor grip, body weight and sensor errors for the whole of training, while its balance point, its motors' strength and the little delays in its senses change each time it starts over, and every duck gets pushed now and then.

We tested cards meant to walk one square in Pollen Robotics' own simulator, and published every result. Short rows nearly pass, but long rows do not yet, because small errors add up with time, and the evidence page has every number.

Something in this lesson unclear, or wrong? Tell us about it.

Where these numbers come from

"Walk and Turn cards on a duck are timed for now: two seconds each, and the duck does not land on board squares" (Duck Card packs). The squares of 30 cm are from the same page.
4 cards and 5 tries are the activity's own counts.

Lesson 9 · Living with people

Kind company

A robot that lives with people needs more than legs. It needs good manners.

Kith is a game you play in a web browser, made by the same person as this site. You walk with Microducks through a paper valley, or along a real park path. In Kith's park, the ducks keep the real robot's pace, as measured in simulation. It is slow: about as fast as a very old dog. A good friend slows down to walk with you.

Good manners are skills too, and the duck can practise them.

Keep pace, and wait for each other.

Predict

The duck comes to steps it cannot climb. What is the clever, kind thing for it to do?

Keep trying to hop up, again and again. Wait quietly until someone sees it. Stop and ask a person for help.

Check your prediction

Stop and ask for help. In Kith the duck stops and asks with a quack. The person lifts it over the steps and sets it down, and they carry on together. Waiting, asking and helping are all skills the duck can practise: keep pace, wait, ask for help, and help back.

The pop-up

Together. A person and a duck walk side by side, at the duck's pace.
Too fast. The person hurries ahead. The gap grows, and the duck cannot keep up.
Wait. The person stops and waits. They walk on together.
Steps. The duck reaches steps it cannot climb. It stops and asks.
On they go. The person lifts the duck over the steps and sets it down, and they walk on together.

A walk in Kith, sped up. Measured in simulation, on the level, the real robot walks about 0.12 metres a second, about the length of your hand each second.

Big idea: good company is a skill: match the pace, wait, ask, help.

Try it for real

Play Kith in a web browser. Walk as the traveler, or choose to walk as a Microduck and see the world at its pace.

Then go for a walk with someone much slower than you, a small child or an old dog. What did you do differently?

For parents and teachers

Kith gives points for keeping company, a little for every moment the duck stays near the traveler. The walks people choose to give become an open set of examples that a real Microduck could learn from.

Kith has no chat and no names. Everyone is renamed in what it records, and anyone can choose not to share their play.

Something in this lesson unclear, or wrong? Tell us about it.

Where these numbers come from

The pace: the real robot as it walks today, "measured in simulation (about 0.12 m/s on the level and 0.15 at most...)". The old dog walks "0.14 m/s, a Microduck's own pace" (Kith's notes for agents).
The points for company are listed at Kith's rewards.

Lesson 10 · Making

Your duck, your idea

The best Duck Cards are the ones you invent. The deck comes with blank cards, for moves you make up. Each one has its own little square code.

To make a move, pose the duck a few times. Each pose is a keyframe: where every joint should be, and when. The duck moves smoothly from one keyframe to the next.

Every joint has a limit, like your elbow. A good move stays inside the limits. Microduck Studio checks every angle in a move against the real duck's joints.

A move is poses in time, inside the joints' limits.

Predict

You try to tip the duck's head sideways much farther than it can go. What happens?

The whole duck tips over sideways. The head stops at its limit. Studio throws the whole move away.

Check your prediction

The head stops at its limit. The duck's head can tip about 20 degrees each way. Studio checks every angle in a move against the robot's joints, so it never asks for more. The robot's own software only catches broken orders, so staying inside the limits is the move's job.

The pop-up

Pose one. The duck lifts its head. Click: a keyframe drops onto the timeline.
Pose two. The duck bows. Click: a second keyframe.
Play. The duck moves smoothly from one pose to the other and back.
Too far. Ask the head to tip past its limit. The joint stops at the edge and glows red.
Name it. A blank card gets the move's name: Bow.

Drag the head-tip slider past the limit to see what the joint does.

Big idea: you can invent a move: poses in time, inside the limits. Then give it a card.

Try it for real

In Studio's Play screen, Mimic puts the duck in the pose of whoever stands in front of the camera. Strike a pose, keep it as a motion, then draw a card for it and give it a name.

Share your class's deck back: photograph it and open an issue on the site's repository.

For parents and teachers

A keyframe animation does not know about gravity. Studio's preview "interpolates your poses and times; it does not calculate whether the robot could perform the move", so a move can look fine on screen and still topple a real duck, and the physics bench is where to find out, in simulation, before a real duck tries it.

The robot's safety code holds each motor inside its range of travel, but that is not the same as what is safe for each joint, so staying inside the joints' limits is the move's own job. Its trainers never ask the head to go all the way, either: their head commands stop about a tenth short of each joint's real range.

Something in this lesson unclear, or wrong? Tell us about it.

Where these numbers come from

About 20 degrees: "head_roll : ±0.31 rad (limit ±20°)", and the commands' "~10% safety margin" (the walking task's settings). The robot's control loop will "clamp to actuator range" (robotd design).
Studio checks angles: "Every angle it writes is checked against the robot's own joint travel before you see it" (Microduck Studio). The preview sentence is from the docs.

Next: Book 2, How a Microduck Learns, opens the duck's brain: what it senses as numbers, what it does, and how 4,096 pretend ducks train it. All four books are on the shelf.