Train your first duck on an iPhone or iPad

What you need

• An iPhone or iPad on iOS or iPadOS 17 or later, with Apple's free TestFlight app.
• For the rented training only, a free Hugging Face account with credit on it. Hugging Face Jobs runs for any account with a positive credit balance, which you add with a card in your billing settings; a PRO plan's monthly credits count too. The card Studio rents, one NVIDIA L4, is $0.80 an hour, billed by the minute.

Everything else on this page is free, and runs on the device: MuJoCo, the physics engine Pollen trains in, runs inside Studio.

Install Studio and turn on the tools

1. Install Apple's TestFlight app, then join the Microduck Studio beta. There is no App Store release yet.
2. Open Learn and work through it; most lessons have a button that opens the real screen. The training tools stay hidden until its last lesson, Show every tool, turns them on, and Settings, then Detail, turns them on as well.

Be the judge

What does a good walk look like to you? In Studio, under Make, Compare plays two networks walking under the same command, and you pick the better one.

Your picks are the human feedback in RLHF: labels for what a good walk looks like to you. With a hundred picks between networks, Studio can learn your taste on the device and use it for a rented training run: see Train from your picks.

Train on the device

Under Train and measure, these improve a network on the device itself, against physics it runs, with nothing sent anywhere:

• Tune it on this phone nudges a network toward a command, and keeps what scores better on Pollen's own reward.
• Search its weights on this phone searches every number in a network, all 197,774 of them in Pollen's walker. Each generation tries pairs of small random changes, scores every one, and steps toward the better change of each pair.

Search a move's keyframes, beside them, works on a move rather than a network: it starts from a published stairs-challenge move and searches its poses and timing on the challenge's own grid. Studio says plainly that this is not training, since it keeps the version that scored best.

And in Studio, under Play, Train a duck to shoot builds the controller that walks the duck to a ball and joins Pollen's walker to its kick, then trains its numbers on goals scored.

None of these learns a network from nothing: they start from one that already walks or kicks. The device is a bench already, and for a faster one, run the free desk bench on a Mac, PC or Raspberry Pi of yours.

Train with Pollen's trainer on a rented card

Train a network on Hugging Face, under Train and measure, runs PPO on a rented card with a reward you set on the device, and brings the network back. PPO is the reinforcement learning Pollen trains the Microduck's own networks with.

In Studio's own words, this training is simulation only: it fine-tunes Pollen's own walking network, distilled to a smaller student, it is not a gait learned from nothing, and a pass is not a claim about a physical duck.

1. On Hugging Face, add credit in your billing settings, and make a token at huggingface.co/settings/tokens: a Write one, or a fine-grained one with the Jobs permissions and write access to your repositories. The job is started with it, and uploads its records with it.
2. In Studio, open Train a network on Hugging Face. Paste the token under Your Hugging Face token and tap Check this token, and Studio says which account it signed in as, the account jobs are billed to. Studio sends the token as a job secret, which Hugging Face encrypts and keeps out of the job's visible settings; the job uses it to upload its records and does not print it.
3. Under The reward, name the run and leave Full run off. Your first is a pilot: 100 iterations, about a quarter of an hour, judged only on whether it runs. A full run is 1,500 iterations, about an hour and a half. Pay for walking and Pay for turning set how much the reward pays for going, and nothing pays for standing still; leave them at 1 for now.
4. Records dataset is filled in as your account's duckbatch-records: the job writes what it trained and measured there. Hugging Face makes a new dataset public, so to keep yours private, create a private dataset with that name on Hugging Face before your first run.
5. Tap Train on Hugging Face. The screen says Waiting for a GPU, then Training, and you can leave it: the job carries on at Hugging Face, and the screen picks it up when you come back. Open the job and its log shows it on Hugging Face, where you can also cancel it.
6. When it says Finished, tap Add the trained network to Behaviours. It brings back the trained network and the one it started from, and asks you to Compare the two: the trained one should win at least 12 of 20 picks. A pilot only has to run. A full run is also judged on pass lines written into its job file before it started (Show the job file, on the same screen), which the job measures itself.

What it costs. One L4 at $0.80 an hour, for as long as the job runs, and Studio stops a pilot after an hour and a full run after three, so neither can run on.

Train from your picks

With a hundred picks in Compare, Train from your picks on the same screen offers a switch to train from them. Studio fits your taste on the device and scales each part of Pollen's reward by it, between ×0.5 and ×2, so the reward is still Pollen's, weighted your way.

Kick duels in Compare do the same for a kick, and Train a kick from your picks starts from Pollen's own kick. Either run is judged the same way, by you, against the network it started from.

Drive what you trained

On the Play tab, tap Show the controls. Below the Bench picker, Put one of your networks on this bench sends a network from Behaviours to the bench, and the sticks steer by it.

When your duck arrives, the same app drives it through a small bridge on a computer next to it, and can put a policy file on that computer's disk: see Connect a real duck.

When something goes wrong

The training tools are not in Studio

They are hidden in Simple, so in Settings, then Detail, choose Everything, or finish Learn.

Train on Hugging Face stays grey

The line under it says why: check the token first, name a records dataset, or wait for the run before it to finish.

Hugging Face refuses the job

Studio shows Hugging Face's answer. A 403 means the token can sign in but not start Jobs, so make one as in step 1 of the rented training and paste it in again. An answer about credit or billing means the account needs credit, which you add in your billing settings.

The job stopped with an error

Its log on Hugging Face says where: Open the job and its log. A token that cannot write to your repositories passes Check this token and fails at the end, when the job uploads its records, so make one as in step 1.

Then

The books explain what is happening while it trains. On a Mac, the same app does everything here, and the Mac guide adds Pollen's own workshop, which trains for the Arena's events. Stuck on a step, or found a better way? Tell us, and say which step.