5-2 From Recognizing to Creating — Flipping the Arrow of the Feature Table

1Mission

The second order — "Create a throw"

The litterer has been caught. But HQ still has a second question. Using the same numbers we used to recognize motions, can we now create one?

[SYSTEM] 03:40 · 5-1 completion report received. Next order: generate stick-figure motion. Attached: 100 throwing scenes.

the DoctorA discriminative model takes a motion and gives a label. Now we flip the arrow. A generative model takes a label and gives a motion!

the AssistantWe have as many as 100 throwing scenes… If we average them all, won't we get a "typical throw"?

the Fruit Fly ResearcherTry it. I bet the arms will fold and the power will drain away. The question is how you mix the numbers. And we need the Evaluator to score it, too.

the Lab RobotI have prepared four generators: average, dice, copy, and knob (latent variable) sampling[note]. The Evaluator has five scoring items.

A wall covered in stick-figure motion cards; the Assistant examines one card with a magnifying glass while the Fruit Fly Researcher and a box-shaped robot inspect cards on a conveyor belt — The motion data warehouse. Each card is one scene: 1,196 numbers.

2Flip

Flip the arrow — discriminative and generative models

The same three boxes; only the direction of the arrows differs. Try switching between the two directions.

Direction

Scene

Discriminative model (recognizing)

Question: "What is this motion?"
Direction of information: many numbers → few numbers. Reducing is easy.
Answer: one. The same motion always gets the same label.
Did it do well? Just check whether it matches the correct answer.

Generative model (creating)

Question: "Show me a motion like this."
Direction of information: few numbers → many numbers. The blanks have to be filled in.
Answer: many. Each time, a slightly different throw should come out.
Did it do well? There is no correct answer. Whether it looks real, whether it is diverse, and whether it was copied must each be measured separately (Section 6).

Explanation for beginners

A common summary is that a discriminative model learns "the probability of the answer given the input," while a generative model learns "the probability of what the data itself looks like" [R1]. If you know the shape of the data, you can draw new data that follows that shape. Today's generative AI (GANs [R2], VAEs [R3], diffusion models [R5]) all tackle this "learn the shape, draw new points" task in different ways.

3Motion is a table

A motion is a table — 46 rows × 26 columns, 1,196 numbers

Here a stick-figure scene from 5-1 is spread out as a table. These 1,196 cells are exactly the blanks a generative model must fill in. Hover the mouse over the cells.

Scene

The 1,196 numbers are tied together

The cells of the table are not random. In the same frame, the "right elbow" and the "right wrist" are always 0.16 apart (bone length), and the values in two neighboring frames are almost the same (motion is continuous). A discriminative model can give an answer without knowing these ties, but a generative model that fails to keep them gives itself away immediately. In the next section we break them ourselves.

4Mix

Mixing two motions — averaging coordinates folds the arms

The easiest way to "make a new motion" is to mix two existing ones. Try changing what gets mixed. The bars below show how far each of the ten bones in the current frame strays from its original length (the longer the bar, the more it is distorted; red above 5%).

AB

Share of B50%

What to mix

Results of Code 1 (50:50 mix)

Mixing throw + wave by averaging coordinates gives an average bone-length error of 11.4%, and 99.9% at the worst moment. At the moment when the two wrists were on opposite sides, their midpoint almost overlapped the shoulder, and the forearm vanished. Drop-in + walk-by gives 1.9% (worst 25.7%), and throw + pick-up gives 1.4% (worst 20.4%). Averaging angles gives 0.0% for all three pairs. It is the same "average" in name, but what you average decides the result.

5Data

100 throws — same name, different shapes

What a generative model learns is not a single scene but the shape in which scenes are scattered. Here are 100 throws, varied in speed, position and size in the same way as in 5-1. Click a dot on the left to pick a scene.

6Generators

Four generators and the Evaluator's score sheet

We make six new throws from the 100 throws. The Evaluator checks five things: does it look real (bone length, jitter, throwing power), is it diverse, and was it not copied. Switch generators and press "Generate again."

Evaluator stick figures in neckties, holding clipboards and whistles, pointing at a star-rating board and giving a thumbs-up — The Evaluator. In real training, a loss function or a discriminator plays this role.

Generator

Code 2 score sheet — 20 per generator

The screen above redraws six at a time in the browser, so its numbers differ slightly. The table below is the result of actually running Code 2 (100 training throws, 20 generated). "Novelty" is the distance to the nearest training scene (in units of body height, root mean square over coordinates).

Generator	Bone-length error	Jitter	Max wrist speed	Diversity	Novelty	Evaluator's one-line review
Training data (reference)	4.7%	0.0288	2.55	0.126	0.000	Because of pose-estimation jitter, even the real data's bone lengths wobble by 4.7%
A One average	1.7%	0.0045	1.36	0.000	0.033	Smooth, but always the same, and half the power is gone
B Dice per coordinate	44.2%	0.2659	10.00	0.149	0.109	Diverse, but not human. The wrists teleport
C Pick one and copy	4.7%	0.0300	2.75	0.125	0.000	Looks real, but it is a copy
D Sample 3 knobs	0.0%	0.0125	2.71	0.123	0.022	Passes all five items. But it knew the knobs in advance

Using the Evaluator in training

The Evaluator in this part only scores the results after they are made. Real generative models feed the scores straight back into training. They either adjust the weights to reduce a loss function, a single number measuring how different the results are from the training data (VAE [R3]), or they set up a discriminator that tells real from fake as the Evaluator and make it compete with the generator (GAN [R2]). In effect, the discriminative model from 5-1 comes back as the Evaluator in 5-2.

7Next mission

Where do the knobs come from? — On to the next mission

D won because it sampled from 3 numbers (speed, position, size) instead of 1,196. However you change those 3 numbers, they pass through the skeleton (fk) and unfold into a human shape.

What we learned in this part

Generation is sampling from a distribution: instead of a single correct answer, it learns the shape of the believable answers.
What you mix and sample is half the battle: in coordinates the motion breaks; in angles (knobs) it holds together.
Scoring has three branches: realism, diversity and novelty. Look at only one, and A (safe and bland) or C (copying) wins.

Open problems

Who tells us the knobs? Real motion data does not come with knobs like "speed ×1.03, size ×0.98." Motions are too varied for people to build angle models by hand. Such hidden knobs are called latent variables.
So the next question: just by looking at the 1,196 numbers of data, can we find a few knobs on our own? The part that folds 1,196 → a few is the encoder, and the part that unfolds a few → 1,196 is the decoder.

Stick-figure film strips and waves of numbers get sucked into a giant funnel and drop out as one small capsule; the Assistant measures the capsule with calipers while the Doctor gives a thumbs-up — Next mission preview · The funnel that folds motion

the DoctorWe put the 1,196 numbers into a funnel and fold them into a capsule of just a few numbers. Then we unfold them again from that capsule. The capsule's numbers become the knobs!

the AssistantIf we fold it and unfold it and the original motion comes back… that means the whole motion fits in that capsule?

the Fruit Fly ResearcherThat's 5-3: the encoder and the decoder. And the littering detection from 5-1 will get better along the way.

Colab code — run this page yourself

Both code notebooks use only libraries preinstalled in Colab. After "Download notebook (.ipynb)", open it in Colab via File → Upload notebook, or copy the code and paste it into a new notebook cell.

Code 1 · A motion is a table — why mixing breaks itLinked to Sections 3·4 · numpy, matplotlib · under 1 second on CPU

We added a bone-length check (bone_error) and mixing (blend) to the scene functions from 5-1. Three pairs are mixed by coordinate averaging and by angle averaging, and their bone-length errors are compared. To use your own data, put real motion coordinates in the two scene slots of blend (the fact that only coordinate averaging is possible there is exactly the problem of this part).

Output (run on 2026-10-05, numpy 2.4 · matplotlib 3.10 · CPU). It uses no random numbers, so it gives the same result every time. Three graphs (the motion table, the coordinate average and the angle average) are also produced.

Code 2 · Four motion generators and the Evaluator's score sheetLinked to Sections 5·6 · numpy · about 10 seconds on CPU

Builds generators A–D from the 100 throws, samples 20 from each and scores them. The score sheet under Section 6 is made directly from this output. To test a new generator, pass a list of scenes shaped (46, 13, 2) to judge("name", list of results).

Output (run on 2026-10-05, numpy 2.4 · CPU). The seed is fixed, so it gives the same result every time.

AI prompts — using this code as a starting point

These are prompts to paste into AI assistants such as ChatGPT, Qwen, or Claude. When you press "Copy," the selected code is attached after the prompt. Just replace the parts in 〔〕 with your own situation.

Code to attach

The AI's answer is only a starting point. Always check suggested papers, figures and library functions against the original sources and official documentation. When using real human motion data, first check filming consent and the dataset's terms of use (such as whether commercial use is allowed).

Glossary

The same dictionary as the yellow dotted underlines in the text. Click a section name to jump to where the term first appears.

Analogy collection — and where the analogies break

Analogies are a first step toward understanding, but they can also plant misconceptions. Analogies to human perception are marked by strength. Direct there is evidence that it actually does the same thing · Partial only some properties are similar · None no known counterpart

Concept	Everyday analogy	Human perception analogy	Where the analogy breaks
Discriminative vs. generative	Grading a test (is the answer right?) vs. setting a test (writing believable new questions)	Partly For people, recognizing a face and drawing one are different skills	A test setter writes questions with intent, but a generative model just samples following the shape of the data
Distribution	How the heights of students in a class cluster together	None	Height is a single number, but a motion is 1,196 numbers moving together, so its shape cannot be drawn by eye
Coordinate average	Overlaying two people's photos semi-transparently	None	Overlaid photos still show both people, but a coordinate average makes one person halfway between them, and that person's arms are folded
Latent variable (knob)	A few volume and tone knobs on an audio mixer	Partly Remembering a motion in a few words, like "fast, big"	Mixer knobs come with labels, but knobs found by learning have no names and mix several meanings (5-3)
Evaluator	Judges at an audition	None	Judges have tastes, but the Evaluator in this part looks only at five fixed numbers. It misses any awkwardness the numbers do not capture

About the author and references

E-1 About the author · E-2 Academic references · E-3 Sources of figures and numbers

E-1 About the author

Ahn Sangsun (CEO, M-Robo Co., Ltd.)

After working at the Korea Electrical Safety Corporation and the Maeil Business Newspaper, he founded M-Robo Co., Ltd., a spin-off venture of Maeil Business Newspaper, where he currently serves as CEO. M-Robo began as an in-house venture at a news company and is a media venture that has provided a robot reporter service (StockBot) to Kookmin Ilbo and carried out numerous projects for the Korea Internet & Security Agency, the Korea Press Foundation, the Korea Communications Agency, and others. It now runs its own AI news service, Money-Robo. While serving as CEO, he won the Asia Open Data Hackathon and twice received the Minister of the Interior and Safety Award for contributions to data entrepreneurship, and he is the inventor and registrant of 5 patents in artificial intelligence and blockchain. He is the author of "AI Crime and Self-Defense" and has developed e-learning content on fintech, statistics, and artificial intelligence for the Online Lifelong Education Institute of Korea University of Technology and Education. He holds a Ph.D. in Economics (Dongguk University, microeconomics and mathematical economics). He teaches data and AI courses at several universities, including Kookmin University and Hoseo University. Full profile

Lectures

Ahn Sangsun (2025). "Data Science for Business" lecture notes. Department of AI Convergence, Open Cyber University of Korea.
Ahn Sangsun (2025). [K-Digital] AI Natural Language Processing. STEP (Smart Training Education Platform), Online Lifelong Education Institute, Korea University of Technology and Education.
Ahn Sangsun (2024). [K-Digital] Fintech Data Analysis Basics. STEP (Smart Training Education Platform), Online Lifelong Education Institute, Korea University of Technology and Education.

STEP course links: link.step.or.kr/1qyVoJ · link.step.or.kr/M9na9N · link.step.or.kr/brqEyc
If the links do not work because of a site redesign, please visit www.step.or.kr.

Papers and presentations

Ahn Sangsun (2025). A study on a household expenditure prediction model using the Korean Retirement and Income Study panel data: An AI model-based approach. Future Society, 16(2). Full text
Ahn Sangsun (2025). A study on a household expenditure prediction model using the Korean Retirement and Income Study panel data: An AI approach. Presentation at the KReIS Panel Conference (2025.05.13), National Pension Research Institute, National Pension Service. Slides
Ahn Sangsun (2025). A study on an algorithm for detecting suspected drug transactions based on SNS text: Focusing on pattern and context analysis. Future Society, 16(1). Full text
Ahn Sangsun (2024). Statistics-based countermeasures against adversarial data for securing anomaly detection models in finance: Empirical analysis and simulation results. Future Society, 15(3). Full text
Ahn Sangsun (2024). An AI-based solution for detecting SNS posts based on virtual character traits. Poster presentation at the 2024 Summer Newmedia Party (2024.06.19), Seoul Media Institute of Technology.
Ahn Sangsun (2024). Machine learning models using industrial accident panel data. Poster presentation at the 10th Panel Study of Workers' Compensation Insurance Conference (2024.05.17), Labor Welfare Research Institute, Korea Workers' Compensation & Welfare Service.
Ahn Sangsun (2024). Analysis and forecasting of virtual asset time-series data: Comparing time-series models and AI models, and suggesting improvements. Future Society, 15(1). Full text

E-2 Academic references

[R1] Ng, A. Y., & Jordan, M. I. (2002). On discriminative vs. generative classifiers: A comparison of logistic regression and naive Bayes. Advances in Neural Information Processing Systems 14. — Section 2
[R2] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. Advances in Neural Information Processing Systems 27. arXiv:1406.2661 — Sections 2·6
[R3] Kingma, D. P., & Welling, M. (2014). Auto-encoding variational Bayes. ICLR 2014. arXiv:1312.6114 — Sections 2·6, next part 5-3
[R4] Holden, D., Saito, J., & Komura, T. (2016). A deep learning framework for character motion synthesis and editing. ACM Transactions on Graphics, 35(4), Article 138. doi:10.1145/2897824.2925975 — Section 7 (representation learning that folds motion)
[R5] Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems 33. arXiv:2006.11239 — Section 2

E-3 Where the pictures and numbers come from

The scene illustrations (motion data warehouse, generative learning, the Evaluator, the encoding funnel) are pixel art made by the author
All motion scenes are the same synthetic data as in 5-1. The numbers under Section 4 and the score sheet under Section 6 are results of actually running Codes 1·2
The screens in Sections 5·6 are redrawn in the browser in the same way, but with different random numbers from Code 2, so the numbers differ slightly

E-4 Notes

[note] Knob (latent variable) — the analogy this series uses as an easy name for a latent variable. It means a small set of numbers that change the whole data together when you turn them. English-language explanations also often call each dimension of a latent space a "knob," "dial" or "slider" as a figure of speech. The formal term in papers is latent variable, and each knob the data finds in 5-3 is a principal component. Unlike a real knob, a single one can mix several meanings.

Tips for educators

A teaching guide for instructors, on-the-job (OJT) trainers and tutors. Times are estimates, so adjust them to your actual class.

Audience	University students and working professionals who have finished 5-1. Planners and practitioners who want a feel for what generative AI "learns"
Prerequisites	The 13-point stick figure and six kinds of scenes from 5-1; mean and standard deviation
Learning goals	① Explain the difference between discriminative and generative models in terms of the arrow's direction and "the number of answers" ② Give an example of why averaging coordinates breaks bone lengths ③ Evaluate generated results on three branches: realism, diversity and novelty
Estimated time	Reading 30–40 min · Animation demos 20 min · Colab practice 25 min (Code 1 10 min, Code 2 15 min)

Three ways to run it

Lecture, 50 min

Concept-focused

Intro 5 min (Sections 1·2, flipping) → Section 3 motion table 8 min → Section 4 mixing experiment 12 min → Section 5 data cloud 7 min → Section 6 four generators 13 min → check questions 5 min.

On-the-job training, 90 min

Focus on evaluation criteria

Section 2, 15 min → apply the Section 6 score sheet to your own work's generated outputs (documents, images, motions) and build "realism · diversity · novelty" criteria, 30 min → discussion on "choosing a representation" with Section 4, 15 min → run Code 2 and add a new criterion, 30 min.

1:1 tutoring

Pace by level

Beginners: Sections 2·4·6 only. Intermediate learners: have them build generator E (mixing two training scenes by averaging angles) in Code 2 themselves and add it to the score sheet.

Teaching points by section

Section	Question to pause and ask	Hands-on tip
2 Flip	"How many correct answers does creating have?"	Switch back and forth between Recognize → Create
3 Motion is a table	"What rules between cells in this table must not be changed freely?"	In the throw, hover over the columns from 0.8 to 1.2 seconds
4 Mix	"We mixed half and half — why did the arm disappear?"	With throw + wave and coordinate averaging, find the frames where the red bars grow long
5 Data	"If we draw a new point from this cloud, what kind of throw will come out?"	Click near the boundary between the red and green dots
6 Generators	"On which scoring item do A, B and C each fail?"	Press "Generate again" two or three times for each generator

Top 5 common misconceptions

Misconception	Correct explanation	Scene to show
A generative model produces one correct answer	It samples one from a distribution of believable answers. It is normal for it to differ every time	Section 2, Section 6 "Generate again"
The average is the safest generation	The average has zero diversity, and averaging motions with different timing drains the power (wrist speed 2.55 → 1.36)	Section 6, generator A
The more numbers, the better the representation	The 1,196 numbers are tied together. A few numbers that keep those ties (angles, knobs) are better for generation	Section 4 angle average, Section 6 generator D
Being identical to the training data is the best generation	Copying has zero novelty. It also risks putting out the individuals in the data as they are	Section 6, generator C
If it looks real, the evaluation is done	Realism, diversity and novelty must be checked together. Look at only one, and A or C wins	Score sheet under Section 6

Check questions (click to expand the answers)

Q1. How many numbers does it take to represent a 3-second (15 fps) scene with 13 joints? What if it is extended to 5 seconds?

3 seconds is 46 frames (including 0 s) × 26 = 1,196 numbers. 5 seconds is 76 frames × 26 = 1,976 numbers.

Q2. An arm of length 0.33 attached at the shoulder (0, 1) points horizontally left (−0.33, 1) in one scene and horizontally right (0.33, 1) in another. If you average the hand coordinates, what is the arm length?

Hand = (0, 1), the same point as the shoulder. The arm length is 0, a bone-length error of 100%. The "worst 99.9%" in Section 4 is exactly this kind of moment. If you average the angles instead (the average of −90° and +90° is 0°), you get an arm of length 0.33 hanging straight down.

Q3. Generator C (pick one and copy) passes both realism and diversity. Why is it not a good generator?

Its novelty is 0. It only memorizes and hands back the training data; it cannot make new motions. If the data contains real people's motions or faces, it also causes the problem of putting out personal information as it is.

Q4. Why is generator A's max wrist speed (1.36) lower than the training data's (2.55)?

The moment of the throw (timing) differs from scene to scene, so averaging coordinates spreads the fast swing thinly over many frames. The peak gets smeared out, and the result is a throw with no power.

Q5. Generator D passed all five items, so why did we say it is "hard to use in practice"?

D knew in advance the knobs that made the scenes (speed · position · size). Real motion data does not come with such knobs. Finding the knobs just by looking at the data is the job of the encoder and decoder in 5-3.

Colab assignments

Basic

In Code 1, change the mixing ratio to 0, 0.25, 0.5, 0.75 and 1, and make a table of the bone-length error of throw + wave under coordinate averaging. Write two sentences on which ratio gives the largest error and why.

Challenge

Add generator E, "pick two training scenes and average their coordinates," and F, "sample the 3 knobs outside the training range (size ×1.6)," to Code 2 and score them. Report what problem F's results have even if they pass realism (where does the can fly?).

Adjusting difficulty and going further

Beginners	Grasp just one thing firmly, "coordinate average vs. angle average" in Section 4, and you get half of this part
Advanced	Have students research and present how GANs and VAEs bring the "Evaluator" into training. Check the papers in the original
Next episode	5-3 Encoder and decoder — finding the knobs (latent variables) on our own, just from the data