5-1 CCTV Littering — Turning People into Stick Figures to Catch Them

1Report

A report at 2 a.m. — "Show me who dumped it"

A CCTV control room has dozens of screens. No one can watch every screen all night. The lab has received a single mission: make AI spot the "motion" of dumping trash first.

[SYSTEM] 02:13 · CAM-07 alley · 1 report of suspected illegal dumping. Attached: 3 seconds of footage, one page of control-room logs.

DoctorA single video has millions of numbers. If you try to find "littering" in it directly, you'll get lost. Let's set an order: find the person → stick figure → movement features → decision.

AssistantA stick figure? So we drop the face and keep only the skeleton? Then we don't even need to know who's who.

Fruit Fly ResearcherThere's a trap. Someone waving their hand hard and someone throwing a can look equally fast if you only look at the wrist. You have to check where the can landed, too.

Lab RobotI have prepared six kinds of scenes and 300 test scenes. I'll try out two rules and three learned models.

On the left, a CCTV screen in which a stick figure throwing a can in an alley is marked with a red box; on the right, a control-room operator stick figure pointing at a warning on the monitor, with a delighted Doctor and the Fruit Fly Researcher — The control room's warning screen. Sections 2–6 build the path to this scene step by step.

Sections 2 · 3Screen → person → 13 joints

Find the person among 6 million numbers and reduce the pose to 26 numbers

Sections 4 · 5Time → 4 features → decision

Summarize 46 frames of movement as 4 features and compare rules with learning

Sections 6 · 7Control room → limits

Live scores and abstaining, and where this method goes wrong

2Screen = numbers

A CCTV screen is numbers — 6 million of them, with no labels

What the computer receives is not a photo but the intensity (0–255) of red, green, and blue in each cell. Switch the view mode and hover your mouse over the screen.

View

Cell size

Explanation for beginners

One frame of Full HD CCTV is 1,920 cells wide × 1,080 cells tall, and each cell holds three numbers: red, green, and blue. 1,920 × 1,080 × 3 = 6,220,800. At 30 frames per second, that is 186.62 million numbers every second. Nowhere among these numbers is there a label saying "person" or "trash can." So the first step is to find what is where in the pile of numbers. How a photo becomes a matrix of numbers is covered in detail in 4-2.

3Finding people

Find the person and turn them into a stick figure — object detection and pose estimation

There are two steps. First, object detection finds the boxes (bounding boxes) around the person, the trash can, and the can. Then pose estimation finds the coordinates of 13 joints inside the person's box. Try turning the layers on and off.

Night-time alley CCTV view. A glowing stick figure stretches out its right arm and throws a can, which flies along a dotted line toward the trash can on the left

The coordinates were read directly from the joint points in the illustration (1402 × 1122). The top left is (0, 0), and y increases going down. Because we are facing the person, the arm on the left side of the screen is their "right arm."

Layers

Why a stick figure?

It's small: One pose is 26 numbers. The same screen has more than 6 million pixels.
Less affected by clothing and lighting: Black hoodie or white shirt, daytime or under a streetlight, the arm angle comes out as the same number.
It doesn't have to keep faces: What the decision needs is the motion. Storing only joints reduces information that could identify individuals.

What you lose instead

Objects: A stick figure has no can and no trash can. So the result of object detection (the can's position) must be carried along with it. In section 5, this turns out to be decisive.
Occlusion and darkness: If an arm is hidden by the body or it is too dark, joints are missed or placed in the wrong spot.
Hand shape: The 13 points have no fingers. Whether the hand is "closed or open" has to be checked separately.

What tools are used in practice?

For object detection, the YOLO family [R3] is widely used; for pose estimation, OpenPose [R1] or MediaPipe [R5]. For joint numbering, the 17-point scheme of the COCO dataset [R2] is the de facto standard. This page uses 13 points: the COCO 17 points minus the two eyes and two ears, with the head represented by the single nose point.

4Over time

Unrolling the stick figure over time — motion is a sequence of numbers

Littering is not a single frame but movement over 3 seconds. Pick from six scenes. The yellow dotted line is the right wrist's trajectory, and the graph below shows wrist speed and the can's state.

Scene

Where did these scenes come from?

The six scenes are not real footage but synthetic data made by changing joint angles over time. They use units where body height = 1 (based on a height of 1.7 m), and a thrown can flies in a parabola under gravity. The function in Code 1 and this figure perform the same calculation. With real footage, the pose estimation from section 3 would produce these coordinates for every frame.

5Rules vs. learning

4 features, rules vs. learning — a fast arm does not mean littering

We reduce 1,196 numbers to 4 features chosen by people. Then we decide in two ways: rules set by people, and a model learned from data. Move the threshold and find where the rules go wrong.

Rule

Wrist speed threshold2.0heights/s

The Code 2 experiment — 300 test scenes, five detectors

We varied the six scenes in speed (×0.8–1.25), position, and motion size (×0.75–1.25) to make 600 training and 300 test scenes, and added jitter (pose estimation error) to the joint coordinates. The correct answer is set by the outcome: if the can remains on the ground outside the trash can, it is littering. So even "Throw far" is normal if a hard throw lands in the can (only 52% of the throws in the test set are littering). The numbers in the cells are the share of that scene judged as littering, and those that differ from the correct answer by more than 5 percentage points are marked in red.

Rule 1 (60.7%): It flagged 94% of hand waves as littering (false alarms). It missed 98% of slow drops.

Rule 2 (76.7%): Adding "did it let go?" removed the hand-waving false alarms. But it still misses drops, and it even flags normal scenes where the can was thrown into the bin.

4 features + logistic regression (99.7%): The learned weights are [speed −0.33, release 1.16, distance outside the bin 3.93, moving away 1.19]. Wrist speed actually got a small negative weight. The strongest clue the model picked is "did the can remain on the ground outside the bin?"

MLP fed raw coordinates (92.3%): It learned the "shape" of throwing, but it also raised alarms for throws that landed in the bin (19 false alarms). Adding 2 object coordinates did not help. Among 1,288 numbers, finding "is the can's x-coordinate in the last frame outside the bin?" from only 600 training examples is hard.

How to read these results

If you choose features well, the model can be simple. The 99.7% is because the feature "distance outside the bin" is almost the same as the definition of the correct answer. In exchange, this feature relies entirely on object detection finding the can and the trash can accurately.
Learning from raw coordinates means people don't have to choose features, but it needs much more data. Real research reduces this problem with graph neural networks (ST-GCN and others) that use the connection structure of the joints directly [R4].
These numbers are results on synthetic data. In real alley footage, occlusion, darkness, and multiple people lower the scores of every detector. Look at the pattern of "which scenes it got wrong" rather than at the numbers themselves.

6Control room

Live control room — a score recalculated every frame

This uses the weights learned by the logistic regression in Code 2 as they are. Using only the frames seen so far, it recomputes the 4 features and assigns a score. At 0.6 or above it raises an alarm; between 0.4 and 0.6 it abstains and hands the case to a person.

Littering score

Scene

Why abstain?

False alarms make innocent people suspects, and misses make enforcement toothless. Because the two mistakes carry different weights, it is hard to settle on a single threshold. So scores in the ambiguous zone (here 0.4–0.6) are not decided automatically; instead a control-room operator watches the footage directly. AI is used only to set the order of "which screen to look at first," and people make the final judgment and bear the responsibility. Press "Random scene" several times and look for throws that land in the bin and weak throws.

7Limits

Where this method goes wrong, and the next mission

A stick figure and 4 features are easy to understand and lightweight. By the same token, what they miss is clear, too.

Why it works

Small data: A pose is just 26 numbers, so it is light and fast. It can watch dozens of control-room screens at once.
Explainable: It can state the reason for a decision in numbers, such as "the can stayed outside the bin, and the person moved 0.77 away after letting go."
No faces: Because it decides from motion alone, the system can be designed to store less personal information.

Limits and risks

Depends on object detection: The strongest feature is "the can's position." If it is too dark to find the can, the detector collapses with it.
Limits of synthetic data: There are only six kinds of scenes here. In reality there are many scenes with ambiguous definitions, such as setting down a trash bag or moving someone else's trash.
Law and responsibility: In Korea, public CCTV is governed by the video information processing device provisions of the Personal Information Protection Act. Check the installation purpose, signage, and retention period, and have a person make the final check so that no one is punished on the AI's decision alone.
Bias: If pose estimation often fails for certain clothing or body types, false alarms may pile up on those people. Error rates must be checked separately for each group.

A delighted Assistant and the Doctor explaining with a pointer, in front of a stick-figure film strip, a joint number chart, and a feature bar chart — Next mission preview · The scene where a stick figure's movement becomes features

DoctorCatching works. Now let's go in reverse. Instead of looking at features and recognizing "littering," could we give the word "throw" and have the stick figure create a throwing motion?

AssistantFrom an AI that recognizes to an AI that creates? Then we'll need a ton of motion data.

Fruit Fly ResearcherThat's the mission of 5-2. We just flip the direction of the arrows in the same feature table.

Colab code — run this page yourself

Both code notebooks use only libraries preinstalled in Colab. After "Download notebook (.ipynb)", open it in Colab via File → Upload notebook, or copy the code and paste it into a new notebook cell.

Code 1 · Build stick-figure scenes and extract 4 featuresLinked to sections 3–5 · numpy, matplotlib · under 1 second on CPU

Computes the function that draws a stick figure from joint angles (fk), the six scenes, and the 4 features. It produces the same numbers as the table at the top of section 5. To use your own data, put the coordinates from a pose estimation tool where clip() returns joint coordinates, and divide so that the person's height becomes 1.

Run results (run on 2026-10-05, numpy 2.4 · matplotlib 3.10 · CPU). No random numbers are used, so the results are the same every time. Two graphs (wrist speed per scene, and four frames of the throw) are also produced.

Code 2 · Rules vs. learning — comparing five littering detectorsLinked to sections 5 · 6 · numpy, scikit-learn · about 20 seconds on CPU

Creates 600 training and 300 test examples by varying the scenes, and compares 2 rules, logistic regression, and 2 MLPs. The experiment table in section 5 and the weights in the section 6 control room come straight from this output.

Run results (run on 2026-10-05, numpy 2.4 · scikit-learn 1.8 · CPU). The data seed is fixed, but the MLP numbers may change slightly depending on the library version.

AI prompts — using this code as a starting point

These are prompts to paste into AI assistants such as ChatGPT, Qwen, or Claude. When you press "Copy," the selected code is attached after the prompt. Just replace the parts in 〔〕 with your own situation.

Code to attach

The AI's answer is only a starting point. Always verify suggested papers, figures, and library functions against the original sources and official documentation. When using footage of real people, first check filming consent and personal data processing rules.

Glossary

The same dictionary as the yellow dotted underlines in the text. Click a section name to jump to where the term first appears.

Analogy collection — and where the analogies break

Analogies are a first step toward understanding, but they can also plant misconceptions. Analogies to human perception are marked by strength. Direct there is evidence that it actually does the same thing · Partial only some properties are similar · None no known counterpart

Concept	Everyday analogy	Human perception analogy	Where the analogy breaks
Object detection	Sticking a Post-it on each person in a photo	Partial Noticing at a glance that "there's a person over there"	A person places Post-its accurately, but a detector also outputs wrong boxes, along with confidence scores
Pose estimation	Laying a wire doll over a photo	Partial The ability to recognize a walking person from the motion of just a few dots (point-light walkers)	People can guess where a hidden arm is, but when parts are hidden, pose estimation puts points in the wrong place or leaves them out
Feature	The "height, clothing, gait" items on a suspect sketch	None	People choose the items on a suspect sketch. A model that learns from raw coordinates makes up its own items (the MLP in section 5)
Rule-based	A security guard's rule of thumb: "someone running is suspicious"	None	A guard adjusts the rule of thumb based on context, but a rule simply fails on scenes outside its set conditions
Abstaining	A referee requesting a video review	None	Video review is used rarely, but if you widen the abstain zone, the operators' workload grows again

About the author and references

E-1 About the author · E-2 Academic references · E-3 Sources of figures and numbers

E-1 About the author

Ahn Sangsun (CEO, M-Robo Co., Ltd.)

After working at the Korea Electrical Safety Corporation and the Maeil Business Newspaper, he founded M-Robo Co., Ltd., a spin-off venture of Maeil Business Newspaper, where he currently serves as CEO. M-Robo began as an in-house venture at a news company and is a media venture that has provided a robot reporter service (StockBot) to Kookmin Ilbo and carried out numerous projects for the Korea Internet & Security Agency, the Korea Press Foundation, the Korea Communications Agency, and others. It now runs its own AI news service, Money-Robo. While serving as CEO, he won the Asia Open Data Hackathon and twice received the Minister of the Interior and Safety Award for contributions to data entrepreneurship, and he is the inventor and registrant of 5 patents in artificial intelligence and blockchain. He is the author of "AI Crime and Self-Defense" and has developed e-learning content on fintech, statistics, and artificial intelligence for the Online Lifelong Education Institute of Korea University of Technology and Education. He holds a Ph.D. in Economics (Dongguk University, microeconomics and mathematical economics). He teaches data and AI courses at several universities, including Kookmin University and Hoseo University. Full profile

Lectures

Ahn Sangsun (2025). "Data Science for Business" lecture notes. Department of AI Convergence, Open Cyber University of Korea.
Ahn Sangsun (2025). [K-Digital] AI Natural Language Processing. STEP (Smart Training Education Platform), Online Lifelong Education Institute, Korea University of Technology and Education.
Ahn Sangsun (2024). [K-Digital] Fintech Data Analysis Basics. STEP (Smart Training Education Platform), Online Lifelong Education Institute, Korea University of Technology and Education.

STEP course links: link.step.or.kr/1qyVoJ · link.step.or.kr/M9na9N · link.step.or.kr/brqEyc
If the links do not work because of a site redesign, please visit www.step.or.kr.

Papers and presentations

Ahn Sangsun (2025). A study on a household expenditure prediction model using the Korean Retirement and Income Study panel data: An AI model-based approach. Future Society, 16(2). Full text
Ahn Sangsun (2025). A study on a household expenditure prediction model using the Korean Retirement and Income Study panel data: An AI approach. Presentation at the KReIS Panel Conference (2025.05.13), National Pension Research Institute, National Pension Service. Slides
Ahn Sangsun (2025). A study on an algorithm for detecting suspected drug transactions based on SNS text: Focusing on pattern and context analysis. Future Society, 16(1). Full text
Ahn Sangsun (2024). Statistics-based countermeasures against adversarial data for securing anomaly detection models in finance: Empirical analysis and simulation results. Future Society, 15(3). Full text
Ahn Sangsun (2024). An AI-based solution for detecting SNS posts based on virtual character traits. Poster presentation at the 2024 Summer Newmedia Party (2024.06.19), Seoul Media Institute of Technology.
Ahn Sangsun (2024). Machine learning models using industrial accident panel data. Poster presentation at the 10th Panel Study of Workers' Compensation Insurance Conference (2024.05.17), Labor Welfare Research Institute, Korea Workers' Compensation & Welfare Service.
Ahn Sangsun (2024). Analysis and forecasting of virtual asset time-series data: Comparing time-series models and AI models, and suggesting improvements. Future Society, 15(1). Full text

E-2 Academic references

[R1] Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017). Realtime multi-person 2D pose estimation using part affinity fields. CVPR 2017, 7291–7299. arXiv:1611.08050 — Section 3 (pose estimation)
[R2] Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014). Microsoft COCO: Common objects in context. ECCV 2014, 740–755. arXiv:1405.0312 — Section 3 (17-keypoint scheme)
[R3] Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. CVPR 2016, 779–788. arXiv:1506.02640 — Section 3 (object detection)
[R4] Yan, S., Xiong, Y., & Lin, D. (2018). Spatial temporal graph convolutional networks for skeleton-based action recognition. AAAI 2018. arXiv:1801.07455 — Section 5 (skeleton-based action recognition)
[R5] Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.-L., Yong, M. G., Lee, J., Chang, W.-T., Hua, W., Georg, M., & Grundmann, M. (2019). MediaPipe: A framework for building perception pipelines. arXiv:1906.08172 — Section 3

E-3 Where the pictures and numbers come from

The scene pictures (alley CCTV, control room, feature lab) are pixel art made by the author
The joint coordinates in Section 3 were read directly off the alley picture, and the detection-box confidences (0.94 etc.) are example values for explanation
All scenes in Sections 4–6 are synthetic data. Only the experiment table in Section 5 and the weights in Section 6 are results of real training (Code 2)

Tips for educators

A teaching guide for instructors, on-the-job (OJT) trainers and tutors. Times are estimates, so adjust them to your actual class.

Audience	University students and working adults new to video AI. Practitioners considering CCTV analytics for public safety, facility management or store operations
Prerequisites	That a photo is a matrix of numbers (4-2), coordinates and distance calculation (Pythagoras), what accuracy means
Learning goals	① Explain the flow CCTV video → object detection → pose estimation → features → decision ② Give examples of scenes where rules and learning each go wrong ③ Propose an operating plan that weighs false alarms and misses, abstaining ("no call") and privacy together
Estimated time	Reading 35–45 min · animation demos 20 min · Colab practice 25 min (Code 1: 10 min; Code 2: 1 min to run + 14 min to interpret)

Three ways to run it

Lecture, 50 min

Concept-focused

Intro 5 min (Section 1 report) → Sections 2·3, from the screen to the stick figure, 12 min → Section 4 six scenes 8 min → Section 5 rule-threshold experiment 10 min → Section 6 control room 10 min → check questions 5 min.

On-the-job training, 90 min

Focused on adoption review

Sections 1·3 15 min → re-read the Section 5 experiment table in terms of false-alarm costs at your own site 20 min → Section 6 discussion on setting the abstain range 15 min → Section 7 build a legal/bias checklist 15 min → run and interpret Code 2 25 min.

1:1 tutoring

Pace by level

Beginners: only Sections 2·3, the upper table of 5, and 6. Intermediate learners: have them trace the features function in Code 1 by hand, then add one new feature (e.g. the minimum distance between the hand and the bin) and compare the scores in Code 2.

Teaching points by section

Section	Question to pause and ask	Hands-on tip
2 The screen is numbers	"Where is the person among 6 million numbers?"	In "View as numbers", shrink the cell size from 64 → 16
3 Finding the person	"If we keep only the stick figure, what do we lose and what do we protect?"	Move on to the next section with privacy mode still on
4 Into time	"Where do the graphs of waving and throwing differ?"	Click the two scenes in turn and compare the height of the yellow line
5 Rules vs. learning	"Where can we put the threshold so that all six scenes are right?"	Move the slider all the way to confirm "nowhere", then go on to the experiment table
6 Control room	"What happened in the frame where the score suddenly jumps?"	Play the drop scene and pause the moment the can hits the ground
7 Limits	"If the AI sounds an alarm, can we issue a fine right away?"	Connect abstaining with a final check by a human

Top 5 common misconceptions

Misconception	Correct explanation	Scene to show
AI "sees" the CCTV screen like a person does	It receives a grid of numbers and computes step by step. Labels like "person" and "bin" are attached by the detection model	Section 2 "View as numbers"
Swinging an arm fast means littering	Waving is fast too, and dropping is slow. The outcome (where the object ended up) is a stronger clue	Section 5, Rule 1
99.7% accuracy means 99.7% in the field too	That is a result on synthetic data. On real video it drops because of occlusion, darkness and multiple people	Section 5, explanation 3 below
Deep learning on raw coordinates is always better	With little data, a few well-chosen features can win	The two MLP rows in the Section 5 experiment table
If we store only the stick figure, there is no privacy problem	It reduces the risk but does not remove it. People can be recognized by their gait, and the rules on installation and storage still apply	Section 3 privacy mode, Section 7

Check questions (click to expand the answers)

Q1. How many numbers are in one frame of Full HD (1920 × 1080) color CCTV? Compared with a 13-joint stick figure, what fraction is that?

1920 × 1080 × 3 = 6,220,800. The stick figure has 13 × 2 = 26, so 6,220,800 ÷ 26 ≈ 239,262: about 1/240,000.

Q2. In one frame (1/15 s), the right wrist moved from (−0.20, 0.80) to (−0.32, 0.88). What is the wrist speed in heights per second?

Distance = √(0.12² + 0.08²) ≈ 0.144. That took 1/15 s, so 0.144 × 15 ≈ 2.16 heights/s. It exceeds Rule 1's threshold of 2.0.

Q3. Give one littering scene that Rule 2 (wrist speed > 2.0 AND the object is released) misses, and one normal scene it wrongly flags.

Miss: dropping at your feet (the wrist is slow). False alarm: a hard throw that lands in the bin (fast and released, but not littering).

Q4. In the logistic regression weights, wrist speed got −0.33. Does that mean "the faster, the less likely it is littering"?

No. It is its share with the other features (especially the distance outside the bin) included at the same time. Since the distance outside the bin already explains almost all littering, speed is left with only a small role: slightly pushing down scenes that are fast but normal, like waving. Never interpret one weight on its own.

Q5. If the control-room score is 0.52, what should the system do? How is that different from using a single threshold of 0.5?

It falls in the abstain range (0.4–0.6), so the system makes no automatic decision and hands the video to a control-room operator. With a single threshold, 0.52 would immediately become a littering alarm, and false alarms on ambiguous scenes would land straight on people.

Colab assignments

Basic

Add a fifth feature to Code 1: "the horizontal distance between the hand and the bin at the moment the can is released". Tabulate this value for the six scenes and explain in two sentences whether it helps separate putting it in the bin from dropping it.

Challenge

In make_dataset in Code 2, change the number of training scenes 100 → 25 → 400 and compare the test accuracy of logistic regression and "joints + objects + MLP". Report with a graph how the gap between the two approaches changes with the amount of data.

Adjusting difficulty and going further

Beginners	For the graph in Section 4 and the experiment table in Section 5, it is enough to read only "which scene went wrong"
Advanced	Have learners extract joint coordinates with MediaPipe from a short video they shoot themselves (acted, with consent) and feed them into features in Code 1
Next episode	5-2 From Recognizing to Creating — flip the arrow on the same feature table to generate motion